Hi,
I wonder if such configuration is possible :
I have my telegraf running connected to my MQTT broker [inputs.mqtt_consumer], then I tried to configure [outputs.http] to push mqtt data to a stream/custom_stream_id/mqtt
But I can’t figure out where to set the topic in the grafana Measurement configuration panel
May also use grafana MQTT plugin, but as it does not support TLS yet…
Does anyone tried this ?
Thanks
Hello and welcome to the Grafana forum.
I presume you want the MQTT “live” data to stream to a panel in Grafana? See the panel at the bottom of this page.
If yes, then you would need to install the MQTT datasource. It’s not a straightforward exercise, but it can be done with some knowledge and patience. Check out this thread and maybe @antonio79 can intervene if you get stuck.
Hi grant2 and thanks for welcoming me and reply !
Well, I 've already install MQTT data source and here are my topics :
1 : using an external cloud-based broker such as HiveMQ (not NAT, fine), It works well (Just had to recompile client.go from source and modify to set SSL instead of TCP opts.AddBroker(fmt.Sprintf(“ssl://%s:%d”, o.Host, o.Port))
2 : using an" internal" broker (problems will come with NAT for clients…) such as mosquitto in the same host where grafana is running, connection OK too 
question is , regading what’s told in the MQTT DATA source README
“Including multiple topics in a panel is not yet well supported.”
I tried to publish two different topics and I can monitor then using two different query
Can I do the following :
publish my/topic/
publish my1/topic1/
and , in the panel, I had one query and related topic ?
is there a limitation for numbers of topic and levels hierarchy topic/x/y/z/…
Thanks for your support.
Sounds like something that @atifshoukatali or @antonio79 might be able to help with.
Guys: Any advice?
I curently have 11 topics being fed to MQTT datasource as a stat panel here on the right column (using internal MQTT broker)
So, 11 works. Sometimes it takes upto 5 seconds for the number to show up, but thats the only issue(?) i can report.
EDIT: to add to this, all levels are w/x/y/z
@skillDo yeah any level should be okay with the datasource as @antonio79 mentioned - and as far as I know there should be no limitations on the number of topics - but i would assume as you add more and more panels and topics your dashboard might load up a bit slower
@skillDo as for your original post and question for any measurements coming through from the grafana live setup you should be able to see all the channels being published under the grafana datasource live measurement query type under the view as below:
if your streams dont show up then i would double check your telegraf setup since it can be a bit challenging to get it right sometimes
Hi @atifshoukatali , @antonio79 , I am very greatfull you give some much details for help,
Besides, since i just discovered Grafana, influxDB and telegraph few weeks ago, I am little confused about the best option between :
telegraf [input= MQTT broker] => Telegraf HTTP output plugin) , so using grafana datasource live measurement query type
and
MQTT data source plugin [MQTT Broker] => directly (via GPRC) to the grafana panel using topics of my choices
I understand telegraf come with a bunch of plugins (that’s a key-point I presume), but may be MQTT datasource is THE choice when acting with MQTT broker ?
then, I also follow-up grafana discussions about persistence storage (instead pushing to Influx DB…), If you have some material links to read ?
thanks for all guys !
@skillDo yeah indeed if you want to be able to just visualize the data from an mqtt broker on your dashboard i would stick with the mqtt plugin (note this is very limited in functionality right now so hoping to get more features out on this soon) - but if you want to be able to integrate well with the influxdata ecosystem and/or other systems for storage needs the most flexible option would be to use telegraf as the middle man to deal with all this with grafana being the tool for vizualization in that case - within the grafana ecosystem for storage needs prometheus (https://prometheus.io/) could be of interest to you - we are actively working on making such use-cases better served with all the needs you mentioned here - so stay tuned - this will only get better from here on out!
Ok then !
so, I could use Prometheus instead of influx for storage
until now, I was pushing data using python influxdb-client library
I will check on Comparison to alternatives | Prometheus to better understand the key points
Thanks again, and I hope I would be able to contribute some day
yeah do check it out and report back with the final dashboard and the solution you end up implementing!
Yes, I will when eveything’s set up on my side. will take some time…
In between, I would like to get your fedback on a particular matter:
Suppose I have raw data in CSV file with ten’s and ten’s of field (could be hundred…)
Then, using InfluxDB (CSV have been put in bucket) , its very convenient to browse the bucket with the “Data Explorer” by selecting items in the big list , clicking on them…
But using Grafana, I guess there is not such facility right ? , because data are stored outside of grafana.
One must build the query by writing with query language , knowing the exact names of fields right ?
thanks for your feedback
@skillDo the query language for influx is quite flexible (Influx Query Language (InfluxQL) | InfluxDB OSS 1.8 Documentation) - you could structure your query in grafana to output certain fields or all fields matching a certain criteria or to get ALL the fields - but depending on your application the resulting visualizations might not be the best approach if you are looking to experiment with the data - maybe use the explore tab in grafana if you want to play around with the data stored in influx a bit? hope this helps
Hi @atifshoukatali
My dashboard is now working fine using MQTT plugin
Then, I wonder about the refreshing value which I can not set under 5 s using GUI. Is 5s the best I can expect, or is there a programmatic way to set it under 5s
Thanks for your feedback
^^ woops looks like the message got deleted - umm well i guess i’ll provide a possible solution either way - so to change the default refresh rate options you will have to make a change in the ini file - admittedly can be confusing: Lowest Refresh Rate Now 5s?
also maybe if the panels are mostly just streaming in real-time i would suggest just disabling refresh since that is not needed with the built in live streaming setup introduced in 8.0
Hi thanks it works
But I have another concern : I can see in this demo :
There are multiple different Topics aX, aY and aZ in the same panel
I can’t figure out how to have multiple : when I had a new query on a new topic, grafana panel display ‘value’ name twice and when I change the nane, It changes for both.
I see what you mean. In my example below, both the Setpoint and Actual are called “Value” in the legend. @atifshoukatali Is there a way to correct this?
Also, I have run the same MQTT subscription using telegraf input mqtt consumer
and in this way, do you think it is easier to filter about topics in the SAME panel ?
@skillDo can you please open up an issue for this here: GitHub - grafana/mqtt-datasource: MQTT Datasource for Grafana allows streaming data from any MQTT broker running either locally or remotely. - make sure to include as many setup details as possible so we can easily reproduce this
Hi I am not sure to get the point : are you talking about the issue : not able to display different value in the same grafana panel ?
yes indeed - some details on how to quickly reproduce it would be very useful if possible
