t=2017-04-29T00:16:04+0000 lvl=info msg=“Sending notification” logger=alerting.notifier type=webhook id=0 isDefault=false
t=2017-04-29T00:16:04+0000 lvl=info msg=“Sending webhook” logger=alerting.notifier.webhook
t=2017-04-29T00:16:04+0000 lvl=eror msg=“Failed to send webhook” logger=alerting.notifier.webhook error=“Post https://hooks.slack.com/services/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: read tcp 172.17.0.3:54532->172.17.0.1:443: read: connection reset by peer” webhook=TestAlert
t=2017-04-29T00:16:04+0000 lvl=eror msg=“Failed to send alert notifications” logger=context userId=1 orgId=1 uname=admin error="Post https://hooks.slack.com/services/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: read tcp 172.17.0.3:54532->172.17.0.1:443: read: connection reset by peer"
t=2017-04-29T00:16:04+0000 lvl=eror msg=“Request Completed” logger=context userId=1 orgId=1 uname=admin method=POST path=/api/alert-notifications/test status=500 remote_addr=172.17.0.1 time_ms=2ns size=48
[8:26]
hi all, im trying to see which port to forward from grafana onto vm that hosts container but it lookslike slacks webhook is using multiple ports which seems to be causing the issue, please help out if you have any idea on this, thanks!
Calls to Slack are normal http requests on port 80 for http or port 443 for https. This sounds like a firewall problem.
- Can you explain a bit more about your setup?
- Are you able to access
https://hooks.slack.com/services/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 from the server/container where Grafana is installed?
thanks for your help, yes that was a proxy issue and has been resolved thanks.
1 Like
Although can you help me pout with the below mentioned error, this is whenever is try to write a alert rule it throws me this error?
sdb.HandleRequest() error Get http://10.205.79.78:4040/query?db=telegraf&epoch=s&q=SELECT+mean("available_percent")+FROM+"mem"+WHERE+time+>+now()+-+1s+GROUP+BY+time(1ms)+fill(null): EOF
Can you show more of the error? See this guide on how to check the response in the network tab in Chrome. Looks like a connection problem with your InfluxDB.
i dont know what is throwing that err, please help
Have you checked the Grafana server logs? It looks like the Grafana server does not have access to your InfluxDB database. Did you choose Direct mode instead of Proxy mode for your InfluxDB data source in the data source settings?
It could also be an error in the query. Does that query work if you test it directly?
just checked the grafana server logs, it says the same err but prior to that err it says “could not find datasource”, Yes i did choose direct mode in datasource settings, is that a problem?
and i did not test the query directly, should i directly query that thru influxdb ui?
If you want alerting then you will have to choose Proxy mode. Any special reason for choosing Direct (security maybe?)
ok so if i choose proxy mode do i need any one of the http auth methods? and where do i setup those credentials then
?
the only way datasource working for me is thru the direct mode when i set it to proxy mode it throws the err influxdb error undefined
Alerts run in the browser can use Direct mode and use your authentication but that will not work on the server. Alerting runs the queries against InfluxDB from the server and is not authenticated. So it needs a username and a password. Skip the http auth methods:
ok so here is the thing i’ve given the url and proxy mode and the user name password i’ve provided is the once which i created in influxdb?
Now you are getting the same error that the alerting was throwing before. Can you use the guide that I linked to above to troubleshoot the error with the Chrome developer tools. Can you access the InfluxDB server from your Grafana server? If you ssh (or remote desktop) into the grafana server and try access the InfluxDB database, does it work?
1 Like
so what we have here is on our VM we have installed docker containers for influxdb, kapacitor and grafana.i logged into my grafana container , is that what you want me to do and check if i can access influxdb from the grafana container? i dont think i will be able to do that ??
Why not? Why does your Grafana server not have access to the InfluxDB database? Or do you not know how to check the connection -> check these docs for how to query via the InfluxDB HTTP API: https://docs.influxdata.com/influxdb/v1.2/guides/querying_data/
throws me an err unabel to parse authentication credentials
Are the credentials correct? Can you log into the influxdb admin ui with them? (It’s usually on port 8083)
yes i can . i created them thru the influx UI and updated them in grafana as well.
Hard for me to say why it doesn’t work on your Grafana server. Firewall settings?
