What do you mean by data source? Im using InfluxDB…
If I want to make the same query as Template a ? How can I do it?
In Grafana manual you can see that they are using template variables. They are using template A in their example.
there is no use of templating in that example, just refering to query A from your metrics tab.
Your query A seems to be using template variables, hence the big red alert error message at the top of your alert tab informing you that this is not supported.
@daniellee, i just did a fresh install of grafana 4.4.2 and i am trying to test some alerts. Eventhough i see the alerts in the grafana dashbord, i am unable to receive any alerts. I am able to ping/connect to influxdb machine from grafana and i have enabled debug level logging and i dont see any errors. Below are the details:
Query:
Alert Panel:
Could you please take a look where i am going wrong.
Thanks,
@torkel/@daniellee
Grafana is able to send me emails for OK status or NO DATA status, but after that if an issue occurs… i am able to see it in grafana dashboard but i dont receive any alert.
Below is an example:
Check your group by time interval? What does test rule return?
Also, you are checking the latest minute (now - 1m) in your alert condition, is there any data for the latest minute? Just so you know, you can also do something like this: query (A, 1m, now-1m) to check for the interval now-1m and now-2m.
@torkel its returning null.
@daniellee, could you please help me understand this.
so if want to send an alert if i have issues in the last 24 hours, i should change my condition to ‘query(A,1m,now-24h)’.
Is this correct?
No, that would give you a range of now-24h-1m to now-24h. In that case just use: query(A,24h,now). The extra minute at the end does not matter in this case.
@daniellee, ok so if i want to check if there were any issue in the last 8 minutes. Then this should be the format 'query(A,10m,now-2m)’.
correct?
Below is my current and its checking for every 1 minute
I have one last question, if i want to send alerts continuously every 5 minutes (in case of an outage). How should i configure the Alerts?
This will check the range 10 minutes ago to 2 minutes ago.
We don’t have support for this yet. Feature request here: https://github.com/grafana/grafana/issues/7330
@daniellee, Thanks for the correcting me.
I am again having some issues.
I am running my query to check for any errors for every 5m with the condition of query(A,10m,now) OR query(B,10m,now) to check if there were any issues in the last 10m from now. Below is the image of alert config:
According to the first image, i received a OK alert at 12:06:40 and after that there were no errors so there was no alert sent at 12:11 (after 5m).
At 12:15 i got an error and according to my alert config, alert check should happen on 12:16. I was expecting an alert at 12:16:40 as there were errors matching the my alert config but i didnt receive any alerts.
I checked the logs and below is what i found at 12:16:
t=2017-08-04T17:16:40+0000 lvl=info msg=“Alert Rule returned no data” logger=alerting.evalHandler ruleId=6 name=“ISSUES XXXXXX” changing state to=ok
t=2017-08-04T17:16:40+0000 lvl=dbug msg=“Job Execution completed” logger=alerting.engine timeMs=9.606 alertId=6 name=“ISSUES XXXXXX” firing=false
I am unable to understand this behavior. Could please take a look and let me know if i need to make any changes in my config.
Thanks.
@torkel @daniellee
Another occurrence with the alerts.
I dont know why its sending an alert as OK when i there are issues coming up and not sending an alert after the 1m when the query runs again.
Below is the image of the logs:
@daniellee @torkel
Anything on the above mentioned behaviour?
looks like your query is not returning any data.
You can enable morel logging if you feel if you want to dig deeper.
http://docs.grafana.org/alerting/rules/#troubleshooting
Maybe it’s a problem with timezones, or time range intervals? the interval for a 2m becomes very small and influxb may influxdb resturns nothing.
Thanks @torkel @daniellee
I increased the time range to 15m and now i am getting alerts for every 5m so that worked!
But when i try to set my alert level to send me an email when the number of issues is more that 10… i am receiving and email even if there is only one issue.
The test rule shows me a Value of 16, which i think is the time range its checking for but its not checking if the count is greater that 10 or not.
In the below screenshot, you can see that there is only one issue in the last 15m but still i am receiving an email
This is my query:
SELECT count(“request”) FROM “Tab” WHERE “responseType” = ‘5XX’ AND $timeFilter GROUP BY time(1m) fill(null)
Could you please take a look?
Nevermind,
I changed the condition from count() to sum().
Now its giving me the number of issue for the last 15m.
thanks,
1 Like