Unified alerts query missing "no data" data points

Grafana
1

  • What Grafana version and what operating system are you using?
    v8.2.0

  • What are you trying to achieve?
    Setting up an alert based on Loki data source, which would alert if the value is 0.

  • How are you trying to achieve it?
    Using unified alerting tool, passing the necessary query below and using a classic alert conditional.

  • What happened?
    Query doesn’t take in consideration “no data” values and doesn’t stack them properly, where “no data” would equal to a 0. As you can see in the screenshot below, data points are directly connected and there is no data point present when the query doesn’t return data.

image
image1612×246 18.2 KB

  • What did you expect to happen?
    Where query provides no data it should be an option to transform it to 0.

  • Can you copy/paste the configuration(s) that you are having problems with?

sum(count_over_time({namespace="$namespace"} | json | source="$source" | type="$type"[1m])) by (app)

When there are no such events to fall under the query above, query returns no data. Once that happens Grafana query doesn’t take “no data” value as a separate data point, but just skips the data point all together and connects only the existing data points.

  • Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.
    N/A

  • Did you follow any online instructions? If so, what is the URL?
    N/A

2

did you manage to find a workaround for this? Im facing the same problem

3

No, I haven’t found the solution.

In fact loki event visualization doesn’t work very well in Grafana at all, since no events equals empty strings and Transformations do not work.

Only way to see that something is 0 is to stack series, but you can only do that if you have only 1 series, once you get more = graphs are once again unusable.

4

I ended up creating a new alert with the following prometheus query

up{environment="prod"} == 0

in this way, the targets which are not reporting will be displayed on the multidimensional alerting in Grafana 8+

5

This topic was automatically closed after 365 days. New replies are no longer allowed.