Potential Bug: diff() alerts do not work, please review

Grafana Alerting
1

  • What Grafana version and what operating system are you using?
    6.7.4

  • What are you trying to achieve?
    Use the diff() function for alerting

  • How are you trying to achieve it?
    Eval montonic increasing counter every 1m for 5m. Evaluate points from (1h, now). Get the diff and check if there has been any change. IE, diff() > 0.

  • What happened?
    diff() gets negative values when I know it’s not appropriate

  • What did you expect to happen?
    ^^

When I test my alert, I see that the points do not come back in sorted time order. This seems like a problem. As I can’t find any docs on diff() , I checked the code. In summary: it gets the first and last elements of the data array and diffs them.

It does seem to be doing what I expect with the logic above. IE query(t1) - query(t0) , except t1 and t0 here are not as expected, they are random order. The test data looks like this -

0:Array[15172,1626882120]
1:Array[15192,1626882900]
2:Array[15257,1626884820]
3:Array[15159,1626881460]
4:Array[15170,1626881940]

Clearly Point 3 and 4 occurred before 0 and 1?

What is going on here?

I posted this last week but got no replies - Why is test data not in sorted time order?. I tried to contact through support email.

I can get permission to work on this if someone can please confirm this is an actual bug.

2

@davidparrott I see you replying to alerting questions, could you please take a quick look at this? I really need an answer. I am sorry to bother you all, but this is blocking my work, and I need to know if I need to try to work on it as an individual contributor.

3

@sarahmarshy I don’t know much about the legacy alerting system. I work on the new alerting engine but have not spent a lot of time with anything that existed prior to the 8.0 release. You’re going to have the best luck using regular support channels and I would create a github issue for it as well. I wish I was better able to look into this for you. Is it possible to use a newer version of Grafana?

4

Here is the newly created issue for anyone potentially following along. Thanks for creating that @sarahmarshy

5

This topic was automatically closed after 365 days. New replies are no longer allowed.