Inconsistent data from Graphite when changing timeframes to 6+ hours

Grafana Graphite
1

  • What Grafana version and what operating system are you using?
    9.4.7 running in a docker on Ubuntu, and Graphite 1.1.8

  • What are you trying to achieve?
    Show a movingSum of a metric for multiple nodes. I have X devices sending data and a metric and follows how many data messages each device sent. I want to sum how many messages each device has sent over the past N minutes (N could be anywhere from 5 to 30 minutes) and show how many devices have sent a number of messages above a certain threshold during those N minutes, indicating they are ‘alive’.

  • How are you trying to achieve it?
    alias(sum(divideSeriesLists(movingSum(removeBelowValue(my.metric, 10), “10minute”), movingSum(my.metric, 10), “10minute”))), “Kafka”)

  • What happened?
    When changing my graph’s timeframe to anything above 6 hours I see results MUCH lower than expected.

  • What did you expect to happen?
    I expected to see 20 devices, as I do when I configure the timeframe to anything 3 hours or lower, but only saw 3 devices. The value displayed in the graph changes based on the timeframe.
    To check if my overly complicated query is the problem, I tried to simply run sum(my.metric) and that has also showed different results for the different timeframes.

I’d greatly appreciate any guidance to help me configure this graph properly!