Dashboard failing to retrive data

Grafana Dashboards
1

I am trying to use Node Exporter Full dashboard for Grafana | Grafana Labs dashboard. Sometime it loads the data, but other times it doesn’t. I see following error message in the Grafana logs.

logger=tsdb.prometheus t=2022-06-09T02:25:47.66+0000 lvl=eror msg="Range query failed" query="sum by (instance)(rate(node_cpu_seconds_total{mode=\"system\",instance=\"10.0.17.50:9100\",job=\"node-exporter\"}[1m0s])) * 100" err="Post \"http://prometheus:9090/api/v1/query_range\": context canceled"
logger=tsdb.prometheus t=2022-06-09T02:25:47.66+0000 lvl=eror msg="Range query failed" query="sum by (instance)(rate(node_cpu_seconds_total{mode='user',instance=\"10.0.17.50:9100\",job=\"node-exporter\"}[1m0s])) * 100" err="Post \"http://prometheus:9090/api/v1/query_range\": context canceled"
logger=tsdb.prometheus t=2022-06-09T02:25:47.66+0000 lvl=eror msg="Range query failed" query="sum by (instance)(rate(node_cpu_seconds_total{mode='iowait',instance=\"10.0.17.50:9100\",job=\"node-exporter\"}[1m0s])) * 100" err="Post \"http://prometheus:9090/api/v1/query_range\": context canceled"
logger=tsdb.prometheus t=2022-06-09T02:25:47.66+0000 lvl=eror msg="Range query failed" query="sum by (instance)(rate(node_cpu_seconds_total{mode=~\".*irq\",instance=\"10.0.17.50:9100\",job=\"node-exporter\"}[1m0s])) * 100" err="Post \"http://prometheus:9090/api/v1/query_range\": context canceled"
logger=tsdb.prometheus t=2022-06-09T02:25:47.66+0000 lvl=eror msg="Range query failed" query="sum (rate(node_cpu_seconds_total{mode!='idle',mode!='user',mode!='system',mode!='iowait',mode!='irq',mode!='softirq',instance=\"10.0.17.50:9100\",job=\"node-exporter\"}[1m0s])) * 100" err="Post \"http://prometheus:9090/api/v1/query_range\": context canceled"
logger=tsdb.prometheus t=2022-06-09T02:25:47.66+0000 lvl=eror msg="Range query failed" query="sum by (mode)(rate(node_cpu_seconds_total{mode='idle',instance=\"10.0.17.50:9100\",job=\"node-exporter\"}[1m0s])) * 100" err="Post \"http://prometheus:9090/api/v1/query_range\": context canceled"

What is wrong here ?

Versions
Grafana : 8.5.5
Prometheus : 2.36.0

2

Hi @rp346

It’s possible that the query is trying to fetch too much data and is timing out, does that seem to make sense with what you’ve seen? What time period are you querying (1 day, 1 month)?

Can you share more about the differences between the queries that are returning and the ones that are failing?