How can we read variables from file and use them in promql query?

Grafana Configuration
,
1

I have configured docker container and host dashboard in grafana. By default it shows data for all docker containers:

enter image description here

Note that their are two docker stacks / swarms: xyz_stack_1 and xyz_stack_1_monitoring. xyz_stack is docker swarm of my react web app while xyz_stack_monitoring is docker swarm of monitoring the web app and it contains prometheus, grafana, node exporter and cadvisor. As it can be seen, the panel shows information about containers from both swarms.

Now, I have embedded this panel in my react web app and obviously it also shows all docker containers from both swarms. I need to put a toggle button in my web app, so that it will allow toggling between information of just xyz_stack_1 or both swarms (above panel).

Promql query of above panel is:

sum(rate(container_cpu_usage_seconds_total{name=~".+"}[$interval])) by (name) * 100

For panel containing information of just xyz_stack_1 containers, I can duplicate the panel and change its query to:

sum(rate(container_cpu_usage_seconds_total{container_label_com_docker_stack_namespace="xyz_stack_1"}[$interval])) by (name) * 100

I tested this query in grafana and it works as intended.

Now on toggle click, I can show / hide panels.

However I am not able to figure it out, how can I feed the string xyz_stack_1 to grafana. I have setup docker-compose.yaml file to start up all the containers. The configuration is done through prometheus.yaml, grafana.ini, dashboards.yaml and datasources.yaml

My guessing is that I will set variable with value xyz-stack-1 in some of these files and refer in grafana dashboard promql query. But how do I exactly do this? Am quite new to prometheus, grafana and promql.

PS: above various yaml and ini file links are for example. These are not exactly my files. But I follow similar appoach.

PS: I asked bing chat “how can I read variables from text files in grafana?”. It says: Create a new data source of type “File”… While chatgot say: You can do this by going to the Plugins section in the Grafana UI, searching for “Text File” in the plugin list, and installing/activating the plugin. But I guess none of them exist right?

2

Use a dashboard variable that queries prometheus and gets only the stacks /swarms names
xyz_stack_1 and xyz_stack_1_monitoring
Name that variable filters

Then use that variable like so

sum(rate(container_cpu_usage_seconds_total{container_label_com_docker_stack_namespace=$filter}[$interval])) by (name) * 100

I am not too well versed in promql but you can sort out the details yourself

3

I created dashboard “Custom” variable app_stack_name with single value xyz_stack_1, used it in promql:

sum(rate(container_cpu_usage_seconds_total{container_label_com_docker_stack_namespace="$app_stack_name"}[$interval])) by (name) * 100

exported dashboard json file and put it in dashboards directory so that this pre-configured dashboard will get provisioned automatically. If someone changes the stack name from xyz_stack_1 to say abc_stack_1, one has to replace all instances of xyz_stack_1 with abc_stack_1 in json file and then restart grafana container.

1 Like