I am attempting to auto-populate some singlestat panels based on the existence of specific services that I am collecting metrics on using the Prometheus node_exporter (i.e. the node_systemd_unit_state metric). If I populate each of the singlestat panel queries with service names manually it works fine (e.g. name=“tomcat.service”), but since each host may or may not have a given service I only want to display singlestat panels for each service that is present. To accomplish this I variablized the service query and then used the “Repeat by variable” option to create singestat panels that correspond to a given host. The variable $service was added to the singlestat panel query and “Panel title”. When I select a host that has multiple services it generates singlestat panels for each of the services and displays the Panel title correctly. However, the panels themselves return “No Value”.
Here are the variables set on the dashboard as well as the query and panel title.
label_values(alias)
label_values(node_systemd_unit_state{alias="$alias"},name)
Query
scalar(node_systemd_unit_state{alias="$alias",name="$service",state="failed"}*1) + scalar(node_systemd_unit_state{alias="$alias",name="$service",state="inactive"}*2) + scalar(node_systemd_unit_state{alias="$alias",name="$service",state="deactivating"}*3) + scalar(node_systemd_unit_state{alias="$alias",name="$service",state="activating"}*4) + scalar(node_systemd_unit_state{alias="$alias",name="$service",state="active"}*5)
Panel Title
$service
