Does the Grafana 9 Alert have an annotation for template variables? (Image Rendering)

Grafana Alerting
, ,
1

What Grafana version and what operating system are you using?

  • Grafana 9.0.0

What are you trying to achieve?
I use jmx exporter and set alarm-rule by using Reduce Expression .
Preview Alert state and Annotations setting is like this.

image
image1466Ă—193 13.7 KB

image
image706Ă—240 9.44 KB

But When the alarm state changes and the image is sent to the Slack, the image is sent only to A instance img as the representative template variable in the dashboard, regardless of the instances A, B, or C.

Regarding this, I wonder if there is an Annotation setting or other setting method for sending the correct image when rendering the image after setting the alarm.

2 Likes
2

@yeongun1026 welcome to the :grafana: community!

This could be due to a limitation of image rendering noted here

  • When alerts generated by different alert rules are sent in a single notification, there may be screenshots for each alert rule. This happens if an alert group contains multiple alerting rules. The order the images are attached is random. If you need to guarantee the ordering of images, make sure that your alert groups contain a single alerting rule.
  • Some contact points only handle a single image. In this case, the first image associated with an alert will be attached. Because the ordering is random, this may not always be an image for the same alert rule. If you need to guarantee you receive a screenshot for a particular rule, make sure that your alert groups contain a single alerting rule.
3

Hi,
Use case @yeongun1026 provided is not exactly related to limitations you’re referring to @melori.arellano .
Limitations are about alert groups while yeongun’s example uses single alert rule, but this is multi dimensional one.
I’m facing the same problem and let me elaborate how it looks like.

In my case data source is Google Cloud Monitoring.
I have dashboard containing graphs showing some VM Instances metrics (CPU, memory, etc.) Each graph is grouped by instance name. There are two variables for this dashboard:
project - as it’s used for a few GCP projects
host - as I want to filter graphs for specific instance from time to time.

image
image917Ă—875 73 KB

I also created multi dimensional single alert rule for VM instances CPU usage.
This one is associated to “CPU usage of each VM Instance” graph shown above, using DashoardID and PanelID.
Now, assuming I have 10 VM instances I may receive one email with 1-10 notifications when alert is triggered.

image
image1137Ă—131 15.7 KB

And each of this notification will have the same rendered picture attached as it is not possible to pass VMs instance name to rendered graph as a variable. It doesn’t make sense.

Another thing. As I’ve mentioned above I’ve “project” variable in my dashboard for viewing VMs for a few projects. Having no option for passing project name to graph variable “project” when rendering I would need to create separate dashboards for each project. Same situation - It doesn’t make sense.

So how it should work for multi dimensional alert rules in my opinion ?
If images rendering is enabled and alert rule is correctly associated to dashboard/panel we should have additional configuration option determining values pairs saying which label in alert rule should be passed to which dashboard variable, for example in my case:
resource.label.project_id → $project
metadata.system_labels.name → $instance

What we’ll achieve this way ?

  1. Using the same dashboard/panel for many projects avoid multiplying them
  2. Each notification in single alert email having attached image showing CPU usage only for related VM instance

Additionally I would consider adding one more option for multi dimensional alert, let’s call it “Single image” to avoid rendering and attaching many pictures to one email.
How would it work in my use case ?
Regardless of VM instances number alerting high CPu usage I can show it on one picture (based on panel showing CPU usage for each VM instance) as in fact I don’t need to have separate pictures for each alerted VM instance.
In such case I would only to pass only one label to dashboard variable:
resource.label.project_id → $project
for selecting GCP project alert rule is related to and enable “Single image” option.

Summarizing: My understanding is currently image rendering functionality is useful only for single alerts (not multi dimensional) not grouped, but definitely is great one.
Just should be improved a little bit :slight_smile:
Hoping someone will review this as Feature Request.
If it’s not a correct place for such a request please provide the correct one I can post it to.

Have a nice day !

Ged

4

Moved to GitHub discussion.

1 Like