When checking the dashboards for a synthetic monitoring check, the template variables of the dashboard, job and instance, indicate a 1:n mapping between a single job and multiple instances. And indeed, that’s what I would like to achieve. However, I don’t see a way to configure a check in a way, that it queries multiple targets (target translates to instance in the dashboard).
On a related note, is there a way to even load these targets dynamically, e.g. from a datasource?
Howdy! At the moment you can only have one target/instance per check. You could create multiple checks with the same job name querying different targets if that’s something you’d like to do, but the combination of job and instance labels is what identifies a check in the metrics produced by the synthetic monitoring agent.
is there a way to even load these targets dynamically, e.g. from a datasource?
If you’re looking to get a list of the targets that are already producing metrics, you can do so via the Prometheus datasource. The instance value is on the sm_check_info metric, so you could do something like count(sm_check_info) by (instance).
Hi, thanks a lot for the quick response, this was already very helpful.
Regarding the dynamic list of targets, I want to create the checks dynamically. So something like Run job X for all instances described by list Y. As I understand, this is currently not possible?
You could create multiple checks with the same job name querying different targets if that’s something you’d like to do
This is already important information, thanks! Is there a way to label the instances, so that I can get a name instead of the target value in the dropdown list of the dashboard?
I want to create the checks dynamically.
I believe the only way to do that at the moment would be to manage your checks programatically, either through the API directly or through the grafana terraform provider.
Is there a way to label the instances, so that I can get a name instead of the target value in the dropdown list of the dashboard?
There isn’t any way to do this out of the box, unfortunately. Those variables are informed by a prometheus query, so those dropdowns are showing the values of metric labels. In order to display something different you’d have to make a copy of the dashboards and edit the variable query. I don’t think it’s possible to alias prometheus based dashboard variables, but I could be wrong about that.
Thanks again for the quick and informative answer.
I believe the only way to do that at the moment would be to manage your checks programatically, either through the API directly or through the grafana terraform provider.
Using the API was indeed what I have done now and it worked well for our use case for now. Thanks for confirming this approach.
In order to display something different you’d have to make a copy of the dashboards and edit the variable query.
I was also thinking about making an editable copy if we feel the need for this.