How to filter result of label_value?

Grafana Configuration
1

Hi, we are running a platform of jobs, there are one or more tasks in single job, so we are exporting metrics like:

task_mem_usage{"job"="a", "task_name"="1"} 2
task_mem_usage{"job"="a", "task_name"="2"} 3
task_mem_usage{"job"="b", "task_name"="1"} 2

we want users can see both job-level metrics and task-level metrics.

So we defined two variables: job and task, the query for job is easy: label_values(task_mem_usage, job). But if task is defined as label_values(task_mem_usage, task_name), it would be hard to select, especially when task_name is user defined and there are too many jobs.

From this, it seems we can use variable in other template’s definition, so I defined task’s query as label_values(query_result('task_mem_usage{"job"="$job"}'), task_name), but grafana gives error of Template variables could not be initialized: parse error at char 13.

Not sure if grafana support this? Maybe label_values(task_mem_usage, task_name, ["job"="$job"]) is more clear.

Thanks,
Di

2

Please use this if you allow selection of multiple jobs:

label_values(task_mem_usage{job=~"^$job"}, task_name)

Please use this if you allow single selection job:

label_values(task_mem_usage{job="$job"}, task_name)
3

task_mem_usage{“job”=“a”, task_name=1}
task_mem_usage{“job”=“a”, task_name=2}
task_mem_usage{“job”=“a”, task_name=3}
task_mem_usage{“job”=“a”, task_name=4}

how can I filter the label values task_name < 3?

4

How about task_mem_usage{“job”=“a”, task_name=~"^(0|1|2)$"

5

Can someone help me in this case?

I was trying to use the below query with labels :
label_dp_language=“eng-USA”
label_dp_topic=“GEN”

I need to get the custom label => label_dp_language + “-” + label_dp_topic
I am expecting output as “eng-USA-GEN” and others values in variable from below query. The issue is same label_values doesnt work with query_result, can someone suggest me how can I append two labels and store in grafana template variable? -

query_result(label_join(kube_pod_labels{namespace=~“$Namespace”,label_app=“krypton”},“data_value”,“-”,“label_dp_language”,“label_dp_topic”))

Above query give me all query results with data_value label added to results, now I want this label to be extracted from query so label_values(query_result(…), data_value) should have worked here.