Tempo provisioning with custom query does unintended replacement of values

Grafana Configuration
1

  • What Grafana version and what operating system are you using?
    grafana:9.4.7, Ubuntu 22.04

  • What are you trying to achieve?
    Provisioning Grafana Tempo with a custom query to enable traces to logs.

  • How are you trying to achieve it?
    I prepared a yaml file to provision the tempo data source.

  • What happened?
    The data source is initialized but the custom query field is not set correctly. Tokens like ${__tags} and ${__trace.traceId} might be being interpreted as environment variables which causes them to be replaced by empty strings.

  • What did you expect to happen?
    For the fields to not get replaced while they are being put to the custom query section.

  • Can you copy/paste the configuration(s) that you are having problems with?
    This is the config I’m using:

tracesToLogsV2:
        datasourceUid: 'rmH_crCVz'
        spanStartTimeShift: '1h'
        spanEndTimeShift: '-1h'
        tags: [ { key: "service.name", value: "job"} ]
        filterByTraceID: false
        filterBySpanID: false
        customQuery: true
        query: '{${__tags}} | json | traceid ="${__trace.traceId}" and spanid ="${__span.spanId}"'

  • Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.
    No errors, I can see the resulting custom query which is wrong.

  • Did you follow any online instructions? If so, what is the URL?
    Grafana tempo provisioning docs

I can’t seem to find a way to escape these sequences, what should I do?