Grafana Agent behind network firewall cant specify proxy

  • What Grafana version and what operating system are you using?
    ./agent-linux-amd64 --version
    agent, version v0.24.1 (branch: HEAD, revision: 35015aa4)
    build user: root@4500bb0cda4b
    build date: 2022-04-14T15:27:31Z
    go version: go1.17.6
    platform: linux/amd64

  • What are you trying to achieve?
    Connect machine behind firewall to push metrics to Grafana Cloud

  • How are you trying to achieve it?
    Run the agent configured on the local machine

  • What happened?
    It can’t connect to the internet because its requires a web proxy

  • Did you follow any online instructions? If so, what is the URL?
    Trying to follow (Grafana Agent | Grafana Cloud documentation) to install agent.

The agent config for metrics (see metrics_config | Grafana Agent documentation) does not have a proxy_url option like Promtail does (see grafana .com/docs/loki/latest/clients/promtail/configuration/#clients_)

There are no command line options that allow me to configure a proxy.

It looks like go should support the environment variable https_proxy and use that transparently. But that hasn’t worked either.

ts=2022-05-03T04:56:35.229954187Z caller=dedupe.go:112 agent=prometheus instance=XXX component=remote level=warn remote_name=XX url=https://prometheus-prod-10-prod-us-central-0.grafana.net/api/prom/push msg="Failed to send batch, retrying" err="Post \"https://prometheus-prod-10-prod-us-central-0.grafana.net/api/prom/push\": context deadline exceeded"

What are my alternatives?

So I searched through the Agent and Loki looking for proxy_url

https://github.com/grafana/agent/search?q=proxy_url

There is at least one test https://github.com/grafana/agent/blob/378ffeda5f9107b8acc0e24b21bd2787bc9d6d78/pkg/operator/config/metrics_templates_test.go that looks like it sets the proxy_url

metrics:
  wal_directory: /tmp/wal
  global:
    remote_write:
      - url: https://prometheus-prod-10-prod-us-central-0.grafana.net/api/prom/push
        basic_auth:
          username: XXX
          password: XXX
        proxy_url: http://localhost:3128

ran it and crossed my fingers…

That works.

Looks like the documentation is not up to date.