-
What Grafana version and what operating system are you using?
Grafana 7.5.7 + Debian 10 Buster inside a Docker container -
What are you trying to achieve?
Containerized Grafana instance, able to spin up at a moment’s notice -
How are you trying to achieve it?
Running a grafana instance inside a docker container with file-provisioned dashboards and a Prometheus data source - Available over network on the Docker host server. -
What happened?
Accessing the Grafana instance, every provisioned item gets imported correctly, however, dashboards only show interrupted data segments -
When I take the widget’s PromQL query and plug it straight into the prometheus data source, it returns a continual data graph, so the data is physically alright inside the prometheus instance.
Investigating network requests my browser makes as I load the dashboard shows a few odd POST requests ending with HTTP 405 - Method not Allowed. the URL at fault here is /api/datasources/proxy/1/api/v1/series, but when I go over to edit a widget, I also get the error from /api/datasources/proxy/1/api/v1/metadata
Grafana’s logs also show no apparent source of the issue, only thing I get is a log like this:
t=2021-06-25T13:35:17+0000 lvl=info msg="Request Completed" logger=context userId=1 orgId=1 uname=admin method=GET path=/api/datasources/proxy/1/api/v1/metadata status=405 remote_addr=my.ip time_ms=1 size=19 referer="https://domain.tld/d/000000033/apache?orgId=1&refresh=1m&editPanel=5"
I am accessing the Grafana instance through the master admin account, and proxy the requests through an Nginx instance that acts as a SSL terminator and HTTP authentication provider.
To proxy the requests, I am using the built-in proxy_pass functionality to a localhost-bound port opened by the docker container.
I tried having the prometheus datasource instance bound on the docker network range (172.17.0.0/16), then rebound it on the server’s public WAN… Neither fixed the issue.
-
What did you expect to happen?
Graphs to render correctly and fully -
Can you copy/paste the configuration(s) that you are having problems with?
Prometheus provisioning configuration:
apiVersion: 2
datasources:
- version: 4
name: Prometheus
type: prometheus
access: proxy
url: http://ip-address:9091
jsonData:
httpMethod: 'POST' #Note - I added this as a debug meassure, to find out if a POST would stop the HTTP 405 errors from happening, initially I had the default GET. Neither helps with my issue
Grafana configuration (Stripped of comments and unused configuration parameters / sections):
app_mode = production
[paths]
data = /var/lib/grafana/
logs = /var/log/grafana/
plugins = /var/lib/grafana/plugins
[server]
protocol = http
http_port = 3001
domain = *domain*
root_url = https://*url*
[security]
admin_user = *username*
admin_password = *password*
disable_brute_force_login_protection = true
[smtp]
enabled = true
- Did you follow any online instructions? If so, what is the URL?
Nope, set up a grafana instance in the same manner as other instances we run and work correctly, only difference is it being inside docker container.