Hi guys! I’m trying to setup grafana behind nginx reverse proxy. Configuration is as follows:
nginx.conf:
server {
listen 80;
root /usr/share/nginx/www;
index index.html index.htm;
server_name grafana.domain.com;
location / {
proxy_pass http://grafana.staged-by-discourse.com/;
}
}
grafana.ini
[server]
domain = grafana.domain.com
If l’m trying to change my password/add user to org, give user admin rights iI’m getting
Possibly unhandled rejection: {"message":"Unexpected error"} [angular.js:14700](webpack:///node_modules/angular/angular.js)
Possibly unhandled rejection: {"data":null,"status":-1,"config":{"method":"PUT","transformRequest":[null],"transformResponse":[null],"jsonpCallbackParam":"callback","url":"api/admin/users/2/permissions","data":{"isGrafanaAdmin":true},"retry":0,"headers":{"X-Grafana-Org-Id":1,"Accept":"application/json, text/plain, */*","Content-Type":"application/json;charset=utf-8"}},"statusText":"","xhrStatus":"error"}
I’m struggling with it second day already (I tried to chage headers, unix socket instead of http port, with and without subpath ) but still no luck. without reverse proxy it works completely fine. Any ideas?
nginx’s error logs in debug mode looks like this:
2019/10/25 10:36:08 [debug] 16329#16329: *450 write new buf t:1 f:0 00005571295AEF10, pos 00005571295AEF10, size: 353 file: 0, size: 0
2019/10/25 10:36:08 [debug] 16329#16329: *450 http write filter: l:0 f:0 s:353
2019/10/25 10:36:08 [debug] 16329#16329: *450 http cacheable: 0
2019/10/25 10:36:08 [debug] 16329#16329: *450 http proxy filter init s:200 h:0 c:0 l:23
2019/10/25 10:36:08 [debug] 16329#16329: *450 http upstream process upstream
2019/10/25 10:36:08 [debug] 16329#16329: *450 pipe read upstream: 0
2019/10/25 10:36:08 [debug] 16329#16329: *450 pipe preread: 23
2019/10/25 10:36:08 [debug] 16329#16329: *450 pipe buf free s:0 t:1 f:0 0000557129598470, pos 000055712959859A, size: 23 file: 0, size: 0
2019/10/25 10:36:08 [debug] 16329#16329: *450 pipe length: 23
2019/10/25 10:36:08 [debug] 16329#16329: *450 input buf #0
2019/10/25 10:36:08 [debug] 16329#16329: *450 pipe write downstream: 1
2019/10/25 10:36:08 [debug] 16329#16329: *450 pipe write downstream flush in
2019/10/25 10:36:08 [debug] 16329#16329: *450 http output filter "/login?"
2019/10/25 10:36:08 [debug] 16329#16329: *450 http copy filter: "/login?"
2019/10/25 10:36:08 [debug] 16329#16329: *450 http postpone filter "/login?" 0000557129598008
2019/10/25 10:36:08 [debug] 16329#16329: *450 write old buf t:1 f:0 00005571295AEF10, pos 00005571295AEF10, size: 353 file: 0, size: 0
2019/10/25 10:36:08 [debug] 16329#16329: *450 write new buf t:1 f:0 0000557129598470, pos 000055712959859A, size: 23 file: 0, size: 0
2019/10/25 10:36:08 [debug] 16329#16329: *450 http write filter: l:0 f:0 s:376
2019/10/25 10:36:08 [debug] 16329#16329: *450 http copy filter: 0 "/login?"
2019/10/25 10:36:08 [debug] 16329#16329: *450 pipe write downstream done
2019/10/25 10:36:08 [debug] 16329#16329: *450 event timer: 8, old: 7308247, new: 7308275
2019/10/25 10:36:08 [debug] 16329#16329: *450 http upstream exit: 0000000000000000
2019/10/25 10:36:08 [debug] 16329#16329: *450 finalize http upstream request: 0
2019/10/25 10:36:08 [debug] 16329#16329: *450 finalize http proxy request
2019/10/25 10:36:08 [debug] 16329#16329: *450 free rr peer 1 0
2019/10/25 10:36:08 [debug] 16329#16329: *450 close http upstream connection: 8
2019/10/25 10:36:08 [debug] 16329#16329: *450 free: 0000557129574690, unused: 48
2019/10/25 10:36:08 [debug] 16329#16329: *450 event timer del: 8: 7308247
2019/10/25 10:36:08 [debug] 16329#16329: *450 reusable connection: 0
2019/10/25 10:36:08 [debug] 16329#16329: *450 http upstream temp fd: -1
2019/10/25 10:36:08 [debug] 16329#16329: *450 http output filter "/login?"
2019/10/25 10:36:08 [debug] 16329#16329: *450 http copy filter: "/login?"
2019/10/25 10:36:08 [debug] 16329#16329: *450 http postpone filter "/login?" 00007FFF190B4900
2019/10/25 10:36:08 [debug] 16329#16329: *450 write old buf t:1 f:0 00005571295AEF10, pos 00005571295AEF10, size: 353 file: 0, size: 0
2019/10/25 10:36:08 [debug] 16329#16329: *450 write old buf t:1 f:0 0000557129598470, pos 000055712959859A, size: 23 file: 0, size: 0
2019/10/25 10:36:08 [debug] 16329#16329: *450 write new buf t:0 f:0 0000000000000000, pos 0000000000000000, size: 0 file: 0, size: 0
2019/10/25 10:36:08 [debug] 16329#16329: *450 http write filter: l:1 f:0 s:376
2019/10/25 10:36:08 [debug] 16329#16329: *450 http write filter limit 0
2019/10/25 10:36:08 [debug] 16329#16329: *450 writev: 376 of 376
2019/10/25 10:36:08 [debug] 16329#16329: *450 http write filter 0000000000000000
2019/10/25 10:36:08 [debug] 16329#16329: *450 http copy filter: 0 "/login?"
2019/10/25 10:36:08 [debug] 16329#16329: *450 http finalize request: 0, "/login?" a:1, c:1
2019/10/25 10:36:08 [debug] 16329#16329: *450 set http keepalive handler
2019/10/25 10:36:08 [debug] 16329#16329: *450 http close request
2019/10/25 10:36:08 [debug] 16329#16329: *450 http log handler
2019/10/25 10:36:08 [debug] 16329#16329: *450 free: 0000557129598470
2019/10/25 10:36:08 [debug] 16329#16329: *450 free: 0000557129590960, unused: 2
2019/10/25 10:36:08 [debug] 16329#16329: *450 free: 0000557129597460, unused: 16
2019/10/25 10:36:08 [debug] 16329#16329: *450 free: 00005571295AED70, unused: 2604
2019/10/25 10:36:08 [debug] 16329#16329: *450 free: 0000557129599690
2019/10/25 10:36:08 [debug] 16329#16329: *450 hc free: 0000000000000000
2019/10/25 10:36:08 [debug] 16329#16329: *450 hc busy: 0000000000000000 0
2019/10/25 10:36:08 [debug] 16329#16329: *450 tcp_nodelay
2019/10/25 10:36:08 [debug] 16329#16329: *450 reusable connection: 1
2019/10/25 10:36:08 [debug] 16329#16329: *450 event timer add: 3: 75000:7323275
2019/10/25 10:36:08 [debug] 16329#16329: timer delta: 28
2019/10/25 10:36:08 [debug] 16329#16329: worker cycle
2019/10/25 10:36:08 [debug] 16329#16329: epoll timer: 75000
2019/10/25 10:36:21 [debug] 16329#16329: epoll: fd:6 ev:0001 d:00007F8FFA328010
2019/10/25 10:36:21 [debug] 16329#16329: accept on 0.0.0.0:80, ready: 0
2019/10/25 10:36:21 [debug] 16329#16329: posix_memalign: 0000557129594B60:512 @16
2019/10/25 10:36:21 [debug] 16329#16329: *452 accept: 94.140.208.58:33916 fd:8
2019/10/25 10:36:21 [debug] 16329#16329: *452 event timer add: 8: 60000:7320992
2019/10/25 10:36:21 [debug] 16329#16329: *452 reusable connection: 1
2019/10/25 10:36:21 [debug] 16329#16329: *452 epoll add event: fd:8 op:1 ev:80002001
2019/10/25 10:36:21 [debug] 16329#16329: timer delta: 12717
2019/10/25 10:36:21 [debug] 16329#16329: worker cycle
2019/10/25 10:36:21 [debug] 16329#16329: epoll timer: 60000
grafana’s log is also not that interesting:
t=2019-10-25T10:23:17+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=0.0.0.0:3000 protocol=http subUrl= socket=
t=2019-10-25T10:23:17+0000 lvl=info msg="Backend rendering via phantomJS" logger=rendering
t=2019-10-25T10:23:17+0000 lvl=warn msg="phantomJS is deprecated and will be removed in a future release. You should consider migrating from phantomJS to grafana-image-renderer plugin." logger=rendering
t=2019-10-25T10:25:25+0000 lvl=info msg="Request Completed" logger=context userId=0 orgId=0 uname= method=GET path=/admin/users/edit/2 status=302 remote_addr=127.0.0.1 time_ms=0 size=29 referer=
t=2019-10-25T10:25:30+0000 lvl=info msg="Successful Login" logger=http.server User=admin@localhost
t=2019-10-25T10:27:09+0000 lvl=info msg="Request Completed" logger=context userId=0 orgId=0 uname= method=GET path=/ status=302 remote_addr=127.0.0.1 time_ms=0 size=29 referer=
t=2019-10-25T10:30:08+0000 lvl=info msg="Successful Logout" logger=http.server User=admin@localhost
t=2019-10-25T10:30:08+0000 lvl=info msg="Request Completed" logger=context userId=1 orgId=1 uname=admin method=GET path=/logout status=302 remote_addr=127.0.0.1 time_ms=7 size=29 referer="http://grafana.amaiz.com/?orgId=1"
t=2019-10-25T10:36:08+0000 lvl=info msg="Successful Login" logger=http.server User=admin@localhost
alexey_ruzhitsky_amaiz_com@grafana-1:~$
Versions:
Grafana: v6.4.3 (commit: 3a2bfb7) (v 6.0.0. gives the same)
Nginx: nginx/1.14.0 ( 16.1 gives the same )