Mkdir: can't create directory '/var/lib/grafana/plugins': Permission denied

Grafana Configuration
, ,
1

I am attempting this tutorial: GitHub - questdb/questdb-slack-grafana-alerts: Example code for a tutorial for sending Slack alerts based on market data streamed to QuestDB

My repo is here: GitHub - coding-to-music/questdb-slack-grafana-alerts: send alerts to Slack based on changes in market data streamed to QuestDB. Stock prices are fetched from the IexFinance API using the iexfinance Python package, streamed into QuestDB, a time series database, and alerts are set up in Grafana based on the metrics we care about.

When I run docker-compose up I get these messages:

Errors and Messages

Attaching to grafana_alerts, questdb_alerts
grafana_alerts | mkdir: can't create directory '/var/lib/grafana/plugins': Permission denied
grafana_alerts | GF_PATHS_DATA='/var/lib/grafana' is not writable.
grafana_alerts | You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migrate-to-v51-or-later

Steps attempted to fix

When I run this

id -u
1000

I see that I am user 1000, so I tried that in addition to user 472

Per the recommendation of this page: Run Grafana Docker image | Grafana documentation

I have added this line to the docker-compose.yml

user: "472"

I have attempted to remove the directory /var/lib/grafana and also change it’s ownership

I have tried changing the target directory ownership

and run docker-compose up as that user specified in the grafana–>user string

sudo chown 472:472 /var/lib/grafana 
sudo chown 472:472 /var/lib/grafana/plugins 

sudo chown 1000:1000 /var/lib/grafana 
sudo chown 1000:1000 /var/lib/grafana/plugins

Similar problem reports:

https://localhost:3000/t/gf-paths-data-var-lib-grafana-is-not-writable/31369

Any assistance is appreciated

1 Like
2

Any ideas? I’m stopped and could use some help. Thanks.

3

Cross post issue Mkdir: can’t create directory ‘/var/lib/grafana/plugins’: Permission denied · Issue #51931 · grafana/grafana · GitHub

4

docker-compose.yml

version: "3"

services:
  questdb:
    image: questdb/questdb
    container_name: questdb_alerts
    restart: always
    ports:
      - "8812:8812"
      - "9000:9000"
    networks:
      - "questdb"
    volumes:
      - ./data/db:/root/.questdb/db

  grafana:
    image: grafana/grafana
    container_name: grafana_alerts
    restart: always
    ports:
      - 3000:3000
    networks:
      - questdb
    volumes:
      - ./grafana/provisioning/:/etc/grafana/provisioning/
      - ./data/grafana:/var/lib/grafana/
    user: "472"

networks:
  questdb:
5

I had the same problem when trying to use grafana v9.
Starting the container with grafana v8 then upgrading to v9 seemed to work for me.

1 Like
6

More of a workaround than a solution tho.

7

Thanks for the info. Hopefully someone from Grafana staff can review and advise. Sounds like a bug!

8

To be honest, this sounds more like an issue related to basic understanding of permissions, docker and user IDs than to grafana, but I may be wrong.

Where exactly are you running the chown commands? This isn’t clear from your initial post.

[later edit:]
You are using a bind mount. That means that the permissions on the host need to be adapted accordingly. Now, depending on where you’re actually running this, the host might be the computer you’re running this on if it’s linux or a virtual machine if you’re running docker desktop on windows/mac.

2 Likes
9

Basic example is to open my repo in codespaces, type docker-compose up, and see the errors. I don’t see anything in my actual code that could cause the errors.

10

I agree that docker is trying to access the local file system.

I see the error also in this different repo GitHub - RedisTimeSeries/redis-weather

open via codespaces, type docker-compose up, and see this in the log:

redis-weather-grafana | logger=provisioning.datasources t=2022-07-13T18:01:15.217956091Z level=info msg="inserting datasource from configuration " name=Redis uid=PA7F6415749A3297A
redis-weather-grafana | logger=provisioning.plugins t=2022-07-13T18:01:15.297242464Z level=error msg="Failed to read plugin provisioning files from directory" path=/etc/grafana/provisioning/plugins error="open /etc/grafana/provisioning/plugins: no such file or directory"
redis-weather-grafana | logger=provisioning.notifiers t=2022-07-13T18:01:15.297287565Z level=error msg="Can't read alert notification provisioning files from directory" path=/etc/grafana/provisioning/notifiers error="open /etc/grafana/provisioning/notifiers: no such file or directory"
11

Yeah, well, you still haven’t answered my questions (the second one, related to the OS, was implied, of course), so I’m not sure how I can help you further.

12

Your question is: " Where exactly are you running the chown commands? This isn’t clear from your initial post."

Answer - in a DigitalOcean droplet - but you don’t even need to do any chown or anything to see the errors. The error is visible via clone the repo or open via cloudspaces (browser based vm) and do docker-compose up and see the permissions errors.

13

You cannot be running the chown commands in the droplet, because those directories don’t exist on the host in that path /var/lib/grafana or /etc/grafana/provisioning. If you were to do that, then you’d have received a clear message “no such file or directory” (I’m not talking about the moment you run docker-compose up, which is different). And if they do exist, they have nothing to do with your grafana running inside the container.

So first you have to understand that this ./grafana/provisioning/:/etc/grafana/provisioning/ (in the volumes) is a relative path. It’s relative to the place where you run docker-compose up.

So you have to run the chown on those directories in the git repository that you’ve cloned.

I also see that the second pair chown commands overwrite the first one - why are you changing it again like that? It doesn’t make any sense. You need to change it to 472 only.

but you don’t even need to do any chown or anything to see the errors.

You need to run the chown commands, otherwise it doesn’t work. This is not related to grafana at all, this is plain docker.

2 Likes
14

the chown were attempts to fix the errors.

I did not want to find an error and try nothing and just report the bug here.

So, ignore the chown stuff - the basic point is out of the box, the repo does not work because of the permissions errors.

Example

Errors:

grafana_alerts | mkdir: can't create directory '/var/lib/grafana/plugins': Permission denied
grafana_alerts | GF_PATHS_DATA='/var/lib/grafana' is not writable.
grafana_alerts | You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migrate-to-v51-or-later

As noted, I did try several ways to fix but could not solve. Any ideas? Are you able to get it to launch?

1 Like
15

This is a 3rd party tool?

16

basic hello world, docker-compose up and it should work, but is showing a grafana permissions error.

17

What does it have to do with grafana? From what I can see it has nothing to do with grafana? there is a folder called grafana :smile Have you looked into not using it at all and spinning up your own vanilla grafana and connecting it to questdb and using slack notification?

18

I have many projects and it is much cleaner to use docker and Terraform etc.
Doing all this via the UI will surely be possible, or ideally the Cloud Grafana, but the basic case of docker-compose up should be a prudent start.

19

I whole heartedly agree, auto provisioning via docker or podman is the way to go. but again this has nothing to do with grafan other than the mention of a folder called grafana :wink:

20

The error supplies this as a solution Run Grafana Docker image | Grafana documentation