Issue with MQTT plugin when /var/lib/grafana is mapped to external drive

Grafana Configuration
1

  • What Grafana version and what operating system are you using?
    10.0.1 on a Raspberry Pi 4, Raspi OS with Docker

  • What are you trying to achieve?
    I want to map the database directory to an external drive

  • How are you trying to achieve it?
    With the docker -v flag on starting the Grafana container

  • What happened?
    Grafana is able to use the directory on the external drive (writes the database and other data) and installs the plugin successfully, but cannot use it (Plugin unavailable). When copying the whole directory to the SD on the Raspberry where Raspi OS and Docker runs from, everything is fine and I can use the MQTT plugin without issues

  • What did you expect to happen?
    Since grafana in general can use the directory on the external drive, the plugin should be able to work from there, too

  • Can you copy/paste the configuration(s) that you are having problems with?
    This is the startup command I use: docker run -d --name=grafana -p 9090:3000 -v /mnt/SSD_240GB/grafana:/var/lib/grafana --restart=unless-stopped grafana/grafana-oss

  • Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.
    “Plugin unavailable”. In the logs I also only see that error without any more information.

  • Did you follow any online instructions? If so, what is the URL?
    Run Grafana Docker image | Grafana documentation

2

interesting issue. it might be a docker issue in relation to an external drive

3

Thanks for the link, going to try if that might solve it!
However, the strange part about it that remains is that grafana is able to write to that dir on the external disk already, only when I try to use the plugin I run into an issue.
Without knowing too much about grafana yet, I was wondering about something:
On this page Configure a Grafana Docker image | Grafana documentation it shows that you can map the plugins directory independently from the data directory, yet the plugins dir is a subdirectory of the data dir /var/lib/docker
Could it be that there is some weirdly specific bug connected to plugins, that leads to grafana being able to use the in my case not specifically configured plugins directory for installation, but keeps it from executing plugins there? It somehow feels logical, so one part of grafana is able to install it and think it’s there, but another can’t actually use it.

4

Ha it seems the error has indeed somehow to do with the mapping of the plugins directory, here’s what happened now:

  • created only a docker volume for /var/lib/grafana
  • mapped that volume: same issue

Then I tried this:

  • created a docker volume for /var/lib/grafana
  • created a docker volume for /var/lib/grafana/plugins
  • mapped both docker volumes with the docker run command

Now this happens:

  • the directory for the /var/lib/grafana docker volume is being used as before
  • HOWEVER the directory mapped to /var/lib/grafana/plugins remains empty, and when I install the plugin it’s also not being installed in the plugins directory inside the mapped /var/lib/grafana but somewhere else

So it works now, but it seems there is some bug that needs to be fixed :slight_smile:

1 Like
5

Good firensic work!

Also this is interesting

Note: Though both methods are similar, there is a slight difference. If you want your storage to be fully managed by Docker and accessed only through Docker containers and the Docker CLI, you should choose to use persistent storage. However, if you need full control of the storage and want to allow other processes besides Docker to access or modify the storage layer, then bind mounts is the right choice for your environment.

6

Thanks for this info! Have to make myself more familiar with persistent storage which might also be a workaround for this.
In the meantime I think I came to the bottom of this bug! Filed an issue on github: Bug with plugin when mapping docker volume on external drive for /var/lib/grafana · Issue #81 · grafana/mqtt-datasource · GitHub

This subdirectory structure really seems to be the issue - I found out that grafana for some reason creates another volume called grafanadataplugins and uses that instead of the mapped directory for plugins

1 Like