How I use Docker for plugin development

marcusolsson · January 5, 2022, 10:09am

I thought I’d share how I use Docker for developing plugins. Docker makes it really easy for me to test my plugin against multiple versions of Grafana. For example, to make sure that the plugin still works with the earliest supported version of Grafana (as defined by grafanaDependency in plugin.json).

Here’s a one-liner that starts a Docker container running Grafana:

docker run -d \
    -e GF_DEFAULT_APP_MODE=development \
    -p 3000:3000 \
    -v ~/src/github.com/marcusolsson/grafana-plugins:/var/lib/grafana/plugins \
    grafana/grafana:8.3.3

-d makes the container run in the background.
-e GF_DEFAULT_APP_MODE=development starts Grafana in development mode, which lets me run unsigned plugins.
-p 3000:3000 exposes Grafana at port 3000 on my local machine.
-v ~/src/github.com/marcusolsson/grafana-plugins:/var/lib/grafana/plugins mounts my plugins in the default plugin directory inside the container.

This lets me develop my plugins for multiple Grafana versions simultaneously. For example, I can run version 8.0.0 on a different port:

docker run -d \
    -e GF_DEFAULT_APP_MODE=development \
    -p 3001:3000 \
    -v ~/src/github.com/marcusolsson/grafana-plugins:/var/lib/grafana/plugins \
    grafana/grafana:8.0.0

Now I can switch between browser tabs to test whether a change introduces a regression!

marcusolsson · January 5, 2022, 10:26am

And while we’re at it, why not add a convenient ZSH helper.

Add the following to your .zshrc file:

start_plugin_dev() {
  docker run -d \
    -e GF_DEFAULT_APP_MODE=development \
    -p "${2:=3000}:3000" \
    -v ~/src/github.com/marcusolsson/grafana-plugins:/var/lib/grafana/plugins \
    grafana/grafana:"${1:=latest}"
}

In a new terminal, run the function with the latest version of Grafana:
```
start_plugin_dev
```
- start_plugin_dev 8.0.0 to run a specific version.
- start_plugin_dev 8.0.0 3001 to run a specific version on a specific port.

zuperzee · January 5, 2022, 1:21pm

Cool. I also want to show my setup

I have a docker-compose file that I update the version number to test different versions and how my plugin interracts with other plugins.

My docker-compose file:

version: "3"
services:
  grafana-dev:
    image: grafana/grafana:8.3.1
    container_name: grafana
    restart: always
    networks:
      - grafana
    volumes:
      - ./grafana-plugins:/var/lib/grafana/plugins
      - grafana-storage-dev:/var/lib/grafana
    ports:
      - 3100:3000
    environment:
      - GF_APP_MODE=development
      - GF_SERVER_ROOT_URL=%(protocol)s://%(domain)s:%(http_port)s/
      - GF_SERVER_SERVE_FROM_SUB_PATH=true
      - GF_DASHBOARDS_MIN_REFRESH_INTERVAL=0.1s
      - GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS=gapit-htmlgraphics-panel,marcusolsson-json-datasource
      - GF_LIVE_ALLOWED_ORIGINS=*
      - GF_SECURITY_ADMIN_PASSWORD=admin
      - GF_USERS_DEFAULT_THEME=dark
      - GF_INSTALL_PLUGINS=marcusolsson-static-datasource

volumes:
  grafana-storage-dev:

networks:
  grafana:
    external: true
    name: grafana

marcusolsson · January 5, 2022, 1:38pm

Nice!

Mounting /var/lib/grafana as a volume is really useful if you want to reuse state. I had this in my earlier configuration, but since I tend to create new dashboard to test things rather than to reuse them, I ended up with a ton of dashboards with gibberish names. I decided I’ll just wipe them until I’ve proved I can handle the responsibility .

Also cool to see you’re using the Static plugin!

mattabrams · January 12, 2022, 9:08pm

oooh, this is SLICK!

mattabrams · January 12, 2022, 9:24pm

I have also found these two aliased commands helpful. A quick way to stop and remove all Docker containers:

alias stopc='docker stop $(docker ps -aq)'
alias rmc='docker rm $(docker ps -aq)'

marcusolsson · January 13, 2022, 9:55am

I’ll be stealing these!

steffanou · March 23, 2022, 10:18am

Hi @marcusolsson ,

When we use

docker run -d -p 3000:3000 -v ~/src/github.com/marcusolsson/grafana-plugins:/var/lib/grafana/plugins grafana/grafana

We have error when installing plugins from within the grafana plugin option web UI.

Probably access right issue but I don’t know how to fix that.
Can you help ?

marcusolsson · March 23, 2022, 10:38am

What OS are you using? I just tried it on macOS and seems to work. Also, what does the error say?

steffanou · March 23, 2022, 11:07am

Hi, we are using Ubuntu 18.04
The error from the web page is basic : failed to install plugin

If I use the docker cmd line including the plugin install request:

sudo docker run -d -p 3000:3000 -v /home/toto/SRC/grafana_plugins/grafana-plugins:/var/lib/grafana/plugins -e GF_INSTALL_PLUGINS="vertamedia-clickhouse-datasource" --name=grafana grafana/grafana

Docker logs

failed to extract plugin archive: could not create "/var/lib/grafana/plugins/vertamedia-clickhouse-datasource", permission denied, make sure you have write access to plugin dir

it works if I use –user root

sudo docker run -d --user root -p 3000:3000 -v /home/toto/SRC/grafana_plugins/grafana-plugins:/var/lib/grafana/plugins -e GF_INSTALL_PLUGINS="vertamedia-clickhouse-datasource" --name=grafana grafana/grafana

but it’s not very clean in the plugin dir with the plugin under root ownership:

drwxrwxr-x 6 toto toto 4096 Mar 23 09:02 my-plugin/
drwxr-xr-x 4 root root 4096 Mar 24 02:08 vertamedia-clickhouse-datasource/

Also the my-plugin created with npx @grafana/toolkit plugin:create is not listed inside grafana.

Is there a cleaner way to do that ?

steffanou · March 29, 2022, 1:08am

We’ve found a way to make it work with Linux (ubuntu for us).

1st, not directly linked to the issue, but to allow the current user to run docker without using sudo, add user to docker group


sudo usermod -aG docker $USER

# then exit / reconnect your linux session for that to take effect

As per link https://grafana.com/docs/grafana/latest/installation/docker/#migrate-to-v73-or-later the Grafana Docker image runs with the root group (id 0) in last versions.

So we changed the folder ownership to group root, keeping the user as is, be sure to have full right for both user and group, and that it apply to all subfolders and files.


chown :root /home/toto/SRC/grafana_plugins/grafana-plugins

chmod 775 /home/toto/SRC/grafana_plugins/grafana-plugins

chmod g+s /home/toto/SRC/grafana_plugins/grafana-plugins

For context, in the user folder, SRC is where I clone my GITs, here the Git project grafana_plugins, in which there is the folder grafana-plugins, where we will create the devellopment pluggins

Then we can create the docker with:

an external mount plugins folder (-v)
with a plugin install (-e GF_INSTALL_PLUGINS= )
for dev, with no need to sign the new pluggins (-e GF_DEFAULT_APP_MODE=development)


docker run -d -p 3000:3000 -v /home/toto/SRC/grafana_plugins/grafana-plugins:/var/lib/grafana/plugins -e GF_INSTALL_PLUGINS="vertamedia-clickhouse-datasource" -e GF_DEFAULT_APP_MODE=development --name=grafana grafana/grafana

With that, the pluggin clickhouse is installed properly, and the my-plugin (new empty plugin created with npx is also available in the grafana web UI.