tam481
July 15, 2020, 9:47pm
1
Hello all,
I am trying to get my Grafana docker container to persist data. I’ve configured the container to run as root to rule out any user account related issues. I’m not sure what I’m doing wrong but I can’t get the data to persist.
I can remove panels from the default Home dashboard. If I create a custom dashboard everything seems to be OK but as soon as I log out and re-login (to the default admin account) it is as if I’m logging in for the first time.
Here’s my docker-compose file
grafana:
image: grafana/grafana:latest
user: root
restart: unless-stopped
container_name: grafana
volumes:
- $PWD/data/grafana:/var/lib/grafana
- $PWD/conf/grafana:/etc/grafana
- $PWD/logs/grafana:/var/log/grafana
- $PWD/data/grafana/provisioning:/etc/grafana/provisioning
environment:
- "GF_INSTALL_PLUGINS=grafana-clock-panel,grafana-simple-json-datasource,grafana-piechart-panel"
networks:
- backend
What am I doing wrong? Initially, the container complained about not finding directories (dashboard, datasources and notifiers) within the provisioning directory which I expected it to create under /etc/grafana so to overcome that I added the last binding and manually created these directories under the provisioning directory.
Thank you in advance
1 Like
Have you made the data directories first? Are the permissions on them correct? Personally i hate using $PWD and always set a full path. I like using /docker_data as it makes it obvious to anyone after me what it is.
Try this…
version: '3'
services:
grafana:
image: grafana/grafana
container_name: grafana
depends_on:
- influxdb
ports:
- 3000:3000
volumes:
- /docker_data/grafana_data:/var/lib/grafana
- /docker_data/certs:/certs
env_file:
- /docker_data/grafana_conf/config.monitoring
environment:
- HTTP_USER="{{ grafana_user }}"
- HTTP_PASS="{{ grafana_passwd }}"
- INFLUXDB_HOST=influxdb
- INFLUXDB_PORT=8086
- INFLUXDB_NAME="{{ db_name }}"
- INFLUXDB_USER="{{ influxdb_user }}"
- INFLUXDB_PASS="{{ influxdb_passwd }}"
restart: always
prometheus:
image: prom/prometheus
container_name: prometheus
volumes:
- /docker_data/prometheus_conf:/etc/prometheus/
- /docker_data/prometheus_data:/prometheus
command:
- '--config.file=/etc/prometheus/prometheus.yml'
- '--storage.tsdb.path=/prometheus'
- '--web.console.libraries=/usr/share/prometheus/console_libraries'
- '--web.console.templates=/usr/share/prometheus/consoles'
ports:
- 9090:9090
links:
- grafana
- influxdb
restart: always
influxdb:
image: docker.io/influxdb
container_name: influxdb
ports:
- "8086:8086"
volumes:
- /docker_data/influxdb_data:/var/lib/influxdb
env_file:
- '/docker_data/influxdb_conf/env.influxdb'
restart: always
That should give you a stack.
I use Ansible for all the lifting -
docker-install.yml
---
-
connection: localhost
hosts: localhost
tasks:
-
args:
chdir: /home/
command: "curl -fsSL get.docker.com -o get-docker.sh"
name: "Download docker script"
-
name: "run docker script"
script: /home/get-docker.sh
-
name: "start Docker"
service:
name: docker
state: started
-
apt:
name: python-pip
state: present
name: "install pip"
register: results
-
debug:
var: results
-
command: "pip install docker-compose"
name: "install docker-compose"
-
name: "include vars file"
include_vars: docker_install.var.yml
-
name: "create docker data folders"
file:
path: "{{ item }}"
state: directory
mode: 0777
with_items:
- /docker_data/portainer_data
- /docker_data/influxdb_data
- /docker_data/grafana_data
- /docker_data/chronograf_data
- /docker_data/prometheus_data
-
name: "create docker config folders"
file:
path: "{{ item }}"
state: directory
mode: 0775
with_items:
- /docker_data/influxdb_conf
- /docker_data/grafana_conf
- /docker_data/prometheus_conf
-
name: "create telegraf.conf from template values"
template:
dest: /docker_data/telegraf_conf/telegraf.conf
src: telegraf.conf.jk2
-
name: "create telegraf.conf outputs from template values"
template:
dest: /docker_data/telegraf_conf/telegraf.d/telegraf.outputs.conf
src: telegraf.outputs.conf.j2
-
name: "create telegraf.conf inputs from template values"
template:
dest: /docker_data/telegraf_conf/telegraf.d/telegraf.inputs.conf
src: telegraf.inputs.jk2
-
file:
mode: 420
path: /docker_data/telegraf_conf/telegraf.conf
name: "change file permission"
-
name: "create grafanacfg.output file with values"
template:
dest: /docker_data/grafana_conf/grafanacfg.output
src: grafanacfg.input.jk2
-
name: "create influx env file"
template:
dest: /docker_data/influxdb_conf/env.influxdb
src: env.influxdb.j2
-
name: "create grafana config file"
template:
dest: /docker_data/grafana_conf/config.monitoring
src: config.monitoring.j2
-
name: "create docker-compose.yml from template"
template:
dest: /docker_data/docker-compose.yml
src: docker-compose.yml.j2
-
name: "create docker-compose .env from template"
template:
dest: /docker_data/.env
src: docker-compose.env.j2
-
args:
chdir: /docker_data/
command: "docker-compose build"
name: "docker build"
-
args:
chdir: /docker_data/
command: "docker-compose up -d"
name: "docker up"
-
name: "Wait 300 seconds for port 3000 to become open on the host, don't start checking for 10 seconds"
wait_for:
delay: 10
port: 3000
-
name: "Set Grafana admin password"
command: docker exec -t grafana bash -c 'grafana-cli --homepath /usr/share/grafana admin reset-admin-password "{{ grafana_passwd }}"'
-
name: "check if grafana can be accessed"
command: "curl -s http://grafana.staged-by-discourse.com/api/org -u {{ grafana_user }}:{{ grafana_passwd }}"
-
name: "create datasoure in grafana"
command: "curl -L --header 'Content-Type: application/json' --header 'Accept: application/json' -d@//docker_data/grafana_conf/grafanacfg.output http://grafana.staged-by-discourse.com/api/datasources -u {{ grafana_user }}:{{ grafana_passwd }}"
-
name: "create TelegrafHost dashboard in grafana using jsonfile"
command: "curl -L --header 'Content-Type: application/json' --header 'Accept: application/json' -d@/home/Configfiles/dashboards/TelegrafHostMetrics.json http://grafana.staged-by-discourse.com/api/dashboards/db -u {{ grafana_user }}:{{ grafana_passwd }}"
-
name: "create DockerMetrics dashboard in grafana using jsonfile"
command: "curl --header 'Content-Type: application/json' --header 'Accept: application/json' -d@/home/Configfiles/dashboards/DockerMetricsperContainer.json http://grafana.staged-by-discourse.com/api/dashboards/db -u {{ grafana_user }}:{{ grafana_passwd }}"
-
name: "create JVM dashboard in grafana using jsonfile"
command: "curl --header 'Content-Type: application/json' --header 'Accept: application/json' -d@/home/Configfiles/dashboards/jvm-metrics-jolokia-2_rev1.json http://grafana.staged-by-discourse.com/api/dashboards/db -u {{ grafana_user }}:{{ grafana_passwd }}"
docker-install.var.yml (example this wont work!)
grafana_user: !vault |
$ANSIBLE_VAULT;1.1;AES256
grafana_passwd: !vault |
$ANSIBLE_VAULT;1.1;AES256
influxdb_admin_user: !vault |
$ANSIBLE_VAULT;1.1;AES256
influxdb_admin_passwd: !vault |
$ANSIBLE_VAULT;1.1;AES256
influxdb_user: !vault |
$ANSIBLE_VAULT;1.1;AES256
influxdb_passwd: !vault |
$ANSIBLE_VAULT;1.1;AES256
I suggest you use ansible-vault to not save passwords in plain text.
This should give you a good start, there are some files you will need to create, they are mainly your current config files saved as .jk2 or j2 (where I forgot the k). The dashborads I auto deploy can be removed or you can go to the Grafana page for them and download the yml.
tam481
July 23, 2020, 4:37pm
3
Thank you very much for your help.
Since posting, I noticed that I am able to create and save dashboards etc. What I cannot do is reset the admin password (it keeps prompting me to change it every time" I also cannot seem to be able to edit the main “home” dashboard.
I thought it had something to do with Grafana being behind Traefik so I tested a standalone container with port 3000 exposed on http and found that it did the same thing so I know for sure it’s not Traefik.
Any ideas?
alex440
September 18, 2021, 12:25pm
4
@tam481
I have the same problem. How did you solve it?