Hello, I am new to Grafana and Loki. I have just installed the Loki stack on my K8s cluster with Helm 3 using the instructions found here.
I did the first two steps: “Prerequisites” and “Deploy Lokito your cluster”
helm upgrade --install loki loki/loki-stack -n loki-stack --set grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false,loki.persistence.enabled=true,loki.persistence.storageClassName=standard,loki.persistence.size=25Gi
I didn’t do the " Deploy Grafana to your cluster" part, though, because I had just set grafana=enabled in the helm command. I set up my ingress, letsencrypt and I accessed the Grafana UI, created a few dashboards, etc.
Everything is working fine. Now I want to install the Kubernetes plugin. So I run :
kubectl exec -it -n loki-stack loki-grafana-767cf5c9bc-sbmq2 grafana-cli plugins install grafana-kubernetes-app
which tells me everything is set up correctly and tells me to restart. /var/lib/grafana/plugins contains the new plugin. So I run
kubectl -n loki-stack delete pod loki-grafana-856c649964-sbmq2 --grace-period=0 --force
and I log back in the interface, but the plugin is not installed. Checking the newly created pod, /var/lib/grafana/plugins is empty… It looks like persisting is not working.
Edit: I also tried to run
k scale deployment -n loki-stack --replicas=0 loki-grafana
k scale deployment -n loki-stack --replicas=1 loki-grafana
Same result.
Any idea why? And how I can solve this?