Need Help in Understanding different components created by Loki simple scalable helm chart on EKS

[dkumarkaran]

I have deployed the Loki simple-scalable helm chart and it works well. There are many pods, services, demonsets, and stateful sets that got created and I would like to understand those components to get a complete understanding of how Loki works.

NAME                       DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/loki-logs   6         6         6       6            6           <none>          10d

What is the use of daemonset.apps/loki-logs ?
what is the difference between service/loki-read and service/loki-read-headless?

NAME                STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
data-loki-read-0    Bound    pvc-xxxx   10Gi       RWO            gp2            10d
data-loki-read-1    Bound    pvc-xxxx   10Gi       RWO            gp2            10d
data-loki-read-2    Bound    pvc-xxxx   10Gi       RWO            gp2            10d
data-loki-write-0   Bound    pvc-xxxx   10Gi       RWO            gp2            10d
data-loki-write-1   Bound    pvc-xxxx  10Gi       RWO            gp2            10d
data-loki-write-2   Bound    pvc-xxxx   10Gi       RWO            gp2            10d

We configured S3 as a storage but I found these pvc so can someone help me to understand why do we have these pvc when i have configured s3 as a storage option?

Thanking in Advance.

[b0b]

Hi @dkumarkaran ,

Have you read this Components | Grafana Loki documentation

S3 is your “long-term storage backend” but there is some local storage needed on ingesters and queriers.

The only daemon set I can think of that would run, that is related to Loki, would be Promtail, for log collection.

[dkumarkaran]

Hi @ b0b
Thanks for looking into my Query
I had the same guess that there could be a local pv and a remote storage for the Querier and Ingestor.

As for the demonsets ,they got created while using the helm chart “loki-simple-scalable” and when I tried checking the logs i found the below logs from demonset loki-logs:

ts=2023-01-27T02:09:19.77614216Z caller=filetargetmanager.go:177 level=info component=logs logs_config=monitoring/loki msg="received file watcher event" name=/var/log/pods/monitoring_loki-write-0_d1e3c9b2-ce8a-47b4-a2ce-9436ef1377f4/write/0.log.20230126-162515.tmp op=CREATE
ts=2023-01-27T02:09:20.308809034Z caller=filetargetmanager.go:177 level=info component=logs logs_config=monitoring/loki msg="received file watcher event" name=/var/log/pods/monitoring_loki-write-0_d1e3c9b2-ce8a-47b4-a2ce-9436ef1377f4/write/0.log.20230126-162515.gz op=CREATE
ts=2023-01-27T02:09:20.310105981Z caller=filetargetmanager.go:177 level=info component=logs logs_config=monitoring/loki msg="received file watcher event" name=/var/log/pods/monitoring_loki-write-0_d1e3c9b2-ce8a-47b4-a2ce-9436ef1377f4/write/0.log.20230127-020920 op=CREATE
ts=2023-01-27T02:09:20.310516055Z caller=filetargetmanager.go:177 level=info component=logs logs_config=monitoring/loki msg="received file watcher event" name=/var/log/pods/monitoring_loki-write-0_d1e3c9b2-ce8a-47b4-a2ce-9436ef1377f4/write/0.log op=CREATE
ts=2023-01-27T02:09:20.351424006Z caller=log.go:168 component=logs logs_config=monitoring/loki level=info msg="Re-opening moved/deleted file /var/log/pods/monitoring_loki-write-0_d1e3c9b2-ce8a-47b4-a2ce-9436ef1377f4/write/0.log ..."
ts=2023-01-27T02:09:20.351511553Z caller=log.go:168 component=logs logs_config=monitoring/loki level=info msg="Successfully reopened /var/log/pods/monitoring_loki-write-0_d1e3c9b2-ce8a-47b4-a2ce-9436ef1377f4/write/0.log"
ts=2023-01-27T04:31:28.950867667Z caller=reporter.go:131 level=info msg="reporting cluster stats" date=2023-01-27T04:31:28.950861467Z
ts=2023-01-27T04:31:29.237841318Z caller=reporter.go:159 level=info msg="usage report sent with success"

The above logs are from daemonset.apps/loki-logs

[b0b]

You could check what container image the daemonset is running. Those logs would make sense to come from a logging agent like promtail or grafana-agent