Loki is deleting logs of killed/deleted containers before the retention policy

  • What loki version and what operating system are you using?
    loki:2.6.1

  • What are you trying to achieve?
    keep logs in the selected namespace for configurable time

  • How are you trying to achieve it?
    we have configured limits_config

  • What happened?
    Logs are correctly retained for the containers that are still running.
    If the user manually kills or deletes a container these logs are deleted before the configured retain policy.

  • What did you expect to happen?
    All logs linked to containers matching particular selectors should be kept by Loki until the retain policy.

  • Can you copy/paste the configuration(s) that you are having problems with?
    we are using loki-grafana-agent-operator and helmfiles

loki:
commonConfig:
replication_factor: 1
storage:
type: ‘filesystem’
rulerConfig:
storage:
type: local
auth_enabled: false
limits_config:
retention_period: 168h
retention_stream:
- selector: ‘{namespace=“gate”}’
priority: 1
period: {{ .Values.logging.retentionPeriod | default “744h” }}
monitoring:
selfMonitoring:
lokiCanary:
enabled: false
global:
dnsService: rke2-coredns-rke2-coredns
singleBinary:
persistence:
storageClass: {{ .Values.storage.usage.logs }}

  • Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.
    No errors

  • Did you follow any online instructions? If so, what is the URL?
    yes but this part is not covered: Retention | Grafana Loki documentation

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.