Loki is not working. Loki Ingester container failing with no error logs on container

Grafana Loki
1

I have deployed loki with loki-distributed 0.69.11 version.
I am not getting any error logs on loki-ingester container but still ingester container terminating and shutting down.
All other componant pods are running and up.
Here is loki-distributed.yaml file

loki:
      structuredConfig: 
        server:
          http_server_read_timeout: 600s
          grpc_server_max_send_msg_size: 666561028
          grpc_server_max_recv_msg_size: 666561028
#          http_server_write_timeout: 600s
        limits_config:
          max_streams_per_user: 10000000
          max_global_streams_per_user: 10000000
          max_entries_limit_per_query: 4567587326
        schema_config:
          configs:
          - from: "2020-09-07"
            store: boltdb-shipper
            object_store: azure
            schema: v11
            index:
              prefix: loki_index_
              period: 24h
        storage_config:
          boltdb_shipper:
            shared_store: azure
            active_index_directory: /var/loki/index
            cache_location: /var/loki/cache
            cache_ttl: 168h
          azure:
            container_name: xxx
            account_name: xxx
            account_key: ${STORAGE_ACCOUNT_KEY}
            chunk_delimiter: ":"
    ingester:
      extraArgs:
        - -config.expand-env=true
      extraEnvFrom: 
        - secretRef:
            name: xxx
      persistence:
        enabled: true
        storageClassName: standardssd-lrs
        size: 5Gi
      resources:
        requests:
          cpu: 1000m
          memory: 1024Mi

loki-ingester logs.

level=warn ts=2023-06-23T07:20:10.674992747Z caller=loki.go:286 msg="per-tenant timeout not configured, using default engine timeout (\"5m0s\"). This behavior will change in the next major to always use the default per-tenant timeout (\"5m\")."
level=info ts=2023-06-23T07:20:10.676055145Z caller=main.go:108 msg="Starting Loki" version="(version=2.8.0, branch=HEAD, revision=90888a0cc)"
level=info ts=2023-06-23T07:20:10.676772511Z caller=server.go:323 http=[::]:3100 grpc=[::]:9095 msg="server listening on addresses"
level=warn ts=2023-06-23T07:20:10.683785555Z caller=experimental.go:20 msg="experimental feature in use" feature="Azure Blob Storage"
level=info ts=2023-06-23T07:20:10.683781355Z caller=memberlist_client.go:437 msg="Using memberlist cluster label and node name" cluster_label= node=loki-distributed-ingester-0-17e72081
level=warn ts=2023-06-23T07:20:10.684012176Z caller=experimental.go:20 msg="experimental feature in use" feature="In-memory (FIFO) cache - embedded-cache"
level=warn ts=2023-06-23T07:20:10.684107285Z caller=cache.go:114 msg="fifocache config is deprecated. use embedded-cache instead"
level=warn ts=2023-06-23T07:20:10.684117786Z caller=experimental.go:20 msg="experimental feature in use" feature="In-memory (FIFO) cache - chunksembedded-cache"
level=warn ts=2023-06-23T07:20:10.684220895Z caller=experimental.go:20 msg="experimental feature in use" feature="Azure Blob Storage"
level=warn ts=2023-06-23T07:20:10.684266999Z caller=experimental.go:20 msg="experimental feature in use" feature="Azure Blob Storage"
level=info ts=2023-06-23T07:20:10.684317704Z caller=shipper.go:131 msg="starting index shipper in WO mode"
level=info ts=2023-06-23T07:20:10.684446616Z caller=table_manager.go:225 msg="loading table loki_index_19527"
level=info ts=2023-06-23T07:20:10.685135879Z caller=memberlist_client.go:543 msg="memberlist fast-join starting" nodes_found=1 to_join=4
level=info ts=2023-06-23T07:20:10.685477611Z caller=table.go:318 msg="handing over indexes to shipper loki_index_19527"
level=info ts=2023-06-23T07:20:10.685533916Z caller=table.go:334 msg="finished handing over table loki_index_19527"
level=info ts=2023-06-23T07:20:10.685564319Z caller=table_manager.go:225 msg="loading table loki_index_19528"
level=info ts=2023-06-23T07:20:10.686062364Z caller=table_manager.go:134 msg="uploading tables"
level=info ts=2023-06-23T07:20:10.686080066Z caller=index_set.go:86 msg="uploading table loki_index_19527"
level=info ts=2023-06-23T07:20:10.686195377Z caller=table.go:318 msg="handing over indexes to shipper loki_index_19528"
level=info ts=2023-06-23T07:20:10.699422192Z caller=memberlist_client.go:563 msg="memberlist fast-join finished" joined_nodes=2 elapsed_time=14.288113ms

Please let me know if i can try any solution here.

2

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.