Failed to load rules from Loki: 404 from rule state endpoint

Grafana Loki
1

I don’t have any rules created on loki but upon opening the alerts page on grafana, the errors shown below pop up.

Failed to load rules state from Loki: 404 from rule state endpoint. Perhaps ruler API is not enabled?
Failed to load rules config from Loki: 404 from rules config endpoint. Perhaps ruler API is not enabled?

I couldn’t understand why it was trying to load the rules in the first place if I don’t have any rules. However, I went ahead and created a test rule but that didn’t work as well. I would really appreciate if anyone can share how to go about solving this problem.

/usr/local/bin/loki-local-config.yaml

auth_enabled: false

server:
  http_listen_port: 3100

ingester:
  lifecycler:
    address: 127.0.0.1
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
    final_sleep: 0s
  chunk_idle_period: 5m
  chunk_retain_period: 30s
  max_transfer_retries: 0

schema_config:
  configs:
    - from: 2018-04-15
      store: boltdb
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 48h

storage_config:
  boltdb:
    directory: /disk/data/mltmp/loki/index

  filesystem:
    directory: /disk/data/mltmp/loki/chunks

limits_config:
  enforce_metric_name: false
  reject_old_samples: false
  reject_old_samples_max_age: 48h
  ingestion_rate_mb: 25
  ingestion_burst_size_mb: 30
  max_entries_limit_per_query: 2100000000
  max_streams_matchers_per_query: 21000
  #unordered_writes: false

chunk_store_config:
  max_look_back_period: 0s

table_manager:
  retention_deletes_enabled: true
  retention_period: 48h

ruler:
  storage:
    type: local
    local:
      directory: /etc/loki/rules/rules1.yaml
  rule_path: /tmp/loki/rules-temp
  alertmanager_url: http://alertmanager:9093
  ring:
    kvstore:
      store: inmemory
  enable_api: true
  enable_alertmanager_v2: true

The ruler section in the end wasn’t there before, so I added to it to see if that would allow grafana to load a test rule. The rule I created is as follows

/etc/loki/rules/rules1.yaml

rules:
    - alert: InstanceDown
      expr: 0
      for: 30s
      labels:
        severity: page
      annotations:
        summary: "Testing rule"
        description: "Testing rule"

Do you think this could be a permission issue because I am not able hit the loki endpoints manually except the ready endpoint? All other endpoints throw “404 page not found” error.

2

I was able to resolve the issue by providing the correct url and port in alertmanager_url.

1 Like
3

Hi @dev77 , how are you?
Can you share how you discovered the correct url, I’m getting the same error, but any url that I configure don’t work.

4

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.