Loki OSS and Loki Cloud Differences in Behavior

Context

We are migrating from an ELK stack to a Loki/Promtail/Grafana (LPG?) stack, and we are evaluating Grafana Cloud Loki versus self-managed OSS Loki in Docker.

As such, and coming from ELK, we’re still using Filebeat and Logstash. We removed the Elasticsearch output from Logstash, and added outputs to both our on-prem OSS Loki and Grafana Cloud Loki (currently the free tier).

Problem

We have seen times when the on-prem Loki is getting data from Logstash, but Cloud Loki is not. You can see from this 90 minute window the differences.

On Prem:

Screen Shot 2023-06-15 at 10.46.58 GMT+2

Cloud:

loki-cloud-byte-rate

We don’t have indications that something else is causing this issue (network, etc…), although that’s always a possibility.

Question

Is there some throttling or limitation applied to Grafana Cloud Loki (free version or paid) that would cause this? And if so, is there a way to find or verify that we’re hitting these limits?

Many thanks!

As it turns out, the issue was not with Loki, but with Logstash.

Logstash was poorly configured, causing it to stop sending data.

Fixing Logstash fixed the issue.

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.