We have some long error messages in our pod log and Grafana / Loki doesn’t seems to show / transfer everything.
It stopped at 8191 character. Is there a limit which I need to configure? The logs lines are fine. It is only that log length doesn’t seem to pass.
Fairly certain the only thing that would truncate a log message is
max_line_size_truncate. The default is not to truncate. There are also other limits configuration you can explore (see Grafana Loki configuration parameters | Grafana Loki documentation), although they don’t fit the behavior you are seeing which is logs being truncated.
You also can’t rule out that this may be a problem with your log pipeline. I’d check your Loki ingester log and see if there is anything obvious, check your log pipeline and run some sort of local debug check on this specific message.
Thanks for your support @tonyswumac. The values where unset but after some digging i found out, that Loki also saved the rest of the log. Only as a new line and because of the search querier we didn’t noticed that
Do you have an idea why this happens?
I don’t think it’s a problem with Loki. I am not sure what your log pipeline looks like, but I’d recommend you to try and reproduce it with your log pipeline in a debug / test environment.