Hello!
I would like to alert on every log line that I query in loki like {service="xxx", level="ERROR"}.
Or maybe there is a tool like elastalert, but for loki?
1 Like
Hi! This is possible using Alerting rules:
Yes, I know that I can alert with the ruler, but it can alert only on vector or scalar expressions and I can’t get a single log line in alert with it.
Can You provide some example of how to do it?
You can only alert on metric queries in Loki, yes.
I’m quite clear on what you want, but if you want to be alerted whenever a new log line appears for this stream, you might consider defining an alert expression like count_over_time({service="xxx", level="ERROR"}[1m]) > 0
Ok, thank you. I will try.
1 Like
I’m trying to do essentially the same thing – alert on the existence of a line, though in this case I’m using logql to parse the contents of the line.
I’ve tried a query like:
count_over_time({filename="/opt/kafka_2.13-3.0.0/logs/connect.log"} |~ received LSN LSN.* identified as already processed [5m]) > 0
with an expression “When last() of A is > 0”
but the problem I’m seeing is that when there is no such logline (which is the normal case), the query returns No Data, and if I create an alert using this query, the alert bogusly fires with a No Data condition.
So I need something like the ability to wrap this in a min(XX, 0) function, so we get 0 rather than no data when we don’t match the logline.
this feels like such a common use case – fire on a particular, rare log line – I feel like I must be missing something obvious which makes this easy-peasy.
ahh, for the edification of others, the answer is to configure “Alert state if no data or all values are null” as OK, under the dropdown “Configure no data and error handling”, all of which is in the “2 Alert evaluation behavior” section.
1 Like
This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.