Loki derived fields

Grafana Loki
1

Hi,

I’ve previously used ELK to store and process syslogs from the various devices on a network and it has worked well.

I’ve been trying out Loki/Promtail/Grafana as an alternative. In the Promtail documentation it says to avoid creating labels for data that has high cardinality.

What I am struggling to do is to do things like grouping query results by say user name or IP without creating a label for these fields. Processing invalid user requests in /var/log/secure to identify bad IPs etc.

I have instead created derived fields for these in Grafana. These work in the table panel but they do not appear to work for other visualisations, e.g. PieChart, Graph etc.

I’d really like to be able to use these derived fields somehow in the LogQL or somehow to specify that they should be used to process/filter the query in Grafana in order to visualise the result.

I hope that makes sense!

Cheers,

Neil.

1 Like
2

It makes sense to me.

My use case was to get a picture of instances claimed by AWS to be terminated. I could extract from the message, as Derived Fields, the instance type, and instance name but I couldn’t use those fields to get a count by instance type.