Run query on daily basis on grafana for each data source

Grafana Configuration
1

Hey everyone,

I have come across an issue, where I want to detect if any of my grafana data sources have secrets in their logs.
Is there any way or query that we can run on grafana on a daily basis and it will check any of the secrets or sensitive information of each data source logs?

Are there any plugins we can install into grafana that we can connect with each data source that plugin will run daily because grafana actually accesses Cloudwatch logs or google cloud logs in real-time, it doesn’t store logs.

Why I want because the tomcat container is printing environment variables into logs and grafana will access those logs in real-time, so prevent can we run such a query?

2

Hi @sonit, this is not an answer to your question, but you don’t want to fix it by modifying the logging in tomcat itself?

3

But what if my grafana data source logs have any secrets mistakenly? So is there any way that we can secure our grafana more secure?

4

I understand that. But a systemically more appropriate solution is to solve it at the level of the source itself and not at the end of the chain - besides, individual implementations of plugins and datasources themselves do not reflect changes so quickly.

E.g. Database passwords being leaked in to logs in connection strings · Issue #16001 · grafana/grafana · GitHub

You can check out the capabilities of Grafana Loki. There will even be a webinar How to manage privacy in log data with Grafana Loki. You might find it useful.

1 Like
5

Hey, you are saying correct. I am just trying to use a method for my own satisfaction level we can set an alert for every data source and run a query which detects some “” from logs.
But little bit confused that if a data source has many log groups, would the query that I am writing to detect “” run for every log group of that data source?

You can check from my below screenshot. I am just fetching “Runtime Version ARN” from the logs lines and setting it as an alert.

I just want to know the method I am implementing , is this correct method or not?