Monitoring RDS logs separately through Loki

Grafana Loki
1
  1. How can we monitor AWS RDS logs specifically from the Loki-Grafana Dashboard? Currently, we monitor logs from CloudWatch, but we are interested in monitoring RDS logs separately through Loki.
  2. If there is a way to achieve this, I would also like to inquire about the associated costs. Could you please provide information on the pricing for 1 API call per log?
2

  1. You can write a simple python script to extract logs from RDS API and then forward to Loki. It’s pretty straight forward.

  2. What pricing are you referring to? Grafana Cloud or self hosted?

3

But is there a way to do that via GUI as I do not have the knowledge of scripting. OR you can guide me in any way?

Steps found but unsure:

  1. Install and Configure Promtail: Install Promtail, the log shipping agent for Loki, on the same server or instance where your Grafana-Loki setup is running. Configure Promtail to scrape logs from CloudWatch Logs using the CloudWatch Logs API.
  2. Configure Loki Data Source in Grafana: In your Grafana instance, configure Loki as a data source. Provide the necessary details such as the Loki URL, authentication settings, and other relevant parameters.
  3. Create Grafana Panels: In Grafana, create panels and queries to visualize the RDS logs. Use LogQL to query and display the logs from Loki. Utilize the Loki data source and the configured Promtail to fetch and display the log data in your Grafana-Loki dashboard.

2nd Question:
As for Loki, I see that the costs are primarily associated with storage, processing, and data transfer. I am referring to the Grafana SELF Hosted.

4

  1. You’ll have to learn some scripting then. Or, if you really don’t want to learn scripting perhaps look at using fluentd instead of promtail, it has a plugin for RDS input: GitHub - shinsaka/fluent-plugin-rds-pgsql-log: fluentd plugin for Amazon RDS for PostgreSQL log input

  2. Fo self hosted instances you should be able to calculate the cost yourself. You already mentioned multiple cost metrics, that’s a good start.

5

Hey @tonyswumac, thanks for that.

The thing is I want to know how much does 1api cost per log. For example I want to capture cloudwatch logs (RDS instance error logs) via loki. How much it would cost per single log. 1api=$?

These are the below references but I was not able to figure out and need some assistance. If you got any, kindly let me know. Thanks in advance.

Pricing for CloudWatch Logs is based on the amount of data ingested, archived, and analyzed via CloudWatch Logs Insights queries. Each time you select a dimension in the query editor, Grafana issues a ListMetrics API request. Each time you change queries in the query editor, Grafana issues a new request to the GetMetricData API.

6

If your question is regarding cost for CloudWatch then I would recommend you to calculate the estimated cost with the price calculator provided by AWS.

If your intent is to have logs be stored in Loki then it wouldn’t make sense to have them pass through CloudWatch.

7

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.