Grafana 8 - Cloudwatch logs broken

Grafana Cloudwatch
1
  • What Grafana version and what operating system are you using?

8.0.2

  • What are you trying to achieve?

Query cloudwatch log groups

  • How are you trying to achieve it?
    I’ve added a cloudwatch data source which is successfully plotting CloudWatch Metrics queries

I then configure a Cloudwatch Logs query, which lists my log groups, I pick a log group and enter a simple query

fields @message,@logStream
| sort @timestamp desc
| limit 10

  • What happened?
    No data is displayed

  • What did you expect to happen?
    Data to be displayed. Running this same query directly in cloudwatch against the same log group and time window produces results

  • Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.
    There were no errors in either the query inspector or in the grafana server log - set to log level debug

2

This appears to be resolved in 8.0.3

3
4

This issues doesn’t appear to be fixed. My scenario:

  1. Using 8.0.3 image
  2. Deployed in Fargate stack
  3. CloudWatch metrics work fine
  4. CloudWatch logs however, never complete (spins forever).
  5. There doesn’t appear to be anything that would diagnose issues in the logs.
  6. I have all the required permissions in my IAM role. Even tried specifying “cloudwatch.*” and that didn’t make a difference.
  7. I tried going into go into developers tools (firefox) while the query is running (I’m trying to explore the logs) and it looks like it’s throwing a 400 error. See screen shot(my Grafana domain name has been redacted).
  8. Reverted back to 7.5.9, no issues.

Screen Shot 2021-06-27 at 10.50.44 AM
Screen Shot 2021-06-27 at 10.50.44 AM3826×926 380 KB

5

I have the same issue.

  1. I am using Grafana 8.0.5
  2. Running inside an AWS Fargate Container
  3. Successfully set up CloudWatch Metrics
  4. Switch to CloudWatch Logs. The query spins forever and never returns.
  5. To verify this is not a permissions issue, I used the same Access Key Id and Secret Key Id specified in the Grafana datasource configuration. The query is “fields @timestamp, @message | limit 20” The aws-cli query completes successfully leading me to believe it is NOT a permissions issue.
6

+1

Same issue here:

  • I’m using Grafana 8.06
  • Running at AWS EC2
  • CloudWatch Metrics are ok… but queries of CloudWatch Logs never returns and spins forever
7

+1 Here.

We are having the same issue. It was working until last night and suddenly queries of CloudWatch Logs never return and spins forever. CloudWatch Metrics are ok…

We are using Grafana v8.0.6

8

This is still happening on Grafana 8.1.
The first few minutes after starting Grafana retrieving the logs worked, but any subsequent attempts fail.

9

Hi, using Grafana 8.1.1 and still experiencing this issue.
Anyone knows what’s up or if a fix is on it’s way? Anything I can do to help?

10

I can confirm this is still and issue with all 8.x versions. I am not sure if this being worked on or not but it’s probably holding quite a few people back from using 8.x. Please bring back the CloudWatch functionality as it was in 7.x.

11

We are experiencing this issue too and would be happy if it was fixed.

12

Can we re-open this Issue?

It looks like we are still having the same issue.

13

Any news on this issue?

14

Just updated to v8.1.2 enterprise (free edition) and experience the same issue. Query inspector shows an error of: “Cannot read property ‘meta’ of undefined”.

Chrome dev tools shows:
annotations_srv.ts:88 AnnotationSrv.query error TypeError: Cannot read property ‘to’ of undefined
at Module.j (app.f2e9788d35f3ceef8811.js:6158)
at y (app.f2e9788d35f3ceef8811.js:5021)
at app.f2e9788d35f3ceef8811.js:5021
at async Promise.all (/d/zUspjyEZz/index 0)
(anonymous) @ annotations_srv.ts:88
along with:
runRequest.ts:149 runRequest.catchError TypeError: Cannot read property ‘meta’ of undefined
at t.project (/public/build/cloudwatchPlugin.f2e9788d35f3ceef8811.js:1)
at t._tryNext (vendors~app.f2e9788d35f3ceef8811.js:2)
at t._next (vendors~app.f2e9788d35f3ceef8811.js:2)
at t.next (vendors~app.f2e9788d35f3ceef8811.js:2)
at t._next (vendors~app.f2e9788d35f3ceef8811.js:2)
at t.next (vendors~app.f2e9788d35f3ceef8811.js:2)
at t._next (vendors~app.f2e9788d35f3ceef8811.js:2)
at t.next (vendors~app.f2e9788d35f3ceef8811.js:2)
at t._next (vendors~app.f2e9788d35f3ceef8811.js:2)
at t.next (vendors~app.f2e9788d35f3ceef8811.js:2)

and others.

1 Like
15

Any update on this? I am try to use this dashboard: Cloudwatch Logs dashboard for Grafana | Grafana Labs
But using variables for log group queries no longer appears to work. Can we [;ease get am update?

16

Same here when trying to view logs “no log groups available”

17

I’m getting “Query error:Log group is required” when switching to cloud watch logs in grafana 8.x version.
Any fix in grafana 8 versions for viewing cloudwatch logs ?

image
image1046×513 27.3 KB

18

This topic was automatically closed after 365 days. New replies are no longer allowed.