I’m using Grafana as our main observability tool for monitoring resources in AWS via AWS Managed Grafana.
We are using AWS Managed Grafana in 2 AWS Accounts: 1 for production and 1 for dev.
We have also set-up CloudWatch cross-account observability, having Cloudwatch from dev reporting to prod, as to make devs logs available to prod.
First we set-up all of our dashboards in dev and everything is fine, but when trying to set up the same dashboards in prod, it is just not loading the dimension values for some services, making them useless to us as it does not show data, or at least the way we wanted and previously worked on the dev environment.
The cloudwatch metrics we want to show in prod are from the prod account + the ones from dev. The issue we are having are from the resources from prod itself, not being shown in the grafana instance for prod.
To get the issue: We are trying to set a dashboard to monitor the ELB in the prod account, following what we did in the dev account using grafana’s and AWS documentation. Below is the expected result of the dev account.
Now, when we do this on the prod account these variables results are not loading. See below image for the prod account:
And when we navigate to cloudwatch in the AWS account for prod and look for the ApplicationELB namespace (below image), we clearly see the that the data exists and its there, but it’s not reachable? from Grafana?
We tried all sort of stuff but me and my team are new to grafana, so some ideas into why this is happening would be greatly appreciated.
Any extra info or logs we would need to provide pls let us know. Thanks!!
