Hi Team,
We had designed a dashboard in grafana with Elasticsearch as datasource. In metricbeat agent, a new field is added via metricbeat.yml with below config
In grafana, we had created a variable to list applications for filtering,
while we filter any one application to get unique count, it by default includes other 2 apps count and display value as “3”. This may be due to cardinality aggregation feature in ES, But we wanted to filter and display value as “1”. is it feasible?
Yeah you can do this. Like you have created variable with the name of application. Put this variable in your query in where clause like
where application in $Variablename ( Name which you have assign to the variable). I wish this will will be helpful.
Thanks.But this is for Elasticsearch as source and use lucene query in grafana. When we filter one application, it shows unique count as 3 (including other 2 in array) as shown below.
I have a plan to work on Elastic search for log files but right now i didn’t use it. Anyways thanks for update. when you’ll find out the solution let me know . This will be helpful for me. Thanks.
As far as I know, Lucene/Elasticsearch does not work quite like that.
I’m sure you will see the same result if you query Elasticsearch directly like this
You will search the application field for $APPLICATION and the result will be how many unique values for $APPLICATION were found in all results combined that matched application:$APPLICATION.
The result is application:[“exxS-e11”,“eBxxxxH-e11”,“exxS-e10”] which is evaluated as 3.
That is exactly what I would expect Grafana to return for that query.
Hi @b0b,
Thanks, but our team expect to see response value as “1”. Is there any alternate options that I can use to get this exact unique count (ignoring combination) as “1” at elasticsearch level or in Grafana?
I will check on using Hash (also need to check its flexibility with automated deployment scenarios)
From two ES forum references which you shared, I could try 1st reference and check if below option may be helpful. But it may be challenging to handle via grafana template variable filter. Also we end up ensuring that our filter or search always match expected response structure.
Thanks @b0b. Initially I was planning to use this process metadata. But refrained and used fields for unique tracking of certain additional details like application etc. What would be the major difference between metadata & fields? Could you please clarify? or please share reference to understand the difference in better way.
Not sure how well this works for metricbeat… Metadata is mostly internal only to the service in question.
When I use metadata fields in Logstash I can set them on different inputs for different kinds of logs. Then I use the metadata to set to which Elasticsearch index the messages will be routed. My Logstash output looks like this
Something like that but I can not find if it is possible in metricbeat or not… And how you would assign the value for the metadata field…
Which is why I suggested Add process metadata as that is something that is documented. Or if you are running containers there is also Add Docker metadata.
application - for Hash
application1 - array retained to handle “template variable” filtering only
But it is not helpful while I try to get count dynamically with filter option (from array based field
only for template variable usage) as referred below
Thanks @b0b. Grouping which are referring is about choosing/selecting multiple apps from template variable and getting the exact unique count based on our multiple selection?
I should have written “Group by” instead of grouping
The third row in the query editor. With a short interval when I used “Group by” Date Histogram I got a float instead of 0 or 1. I guess it was the average over the time range when split into interval sized buckets, if that makes sense…
This is unfortunately not a problem I personally need solving at the moment and I don’t have time for more testing as I have other proprieties…
As I mentioned before, I have no direct experience of doing exactly what you are so everything I have written have been theoretical suggestions of what could work.
Good luck Hopefully you get it to work the way you expect it to.
Thanks @b0b, Did u configured those test data directly in metricbeat yml or via API? I tried similar configuration in metricbeat.yml, but config file is not loading while starting metricbeat.