Elastic Search Queries doesnt work consistantly

Hi,

I am trying to template a variable based on _type.

The below query get preview.

Where as the below query doesnt

Reference - https://localhost:3000/t/templating-and--type-value-in-elastic-search/1671
Any help appreciated

This is the elastic search mapping

“Solutions_DelNTransf_compute_summary” : {
“properties” : {
“bu_name” : {
“type” : “text”,
“fields” : {
“keyword” : {
“type” : “keyword”,
“ignore_above” : 256
}
}
},
“timestamp” : {
“type” : “date”
},
“total_cpu” : {
“type” : “text”,
“fields” : {
“keyword” : {
“type” : “keyword”,
“ignore_above” : 256
}
}
},
“total_disk_gb_with_snaps” : {
“type” : “text”,
“fields” : {
“keyword” : {
“type” : “keyword”,
“ignore_above” : 256
}
}
}

terms query on _type works for me. The _type field is a built in field. somehow it is not indexed in your case (only reason I can see why its not showing any results in the terms query)

1 Like

As long as a single index and single type with a single elastic search source, it was working …when I added a second elastic search source and new index and new type / i added new index and new type to same elastic search source that was working, this query stopped working :frowning:

strange, better check your ES setup/data, maybe some error in ES log or response?

The above comment was a knee jerk reaction i blive !

i have two template variables defined as below, test_es_var1 from one es source and prod_es_var1 from another …prettty much same query used

the prod_es_var1 works and test_es_var1 doesnt … dont know whats wrong here !

Where as when I change the test_es_var1 query , it fetches the fields

Confusing :roll_eyes::roll_eyes::roll_eyes:

I would look at the Elasticsearch index and compare the two elastic search servers / indices, something must be different about them

ES logs have no clue ! And response from ES over API is pretty normal

Really cant differentiate any stuff! Anyway thanks for the hints …I will keep looking and troubleshooting

Fixed the issue ! The fields had STRING values that Lucene queries missed to fetch … I had to convert to float and it solved the issue! The doc body was constructed from a sql query results and hence this type mismatch of data.

@torkel your comparison hint really helped to drill down the issue ! Thanks

I cant get anything out of this … Where are the numbers to plot ? Can u describe _type , _index and _source on u r elastic search?

A terms query on the color field?

1 Like

@coolshiva123 @torkel can you please find the data? please help me out to get a pie chart

{"_id":“AV2InfytmKC6_kxk1gvJ”,"_type":“logs”,"_index":“logstash-2017.07.28”,"@timestamp":[1501235641486],“jobs”:[{“color”:“blue”,“name”:“amc-bootstrap-pl”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“blue”,“name”:“archiva-demo-app”,"_class":“hudson.model.FreeStyleProject”},{“color”:“blue”,“name”:“aris-analytics-pipeline”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“blue”,“name”:“aris-core-ui”,"_class":“hudson.model.FreeStyleProject”},{“name”:“aris-core-ui-pl”,"_class":“org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject”},{“color”:“blue”,“name”:“aris-demo-app”,"_class":“hudson.model.FreeStyleProject”},{“color”:“notbuilt”,“name”:“aris-web-base-app”,"_class":“hudson.model.FreeStyleProject”},{“color”:“blue”,“name”:“aris-web-platform”,"_class":“hudson.model.FreeStyleProject”},{“color”:“red”,“name”:“aris-web-platform-backup”,"_class":“hudson.model.FreeStyleProject”},{“color”:“red”,“name”:“awa-emr-trigger”,"_class":“hudson.model.FreeStyleProject”},{“color”:“red”,“name”:“config-server-build”,"_class":“hudson.model.FreeStyleProject”},{“color”:“blue”,“name”:“DevOps_Course1”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“blue”,“name”:“Devops_course2”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“blue”,“name”:“DevOps_Sample”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“blue”,“name”:“ebiz-analytics-pipeline”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“red”,“name”:“ebiz-app-pipeline”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“yellow”,“name”:“ebiz-test”,"_class":“hudson.maven.MavenModuleSet”},{“color”:“red”,“name”:“inventory-app-pipeline”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“red”,“name”:“inventory-db-pipeline”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“red”,“name”:“inventory-test-pipeline”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“blue”,“name”:“inventory_app_pl_trigger”,"_class":“hudson.model.FreeStyleProject”},{“color”:“notbuilt”,“name”:“inventory_db_pl_trigger”,"_class":“hudson.model.FreeStyleProject”},{“color”:“blue”,“name”:“inventory_test_pl_trigger”,"_class":“hudson.model.FreeStyleProject”},{“color”:“red”,“name”:“location-app-pipeline”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“red”,“name”:“location-db-pipeline”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“blue”,“name”:“location-test-pipeline”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“notbuilt”,“name”:“location_app_pl_trigger”,"_class":“hudson.model.FreeStyleProject”},{“color”:“blue”,“name”:“location_db_pl_trigger”,"_class":“hudson.model.FreeStyleProject”},{“color”:“blue”,“name”:“location_test_pl_trigger”,"_class":“hudson.model.FreeStyleProject”},{“color”:“blue”,“name”:“locationService”,"_class":“hudson.model.FreeStyleProject”},{“color”:“red”,“name”:“magellan-ape-pipeline”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“red”,“name”:“ofbiz_build”,"_class":“hudson.model.FreeStyleProject”},{“color”:“red”,“name”:“ofbiz_override_app_pipeline”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“notbuilt”,“name”:“ofbiz_override_app_pipeline_trigger”,"_class":“hudson.model.FreeStyleProject”},{“color”:“red”,“name”:“ofbiz_pipeline”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“red”,“name”:“openshift-location-app”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“red”,“name”:“openshift-location-app-demo”,"_class":“hudson.model.FreeStyleProject”},{“color”:“blue”,“name”:“openshift-product-app”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“red”,“name”:“order-service_app-pipeleine”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“notbuilt”,“name”:“order-service_app_pl_trigger”,"_class":“hudson.model.FreeStyleProject”},{“color”:“notbuilt”,“name”:“order-service_db-pipeleine”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“notbuilt”,“name”:“order-service_db_pl_trigger”,"_class":“hudson.model.FreeStyleProject”},{“color”:“notbuilt”,“name”:“order-service_test_pl_trigger”,"_class":“hudson.model.FreeStyleProject”},{“color”:“red”,“name”:“product-service-app-pipeline”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“red”,“name”:“product-service-db-pipeline”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“notbuilt”,“name”:“product-service-test-pipeleine”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“blue”,“name”:“product-service-test-pipeline”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“notbuilt”,“name”:“product-service-test_pl_trigger”,"_class":“hudson.model.FreeStyleProject”},{“color”:“blue”,“name”:“product_service_app_pl_trigger”,"_class":“hudson.model.FreeStyleProject”},{“color”:“blue”,“name”:“product_service_db_pl_trigger”,"_class":“hudson.model.FreeStyleProject”},{“color”:“blue”,“name”:“product_service_test_pl_trigger”,"_class":“hudson.model.FreeStyleProject”},{“color”:“blue”,“name”:“Sample Build”,"_class":“hudson.model.FreeStyleProject”},{“color”:“blue”,“name”:“Sample Deploy”,"_class":“hudson.model.FreeStyleProject”},{“color”:“blue”,“name”:“Sample Test”,"_class":“hudson.model.FreeStyleProject”},{“color”:“red”,“name”:“selenium_grid_demo”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“notbuilt”,“name”:“selenium_grid_demo_trigger”,"_class":“hudson.model.FreeStyleProject”},{“color”:“blue”,“name”:“t_bot_build”,"_class":“hudson.model.FreeStyleProject”},{“color”:“blue”,“name”:“test-aws-pipeline”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”},{“color”:“blue”,“name”:“test-job-creation-config”,"_class":“hudson.model.FreeStyleProject”},{“color”:“blue”,“name”:“Test-mail-job”,"_class":“hudson.model.FreeStyleProject”},{“color”:“red”,“name”:“test-rest-api-script”,"_class":“org.jenkinsci.plugins.workflow.job.WorkflowJob”}],"@version":“1”,“http_poller_metadata”:{“request”:{“headers”:{“Accept”:“application/json”},“charset”:“UTF-8”,“method”:“get”,“auth”:{“eager”:true,“pass”:“admin”,“user”:“admin”},

@torkel @torkel please find the above elastic search data. and help me to get a pie chart in GRAFANA.

Sorry, you will not need to explain more how your data looks and what you want to show, I am no mind Reader, just dumping your data and saying you want pie chart is not very informative :slight_smile:

1 Like

i want a pie chart and that should be informative. On the basis of color with the values or percentage.

That explanation doesnt seem to answer a lot of questions … Where are the values and numbers coming up? producing colorful reports on grafana is not a big deal as long as the data in elastic search has numbers in right format

Yes @coolshiva123, but here the values are in string format. you can look at the color field. I want to get the count of the particular color.

I am not sure if we can convert a string to a graph !