I’m also having issues on some graphs and template queries after upgrading elasticsearch from 2.4.6 to 5.6.3, not just with text fields.
After having upgraded our ES clusters and having verified that logstash 2.4 is still indexing data into ES 5.6.3 via kibana 5, I fail to get grafana 4.6.0 to show all our dashboards graphs. It though show the old indexed data that were in ES cluster from before upgrading (up till approx 12:00 on 2017.10.26) but not all of the new, specially the queries that are grouped-by a ES field. See attached SDs below.
(Sorry not a true reply, just wanted to add another SD sample)
Also see attached SD which will show query B but not query A.
Wondering why as data are in ES cluster and both queries work fine against ES v.2.4, all I did it grafana after upgrading were to alter all ES Data Sources to be of type 5.x vs former 2.x.
Sorry for my confusion, reading on ES keyword type:
A field to index structured content such as email addresses, hostnames, status codes, zip codes or tags.
They are typically used for filtering (Find me all blog posts where status is published), for sorting, and for aggregations. Keyword fields are only searchable by their exact value.
Our indexes hold many structured fields exactly for the purpose of sorting/aggregating, but our mapping stems from the 2.x era hence got none of them marked as keywords but rather strings or numbers. Would it in ES 5.x now be necessary with keyword field for grafana to query with aggregation on a structured field ‘host’ like:
‘host’ been mapped just as a string field currently.
If we changed such fields to keyword type rather than string, would we still be able then to query on regexp of such, eg. host:/hostname./ when ES doc says:
Keyword fields are only searchable by their exact value
Ok, changing group-by aggregation to eg. host.keyword (without having defined host field as such in ES index) I can then see some data in grafana, only latest index after 01:00 last night seems to round value to integers rather than floats as yesterday index created by ES 2.4. Wondering what’s changed…
Discovered that our template was lost in ES thus mappings has been left to ES to decide, hence it makes sense data only started looking different after auto new indexes were created @GMT00.00 yesterday, has recreated template + mappings, time will tell if things looks better tomorrow when next index sets are created…
Right with old template + mappings now in place everything is back to normal on the newly created indexes any any old indexes+values stored with ES 2.4, also including not having to use .keyword for aggregation/pipelining in queries, as we had already mapped “doc_values”: true on these fields. But maybe we’ll need to dig into the new keyword type later… also new restriction is coming w/+6.x ES in various areas.
