I’m currently using Kibana to visualise my Elasticsearch data and it does the job. Grafana however looks more polished so I’m looking at moving to it. However, when trying to graph the same data that graphs fine in Kibana, I receive the following error in Grafana:
{
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [SourceZone] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory."
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [
{
"shard": 0,
"index": "palo-firewall-2017.05.22",
"node": "nuMi1pLyTvCT7ncGEF6u2w",
"reason": {
"type": "illegal_argument_exception",
"reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [SourceZone] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory."
}
}
],
"caused_by": {
"type": "illegal_argument_exception",
"reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [SourceZone] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory."
}
}
In essence, I’m trying to replicate a simple graph that shows the amount of data pushed out through our firewall from different vlans.
I understand what the error is asking me to fix but what I don’t understand is why it works in Kibana but not in Grafana The query is exactly the same in each. I’m not using a Lucene query, it’s all from the drop down boxes so little room for error.
Thanks for your reply. I had already tested it with a .raw extension. You’re correct in that I have the field set up to be analysed. My understanding (and it’s clearly wrong) was that I needed to have the fields set to analyzed in order to actually use them in the graphs.
Guess I should do some more reading on that particular issue. That aside, can you explain why it works in Kibana but not in Grafana?
I’ve tried using the Terms field again in a graph using a field that isn’t analyzed and get the same result. Is this a limitation of Grafana over Kibana? I’m at a loss as to work out why, using the same settings in one as the other, they don’t work. Been playing for 2 hours with this and the most I can get Grafana to graph is traffic. Trouble is even the traffic graph looks so good that I really want to persevere and get the rest of it working!
No difference, grafana and kibana do the same query, so not sure what is going on, maybe kibana automatically adds the raw suffix, no one else has reported that there is differences here, so it’s strange that you get this in Grafana but the exact same query works in kibana, can you open chrome dev tools network tab and inspect the query kibana sends, compare it to grafana?
I’d be more than happy to if I knew exactly where to look for the query Kibana is pushing. For the sake of perhaps highlighting any (highly lightly), errors on my part, here’s the template file I’m using for the FW. I’ll dip into the DEV tools in Chrome and see if I can find anything helpful.
Right, Firefox Dev Tool results (I find it easier to work with as I’m no pro!). I created a new visualisation of the previously described setup (a bar graph showing highest vlan traffic in bytes).Exactly the same fields were selected in both cases. Kibana displays them, Grafana can’t. If there is anything more specific you need in order to help me diagnose the issue, please let me know.
From Kibana - (_msearch POST)
{“index”:[“palo-firewall-2017.05.23”],“ignore_unavailable”:true,“preference”:1495563036539}
{“size”:0,“query”:{“bool”:{“must”:[{“query_string”:{“query”:"*",“analyze_wildcard”:true}},{“range”:{"@timestamp":{“gte”:1495562287312,“lte”:1495563187312,“format”:“epoch_millis”}}}],“must_not”:[]}},"_source":{“excludes”:[]},“aggs”:{“2”:{“terms”:{“field”:“SourceZone.keyword”,“size”:5,“order”:{“1”:“desc”}},“aggs”:{“1”:{“sum”:{“field”:“Bytes”}}}}}}
From Grafana - (_msearch POST)
{“search_type”:“query_then_fetch”,“ignore_unavailable”:true,“index”:“palo-firewall-"}
{“size”:0,“query”:{“bool”:{“filter”:[{“range”:{"@timestamp":{“gte”:“1495560167705”,“lte”:“1495563767705”,“format”:“epoch_millis”}}},{“query_string”:{“analyze_wildcard”:true,“query”:"”}}]}},“aggs”:{“2”:{“terms”:{“field”:“SourceZone”,“size”:10,“order”:{“1”:“desc”},“min_doc_count”:1},“aggs”:{“1”:{“sum”:{“field”:“Bytes”}}}}}}
kibana seems to add "field": "SourceZone.keyword" in the terms aggregation. Wonder if .keyword is a special field prefix or does this field with that prefix exist in your index mapping?
Hi, We are able to draw graphs from elastic search data, but when i would like to show the raw data, i am getting below error in grafana logs. After getting this error, we could not edit the chart and query got corrupted. May i know how why it this happening? We are using 4.4.1 version.
{
“root_cause”: [
{
“type”: “search_parse_exception”,
“reason”: “failed to parse search source. unknown search element [docvalue_fields]”,
“line”: 1,
“col”: 295
}
],
“type”: “search_phase_execution_exception”,
“reason”: “all shards failed”,
“phase”: “query”,
“grouped”: true,
“failed_shards”: [
{
“shard”: 0,
“index”: “.kibana”,
“node”: “RH4fYv4GSIyzGItQi3I-RQ”,
“reason”: {
“type”: “search_parse_exception”,
“reason”: “failed to parse search source. unknown search element [docvalue_fields]”,
“line”: 1,
“col”: 295
}
}
]
}
