Problem with automated creation of elasticsearch datasource

Grafana
1

I’m attempting to create an elasticsearch datasource via an HTTP POST call from an Ansible playbook, but I cannot get it working correctly. I created the datasource manually and looked at the request payload in chrome to get the required fields, then created a ‘uri’ task in Ansible with the correct fields filled out. However, when I run the playbook it creates the datasource but when I go to edit the datasource none of the fields are filled out, and if I try and use the DS in a graph, it gives me an error and there’s no data for it to graph.

Normally I would assume that my Ansible task is wrong, but I simply copied this from a similar task that sets up both graphite and aws datasources, and they work just fine. The elasticsearch DS is the only one that’s giving me problems.

Here’s screenshots of the DS config after manual creation and automatic creation:

Manual setup: https://ibb.co/hMPrPa
Automatic setup: https://ibb.co/haC6qF

On the datasources main screen (see image below) it actually looks like it has been configured with the correct elasticsearch source url at least, but it doesn’t list this inside the actual DS config. Also, as I mentioned, it doesn’t appear to be working as it can’t find any data and it gives a cryptic error on the graph (screenshot below):

Does anyone know how I can get this working?

Thanks,
Guy

2

Can you show the HTTP POST with the request body. The docs for this call are out of date unfortunately (I’ll fix that). Adding the other Elasticsearch specific fields is done in a different way - it uses a json field called jsonData. Here is an example request body:

{
  "name":"test",
  "type":"elasticsearch",
  "url":"http://localhost:9200",
  "access":"proxy",
  "jsonData":{
    "timeField":"@timestamp",
    "esVersion":5,
    "timeInterval":">10s"
  },
  "secureJsonFields":{},
  "database":"test"
}
3

Sure, here’s the POST request when creating the DS manually:

{  
   "name":"es",
   "type":"elasticsearch",
   "url":"https://search-app-errors-swf7dbaqndkgtpabcdefg123.us-west-2.es.amazonaws.com",
   "access":"proxy",
   "jsonData":{  
      "timeField":"@timestamp",
      "esVersion":5
   },
   "secureJsonFields":{  

   },
   "database":"cwl-*"
}

Here’s the ansible playbook I’m using to try and do automated creation of the DS:

- name: enable elasticsearch datasource
  uri:
    url: https://localhost:3000/api/datasources
    method: POST
    user: admin
    password: "{{ grafana.admin_password }}"
    body:
      name: "es"
      type: "elasticsearch"
      url: "https://search-app-errors-swf7dbaqndkgtpabcdefg123.us-west-2.es.amazonaws.com"
      access: "proxy"
      jsonData: "{ \"timeField\": \"@timestamp\", \"esVersion\": 5 }"
      secureJsonFields: "{}"
      database: "cwl-*"
      isDefault: false
    force_basic_auth: yes
    status_code: 200
    body_format: json
    validate_certs: no
  when: elasticsearch_ds.status == 404