Hello guys
I’m having problem with connecting grafana and graylog.
It says “Elasticsearch error: Bad Gateway
I’m using graylog and elastic search on same machine which ubuntu.
Any who can help please?
Hello @laale10 and welcome to the 
community
Here are a couple of links [1] [2] that could be helpful.
Otherwise, please share the whole Elasticserach settings
Hello @antonio thanks for the replay.
I have seen the links and it didn’t help.
here is my setup
I’m using Graylag and Elasticsearch on the same machine, Graylog is the master.
and Grafana in another machine.
Elastic configuration:-
======================== Elasticsearch Configuration =========================
NOTE: Elasticsearch comes with reasonable defaults for most settings.
Before you set out to tweak and tune the configuration, make sure you
understand what are you trying to accomplish and the consequences.
The primary way of configuring a node is via this file. This template lists
the most important settings you may want to configure for a production cluster.
Please consult the documentation for further information on configuration options:
---------------------------------- Cluster -----------------------------------
Use a descriptive name for your cluster:
cluster.name: graylog
------------------------------------ Node ------------------------------------
Use a descriptive name for the node:
#node.name: node-1
Add custom attributes to the node:
#node.attr.rack: r1
----------------------------------- Paths ------------------------------------
Path to directory where to store the data (separate multiple locations by comma):
path.data: /var/lib/elasticsearch
Path to log files:
path.logs: /var/log/elasticsearch
----------------------------------- Memory -----------------------------------
Lock the memory on startup:
#bootstrap.memory_lock: true
Make sure that the heap size is set to about half the memory available
on the system and that the owner of the process is allowed to use this
limit.
Elasticsearch performs poorly when the system is swapping the memory.
---------------------------------- Network -----------------------------------
Set the bind address to a specific IP (IPv4 or IPv6):
#network.host: 192.168.0.1
Set a custom port for HTTP:
#http.port: 9200
For more information, consult the network module documentation.
--------------------------------- Discovery ----------------------------------
Pass an initial list of hosts to perform discovery when new node is started:
The default list of hosts is [“127.0.0.1”, “[::1]”]
#discovery.zen.ping.unicast.hosts: [“host1”, “host2”]
Prevent the “split brain” by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#discovery.zen.minimum_master_nodes:
For more information, consult the zen discovery module documentation.
---------------------------------- Gateway -----------------------------------
Block initial recovery after a full cluster restart until N nodes are started:
#gateway.recover_after_nodes: 3
For more information, consult the gateway module documentation.
---------------------------------- Various -----------------------------------
Require explicit names when deleting indices:
action.destructive_requires_name: false
Graylog configuration:-
############################
GRAYLOG CONFIGURATION FILE
############################
This is the Graylog configuration file. The file has to use ISO 8859-1/Latin-1 character encoding.
Characters that cannot be directly represented in this encoding can be written using Unicode escapes
as defined in .
For example, \u002c.
- Entries are generally expected to be a single line of the form, one of the following:
propertyName=propertyValue
propertyName:propertyValue - White space that appears between the property name and property value is ignored,
so the following are equivalent:
name=Stephen
name = Stephen - White space at the beginning of the line is also ignored.
- Lines that start with the comment characters ! or # are ignored. Blank lines are also ignored.
- The property value is generally terminated by the end of the line. White space following the
property value is not ignored, and is treated as part of the property value. - A property value can span several lines if each line is terminated by a backslash (‘\’) character.
For example:
targetCities=
Detroit,
Chicago,
Los Angeles
This is equivalent to targetCities=Detroit,Chicago,Los Angeles (white space at the beginning of lines is ignored). - The characters newline, carriage return, and tab can be inserted with characters \n, \r, and \t, respectively.
- The backslash character must be escaped as a double backslash. For example:
path=c:\docs\doc1
If you are running more than one instances of Graylog server you have to select one of these
instances as leader. The leader will perform some periodical tasks that non-leaders won’t perform.
is_master = true
The auto-generated node ID will be stored in this file and read after restarts. It is a good idea
to use an absolute file path here if you are starting Graylog server from init scripts or similar.
node_id_file = /etc/graylog/server/node-id
You MUST set a secret to secure/pepper the stored user passwords here. Use at least 64 characters.
Generate one by using for example: pwgen -N 1 -s 96
ATTENTION: This value must be the same on all Graylog nodes in the cluster.
Changing this value after installation will render all user sessions and encrypted values in the database invalid. (e.g. encrypted access tokens)
password_secret = VBC9MiBZZ2Mkye7VYq1D3zCCCvdxQJmzxsmBvdBMwsHcbtSxsg9kGL6YSIjDPHQxMMFzdoBhNPfmJ0Y3Vllx6RR3yjowcw4o
The default root user is named ‘admin’
#root_username = admin
You MUST specify a hash password for the root user (which you only need to initially set up the
system and in case you lose connectivity to your authentication backend)
This password cannot be changed using the API or via the web interface. If you need to change it,
modify it in this file.
Create one by using for example: echo -n yourpassword | shasum -a 256
and put the resulting hash value into the following line
root_password_sha2 = 66c74b936306c9927d418c5e1ccb3fa137d36c90f52ebf71100a189b9010f85c
The email address of the root user.
Default is empty
#root_email = “”
The time zone setting of the root user. See http:/www.joda.org/joda-time/timezones.html for a list of valid time zones.
Default is UTC
#root_timezone = UTC
Set the bin directory here (relative or absolute)
This directory contains binaries that are used by the Graylog server.
Default: bin
bin_dir = /usr/share/graylog-server/bin
Set the data directory here (relative or absolute)
This directory is used to store Graylog server state.
Default: data
data_dir = /var/lib/graylog-server
Set plugin directory here (relative or absolute)
plugin_dir = /usr/share/graylog-server/plugin
###############
HTTP settings
###############
HTTP bind address
The network interface used by the Graylog HTTP interface.
This network interface must be accessible by all Graylog nodes in the cluster and by all clients
using the Graylog web interface.
If the port is omitted, Graylog will use port 9000 by default.
Default: 127.0.0.1:9000
http_bind_address = 0.0.0.0:9000
#http_bind_address = [2001:db8::1]:9000
HTTP publish URI
The HTTP URI of this Graylog node which is used to communicate with the other Graylog nodes in the cluster and by all
clients using the Graylog web interface.
The URI will be published in the cluster discovery APIs, so that other Graylog nodes will be able to find and connect to this Graylog node.
This configuration setting has to be used if this Graylog node is available on another network interface than $http_bind_address,
for example if the machine has multiple network interfaces or is behind a NAT gateway.
If $http_bind_address contains a wildcard IPv4 address (0.0.0.0), the first non-loopback IPv4 address of this machine will be used.
This configuration setting must not contain a wildcard address!
Default:
#http_publish_uri = http:/192.168.1.1:9000/
External Graylog URI
The public URI of Graylog which will be used by the Graylog web interface to communicate with the Graylog REST API.
The external Graylog URI usually has to be specified, if Graylog is running behind a reverse proxy or load-balancer
and it will be used to generate URLs addressing entities in the Graylog REST API (see $http_bind_address).
When using Graylog Collector, this URI will be used to receive heartbeat messages and must be accessible for all collectors.
This setting can be overriden on a per-request basis with the “X-Graylog-Server-URL” HTTP request header.
Default: $http_publish_uri
#http_external_uri =
Enable CORS headers for HTTP interface
This allows browsers to make Cross-Origin requests from any origin.
This is disabled for security reasons and typically only needed if running graylog
with a separate server for frontend development.
Default: false
#http_enable_cors = false
Enable GZIP support for HTTP interface
This compresses API responses and therefore helps to reduce
overall round trip times. This is enabled by default. Uncomment the next line to disable it.
#http_enable_gzip = false
The maximum size of the HTTP request headers in bytes.
#http_max_header_size = 8192
The size of the thread pool used exclusively for serving the HTTP interface.
#http_thread_pool_size = 16
################
HTTPS settings
################
Enable HTTPS support for the HTTP interface
This secures the communication with the HTTP interface with TLS to prevent request forgery and eavesdropping.
Default: false
#http_enable_tls = true
The X.509 certificate chain file in PEM format to use for securing the HTTP interface.
#http_tls_cert_file = /path/to/graylog.crt
The PKCS#8 private key file in PEM format to use for securing the HTTP interface.
#http_tls_key_file = /path/to/graylog.key
The password to unlock the private key used for securing the HTTP interface.
#http_tls_key_password = secret
Comma separated list of trusted proxies that are allowed to set the client address with X-Forwarded-For
header. May be subnets, or hosts.
#trusted_proxies = 127.0.0.1/32, 0:0:0:0:0:0:0:1/128
List of Elasticsearch hosts Graylog should connect to.
Need to be specified as a comma-separated list of valid URIs for the http ports of your elasticsearch nodes.
If one or more of your elasticsearch hosts require authentication, include the credentials in each node URI that
requires authentication.
Default: http:/127.0.0.1:9200
#elasticsearch_hosts = http:/node1:9200,http:/user:password@node2:19200
#elasticsearch_hosts =
Maximum number of attempts to connect to elasticsearch on boot for the version probe.
Default: 0, retry indefinitely with the given delay until a connection could be established
#elasticsearch_version_probe_attempts = 5
Waiting time in between connection attempts for elasticsearch_version_probe_attempts
Default: 5s
#elasticsearch_version_probe_delay = 5s
Maximum amount of time to wait for successful connection to Elasticsearch HTTP port.
Default: 10 Seconds
#elasticsearch_connect_timeout = 10s
Maximum amount of time to wait for reading back a response from an Elasticsearch server.
(e. g. during search, index creation, or index time-range calculations)
Default: 60 seconds
#elasticsearch_socket_timeout = 60s
Maximum idle time for an Elasticsearch connection. If this is exceeded, this connection will
be tore down.
Default: inf
#elasticsearch_idle_timeout = -1s
Maximum number of total connections to Elasticsearch.
Default: 200
#elasticsearch_max_total_connections = 200
Maximum number of total connections per Elasticsearch route (normally this means per
elasticsearch server).
Default: 20
#elasticsearch_max_total_connections_per_route = 20
Maximum number of times Graylog will retry failed requests to Elasticsearch.
Default: 2
#elasticsearch_max_retries = 2
Enable automatic Elasticsearch node discovery through Nodes Info,
WARNING: Automatic node discovery does not work if Elasticsearch requires authentication, e. g. with Shield.
Default: false
#elasticsearch_discovery_enabled = true
Filter for including/excluding Elasticsearch nodes in discovery according to their custom attributes,
Default: empty
#elasticsearch_discovery_filter = rack:42
Frequency of the Elasticsearch node discovery.
Default: 30s
elasticsearch_discovery_frequency = 30s
Set the default scheme when connecting to Elasticsearch discovered nodes
Default: http (available options: http, https)
#elasticsearch_discovery_default_scheme = http
#elasticsearch_discovery_default_scheme = https
Enable payload compression for Elasticsearch requests.
Default: false
#elasticsearch_compression_enabled = true
Enable use of “Expect: 100-continue” Header for Elasticsearch index requests.
If this is disabled, Graylog cannot properly handle HTTP 413 Request Entity Too Large errors.
Default: true
#elasticsearch_use_expect_continue = true
Graylog will use multiple indices to store documents in. You can configure the strategy it uses to determine
when to rotate the currently active write index.
It supports multiple rotation strategies, the default being “count”:
- “count” of messages per index, use elasticsearch_max_docs_per_index below to configure
- “size” per index, use elasticsearch_max_size_per_index below to configure
- “time” interval between index rotations, use elasticsearch_max_time_per_index to configure
A strategy may be disabled by specifying the optional enabled_index_rotation_strategies list and excluding that strategy.
#enabled_index_rotation_strategies = count,size,time
Provides a hard upper limit for the retention period of any index set at configuration time.
This setting is used to validate the value a user chooses for the maximum number of retained indexes, when configuring
an index set. However, it is only in effect, when a time-based rotation strategy is chosen.
If a rotation strategy other than time-based is selected and/or no value is provided for this setting, no upper limit
for index retention will be enforced. This is also the default.
max_index_retention_period = P90d
ATTENTION: These settings have been moved to the database in 2.0. When you upgrade, make sure to set these
to your previous 1.x settings so they will be migrated to the database!
This configuration setting is only used on the first start of Graylog. After that,
index related settings can be changed in the Graylog web interface on the ‘System / Indices’ page.
(Approximate) maximum number of documents in an Elasticsearch index before a new index
is being created, also see no_retention and elasticsearch_max_number_of_indices.
Configure this if you used ‘rotation_strategy = count’ above.
ATTENTION: These settings have been moved to the database in 2.0. When you upgrade, make sure to set these
to your previous 1.x settings so they will be migrated to the database!
This configuration setting is only used on the first start of Graylog. After that,
index related settings can be changed in the Graylog web interface on the ‘System / Indices’ page.
elasticsearch_max_docs_per_index = 20000000
(Approximate) maximum size in bytes per Elasticsearch index on disk before a new index is being created, also see
no_retention and elasticsearch_max_number_of_indices. Default is 1GB.
Configure this if you used ‘rotation_strategy = size’ above.
ATTENTION: These settings have been moved to the database in 2.0. When you upgrade, make sure to set these
to your previous 1.x settings so they will be migrated to the database!
This configuration setting is only used on the first start of Graylog. After that,
index related settings can be changed in the Graylog web interface on the ‘System / Indices’ page.
#elasticsearch_max_size_per_index = 1073741824
(Approximate) maximum time before a new Elasticsearch index is being created, also see
no_retention and elasticsearch_max_number_of_indices. Default is 1 day.
Configure this if you used ‘rotation_strategy = time’ above.
Please note that this rotation period does not look at the time specified in the received messages, but is
using the real clock value to decide when to rotate the index!
Specify the time using a duration and a suffix indicating which unit you want:
1w = 1 week
1d = 1 day
12h = 12 hours
Permitted suffixes are: d for day, h for hour, m for minute, s for second.
ATTENTION: These settings have been moved to the database in 2.0. When you upgrade, make sure to set these
to your previous 1.x settings so they will be migrated to the database!
This configuration setting is only used on the first start of Graylog. After that,
index related settings can be changed in the Graylog web interface on the ‘System / Indices’ page.
#elasticsearch_max_time_per_index = 1d
Optional upper bound on elasticsearch_max_time_per_index
elasticsearch_max_write_index_age = 1d
Disable checking the version of Elasticsearch for being compatible with this Graylog release.
WARNING: Using Graylog with unsupported and untested versions of Elasticsearch may lead to data loss!
#elasticsearch_disable_version_check = true
elasticsearch_disable_version_check = true
elasticsearch_version = 7
Disable message retention on this node, i. e. disable Elasticsearch index rotation.
#no_retention = false
How many indices do you want to keep?
ATTENTION: These settings have been moved to the database in 2.0. When you upgrade, make sure to set these
to your previous 1.x settings so they will be migrated to the database!
This configuration setting is only used on the first start of Graylog. After that,
index related settings can be changed in the Graylog web interface on the ‘System / Indices’ page.
elasticsearch_max_number_of_indices = 20
Decide what happens with the oldest indices when the maximum number of indices is reached.
The following strategies are availble:
- delete # Deletes the index completely (Default)
- close # Closes the index and hides it from the system. Can be re-opened later.
ATTENTION: These settings have been moved to the database in 2.0. When you upgrade, make sure to set these
to your previous 1.x settings so they will be migrated to the database!
This configuration setting is only used on the first start of Graylog. After that,
index related settings can be changed in the Graylog web interface on the ‘System / Indices’ page.
retention_strategy = delete
How many Elasticsearch shards and replicas should be used per index? Note that this only applies to newly created indices.
ATTENTION: These settings have been moved to the database in Graylog 2.2.0. When you upgrade, make sure to set these
to your previous settings so they will be migrated to the database!
This configuration setting is only used on the first start of Graylog. After that,
index related settings can be changed in the Graylog web interface on the ‘System / Indices’ page.
elasticsearch_shards = 4
elasticsearch_replicas = 0
Prefix for all Elasticsearch indices and index aliases managed by Graylog.
ATTENTION: These settings have been moved to the database in Graylog 2.2.0. When you upgrade, make sure to set these
to your previous settings so they will be migrated to the database!
This configuration setting is only used on the first start of Graylog. After that,
index related settings can be changed in the Graylog web interface on the ‘System / Indices’ page.
Also see
elasticsearch_index_prefix = graylog
Name of the Elasticsearch index template used by Graylog to apply the mandatory index mapping.
Default: graylog-internal
ATTENTION: These settings have been moved to the database in Graylog 2.2.0. When you upgrade, make sure to set these
to your previous settings so they will be migrated to the database!
This configuration setting is only used on the first start of Graylog. After that,
index related settings can be changed in the Graylog web interface on the ‘System / Indices’ page.
#elasticsearch_template_name = graylog-internal
Do you want to allow searches with leading wildcards? This can be extremely resource hungry and should only
be enabled with care. See also:
allow_leading_wildcard_searches = false
Do you want to allow searches to be highlighted? Depending on the size of your messages this can be memory hungry and
should only be enabled after making sure your Elasticsearch cluster has enough memory.
allow_highlighting = false
Analyzer (tokenizer) to use for message and full_message field. The “standard” filter usually is a good idea.
All supported analyzers are: standard, simple, whitespace, stop, keyword, pattern, language, snowball, custom
Elasticsearch documentation:
Note that this setting only takes effect on newly created indices.
try increasing the verbosity of the Grafana server logs to debug and note any errors. For printing to console, set the console logs to debug as well.
Also, when you try to save and test the datasource connection, you can open up your browser’s developer tools and look at the network tab. that can often include useful information about network connection issues