Bad Gateway http: proxy error: x509: certificate is valid for Unknown, not FQDN

I have everthing working happily under non-https but using https I cannot get passed

EROR[03-02|17:43:04] Data proxy error logger=data-proxy-log userId=1 orgId=1 uname=admin path=/api/datasources/proxy/6 remote_addr=199.188.77.23 referer=https://grafana.oninit.com:3000/datasources/edit/6/ error="http: proxy error: x509: certificate is valid for Unknown, not "

This is being triggered via TestDatasource. The URL work via direct https and curl from both the grafana and the database server. I have wildcard DigiCert signed cert deployed across all the servers.

I suspect I have a missed a setting, hence the ‘Unknown’, I just don’t know what is Unknown.

openssl on the pem file, searching for the CN, that is the CN used in server_cert_name but I have tried both the grafana and database server FDQN. Also I have tried pem format certs and crt/key files.

Searching on Google I can’t anythng for “certificate is valid for Unknown”

The working non-ssl config has access : proxy in the Yaml file under provisioning, I have tried direct as well

what reverse proxy are you using? And what grafana version? There was a change to reverse proxy settings in 8.3.5+

I am using 7.5.11. pulled from the Fedora 34 repository. I am not aware of setting any reverse proxy config. Under non https every thing works fine, so it must just be a config issue.

The servers are behind a firewalll and are NAT’d to the real world, although until I can get the https working there are only a limited number of external IPs can access the systems. I can curl the test URL successfully from the internal and external IPs from the internal servers (grafana server, database server and another test server) and the external IPs from the trusted external IPs

Added the following to Apache config

<VirtualHost *:443>
ServerName grafana.oninit.com
ServerAdmin webmaster@oninit.com
DocumentRoot pathto/www.oninit.com
UserDir disabled
SSLEngine on
SSLProxyEngine on
SSLCertificateFile “pathto/star_oninit_com.crt”
SSLCertificateKeyFile “pathto/star_oninit_com.key”
SSLCertificateChainFile “pathto/DigiCertCA.crt”

    CustomLog logs/www.oninit.com/grafana.access combined
    ErrorLog logs/www.oninit.com/grafana.err

    ReWriteEngine on
    ProxyPass "/"  "https://<internal ip:3000/"
    ProxyPassReverse "/"  "https://<internal ip>:3000/"

Same error

Changed Proxy from internal ip to internal FQDN got the proxy working but still Bad Gateway

I have it working but got it wrong at first.
Using nginx as my reverse proxy, I setup a redirect from http to https
Then proxy_pass to the grafana instance which is likely what you’re doing.
But.
I had the grafana instance also serving as https (protocol = h20 and was serving on localhost with the same certificates setup in grafana. Not surprisingly I started getting complaints which amounted to localhost =/= grafana.mydomain.com.

So I reset grafana to protocol=http and removed the certificates and redirected to an http version of grafana.
My nginx config looks like:
server {
listen 80;
server_name grafana.mydomain.com;
return 301 https://grafana.mydomain.com;
}
server {
server_name grafana.mydomain.com;
listen 443 ssl http2;
ssl on;
ssl_certificate /usr/share/ssl/certs/my_cert.crt;
ssl_certificate_key /usr/share/ssl/certs/my_key.key;
location / {
proxy_pass http://127.0.0.1:3000/;
proxy_set_header Host $http_host;
proxy_connect_timeout 600;
proxy_send_timeout 600;
proxy_read_timeout 600;
send_timeout 600;
}
location ~ {{ {
deny all;
}
}

…and all is well.

In my world that doesn’t fix it, it just takes a standard URL request and sends it to Grafana over https. That bit works fine for me and always has.

The issue is how make Grafana use a https request to the database source. I get a Bad Gateway when I make that call.

The fix was to make sure the SSL worked across all the servers. I could not find a way to make Grafana ignore any verification/certifcate issues on the JSON databsource connection