Grafana in Docker, SMTP notification failure

Grafana Configuration
, , ,
1

Hi,

I am trying to get e-mail notifications working.

I am currently working with Grafana 8.4.3 in a Docker container, where Docker runs within wsl2 on Windows 10.

To make my SMTP notifications work I configured the grafana.ini with settings including access credentials (login and password) I got from my corporate smtp provider.

    - GF_UNIFIED_ALERTING_ENABLED=true
    - GF_SMTP_ENABLED=true
    - GF_SMTP_HOST=<server>:25
    - GF_SMTP_USER=user
    - GF_SMTP_Password="""password"""
    - GF_SMTP_startTLS_policy=OpportunisticStartTLS
    - GF_SMTP_FROM_ADDRESS=<e-mail adress>
    - GF_SMTP_FROM_NAME=DTA
    #- GF_SMTP_cert_file=
    #- GF_SMTP_key_file=
    - GF_SMTP_SKIP_VERIFY=false
    - GF_APP_MODE=development
    - GF_SERVER_DOMAIN=<domain>
    - GF_RENDERING_SERVER_URL=http://grafana_renderer:8081/render
    - GF_RENDERING_CALLBACK_URL=http://grafana_com:3000/
    - GF_SERVER_ROOT_URL=http://md2gaahc:8080/
    - GF_LOG_FILTERS=rendering:debug

While testing the e-mail notification in Grafana I am getting 501 error.

Grafana_error

I also get two errors in the Grafana log at the time I test the notification, but I am not sure if they are related.

logger=alerting.notifier.email level=error msg="Missing receiver"
logger=alerting.notifier.email level=error msg="Missing group labels"

The problems began since my SMTP provider started requesting login and password to access the service.

But there is one interesting thing. When I use Grafana for Windows (without Docker) and apply the same SMTP settings, the notifications work.

2

this sounds related to your Alerting version. Are you using the legacy alerting platform or the new alerting platform that was introdced in v8 as an opt-in? It is now default in 9.

3

Thanks for reply, and sorry for the late response.

I’ve just updated Grafana to 9.1.1 and I get exactly the same errors.

But as I stated before, it is only a problem if I run Grafana from Docker container

4

I have met the same problem.

And found it that it is caused by the network mode of the docker.

The issue had been resovled after changing network_mode: bridge to network_mode: host .

2 Likes
5

Unfortunatelly network_mode: host doesn’t work with Windows.
Any other ideas?

6

I’ve just updated Grafana to 9.2.4.
If Grafana runs directly on Windows everything is fine. If I run it in a Docker container on Windows (I use Docker Desktop) I get the 501 error from SMTP server. Is there any explanation for the difference in Grafanas
behaviour?

7

can you reach the smtp server from within docker? ping or run some other smtp command to send email from within the docker grafana?

What does the grafana log say?

8

This is not a Grafana issue but is related to Windows internal networking for Docker as this feature i.e. network_mode: host seems to be not yet implemented

Please check the following links for reference:

1- windows run docker with --network=host and access with 127.0.0.1 - Stack Overflow

2- Support Host Network (--network host) mode on Docker Desktop for Mac and Windows · Issue #238 · docker/roadmap · GitHub

3- Expose all ports for a Docker image - Stack Overflow

4- ruby on rails - Connecting to WSL2 server via local network - Stack Overflow

5- Docker alternative to --network host on macOS and Windows - Stack Overflow

I hope this helps.

9

yes, in general I can reach a SMTP server from within a docker.

The specfic problem with grafana in a docker container, like I described it above, started since my SMTP provider started requesting authentication.

1 Like
10

That’s the log. I get that every time I try to test email notification.

11

Grafana_error

One last thing. Is this error message comming from my SMPT server, or is generated localy by grafana?

12

If your SMTP server is generating an error, it’ll be in that server’s own log
files.

Antony.

1 Like
13

And you are providing the authentication in your grafana config file?

14

not directly, I state user name and password of the SMTP server in the docker-compose.yml

15

I’ll check that! Must find access first

16

Yes, this is the error from the SMTP side as you can see a similar but a bit more detailed message in this reference link.

1 Like
17

Alright! So the problem boils down to the fact that SMTP requests from Grafan in Docker reach the SMTP server but for some reason cannot be processed.

What is so special in the combination of Docker and SMTP authentication?

18

Hi @spectator,

There is no special combinaiton of Docker and SMTP. It is all about the configuration settings so that SMTP is reachable and can send out emails.

The only limitation of Docker for SMTP comes in windows where the host network will not work (which I had posted above with a few reference links).

You will find some similar posts here in the community if you search for e.g. “Grafana SMTP Docker” where different users had proposed different solution strategies which had worked for them.

Also similarly, you can find some more references outside the community forums e.g. here

19

So why does it work when no authentication is required? What is so special in authentication that I need to use host network now?

20

Thanks, I’ve check almost all related post from the past few years. Except for the host network solution (which isn’t working for me) there is little to no explanation of the actual problem.