Grafana monitoring MySQL/Postgres database

Is it possible and if yes, how to configure Grafana for this?

Yes, there are many ways to monitor these databases.

First you need a collector to gather metrics about your database and store those metrics in a time series database. Then you can visualize and alert on this metrics in Grafana.

Here are a couple of ready made dashboards for mysql. You just need to install a metric collector (for example node exporter for Prometheus)