I am very new to PLG (promtail-loki-grafana) and wanted to build a solution for Golang glog format, essentially break apart these into pieces, so that things like W, E, I get converted to warnings, errors, and info type of logs, as well as give meaning to other parts of the log, and to handle multi-line stack traces.
I created a small env based on loki docker install docs, and I am experimenting using this from Logging with Loki: Essential configuration settings video.
docker logs alpha1 2>&1 | promtail --stdin --dry-run --config.file=test.yaml
This actually worked (after with initial hiccup listen tcp :9080: bind: address already in use), but I am stuck on two things, and I am not sure what to research, where go from here.
- how can I build something akin to grok (logstash, fluentd) to break up golang glog format, as well as other formats, like sentry?
- how can I get docker service discovery to work? The install documentation didn’t cover loki datasource setup in grafana config (which is easy enough to do), docker, or any post install material related to docker.
Ultimately, this will end up in Kubernetes, but I wanted to build out a local test environment first to help build better logs for our environment.