Promtail __path__ matches nothing, while files actually exist

[dmitpanov]

Hello everyone,
please assist me in finding a root cause of the following behaviour:

I’m trying to configure promtail 2.4.2

promtail, version 2.4.2 (branch: HEAD, revision: 525040a32)
  build user:       root@5d9e7a4c92e6
  build date:       2022-01-12T16:48:53Z
  go version:       go1.16.2
  platform:         linux/amd64

Installed it as a systemd service:

[Unit]
Description=promtail
After=network-online.target

[Service]
Restart=always
RestartSec=5
TimeoutSec=5
User=promtail
Group=promtail
ExecStart=/usr/local/bin/promtail -config.file=/etc/promtail/config.yml -log.level=debug

[Install]
WantedBy=multi-user.target

Nothing appeared in Loki so I had enabled debug log_level.
That gave me messages like:

no files matched requested path, nothing will be tailed

for all configured patterns.
my config:

server:
  grpc_listen_port: 0
  http_listen_port: 9080


positions:
  filename: /var/lib/promtail/positions.yml


clients:
  - url: http://192.168.152.23:3100/loki/api/v1/push


scrape_configs:
  - file_sd_configs:
    - files:
      - /etc/promtail/file_sd/*.yml
      - /etc/promtail/file_sd/*.yaml
      - /etc/promtail/file_sd/*.json
    job_name: file_sd

  - job_name: tz_elastic-json
    pipeline_stages:
    - json:
        expressions:
          cluster: cluster.name
          component: component
          level: level
          message: message
          node: node.name
          timestamp: timestamp
          type: type
    static_configs:
    - labels:
        __path__: /var/log/elasticsearch/*server.json
        host: tz-elastic-master-1
        instance: 192.168.152.211
        job: tz_elastic
  - job_name: tz_elastic-log
    static_configs:
    - labels:
        __path__: /var/log/elasticsearch/*.ru.log
        host: tz-elastic-master-1
        instance: 192.168.152.211
        job: tz_elastic
  - job_name: tz_elastic-gc-log
    static_configs:
    - labels:
        __path__: /var/log/elasticsearch/gc.log
        host: tz-elastic-master-1
        instance: 192.168.152.211
        job: tz_elastic

files actually exist:

root@tz-elastic-master-1:/var/log/elasticsearch# ls {gc.log,*server.json,*.ru.log}
gc.log  tz-elasticsearch.int.mycompany.ru.log  tz-elasticsearch.int.mycompany.ru_server.json

Also I’ve tested glob library separately GitHub - bmatcuk/doublestar: Implements support for double star (**) matches in golang's path.Match and filepath.Glob.

and it seems to be OK.

Please advise what else should I check? Is there any issue in my configuration?

[dmitpanov]

Managed to solve the issue.
Promtail user had no access to target path and for some reason there’s no error log message on ‘permission denied’.