Not able to create Labels from Promtail static_configs

[hangon]

I’m using the latest promtail and loki and trying to use your static_configs example… but I’m not getting the labels pushed to Loki… all I see below

Discovered labels
__address
__path
job

I do not see labels of facility and hostname am I doing something wrong ?

here is my promtail

server:
http_listen_port: 9080
grpc_listen_port: 0

positions:
filename: C:\loki\tmp\promtail\positions.yaml

clients:

• url: http://localhost:3100/loki/api/v1/push
  scrape_configs:
• job_name: rfc5424
  pipeline_stages:
  • regex:
    expression: ‘.?<(\d+)>(\d+)\s\d±\d±\d+T\d+:\d+:\d+.\d+\S+\s+(\S+)\s+.’
    output:
    source_labels: [facility, hostname]
  • labels:
    facility: “${1}”
    hostname: “${2}”
    static_configs:
  • targets:
    • localhost
      labels:
      job: rfc5424
      path: C:\logs\rfc5424*log

My Log format is

<165>1 2003-10-11T22:14:15.003Z mymachine.example.com evntslog - ID47 [exampleSDID@32473 iut=“3” eventSource=“Application” eventID=“1011”] BOMAn application event log entry…

[tonyswumac]

Couple of things:

1. Your regex doesn’t actually match your log string.
2. In order to retain something regex you need to use capture groups.
3. Once you capture something with regex you can then use it as a reference in subsequent pipeline actions.

Your formation is all over the place, so I am not quite sure which part of the log would be facility, so the example below captures hostname only (not tested):

scrape_configs:
  - job_name: rfc5424
    static_configs:
      - targets: localhost
        labels:
          job: rfc5424
          path: C:\logs\rfc5424*log
    
    pipeline_stages:
      - regex:
          expression: '^\S+ \S+ (?P<hostname>\S+) .*$'
      - labels:
          hostname:

I would also recommend you to capture the timestamp part and turn that into actual timestamp for the log stream.

[hangon]

@tonyswumac

^\S+ \S+ (?P<hostname>\S+) .*$

First, Thanks for the response… really appreciate it… I tried to acheive in different ways… but I’m still not getting the results I need (may be my understanding is wrong ?)

I tried different ways … here is my config and I’m expecting label “identification” show up in the loki but not working…

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: C:\_development\loki\tmp\promtail\positions.yaml

clients:
  - url: http://localhost:3100/loki/api/v1/push

scrape_configs:
 - job_name: systemx
   static_configs:
   - targets:
      - localhost
     labels:
      job: varlogs
      host: testserver
      __path__: C:\_development\logs\rfc5424x\*log

   pipeline_stages:
       - regex:
           expression: '^\S+ \S+ (?P<hostname>\S+) .*$'
       - labels:
           identification: ${hostname}

Here is the screenshot from the Service Discovery Page where I do not see “Identification”

[hangon]

Finally this is working for me… you have to check it in th Grafana - Datasources - Explore

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: C:\_development\loki\tmp\promtail\positions.yaml

clients:
  - url: http://localhost:3100/loki/api/v1/push

scrape_configs:
 - job_name: systemx
   static_configs:
   - targets:
      - localhost
     labels:
      job: varlogs
      host: testserver
      __path__: C:\_development\logs\rfc5424x\*log

   pipeline_stages:
       - regex:
           expression: '^\S+ \S+ (?P<hostname>\S+) .*$'
       - labels:
           hostname: