Error Parsing Syslog Stream

[chrisnieveen]

Expected behavior

Log messages to get forwarded to Grafana / Loki

Actual behavior

Error message:
Apr 24 14:44:50 grafana promtail-linux-amd64[200904]: level=warn ts=2023-04-24T14:44:50.3438799Z caller=syslogtarget.go:105 msg=“error parsing syslog stream” err=“expecting an app-name (from 1 to max 48 US-ASCII characters) or a nil value [col 54]”

All other network devices work properly.

Steps to reproduce the behavior

I am forwarding Meraki logs to Rsyslog to get scraped by Promtail and forwarded into Grafana/Loki. However when forwardiing this is the error I receive.

Environment

• Grafana Version: 9.4.7
• rsyslog version: 8.2112.0
• platform: Ubuntu 22.04.2

[yosiasz]

Welcome

Could you please post sample syslog couple of line, obfuscate any sensitive data you might have?

[chrisnieveen]

rsyslog.conf:

/etc/rsyslog.conf configuration file for rsyslog

Default logging rules can be found in /etc/rsyslog.d/50-default.conf

#################

MODULES

#################

module(load=“imuxsock”) # provides support for local system logging
#module(load=“immark”) # provides --MARK-- message capability

provides UDP syslog reception

#module(load=“imudp”)
#input(type=“imudp” port=“514”)
$ModLoad imudp
$UDPServerRun

provides TCP syslog reception

#module(load=“imtcp”)
#input(type=“imtcp” port=“514”)

provides kernel logging support and enable non-kernel klog messages

module(load=“imklog” permitnonkernelfacility=“on”)

###########################

GLOBAL DIRECTIVES

###########################

Use traditional timestamp format.

To enable high precision timestamps, comment out the following line.

$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

Filter duplicated messages

$RepeatedMsgReduction on

Set the default permissions for all log files.

$FileOwner syslog
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
$PrivDropToUser syslog
$PrivDropToGroup syslog

Where to place spool and state files

$WorkDirectory /var/spool/rsyslog

Include all config files in /etc/rsyslog.d/

$IncludeConfig /etc/rsyslog.d/*.conf

$template remote-incoming-logs, "/var/log/logs.log
. ?remote-incoming-logs
STOP

Promtail Config:
action(type=“omfwd” Target=“localhost” Port=“1514” Protocol=“tcp” Template=“RSYSLOG_SyslogProtocol23Format”)
}

RSyslog Documentation - rsyslog

module(load=“imudp”)
input(type=“imudp” port=“514” ruleset=“remote”)

RSyslog Documentation - rsyslog

module(load=“imtcp”)
input(type=“imtcp” port=“514” ruleset=“remote”)

[yosiasz]

Great. You provided everything except what I asked.

Please provide us a sample actual syslog data from your log file

[chrisnieveen]

Oops!! Sorry.

1379967288.409907239 MS220_8P events port 3 status changed from 100fdx to down

1380653443.857790533 MR18 events type=association radio=‘0’ vap=‘1’ channel=‘6’ rssi=‘23’ aid=‘1813578850’