GROK pattern cannot parse logs - logstash

Question

I want to launch the ELK-stack for gathering syslog from all my network equipment - Cisco, F5, Huawei, CheckPoint, etc. While experimenting with Logstash, writing grok patterns.
Below is an example of messages from Cisco ASR:
<191>Oct 30 16:30:10 evlogd: [local-60sec10.950] [cli 30000 debug] [8/0/30501 cliparse.c:367] [context: local, contextID: 1] [software internal system syslog] CLI command [user root, mode [local]ASR5K]: show ims-authorization policy-control statistics\u0000
<190>Oct 30 16:30:10 evlogd: [local-60sec10.959] [cli 30005 info] [8/0/30501 _commands_cli.c:1792] [software internal system syslog] CLI session ended for Security Administrator root on device /dev/pts/7\u0000
<190>Oct 30 16:30:10 evlogd: [local-60sec10.981] [snmp 22002 info] [8/0/4550 trap_api.c:930] [software internal system syslog] Internal trap notification 53 (CLISessEnd) user root privilege level Security Administrator ttyname /dev/pts/7\u0000
<190>Oct 30 16:30:12 evlogd: [local-60sec12.639] [cli 30004 info] [8/0/30575 cli_sess.c:127] [software internal system syslog] CLI session started for Security Administrator root on device /dev/pts/7 from 192.168.1.1\u0000
<190>Oct 30 16:30:12 evlogd: [local-60sec12.640] [snmp 22002 info] [8/0/30575 trap_api.c:930] [software internal system syslog] Internal trap notification 52 (CLISessStart) user root privilege level Security Administrator ttyname /dev/pts/7\u0000
All of them matching with my pattern here and here.
<%{POSINT:syslog_pri}>%{DATA:month} %{DATA:monthday} %{TIME:time} %{WORD:device}: \[%{WORD:facility}\-%{HOSTNAME}\] \[%{WORD:service} %{POSINT} %{WORD:priority}\] \[%{DATA}\] ?(\[context: %{DATA:context}, %{DATA}\])?%{SPACE}?(\[%{DATA}\] )%{GREEDYDATA:message}\\u0000
But my simple logstash configuration return tag _grokparsefailure (or _grokparsefailure_sysloginput if I use GROK in syslog input plugin), and doesn't parse my log.
Config using GROK-filter
input { udp {
port => 5140
type => syslog } }
filter {
if [type] == "syslog" {
grok {
match => ["message", "<%{POSINT:syslog_pri}>%{DATA:month} %{DATA:monthday} %{TIME:time} %{WORD:device}: \[%{WORD:facility}\-%{HOSTNAME}\] \[%{WORD:service} %{POSINT} %{WORD:priority}\] \
[%{DATA}\] ?(\[context: %{DATA:context}, %{DATA}\])?%{SPACE}?(\[%{DATA}\] )%{GREEDYDATA:response}\\u0000"]
}
}
}
output { stdout { codec => rubydebug } }
Output:
{
"#version" => "1",
"host" => "172.17.0.1",
"#timestamp" => 2018-10-31T09:46:51.121Z,
"message" => "<190>Oct 31 15:46:51 evlogd: [local-60sec51.119] [snmp 22002 info] [8/0/4550 <sitmain:80> trap_api.c:930] [software internal system syslog] Internal trap notification 53 (CLISessEnd) user kiwi privilege level Security Administrator ttyname /dev/pts/7\u0000",
"type" => "syslog",
"tags" => [
[0] "_grokparsefailure"
]
}
Config syslog-input-plugin:
input {
syslog {
port => 5140
grok_pattern => "<%{POSINT:syslog_pri}>%{DATA:month} %{DATA:monthday} %{TIME:time} %{WORD:device}: \[%{WORD:facility}\-%{HOSTNAME}\] \[%{WORD:service} %{POSINT} %{WORD:priority}\] \[%{DATA
}\] ?(\[context: %{DATA:context}, %{DATA}\])?%{SPACE}?(\[%{DATA}\] )%{GREEDYDATA:response}\\u0000"
}
}
output {
stdout { codec => rubydebug }
}
Output:
{
"severity" => 0,
"#timestamp" => 2018-10-31T09:54:56.871Z,
"#version" => "1",
"host" => "172.17.0.1",
"message" => "<191>Oct 31 15:54:56 evlogd: [local-60sec56.870] [cli 30000 debug] [8/0/22400 <cli:8022400> cliparse.c:367] [context: local, contextID: 1] [software internal system syslog] CLI command [user kiwi, mode [local]ALA3_ASR5K]: show subscribers ggsn-only sum apn osmp\u0000",
"tags" => [
[0] "_grokparsefailure_sysloginput"
],
}
What am I doing wrong? And can someone help fix it?
PS Tested on logstash 2.4.1 and 5

Answer 1

Unlike the online-debuggers, logstash's GROK didn't like my \\u0000 at the end of pattern.
With single backslash all is working.
Right grok-filter is:
<%{POSINT:syslog_pri}>%{DATA:month} %{DATA:monthday} %{TIME:time} %{WORD:device}: \[%{WORD:facility}\-%{HOSTNAME}\] \[%{WORD:service} %{POSINT} %{WORD:priority}\] \[%{DATA}\] ?(\[context: %{DATA:context}, %{DATA}\])?%{SPACE}?(\[%{DATA}\] )%{GREEDYDATA:message}\u0000

Answer 2

I have faced similar problem. The workaround is just to received those logs from routers/firewalls/switches to a syslog-ng server and then forward to the logstash.
Following is a sample configuration for syslog-ng,
source s_router1 {
udp(ip(0.0.0.0) port(1514));
tcp(ip(0.0.0.0) port(1514));
};
destination d_router1_logstash { tcp("localhost",port(5045)); };
log { source(s_router1); destination(d_router1_logstash); };