logstash config file - separating out user and message - logstash

Question

New to logstash. I am trying to parse application log lines such as:
2014-11-05 16:59:36,779 ERROR DOMAINNAME\bob [This is an error. ]
My config file looks like this:
input {
file {
path => "C:/tmp/*.log"
}
}
filter {
grok {
match => [
"message", "%{TIMESTAMP_ISO8601:timestamp}\s*%{LOGLEVEL:level}\s*%{DATA:userAlias}\s*%{GREEDYDATA:message}"
]
overwrite => [ "message" ]
}
if [level] =~ "INFO" {
drop {
}
}
}
output {
elasticsearch {
host => "localhost"
protocol => "http"
}
}
The timestamp and level are parsed out fine, but the message displays in Kibana as:
message:
DOMAINNAME\bob [This is an error. ]
The grok pattern for DATA is .*?
so I would assume that it should handle the backslash \ and properly set
userAlias to DOMAINNAME\bob and
message to [This is an error. ]
But this isn't the case. What am I doing wrong here? Thanks.

Answer 1

The problem with your grok pattern is that .*? is non-greedy (i.e. optional) and .* is greedy, so the latter "takes over" the string that could have been matched by the preceding .*? pattern.
I suggest you avoid the DATA and GREEDYDATA patterns except for matching the remainder of the string (like your use of GREEDYDATA here). In this case you could e.g. use the NOTSPACE pattern to match the username. You could use an even more specific pattern that e.g. excludes characters that are invalid in usernames, but I don't see the point of that. This works:
"%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:level}\s+%{NOTSPACE:userAlias}\s+%{GREEDYDATA:message}"
(I also took the liberty of replacing \s* with \s+ since the whitespace between the fields isn't optional.)