Logstash multiline codec ignore last event / line - logstash

Question

Logstash multiline codec ignore my last event (line) until send next package of logs.
My logstash.conf:
input {
}
http {
port => "5001"
codec => multiline {
pattern => "^\[%{TIMESTAMP_ISO8601}\]"
negate => true
what => previous
auto_flush_interval => 15
}
}
}
filter{
grok {
match => { "message" => "(?m)\[%{TIMESTAMP_ISO8601:timestamp}\]\s\<%{LOGLEVEL:log-level}\>\s\[%{WORD:component}\]\s%{GREEDYDATA:log-message}"
}
}
output {
elasticsearch {
hosts => "elasticsearch:9200"
index => "%{+YYYY-MM-dd}"
}
}
Moreover solution with auto_flush_interval don't work.
For example:
input using Postman:
[2017-07-11 22:32:12.345] [KCU] Component initializing
Exception in thread "main" java.lang.NullPointerException
at com.example.myproject.Book.getTitle(Book.java:16)
[2017-07-11 22:32:16.345] [KCU] Return with status 1
output - only one event (should be two):
[2017-07-11 22:32:12.345] [KCU] Component initializing
Exception in thread "main" java.lang.NullPointerException
at com.example.myproject.Book.getTitle(Book.java:16)
I need this last line.
Question:
Am I doing something wrong or there are problems with multiline codec? - How to fix this?

Answer 1

I'm afraid you're using the multiline codec wrong. Let's take a look at your configuration:
codec => multiline {
pattern => "^\[%{TIMESTAMP_ISO8601}\]"
negate => true
what => previous
}
It says if a logline does not (negate => true) start with a ISO timestamp (pattern) append it to the previous log line (what => previous).
But the logline you're missing starts with a ISO timestamp:
[2017-07-11 22:32:16.345] [KCU] Return with status 1
So it will not be appended to the previous log lines but instead create a new document in Elasticsearch.