Logstash - sleep until there is a specific message - logstash

Question

In Logstash, let's say I have the following lines in my logs:
Message: msisdn: 111111111
Message: msisdn: 222222222
Answer: msisdn: 111111111
Answer: msisdn: 222222222
Now, whenever I get the Message, I'd like to wait for X seconds.
If within this time period I get the matching Answer (i.e. with the same msisdn), mark it as OK, else mark it as ERROR.
How can I do that?
Thanks
**EDIT**
Fairy, I tried to work with the aggregate filter but with no success, can you please help me with that?
input {
stdin {}
}
filter {
grok {
match => {"message" => "%{GREEDYDATA:type}: msisdn: %{INT:msisdn}"}
}
if [type] == "Message" {
aggregate {
task_id => "%{msisdn}"
code => "event.set('result', 'OK')"
timeout => 5
timeout_code => "event.set('result', 'error')"
}
}
if [logger] == "Answer" {
aggregate {
task_id => "%{msisdn}"
code => "event.set('result', 'OK')"
end_of_task => true
}
}
if "_grokparsefailure" in [tags] {
drop {}
}
}
output {
stdout { codec => rubydebug }
}
After writing this line to the stdin:
Message: msisdn: 111111111
There was a response immediately (it didn't wait 5 seconds) with status OK
{
"result" => "OK",
"#timestamp" => 2017-03-30T17:58:39.940Z,
"#version" => "1",
"host" => "31634cf481d5",
"message" => "Message: msisdn: 111111111",
"type" => "Message",
"msisdn" => "111111111"
}
Should I write it differently?
Many thanks ;)

Answer 1

The aggregate filter pretty much does what you are describing. First you need to identify the task_id to which you correlate your messages. Then you set up your logstash filter so it get the type of the message and your task_id. Then we have to seperate aggregate filters, one for the first theother for the second event.
The first has the timeout and the code to set the event to error should it timeout. The second filter checks for the Answer message and ends the aggregation.
grok {
match => {%{DATA:type}: %{DATA:name}: %{DATA:id}}
}
if [type] == "Message" {
aggregate {
task_id => "%{id}"
code => "event.set('result', 'OK')"
timeout => 5
timeout_code => "event.set('result', 'error')"
}
}
if [type] == "Answer" {
aggregate {
task_id => "%{id}"
code => "event.set('result', 'OK')"
end_of_task => true
}
}
Check out the documentation for the aggregate filter here.