logstash - dynamic field names - logstash

Question

I have problem with dynamics field names in my Logstash configuration.
This is my test config:
input {
generator {
lines => [ "May 15 13:42:55 logstash puppet-agent[3551]: Finished catalog run in 43",
"May 16 14:57:07 logstash puppet-agent[3551]: Starting Puppet client version" ]
count => 7
}
}
filter {
grok {
match => [ "message", "%{SYSLOGBASE} %{WORD:log}.*" ]
}
if "Starting" in [log] {
metrics {
meter => [ "%{logsource}.%{log}" ]
add_tag => [ "metric" ]
add_field => { "server" => "%{logsource}"
"bad" => "true" }
clear_interval => 5
}
}
}
output {
stdout { codec => rubydebug }
}
and here is my output: (just end of output)
{
"message" => "May 15 13:42:55 logstash puppet-agent[3551]: Finished catalog run in 43",
"#version" => "1",
"#timestamp" => "2016-06-07T07:37:50.138Z",
"host" => "logstash.test.lan",
"sequence" => 6,
"timestamp" => "May 15 13:42:55",
"logsource" => "test",
"program" => "puppet-agent",
"pid" => "3551",
"log" => "Finished"
}
{
"message" => "May 16 14:57:07 logstash puppet-agent[3551]: Starting Puppet client version",
"#version" => "1",
"#timestamp" => "2016-06-07T07:37:50.138Z",
"host" => "logstash.test.lan",
"sequence" => 6,
"timestamp" => "May 16 14:57:07",
"logsource" => "test",
"program" => "puppet-agent",
"pid" => "3551",
"log" => "Starting"
}
{
"#version" => "1",
"#timestamp" => "2016-06-07T07:37:50.288Z",
"message" => "Counting: 7",
"logstash.Starting" => {
"count" => 7,
"rate_1m" => 0.0,
"rate_5m" => 0.0,
"rate_15m" => 0.0
},
"server" => "%{logsource}",
"bad" => "true",
"tags" => [
[0] "metric"
]
}
Why field server don´t have logstash as value from the input logs? %{logsource} works for meter option, so why not for add_field?
Thx for help.

Answer 1

When a log event is received, the SYSLOGBASE information is extracted from the content. This is where the %{logsource} value is defined. If the event isn't coming from a log entry that contains SYSLOGBASE information, then logsource will be undefined.
When you receive a start message, logsource is defined in scope and is added to your message.
The metrics plugin is generating a new message per interval. This message is generated from scratch so it does not have the value of logsource or anything else that would normally be obtained from an individual log entry.