Logstash filter according to fields source declare in filebeat config - logstash

Question

input {
beats
{
port => 5042
}
}
output {
if [source] == "access"
{
elasticsearch {
hosts => ["16.113.56.102:9200"]
index => "logstsh-access-nginxlogs-%{+YYYY.MM.dd}"
}
}
else if [source] == "error"
{
elasticsearch {
hosts => ["16.113.56.102:9200"]
index => "logstsh-error-nginxlogs-%{+YYYY.MM.dd}"
}
}
}
I would like to separate the log file using the fields source and that declared at filebeat input, for the, from kibana side, the log have the source either is access/error, however, the logstash won't pass the log to elastic search, I'm wondering is it the right way to declare the source? I try to use the absolute in the input part, it works like a charm, so I think the issues is with the input filebeat or logstash.

Answer 1

First of all, I recommend you use one output if you don't need to send the data to multiple elasticsearch clusters. For example:
output {
elasticsearch {
hosts => ["16.113.56.102:9200"]
index => "logstash-%{source}-nginxlogs-%{+YYYY.MM.dd}"
}
}
}
In filebeat, you can add fields into each documents with the processor
processors:
- add_fields:
target: project
fields:
name: myproject
id: '574734885120952459'
Ref: https://www.elastic.co/guide/en/beats/filebeat/current/add-fields.html
Note: to understand if the problem because of Logstash you can use the following method.
Why are the logs not indexed in the elasticsearch logstash structure I designed?