I want to be able to track logstash logs in case of an error.
I want to be able to monitor issues when logstash tries to send events to output destination. I've checked the monitor API but it doesn't fulfills my requirements.
Is it a good practice to use file input plugin and set the path to its own log?
Related
I am new to elk stack. Let me explain what i am trying to do. I have a application that is running separately for different users i.e. 5 different users will have 5 independent instance of the same application. I am using filebeats to send the logs from the application to logstash where it will be processed first before being sent to the elasticsearch. What i want is to write the application which enables the users to view the logs of theirs instance of application only. Now what i tried to do is creating the logstash pipeline for each the user with different port which will process the log and send it to elasticsearch with the different index name.
Can you suggest me if this is the best practice or i am doing it wrong? Is there a more better way to do it without having separate pipeline for individual users with separate port? I think the way I am doing it is wrong and it will be harder for me to manage when the instances will grow in numbers.
Thank You
I would suggest if there's no skinning , validation and enrichment involved then skip logstash altogether. You can straight away pass filebeat logs to ES. Now there are two ways from here. Filebeat can additionally send a parameter (any fixed string) along with the scanned message to ES or you can store the meta (like ip) source which filebeat will send along with message. This string can then be used to identify the source of the log message and then on kibana you can configure to show dashboard based on that fixed string / user / meta. This simply the process and avoid unnecessary hops.
I'm currently in front of a structure problem with logstash.
I have a syslog-ng client sending logs from different files through the network to an ELK stack.
I noticed that Logstash is mixing logs, especially multiline with adding exception lines to non error logs from others files. So, i guess the trouble is that my logs are treated without any differentiation from their file origins. I found 2 ways to avoid that, but they are not optimal in my opinion:
Instead of using syslog-ng, use FileBeat and add a tag for each file which identify their origins. Then, parse my log with Logstash depending of this tag. The problem is that using syslog-ng as client is a requirement, and it will bother me if I have to change it
Change my syslog-ng sources to send each log files to a different port on ELK. I found that a little dirty and can be embarrassing with a great quantity of log files
What do you think about that ? Did I miss a better solution?
Is there a way to add a tag just like filebeat in syslog-ng?
I have an existing system that sends me log entries to my server via Syslog protocol. The log entries are written into local files, and then I process these log files with Logstash using its File input plugin.
I like it because even if the Logstash goes down (it happens sometimes), I do not lose any log.
I have just realized today that the Logstash also has a Syslog input plugin that is capable of reading log data on the Syslog protocol.
I am wondering if I turn off my Syslog server, and read the data via the Syslog input plugin of the Logstash, will I have the same reliable system, or If the Logstash goes down, I will lose data during the downtime?
If Logstash goes down you will lose data during the downtime.
Also, the syslog input only works if the messages from your logs are in compliance with the RFC3164, anything different and you will need a grok pattern to parse that message.
If you don't want to use the file input anymore you can create a rule on your syslog server to redirect the messages to your logstash input, in this case, if your logstash goes down, you will still have the files to fill the missing data.
Currently I have a ELK stack in which logs are shipped by filebeat and after some filters in logstash, it is forwarded to ES. As there are a lot of servers and logs, a huge logs are coming to logstash, but I have configured the filter to only process a very specific type of log message. Which it is doing fine, but the logs which are not even matching are logged in logstash.log file. As I mentioned earlier that huge logs are coming, the size of logstash.log file is soon reaching to a high value and there is space issue coming up. How to configure the logstash so that I only log the processed logs, and not all.
You could use logrotate to automatically rotate on either a daily basis or once it hits a certain threshold. You could then set the number of rotations to be 1 or 2. This would allow you time to see what is going to the file in case you need to troubleshoot, but purge before it creates space contention.
Currently we are working on forwarding logs to 2 different logstash servers. We cannot figure out a way with which we can install logstash-forwarder on a single machine. Is it possible with logstash-forwarder forwarding logs to multiple logstash ??
Else how can we do it with filebeat ??
In the LSF config, you can specify a list of hosts, but it will pick one at random and only switch to another in case of failure.
FB has the same system, but it allows you to also load balance across the list of hosts.
AFAIK, neither allows you to send events to multiple logstash instances.
Logstash, on the other hand, will send events to all of its outputs, so you could have FB send to a single LS, and have that LS output to your other LS instances. Note that if one output is unavailable, the system will block.