Although this topic is discussed by other people but I could not get it done through reading explanations of other people here.
I would like to use syslog functions to log into a specific file. I can see the logged message but I could not have the logs printed into a specific file.
What I did is:
#define log_info(...) syslog(LOG_INFO, __VA_ARGS__);
First approach:
openlog("PingWatchdog", LOG_PID|LOG_CONS, LOG_USER);
log_info("[INFO]: PingWatchdog: pingDispatcher thread starting.");
closelog();
in /etc/rsyslog.d there is a config file in which I added this rule :
if:syslogtag, isequal, "PingWatchdog:" /var/log/pingwatchdog.log
&stop
second approach:
openlog("PingWatchdog", 0, LOG_LOCAL1);
log_info("[INFO]: PingWatchdog: pingDispatcher thread starting.");
closelog();
in /etc/rsyslog.d there is a config file in which I added this rule :
local1.info /var/log/pingwatchdog.log
but these two methods could not help me to write into my desired file which is: /var/log/pingwatchdog.log
my program name is PingWatchdog
I also tried this rule but not helpful:
if $programname == 'PingWatchdog' then /var/log/pingwatchdog.log
any Idea what should I do?
Add below in rsyslog conf.
if ($syslogtag contains 'PingWatchdog') then {
*.* /var/log/pingwatchdog.log.log
stop
}
We are a setup with three queues in rabbitmq, handling three different types of logs.
The queues are handled by logstash, and given a tag, and then logstash dumps the message into the appropriate index in elasticsearch.
So my input looks something like this:
input {
rabbitmq {
host => "localhost"
queue => "central_access_logs"
durable => true
codec=> json
threads => 3
prefetch_count => 50
port => 5672
tags => ["central_access_log"]
}
And similar setup for the other two queues:
My output is like this:
if("central_access_log" in [tags]){
elasticsearch {
host => "localhost"
index=> "central_access_logs"
}
}
I suspected for a while that not everything was making it into the central_access_log index (the other two indexes, more of less, seemed fine), so I added this:
file {
path => '/data/out'
}
And let that run for a few weeks.
Recently, I noticed that for the last week and half, nothing has been coming into that index (again, the other two are perfectly fine), however the text file contains all the missing messages.
How can I go about debugging this? Is it an error on logstash's end, or elasticsearch?
I have a nodejs app which has the new relic module installed.
Everything is set up but I don't know how to exclude a route from the new relic tracker.
I have this script at http://www.website.com/match/findMatch which usually takes at least 4 seconds and I want to add it to the exclusion list.
The rest of the scripts usually take 0.1s but this one makes my ApDex give out alerts which aren't correct.
I already have this in my conf file because I use websockets:
rules : {
ignore : [
'^\/socket\.io\/.*\/xhr-polling',
'^\/socket\.io\/.*\/websocket',
]
}
I've read the docs here 3 times but still don't understand how to do it.
You'll want to add a new ignore rule that matches requests to http://www.website.com/match/findMatch.
Rules can be strings or regular expressions (like the socket.io examples). If the request you want to ignore is always http://www.website.com/match/findMatch (not, for example, http://www.website.com/match/findMatch?q=foo, you can do this:
rules : {
ignore : [
'^\/socket\.io\/.*\/xhr-polling',
'^\/socket\.io\/.*\/websocket',
'http://www.website.com/match/findMatch'
]
}
Logstash seems to hang when processing a local file. The logstash process is still alive and everything looks fine, but no data get written to the output (elasticsearch). The index gets written, though.
Logstash seems to "hang" and not process any of the input data for the following reason:
Logstash keeps track of what has previously been processed, so when you run it again on the same input data (as will be the case during testing), Logstash will think it has already seen and processed this data the previous time and will not read it again. To bypass this during testing, specify explicitly the location of the sincedb file where Logstash should keep track of what it has read or not and manually delete this sincedb file before each test run.
Here is an example:
input {
file {
path => "~/logstash/data/input_file"
start_position => "beginning"
sincedb_path => "~/logstash/data/sincedb.db"
}
}
or maybe even better (added based on comment below):
input {
file {
path => "~/logstash/data/input_file"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
I am using the Bunyan module for NodeJS logging. When I try using the rotating-file type, it makes my app crash every time and outputs this error:
Error: ENOENT, rename 'logs/info.log.3'
However, it never happens at the same time so I can't find any logic...
This is how I instanciate my logger:
var log = Bunyan.createLogger(config.log.config);
log.info('App started, ' + process.env.NODE_ENV);
And here is my config.json:
{
"name" : "app",
"streams" : [
{
"type" : "rotating-file",
"period": "5000ms", //Low period is for testing purposes
"count" : 12,
"level" : "info",
"path" : "logs/info.log"
},
{
"type" : "rotating-file",
"period": "5000ms",
"count" : 12,
"level" : "error",
"path" : "logs/error.log"
},
{
"type" : "rotating-file",
"period": "5000ms",
"count" : 12,
"level" : "trace",
"path" : "logs/trace.log"
}
]
}
Can anyone advise how to fix my issue? Thanks in advance.
What I have just done (last night actually) to get around this problem of a master + workers contending over a Bunyan rotating-file is to have the workers write "raw" log records to a stream-like object I created called a WorkerStream. The write method of the WorkerStream simply calls process.send to use IPC to deliver the log record to the master. The master uses a different logger config that points to a rotating-file. The master uses the code shown below to listen for log records from its workers and write them to the log file. So far it appears to be working perfectly.
cluster.on('online', function (worker) {
// New worker has come online.
worker.on('message', function (msg) {
/* Watch for log records from this worker and write them
to the real rotating log file.
*/
if (msg.level) {
log._emit(msg);
}
});
});
ln is your friend.
Existing logging libraries have rotation problem with cluster module. Why doesn't ln have this issue?
Both bunyan and log4js rename the log file on rotation. The disaster happens on file renaming under cluster environment because of double files renaming.
bunyan suggests using the process id as a part of the filename to tackle this issue. However, this will generate too many files.
log4js provides a multiprocess appender and lets master log everything. However, this must have the bottleneck issue.
To solve this, I just use fs.createWriteStream(name, {"flags": "a"}) to create a formatted log file at the beginning instead of fs.rename at the end. I tested this approach with millisecond rotation under cluster environment and no disasters occurred.
I have experienced the same issue without using clustering. I believe the problem is being caused by old files sitting in the log directory. While the main logger can open and append to existing files, the file rotation logic uses rename, which files when it steps on an existing file. (e.g. an existing info.log.3 file).
I'm still digging into the source to figure out what needs to be changed to recover from the rolling error.
One additional thought as I review the source. If you have multiple Bunyan log instances that use the same log file (in my case, a common error.log), the rename calls could be happening nearly concurrently from the OS level (asynchronous and separate calls from a Node.js perspective, but concurrently from the OS perspective).
It's sadly not possible to use multiple rotating file streams against the same file.
If you're in the same process, you must use a single logger object - make sure you're not creating multiple of them.
If you're working across processes, you must log to different files. Unfortunately there's nothing yet that has the IPC in place to allow different rotators to coordinate amongst themselves.
I have a plugin rotating file stream that detects if you try to create 2 rotators against the same file in the a single process and throws an error.
It can't help in the case of multiple processes tho.
bunyan-rotating-file-stream
From my experience, it happens sometimes when the logs directory (or whatever you named it) does not exist.
If you are running through this error in a automation pipeline, for example, you may be ignoring all the files in logs and committing it empty, then it is not created when the repository is cloned by the pipeline.
Simply make sure that logs is created by placing a .gitkeep file inside it (or any other trick).
This may be the case of many of you who come across this question.