I know transports are the places where I want to keep my logs but I don't understand what is the level of logging? I have the following code to create a logger with multiple means of transport.
const logger = winston.createLogger({
format: winston.format.json(),
transports: [
new winston.transports.File({ filename: 'logs/error.log', level: 'error' }),
new winston.transports.File({ filename: 'logs/info.log', level: 'info' }),
],
})
When I want to log, logger.log('error', err), it logs in both info.log and error.log files. Why is this happening? Can somebody explain the idea of levels in logging, please?
Geno's comment is correct; in pretty much all logging platforms (winston, log4js, etc.), the log level represents the maximum level of error to print.
Setting log level to ERROR means "only print FATAL and ERROR messages".
Setting log level to INFO means "print FATAL, ERROR, WARN, and INFO messages".
There is no way (in Winston, at least, but I think it's generally true across the board) to specify a log transport that only carries INFO messages and not ERROR messages. This is by design.
When you set a log level, you are actually specifying a level of detail - FATAL is the least detailed logging, DEBUG is the most detailed logging. It wouldn't make sense to ask for more detail, and then have fatal errors disappear from the log. That is why every error level also includes all messages from levels "below" it.
Related
I am trying to implement logging, for my node.js application, that uses socket.io for connections. My initial goal was to implement a logging mechanism, that stores logs in two different files, based on the authenticated Account and the socket.io connection id, so I may quickly review logs related to a specific account, and/or to a specific session when the Account was active.
For this, I've created a logger, that uses two File type transports, like this:
/** ... some constant declarations ommited for brevity **/
const customerTransport = new winston.transports.File({
dirname: customerSessionDir,
filename: 'main.log',
level: 'info',
});
const sessionTransport = new winston.transports.File({
dirname: path.resolve(customerSessionDir, 'sessions'),
filename: moment.now().toString() + "." + socketSessionId + ".log",
level: 'info'
});
const logger = winston.createLogger({
levels: winston.config.syslog.levels,
level: 'info',
format: loggingFormat,
transports: [customerTransport, sessionTransport]
});
This works as expected, but now, I would want to back reference, the socketSessionId variable, in the main.log file, so that when I am reviewing the log for a specific account, I get a reference in that file, that points to a smaller session log file. I know that this is achiavable, by setting the defaultMeta property, when creating the logger, but if I do it like that, the information will also appear in the session log files, and I would not want to bloat those files with any unneccessary information.
Is it possible somehow, to automagically add meta data to specific transports only? Is this the intended way to achieve my end-goal? Or should I create two separate loggers, and use that for this scenario?
I am developing an express project which will have multiple modules/services in it. The folder structure looks mostly like this:
-- app.js
-- payment_service
-- routes.js
-- index.js
-- models
-- model_1.js
-- model_2.js
APIs in index.js are the only exposed APIs and they work as a gateway for all requests coming for this module/service.
Most of the services can throw operational error under many circumstances, so manual intervention may needed to fix things. So I need to:
Log errors properly with proper context so that some person/script can do the needful.
Figure out the reason of failure.
There will be dedicated teams owning each service. So I should be able to differentiate between error logs for each service so that it can be aggregated and forwarded to concerned person.
I decided to go with ELK stash so that I can generate reports by script.
The main problem that I am facing is that I can't maintain correlation between logs. For example; If a request comes and it travels through five functions and each function logs something then I can't relate those logs.
One way is to create a child logger for each request and pass it to all the functions but that seems to be extra overhead passing logger instance to all the functions.
Another option is to use something like verror and do the logging only at entry point of the service/module so that the whole context can be contained in the log. This approach looks ok for logging errors, however it can't help with info and debug logs - they help me a lot in development and testing phase.
For the sake of differentiating between error logs, I am going to create
A dedicated logger for each service with log level error.
An application wide generic logger for info and debug purpose.
Is this the correct approach?
What will be the best way so that I can achieve all the requirements in simplest way?
I'd recommend you use a logger and you don't need anything too complex. For example:
npm install 12factor-log
Then create a file in your root folder near app.js (or in a /lib folder is where I'd place libraries)
logger.js
const Log = require('12factor-log');
module.exports = (params) => {
return new Log(params);
}
Then in your modules, import your logger and pass in the module name when you instantiate it so you can track where statements come from...
model_1.js
var log = require('./logger')({name: 'model_1'});
// ...
log.info("Something happened here");
// ...
try {
// ...
catch (error) {
const message = `Error doing x, y, z with value ${val}`;
log.error(message);
throw new Error(message);
}
Then handle error gracefully at your controller -> view layer for user-friendly experience.
Your logs would print something like this:
{"ts":"2018-04-27T16:37:24.914Z","msg":"Something happened here","name":"model_1","type":"info","level":3,"hostname":"localhost","pid":18}
As far as correlation of logs, if you see in the output above it includes the hostname of the machine it's running on, and also the name of the module and severity level. You can import this JSON into Logstash and load into Elasticsearch and it will store JSON for easy search and indexing.
See: https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html
Logging is complex and many people have worked on it. I would suggest not doing so yourself.
So, not following my own advice, I created my own logging package:
https://www.npmjs.com/package/woveon-logger
npm install woveon-logger
This prints file and line numbers of errors and messages, has logging levels and aspect-oriented logging, and can dump a stack trace in one call. It even has color coding options. If you get stuck and need some feature in logging, let me know.
let logger1 = new Logger('log1', {level : 'info', debug : true, showname : true};
let logger2 = new Logger('log2', {level : 'verbose', debug : true, showname : true};
...
log1.info('Here is a log message, that is on line 23.');
log1.verbose('Does not show');
log2.verbose('Shows because log2 is verbose logging');
log2.setAspect('IO', true);
log2.aspect('IO', 'Showing aspect IO logging, for logs for IO related operations');
[2018-06-10T10:43:20.692Z] [INFO--] [log1 ] [path/to/myfile:23] Here is a log message, that is on line 23.
[2018-06-10T10:43:20.792Z] [VERBOS] [log2 ] [path/to/myfile:25] Shows because log2 is verbose logging
[2018-06-10T10:43:20.892Z] [IO----] [log2 ] [path/to/myfile:27] Showing aspect IO logging, for logs for IO related operations
Also, some other features like:
log1.throwError('Logs this as both a line of logging, and throws the error with the same message');
log1.printStack('Prints this label next to the stack trace.');
Hope it helps!
You can use grackle_tracking library https://www.getgrackle.com/analytics_and_tracking
It logs errors & traffic to your db.
I try to achieve log level change at run time. i have been following
https://github.com/yannvr/Winston-dynamic-loglevel/blob/master/test . while calling update function it doesn't change actual transport level setting. e.g i updated to info level but info related log doesn't print in file. it seems doesnt work what it says is there any other way to achieve it.i have come across using setLevel provide by winston but i dont know how to call it from out side to change log level at run time?
Easy, suppose that you want to you use the console as transport for your log, you can start as 'debug' level and later change to 'info'.
'use strict';
const winston = require('winston');
const logger = new (winston.Logger)({
transports: [
new (winston.transports.Console)({
level: 'debug'
}),
]
});
logger.debug('DEBUG');
logger.transports.console.level = 'info';
logger.debug('DEBUG ?'); // IGNORED !
I am looking for a logging solution for my node.js app that would allow me to set the logging level via file/folder selectors.
For example I would like to be able to set the logging for all files in /app/schema to 'info'. And all the rest to 'error'.
Exemplary configuration:
{
"*":"error",
"/app/schema":"info" //<--Regex expression would be great too.
}
I constantly comment/uncomment/remove logging statements when I need to examine something. I would rather do that via a configuration change and leave the logging files intact. A global debugging level just creates too way to much noise and volume (which matters when storing logs).
Is there something like this? Apache log4j is similar, you can set logging level on package level.
If I get your question right, I'd suggest Bunyan, it lets you configure as many log streams and log levels as you like, i.e. from the docs:
var log = bunyan.createLogger({
name: 'myapp',
streams: [
{
level: 'info',
stream: process.stdout // log INFO and above to stdout
},
{
level: 'error',
path: '/var/tmp/myapp-error.log' // log ERROR and above to a file
}
]
});
You'll have to set up the logic for each path with each log-level.
For fixing losing logs of transports file in high concurrency, I changed the mode to stream. Issue of losing logs was fixed then. But I have another problem that the transports file can not be created day by day(even I set the transports mode to DailyRotateFile).
I wanna know, is there any option can be set to implement this case? or I have to hack it???
Thanks guys.
Demo codes below:
new (winston.Logger)({
transports: [
new (winston.transports.DailyRotateFile)({
level: 'info',
stream: fs.createWriteStream('performance.log'),
}))
]
});