I am using a third party nodejs application which is quite large and it is using many different things for logging. For instance console.log() and console.error(). I would like to be able to trap all output and log to a specific file. I was thinking about using winston and do something like this:
const winston = require('winston')
const logger = winston.createLogger({
transports: [
new winston.transports.File({ filename: 'error.log', level: 'error' })
]
})
const ce = console.error
console.error = function(...args) {
logger.log.apply('error', args)
ce.apply(args)
}
Is there a better way to handle this kind of situation?
I might also add that some parts of the application is already using winston for logging purposes.
My aim is the create a single file with all errors and warnings generated from within the application.
Related
I am trying to create a daily rotating log with log entries from throughout the application as well as uncaught exceptions:
const { createLogger, format, transports } = require('winston')
import 'winston-daily-rotate-file'
const httpContext = require('express-http-context')
const requestIdFormat = format((info, opts) => {
const requestId = httpContext.get('requestId')
if (requestId){
info.requestId = requestId
}else{
info.requestId = ''
}
return info
})
const allTransport = new transports.DailyRotateFile({
filename: 'logs/%DATE%-application.log',
datePattern: 'YYYY-MM-DD',
maxSize: '20m',
maxFiles: '14d',
level: 'info',
handleExceptions: true
})
const errorTransport = new transports.DailyRotateFile({
filename: 'logs/%DATE%-error.log',
datePattern: 'YYYY-MM-DD',
maxSize: '20m',
maxFiles: '14d',
level: 'error',
handleExceptions: true
})
export const logger = createLogger({
format: format.combine(requestIdFormat(), format.timestamp(), format.printf(i => `${i.timestamp} | ${i.requestId} | ${i.level}: ${i.message}`), format.errors({stack: true})),
transports: [
allTransport,
errorTransport,
],
exitOnError: false
})
But the exceptions e.g. throw Error('hello?') are not logged to the log files.
I've tried other variations c.f. https://github.com/winstonjs/winston#handling-uncaught-exceptions-with-winston e.g. setting exceptionHandlers in createLogger, but that does not work either.
How should I alter the code to include uncaught exceptions in the log?
UPDATE: I now see that an exception thrown on e.g. invalid import IS in fact logged, so maybe the issue is that the exception I test with is thrown in an express service - maybe it is caught in the express framework and that is why it is not logged?
Thanks,
-Louise
I ended up just "manually" logging exceptions, i.e. logger.error(err), as I already have a middleware hook for showing a catch-all error page.
I am still not sure why some exceptions were automatically logged and why I have to manually log exceptions I explicitly throw via "throw new Error('some message')"?
I've faced this kind of situation before, and for some reason, sometimes the node process get killed/exited before all log handling finishes its execution. There are some libs that may cause conflict in handling because they may call process.exit() while other libs are trying to handle the same thing. So, we need to investigate and debug to try to find what is terminating our process before our logs are flushed/written to disk. I strongly recommend not doing any async processing when caughting exceptions. Do everything in a sync way and faster.
In a project I've worked, I spended hours until I found process.exit in a module , probably copied from internet. Process.exit been called everywhere in an app is the root cause for errors and misbehavior of apps.
I have a logging module that I use in many of my projects, which generally exports a single Winston logger, so all I did was define a logger and it's transports, then export it:
module.exports = logger;
when importing using const logger = require('mylogger.js') I then use the various levels built in (logger.info logger.debug etc).
I've now decided that I want to create a second logging function that will write logs to a different file, so I need to create and export a new transport. Thing is, if I switch to module.exports = {logger, mynewlogger}, that will change the way I import and call the functions, and I have that in many places.
Besides creating second file and importing both, is there any other way to add a second export without having to change my code everywhere else?
It's either new module that re-exports both:
logger-and-mynewlogger.js
module.exports = {logger, mynewlogger}
Or a separate module:
mynewlogger.js
module.exports = mynewlogger
Or using existing function as module object:
logger.mynewlogger = ...
module.exports = logger;
The first two options are preferable because they result in reasonably designed modules, while the last one is a quick and dirty fix.
Yes, you can define multiple transports for a single exported logger. When creating your Winston log, the 'transports' property is an array which allows you to define multiple outputs.
Here's an example of one I have that has two transports. Firstly, console and the second a daily rotating log.
const winston = require('winston');
const Rotate = require('winston-daily-rotate-file');
const tsFormat = () => (new Date()).toLocaleTimeString();
const logger = new (winston.Logger)({
transports: [
// colorize the output to the console
new (winston.transports.Console)({
timestamp: tsFormat,
colorize: true,
level: 'info',
}),
new (Rotate)({
filename: `${logDir}/${logName}-app.log`,
timestamp: tsFormat,
datePattern: 'YYYY-MM-DD',
prepend: true,
level: env === 'development' ? 'verbose' : 'info',
}),
],
});
module.exports = logger;
I want to get the console contents of the current running Node.js script.
I've tried to do this event but it doesn't work:
setInterval(function() { console.log("Hello World!") }, 1000);
process.stdout.on('message', (message) => {
console.log('stdout: ' + message.toString())
})
It doesn't listen to the event.
This is not a fully Node.js solution but it is very good in case you run Linux.
Create a start.sh file.
Put the following into it:
start.sh:
#!/bin/bash
touch ./console.txt
node ./MyScript.js |& tee console.txt &
wait
Now open your Node.js script (MyScript.js) and use this Express.js event:
MyScript.js:
const fs = require('fs');
app.get('/console', function(req, res){
var console2 = fs.readFileSync("./console.txt", 'utf8');
res.send(console2);
});
Always start your Node.js application by calling start.sh
Now calling http://example.com/console should output the console!
A part of this answer was used.
NOTE: To format the line breaks of the console output to be shown correctly in the browsers, you can use a module like nl2br.
An advice: The problems aren't always solved the direct way, most of the problems are solved using indirect ways. Keep searching about the possible ways to achieve what you want and don't search about what you're looking for only.
There's no 'message' event on process.stdout.
I want to make a GET in my Express.js app called /getconsole .. it
should return the console of the current running Node.js script (which
is running the Express.js app too)
What you should use is a custom logger, I recommend winston with a file transport, and then you can read from that file when you issue a request to your endpoint.
const express = require('express');
const fs = require('fs');
const winston = require('winston');
const path = require('path');
const logFile = path.join(__dirname, 'out.log');
const app = express();
const logger = winston.createLogger({
level: 'info',
format: winston.format.json(),
transports: [
new winston.transports.Console({
format: winston.format.simple()
}),
new winston.transports.File({
filename: logFile
})
]
});
// Don't use console.log anymore.
logger.info('Hi');
app.get('/console', (req, res) => {
// Secure this endpoint somehow
fs.createReadStream(logFile)
.pipe(res);
});
app.get('/log', (req, res) => {
logger.info('Log: ' + req.query.message);
});
app.listen(3000);
You can also use a websocket connection, and create a custom winston transport to emit the logs.
stdout, when going to a tty (terminal) is an instance of a writable stream. The same is true of stderr, to which node writes error messages. Those streams don't have message events. The on() method allows solicitation of any named event, even those that will never fire.
Your requirement is not clear from your question. If you want to intercept and inspect console.log operations, you can pipe stdout to some other stream. Similarly, you can pipe stderr to some other stream to intercept and inspect errors.
Or, in a burst of ugliness and poor maintainability, you can redefine the console.log and console.error functions to something that does what you need.
It sounds like you want to buffer up the material written to the console, and then return it to an http client in response to a GET operation. To do that you would either
stop using console.log for that output, and switch over to a high-quality logging npm package like winston.
redefine console.log (and possibly console.error) to save its output in some kind of simple express-app-scope data structure, perhaps an array of strings. Then implement your GET to read that array of strings, format it, and return it.
My first suggestion is more scalable.
By the way, please consider the security implications of making your console log available to malicious strangers.
Is there an equivalent of log.IsDebugEnabled in Winston?
I want to use this to skip expensive logging code in a production environment but have it execute in development.
For example:
if(winston.isDebugEnabled){
// Call to expensive dump routine here
dump();
}
Checking winston.debug just checks whether the method is defined, not whether it is enabled.
Many thanks!
Edit: Added code example.
I've added a method to my logger to achieve just that:
logger.isLevelEnabled = function(level) {
return _.any(this.transports, function(transport) {
return (transport.level && this.levels[transport.level] <= this.levels[level])
|| (!transport.level && this.levels[this.level] <= this.levels[level]);
}, this);
};
This goes through each of your logger's transports and checks whether it 'wants' to log the specified level.
Note _.any is lodash, you can replace with for loop.
I'm sure you'd be able to get that directly from winston, but if you want to have different logging levels for different environments, you should pass these in when you're creating the winston.logger.
For example:
// default log file level is info (which is the lowest by default)
var logFileLevel = 'info';
if (process.env.NODE_ENV == 'production') {
// only write logs with a level of 'error' or above when in production
logFileLevel = 'error';
}
var logger = new (winston.Logger)({
transports: [
new (winston.transports.File)({
filename: '/var/log/node-logger.log',
level: logFileLevel
})
]
});
It's also useful to switch out the transports. You might want to use the Console transport whilst in development, and the File transport when in production for example.
More documentation on all this on the winston readme.
Try
if ( logger.levels[logger.level] >= logger.levels['debug'] ) {
// expensive calculation here
logger.debug(...)
}
Since Winston 3.1.0 (PR), you can use the Logger functions isLevelEnabled(string) & isXXXEnabled() for this.
I am using Winston logging with my Node.js app and have defined a file transport. Throughout my code, I log using either logger.error, logger.warn, or logger.info.
My question is, how do I specify the log level? Is there a config file and value that I can set so that only the appropriate log messages are logged? For example, I'd like the log level to be "info" in my development environment but "error" in production.
If you are using the default logger, you can adjust the log levels like this:
const winston = require('winston');
// ...
winston.level = 'debug';
will set the log level to 'debug'. (Tested with winston 0.7.3, default logger is still around in 3.2.1).
However, the documentation recommends creating a new logger with the appropriate log levels and then using that logger:
const myLogger = winston.createLogger({
level: 'debug'
});
myLogger.debug('hello world');
If you are already using the default logger in your code base this may require you to replace all usages with this new logger that you are using:
const winston = require('winston');
// default logger
winston.log('debug', 'default logger being used');
// custom logger
myLogger.log('debug', 'custom logger being used');
Looks like there is a level option in the options passed covered here
From that doc:
var logger = new (winston.Logger)({
transports: [
new (winston.transports.Console)({ level: 'error' }),
new (winston.transports.File)({ filename: 'somefile.log' })
]
});
Now, those examples show passing level in the option object to the console transport. When you use a file transport, I believe you would pass an options object that not only contains the filepath but also the level.
That should lead to something like:
var logger = new (winston.Logger)({
transports: [
new (winston.transports.File)({ filename: 'somefile.log', level: 'error' })
]
});
Per that doc, note also that as of 2.0, it exposes a setLevel method to change at runtime. Look in the Using Log Levels section of that doc.
There are 6 default levels in winston: silly=0(lowest), debug=1, verbose=2, info=3, warn=4, error=5(highest)
While creating the logger transports, you can specify the log level like:
new (winston.transports.File)({ filename: 'somefile.log', level: 'warn' })
Above code will set log level to warn, which means silly, verbose and info will not be output to somefile.log, while warn, debug and error will.
You can also define your own levels:
var myCustomLevels = {
levels: {
foo: 0,
bar: 1,
baz: 2,
foobar: 3
}
};
var customLevelLogger = new (winston.Logger)({ levels: myCustomLevels.levels });
customLevelLogger.foobar('some foobar level-ed message');
Note that it's better to always include the 6 predefined levels in your own custom levels, in case somewhere used the predefined levels.
You can change the logging level in runtime by modifying the level property of the appropriate transport:
var log = new (winston.Logger)({
transports: [
new (winston.transports.Console)({ level : 'silly' })
]
});
...
// Only messages with level 'info' or higher will be logged after this.
log.transports.Console.level = 'info';
I guess, it works similarly for file but I haven't tried that.
If you want to change the log level on the fly. Like for when you need to trace production issue for short amount of time; then revert to error log level. You can use a dynamic logger provided you can expose a service on the web https://github.com/yannvr/Winston-dynamic-loglevel
apart from this you can cleanly achieve this by imlplementing runtime-node-refresh follow this link for more.