currently im zipping the files daily with winston daily rotate. The thing I want to do now is remove the zip files after a week. is there a possibility to accomplish this by using winston daily rotate or do i have to write it myself ?
Code im using:
const transport = new (winston.transports.DailyRotateFile)({
"name": "basic-log",
"filename": `${logDir}/%DATE%-log`,
"datePattern": "YYYY-MM-DD",
"zippedArchive": true,
"colorize": false,
"maxFiles": '2d'
});
transport.on('rotate', function(oldFilename, newFilename) {
// do something fun
console.log(new Date(), oldFilename, newFilename)
});
const logger = new (winston.Logger)({
transports: [
transport
]
});
Thanks in advance.
currently (winston-daily-rotate-file v.3.3.3) doesn't delete zipped files.
Open bug: https://github.com/winstonjs/winston-daily-rotate-file/issues/125
In winston-daily-rotate-file you can set maxFiles: '7d' which will delete the files that are older than a week.
From winston-daily-rotate-file:
maxFiles: Maximum number of logs to keep. If not set, no logs will be removed. This can be a number of files or number of days. If using days, add 'd' as the suffix. (default: null)
read more about it here: https://www.npmjs.com/package/winston-daily-rotate-file#usage
Related
I have a node application that contains several downloadable links (when you click on the link a pdf file is downloaded), and these links are dynamically created/populated. I want to implement a feature where we can somehow download all files from these links in one go. I presume for this I will somehow need to create a zip file from all these links - would anyone know how to go about this?
you could use the fs and archiver module:
var fs = require('fs');
var archiver = require('archiver');
var output = fs.createWriteStream('./example.zip');
var archive = archiver('zip', {
gzip: true,
zlib: { level: 9 } // Sets the compression level.
});
archive.on('error', function(err) {
throw err;
});
// pipe archive data to the output file
archive.pipe(output);
// append files
archive.file('/path/to/file0.txt', {name: 'file0-or-change-this-whatever.txt'});
archive.file('/path/to/README.md', {name: 'foobar.md'});
//
archive.finalize();
I have defined custom levels for my application .They are as follows.
protected levels: Level = {
"error": 0,
"warn": 1,
"info": 2,
"debug": 3,
"trace": 4
};
I am using daily file rotate transport to get daily log in separate files.
const options: Object = {
name: this.level,
filename: logFilePath,
dirname: WinstonLogAgent.DIR_LOG,
datePattern: "yyyyMMdd.",
prepend: true,
level: this.level,
levels: this.levels,
maxsize: this.maxFileSize,
maxFiles: this.maxFileCount,
handleExceptions: true,
humanReadableUnhandledException: true
};
this.transportInstance.push(new (winston.transports.DailyRotateFile)(options));
If i define log level to be 'info', it will create one log file named info.log and will log for levels 'info' , 'warn' and 'error' (trace and debug will be ignored).
But behaviour i wanted was different. If i am specifying level to be 'info' and i am logging levels 'info' , 'warn' and 'error' , then there should be separate files created for each type of log . i.e 'info' level should be logged to info.log and 'warn' level to be logged to warn.log.
I have tried specifying five different daily file rotate transport ,each with unique level. Then the problem i find is that there is duplicate log entries.
For example , if am logging 'error' level , it would log to info.log , warn.log & error.log when logging level is set to info.
How can i achieve my objective?
According to Winston's documentation, the default behavior is to log all the messages which have at least the specifies importance aka logging level.
Winston allows you to define a level property on each transport which specifies the maximum level of messages that a transport should log.
But there are ways to achieve your requirements.
I'll try to show you some of the possibilities, you can choose the method that works the best for you.
1. Custom Transports (Recommended):
You can create a custom transport and log only the levels you want.
Here is an example just to give you an idea:
let mainLogger = new (winston.Logger)({
transports: [
new (winston.transports.Console)(),
]
});
class CustomTransport extends winston.Transport {
constructor(options) {
super(options);
this.name = 'customLogger';
this.level = options && options.level || 'info';
this.levelOnly = options && options.levelOnly;
this.levels = options && options.levels || [];
}
log(level, msg, meta, callback) {
if (!this.levelOnly || this.levels.indexOf(level) > -1) {
mainLogger[level](msg, meta);
}
callback(null, true);
}
}
winston.transports.CustomTransport = CustomTransport;
let myLogger = new winston.Logger({
transports: [
new (winston.transports.CustomTransport)({
levelOnly: true,
levels: ['info'],
}),
]
});
myLogger.info('will be logged');
myLogger.warn('will NOT be logged');
myLogger.info('will be logged as well');
2. Use winston-levelonly
This is a fork of the original winston package. The fork is at https://github.com/damianof/winston
This version adds a levelOnly option to make winston log only the specified level.
In the end, I would like to encourage you to read these relevant discussions:
https://github.com/winstonjs/winston/issues/614
https://github.com/winstonjs/winston/issues/812
https://github.com/winstonjs/winston/pull/628
Winston Logging - separate levels to separate Transports
I'm using WinstonJS for logging to a file, and nodemon to restart when I update my code.
var winston = require('winston');
var logger = new (winston.Logger)({
transports: [
new (winston.transports.File)({
level: 'silly',
filename: __dirname + '/logs/test.log',
json: false
})
]
});
logger.log('info', 'something');
The log file is being appended to, however when I make a code change and save my file, nodemon runs it again and the log file is appended to again. This leads to a log file that just keeps getting longer and longer, and I have to keep manually deleting the contents.
I guess I could do something with the fs module, but it would be far nicer to use Winston to say something like update: replace|append, but I can't see anything like that available.
Is there any way to clear the log file, so I only have log entries for the last time my code was run..?
The
{ flags: 'w' }
property on the options object on the new transport is working for me:
transports: [
new winston.transports.File({ filename: 'myLog.txt', options: { flags: 'w' } })
]
I know this question is a little outdated, but I recently found the answer to this and thought I would share it here for posterity. You can pass fs.createWriteStream options to the File logger:
// Open the file in "write" mode, which truncates first
options: { flags: 'w' },
I'm using node.js with morgan as logger like this:
// create a rotating write stream
var accessLogStream = require("stream-file-archive")({
path: "logs/app-%Y-%m-%d.log", // Write logs rotated by the day
symlink: "logs/current.log", // Maintain a symlink called current.log
compress: true // Gzip old log files
});
app.use(logger('combined', {stream: accessLogStream}));
and I wanted to know how to limit the max file size of the access.log.
Thanks
In my web analytics, I am logging the data in plain text file. I want to rotate the log on a daily basis because its logging too much data. Currently I am using bunyan to rotate the logs.
Problem I am facing
It is rotating the file correctly, but rotated log file are in the name log.0, log.1, etc. I want the file name to be log.05-08-2013, log.04-08-2013
I can't edit the source of the bunyanpackage because we are installing the modules using package.json via NPM.
So my question is - Is there any other log rotation in Node.js that meets my requirement?
Winston does support log rotation using a date in the file name. Take a look at this pull request which adds the feature and was merged four months ago. Unfortunately the documentation isn't listed on the site, but there is another pull request pending to fix that. Based on that documentation, and the tests for the log rotation features, you should be able to just add it as a new Transport to enable the log rotation functionality. Something like the following:
winston.add(winston.transports.DailyRotateFile, {
filename: './logs/my.log',
datePattern: '.dd-MM-yyyy'
});
If you also want to add logrotate (e.g. remove logs that are older than a week) in addition to saving logs by date, you can add the following code:
var fs = require('fs');
var path = require("path");
var CronJob = require('cron').CronJob;
var _ = require("lodash");
var logger = require("./logger");
var job = new CronJob('00 00 00 * *', function(){
// Runs every day
// at 00:00:00 AM.
fs.readdir(path.join("/var", "log", "ironbeast"), function(err, files){
if(err){
logger.error("error reading log files");
} else{
var currentTime = new Date();
var weekFromNow = currentTime -
(new Date().getTime() - (7 * 24 * 60 * 60 * 1000));
_(files).forEach(function(file){
var fileDate = file.split(".")[2]; // get the date from the file name
if(fileDate){
fileDate = fileDate.replace(/-/g,"/");
var fileTime = new Date(fileDate);
if((currentTime - fileTime) > weekFromNow){
console.log("delete fIle",file);
fs.unlink(path.join("/var", "log", "ironbeast", file),
function (err) {
if (err) {
logger.error(err);
}
logger.info("deleted log file: " + file);
});
}
}
});
}
});
}, function () {
// This function is executed when the job stops
console.log("finished logrotate");
},
true, /* Start the job right now */
'Asia/Jerusalem' /* Time zone of this job. */
);
where my logger file is:
var path = require("path");
var winston = require('winston');
var logger = new winston.Logger({
transports: [
new winston.transports.DailyRotateFile({
name: 'file#info',
level: 'info',
filename: path.join("/var", "log", "MY-APP-LOGS", "main.log"),
datePattern: '.MM--dd-yyyy'
}),
new winston.transports.DailyRotateFile({
name: 'file#error',
level: 'error',
filename: path.join("/var", "log", "MY-APP-LOGS", "error.log"),
datePattern: '.MM--dd-yyyy',
handleExceptions: true
})
]});
module.exports = logger;
There's the logrotator module for log rotation that you can use regardless of the logging mechanism.
You can specify the format option to format the date format (or any other format for that matter)
var logrotate = require('logrotator');
// use the global rotator
var rotator = logrotate.rotator;
// or create a new instance
// var rotator = logrotate.create();
// check file rotation every 5 minutes, and rotate the file if its size exceeds 10 mb.
// keep only 3 rotated files and compress (gzip) them.
rotator.register('/var/log/myfile.log', {
schedule: '5m',
size: '10m',
compress: true,
count: 3,
format: function(index) {
var d = new Date();
return d.getDate()+"-"+d.getMonth()+"-"+d.getFullYear();
}
});
mongodb
winston itself does not support log rotation. My bad.
mongodb has a log rotation use case. Then you can export the logs to file names per your requirement.
winston also has a mongodb transport but I don't think it supports log rotation out of the box judging from its API.
This may be an overkill though.
forking bunyan
You can fork bunyan and add your repo's url in package.json.
This is the easiest solution if you're fine with freezing bunyan's feature or maintaining your own code.
As it is an open source project, you can even add your feature to it and submit a pull request to help improve bunyan.