Winston with AWS Cloudwatch on Nestjs - node.js

All the articles and documentation I have read so far talk about the integration of Cloudwatch and Winston on a vanilla Node app, but nothing on Nestjs
So far I have on my app.module.ts:
imports: [
ConfigModule.forRoot({ isGlobal: true }),
MongooseModule.forRoot(
`mongodb://${environment.MONGO_INITDB_ROOT_USERNAME}:${environment.MONGO_INITDB_ROOT_PASSWORD}#${environment.MONGODB_HOST}/${environment.MONGO_INITDB_DATABASE}`,
),
VoucherModule,
ApiKeyModule,
WinstonModule.forRoot(loggerConfig),
],
where loggerConfig are the basic Winston configs depending on the env.
Using winston-cloudwatch package I need to create a new Transporter and add it add it to winston, but can't seem to find a way to do this.

I recently implemented aws-cloudwatch in nestjs and faced similar issue but after some browsing and reading about winston and cloudwatch, came up with this solution.
//main.ts
import {
utilities as nestWinstonModuleUtilities,
WinstonModule,
} from 'nest-winston';
import * as winston from 'winston';
import CloudWatchTransport from 'winston-cloudwatch';
const app = await NestFactory.create(AppModule, {
logger: WinstonModule.createLogger({
format: winston.format.uncolorize(), //Uncolorize logs as weird character encoding appears when logs are colorized in cloudwatch.
transports: [
new winston.transports.Console({
format: winston.format.combine(
winston.format.timestamp(),
winston.format.ms(),
nestWinstonModuleUtilities.format.nestLike()
),
}),
new CloudWatchTransport({
name: "Cloudwatch Logs",
logGroupName: process.env.CLOUDWATCH_GROUP_NAME,
logStreamName: process.env.CLOUDWATCH_STREAM_NAME,
awsAccessKeyId: process.env.AWS_ACCESS_KEY,
awsSecretKey: process.env.AWS_KEY_SECRET,
awsRegion: process.env.CLOUDWATCH_AWS_REGION,
messageFormatter: function (item) {
return (
item.level + ": " + item.message + " " + JSON.stringify(item.meta)
);
},
}),
],
}),
});
Here, we have defined two transports one is transports.Console() which is the default winston transport and another one is the cloudwatch transport.
What this basically does is, replaces the default nestjs Logger(imported from #nestjs/common). So, we don't have to import winston in any module not even in 'app.module.ts'. Instead just import Logger in whichever module you need, and whenever we log anything it will be shown in the terminal and will upload it to the cloudwatch.
EDIT:
With configService..
//main.js
import {
utilities as nestWinstonModuleUtilities,
WinstonModule,
} from 'nest-winston';
import * as winston from 'winston';
import CloudWatchTransport from 'winston-cloudwatch';
import { ConfigService } from '#nestjs/config';
async function bootstrap() {
const app = await NestFactory.create(AppModule);
const configService = app.get(ConfigService);
app.useLogger(
WinstonModule.createLogger({
format: winston.format.uncolorize(),
transports: [
new winston.transports.Console({
format: winston.format.combine(
winston.format.timestamp(),
winston.format.ms(),
nestWinstonModuleUtilities.format.nestLike(),
),
}),
new CloudWatchTransport({
name: 'Cloudwatch Logs',
logGroupName: configService.get('CLOUDWATCH_GROUP_NAME'),
logStreamName: configService.get('CLOUDWATCH_STREAM_NAME'),
awsAccessKeyId: configService.get('AWS_ACCESS_KEY'),
awsSecretKey: configService.get('AWS_KEY_SECRET'),
awsRegion: configService.get('CLOUDWATCH_AWS_REGION'),
messageFormatter: function (item) {
return (
item.level + ': ' + item.message + ' ' + JSON.stringify(item.meta)
);
},
}),
],
}),
);
await app.listen(configService.get('PORT') || 3000);
}

Related

How to sent log to Jaeger in NestJs?

In my nestjs project, I am using Winston as a logger. here is the example of my log.info:
logger.log(
'Error: test',
inout_data,
);
here is the logger:
var winston = require('winston');
const logger = winston.createLogger({
level: 'verbose',
format: winston.format.json(),
transports: [
new winston.transports.Console({
format: winston.format.json(),
}),
],
});
export default logger;
this is how I setup the Jaeger:
import { NodeSDK, tracing } from '#opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '#opentelemetry/auto-instrumentations-node';
import { JaegerExporter, ExporterConfig } from '#opentelemetry/exporter-jaeger';
import { BatchSpanProcessor } from '#opentelemetry/sdk-trace-base';
import { Resource } from '#opentelemetry/resources';
import { SemanticResourceAttributes } from '#opentelemetry/semantic-conventions';
interface Tag {
key: string;
value: TagValue;
}
declare type TagValue = string | number | boolean;
const options = {
tags: [], // optional
host: 'localhost',
port: 6832, // optional
maxPacketSize: 65000,
};
const exporter = new JaegerExporter(options);
export const sdk = new NodeSDK({
resource: new Resource({
[SemanticResourceAttributes.SERVICE_NAME]: 'test,
[SemanticResourceAttributes.DEPLOYMENT_ENVIRONMENT]: '2',
}),
traceExporter: exporter,
instrumentations: [getNodeAutoInstrumentations()],
});
sdk.start();
I want to send the log to Jaeger, but I can not find a way to connect them. Am I missing anything here?

How to get class name in logs in which winston logger is called

I am creating NestJS application in which I am using winston logger for logging purposes.I am getting logs inside log file but along with logs I also want the class name in which logger is called so that it can easily be identified which logs are producing by which class.
Below is my winston setup:
app.module.ts
import { Module } from '#nestjs/common';
import { WinstonModule,utilities as nestWinstonModuleUtilities } from 'nest-winston';
import * as winston from 'winston';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { TestModule } from './test/test.module';
import * as path from 'path';
#Module({
imports: [ WinstonModule.forRoot({
transports: [
new winston.transports.Console({
level: 'info',
format:winston.format.combine(
winston.format.colorize(),
winston.format.timestamp({
format: 'DD-MM-YYYY HH:mm:ss'
}),
winston.format.printf(
info => `${info.timestamp} ${info.level}: ${info.message}`
),
)
}),
new winston.transports.File({
dirname: path.join(__dirname, '../log/info/'), //path to where save logging result
filename: 'info.log', //name of file where will be saved logging result
level: 'info',
format:winston.format.combine(
winston.format.timestamp({
format: 'DD-MM-YYYY HH:mm:ss'
}),
winston.format.printf(
info => `${info.timestamp} ${info.level}: ${info.message}`
)
),
}),
],
}), TestModule,],
controllers: [AppController],
providers: [AppService],
})
export class AppModule {}
app.service.ts
import { Inject, Injectable } from '#nestjs/common';
import { Logger } from 'winston';
#Injectable()
export class AppService {
constructor( #Inject('winston') private readonly logger: Logger){}
getHello(): string {
this.logger.info('Hello logs');
return 'Hello World!';
}
}
Here logger is called by AppService class so I want this classname also in logs.How can I achieve desired result
It's possible by not injecting the logger, but directly creating a new instance.
export class AppService {
private readonly logger = new Logger(AppService.name);
constructor(...)
}
Unfortunately you loose the advantages of dependency injection. For exmaple you need special way to mock it in tests. But it's possible.
It still works if you registered a custom logger. For example when you are using winston logger, the solution will also get you the Winton Logger.:
const app = await NestFactory.create(AppModule, {
logger: WinstonModule.createLogger({
exitOnError: false,
format: format.json(),
transports: [
new transports.Console({
format: format.combine(
format.simple(),
format.timestamp(),
format.ms(),
format.errors({ stack: true }),
utilities.format.nestLike()
)
})
],
level: process.env.WINSTON_LOGGING ?? 'info'
})
});
You can mock it the same way, for example for tests:
const moduleFixture: TestingModule = await Test.createTestingModule({
// ...
}).compile();
const app = moduleFixture.createNestApplication();
app.useLogger(loggerMock);

Unable to inject winston's logger instance with NestJS

I'm using NestJS 7.0.7 and Winston 3.2.1 (with nest-winston 1.3.3).
I'm trying to integrate Winston into NestJS, but so far, I'm unable to inject a logger instance (to actually log anything) into any controller/service.
Since I would like to use Winston across the application AND during bootstrapping, I'm using the approach as the main Nest logger:
// main.ts
import { NestFactory } from "#nestjs/core";
import { WinstonModule } from "nest-winston";
import { format, transports } from "winston";
import { AppModule } from "./app.module";
async function bootstrap(): Promise<void> {
const app = await NestFactory.create(AppModule, {
logger: WinstonModule.createLogger({
exitOnError: false,
format: format.combine(format.colorize(), format.timestamp(), format.printf(msg => {
return `${msg.timestamp} [${msg.level}] - ${msg.message}`;
})),
transports: [new transports.Console({ level: "debug" })], // alert > error > warning > notice > info > debug
}),
});
app.use(helmet());
await app.listen(process.env.PORT || 3_000);
}
bootstrap().then(() => {
// ...
});
I'm not doing anything in regard to the logging in app.module.ts:
// app.module.ts
import { SomeController } from "#controller/some.controller";
import { Module } from "#nestjs/common";
import { SomeService } from "#service/some.service";
#Module({
controllers: [SomeController],
imports: [],
providers: [SomeService],
})
export class AppModule {
// ...
}
// some.controller.ts
import { Controller, Get, Inject, Param, ParseUUIDPipe, Post } from "#nestjs/common";
import { SomeService } from "#service/some.service";
import { WINSTON_MODULE_PROVIDER } from "nest-winston";
import { Logger } from "winston";
#Controller("/api/some-path")
export class SomeController {
constructor(#Inject(WINSTON_MODULE_PROVIDER) private readonly logger: Logger, private readonly service: SomeService) {
// ...
}
...
}
The application tries to start but fails at some point:
2020-04-06T18:51:08.779Z [info] - Starting Nest application...
2020-04-06T18:51:08.787Z [error] - Nest can't resolve dependencies of the SomeController (?, SomeService). Please make sure that the argument winston at index [0] is available in the AppModule context.
Potential solutions:
- If winston is a provider, is it part of the current AppModule?
- If winston is exported from a separate #Module, is that module imported within AppModule?
#Module({
imports: [ /* the Module containing winston */ ]
})
Try importing the WinstonModule in the root AppModule, as explained in the official docs: https://github.com/gremo/nest-winston:
import { Module } from '#nestjs/common';
import { WinstonModule } from 'nest-winston';
import * as winston from 'winston';
const logger: LoggerConfig = new LoggerConfig();
#Module({
imports: [WinstonModule.forRoot(logger.console())],
})
export class AppModule {}
It's probably a good idea to create some kind of factory/Logging-Config in order not to have to duplicate the logger options.
import winston, { format, transports } from "winston";
export class LoggerConfig {
private readonly options: winston.LoggerOptions;
constructor() {
this.options = {
exitOnError: false,
format: format.combine(format.colorize(), format.timestamp(), format.printf(msg => {
return `${msg.timestamp} [${msg.level}] - ${msg.message}`;
})),
transports: [new transports.Console({ level: "debug" })], // alert > error > warning > notice > info > debug
};
}
public console(): object {
return this.options;
}
}
Implement Winston custom logger in NestJs project
Prerequisit:
npm install --save nest-winston winston winston-daily-rotate-file
import { NestFactory } from '#nestjs/core';
import { WinstonModule } from 'nest-winston';
import * as winston from 'winston';
import * as winstonDailyRotateFile from 'winston-daily-rotate-file';
import { AppModule } from './app.module';
const transports = {
console: new winston.transports.Console({
level: 'silly',
format: winston.format.combine(
winston.format.timestamp({
format: 'YYYY-MM-DD HH:mm:ss',
}),
winston.format.colorize({
colors: {
info: 'blue',
debug: 'yellow',
error: 'red',
},
}),
winston.format.printf((info) => {
return `${info.timestamp} [${info.level}] [${
info.context ? info.context : info.stack
}] ${info.message}`;
}),
// winston.format.align(),
),
}),
combinedFile: new winstonDailyRotateFile({
dirname: 'logs',
filename: 'combined',
extension: '.log',
level: 'info',
}),
errorFile: new winstonDailyRotateFile({
dirname: 'logs',
filename: 'error',
extension: '.log',
level: 'error',
}),
};
async function bootstrap() {
const app = await NestFactory.create(AppModule);
app.useLogger(
WinstonModule.createLogger({
format: winston.format.combine(
winston.format.timestamp({
format: 'YYYY-MM-DD HH:mm:ss',
}),
winston.format.errors({ stack: true }),
winston.format.splat(),
winston.format.json(),
),
transports: [
transports.console,
transports.combinedFile,
transports.errorFile,
],
}),
);
await app.listen(4000);
}
bootstrap();
NestJs Custom Logger
NestJs Winston NPM Documentation
Note Log Levels, file names, dateformat you may edit as per your requirement. Follow officials documentations with more option.

nodejs auto maintain winston log

I'm trying to integrate winston to a nodeJs Express app. It works but I don't know how could it be auto maintained. I wanted something like when it reach, for example 50000 rows, it delete the first ones before adding the new ones for terms of space. The idea is doing that without something like cron. Is it possible?
Here is my testing code:
//Winston logger
const { createLogger, format, transports } = require('winston');
const { combine, timestamp, label, printf } = format;
const winstonFormat = printf(info => {
return `${info.timestamp} [${info.label}] ${info.level}: ${info.message}`;
});
const logger = createLogger({
level: 'info',
//maxsize:'10000', //It doesn't works for me
format: combine(
label({ label: config.environment }),
timestamp(),
winstonFormat
),
transports: [
new transports.File({ filename: 'error.log', level: 'error' }),
new transports.File({ filename: 'combined.log' })
]
});
if (process.env.NODE_ENV !== 'production') {
logger.add(new transports.Console({
format: format.simple()
}));
}
You're looking for log rotation, so googling "winston log rotation" points to https://github.com/winstonjs/winston-daily-rotate-file, which is probably just the ticket for you.

Add module name in winston log entries

Is there a possibility to save the current module name in order to be printed automatically in winston log entries when they are called later?
Currently, when I want to print the module name in logs, I have to add it manually:
var logHeader = 'mymodule'
log.info(logHeader + 'Hello')
For example, with debug, you can do (ignore the log format feature %s for now):
var debug = require('debug')('http')
, name = 'My App'
debug('booting %s', name);
This will prin http prefix before the log:
http booting My App
Can this be done in winston? I have searched in the documentation but I couldn't find anything relevant.
I found a better way to do this.
I added an additional layer over the winston logger, in this case a function, that keeps the module name for each module that needs the logger. So when a module requires my new logger, it actually calls the exported function with the module name, in this case __filename.
log.js
var winston = require('winston')
var winstonLogger = new (winston.Logger)({
transports: [
new (winston.transports.File) ({
filename: 'MyLogs.txt',
handleExceptions: true,
humanReadableUnhandledException: true,
level: 'info',
timestamp: true,
json: false
}),
new (winston.transports.Console) ({
level: 'info',
prettyPrint: true,
colorize: true,
timestamp: true
})
]
})
module.exports = function(fileName) {
var myLogger = {
error: function(text) {
winstonLogger.error(fileName + ': ' + text)
},
info: function(text) {
winstonLogger.info(fileName + ': ' + text)
}
}
return myLogger
}
module1.js
var log = require('log')(__filename)
log.info('Info log example')
info: C:\Users\user1\project\module1.js: Info log example
module2.js
var log = require('log')(__filename)
log.error('Error log example')
error: C:\Users\user1\project\module2.js: Error log example
Also, this way, I didn't need to change anywhere the way I submit a text log; log.info('text') stays exactly like before.
This is what Child Loggers are for. They might not have been available when this question was asked, but for future reference, you can create your main logger instance and then export a function that creates a child logger with some default options.
// logging.js
const winston = require('winston')
const logger = winston.createLogger({
transports: [
new winston.transports.Console({
format: winston.format.printf(options => {
// you can pass any custom variable in options by calling
// logger.log({level: 'debug', message: 'hi', moduleName: 'my_module' })
return `[${options.moduleName}] ${options.level}: ${options.message}$`;
})
})
]
});
module.exports = function(name) {
// set the default moduleName of the child
return logger.child({moduleName: name});
}
Then at the top of each module, import the child using:
// my_module.js
const logger = require('./logging.js')('my_module_name');
logger.error('computer says no');
// output:
// [my_module_name] error: computer says no
You can specify a custom log format with Winston -
var moduleName = 'myModule';
var logger = new (winston.Logger)({
transports: [
new (winston.transports.Console)({
formatter: function(options) {
// Return string will be passed to logger.
return moduleName + ' - ' + (options.message ? options.message : '')
}
})
]
});
logger.info('This is a log message');
This will print -
myModule - This is a log message
So your module name will be appended to every log messsage.
With winston v3.3.3 you can do this by using winston.format.label and a customization of winston.format.printf:
const winston = require('winston');
const logger = winston.createLogger({
format: winston.format.combine(
winston.format.label({label: 'mymodule'}),
winston.format.printf(({label, message}) => {
return `${label}: ${message}`;
})
),
transports: [
new winston.transports.Console(),
],
});
logger.info('Hello, World!'); // prints "mymodule: Hello, World!"

Resources