Serilog with ASP NET 5 Razor Pages.
Reducing log verbosity is very useful for Informational logs.
However for debug logs, how to get a MinimumLevel.Override("Microsoft.AspNetCore") to be specific to a debug file sink?
Creating 2 configurations could be a solution, but feels like something more elegant may be possible?
Log.Logger = new LoggerConfiguration()
.MinimumLevel.Debug()
.Enrich.FromLogContext()
// for debug file sink I want the override to be Debug
.MinimumLevel.Override("Microsoft.AspNetCore", LogEventLevel.Debug)
.WriteTo.File("debug.txt", restrictedToMinimumLevel: LogEventLevel.Debug)
// for info and warning file sinks I want the override to be Warning
.MinimumLevel.Override("Microsoft.AspNetCore", LogEventLevel.Warning)
.WriteTo.File("info.txt", restrictedToMinimumLevel: LogEventLevel.Information)
.WriteTo.File("warning.txt", restrictedToMinimumLevel: LogEventLevel.Warning)
.CreateLogger();
Everything works as expected using just one override. But not together.
In the example above the Warning override takes precedence and no AspNetCore Debug event logs are written to debug.txt
Edit
In summary, I'd like my debug log to include Information event level from Microsoft.AspNetCore and my info log file to include Warning event level from Microsoft.AspNetCore
I got the 2 logs files how I wanted by commenting out and in 1. and 2. below
// 1. for debug file sink I want AspNetCore.Information or Debug level override
.MinimumLevel.Override("Microsoft.AspNetCore", LogEventLevel.Information)
.WriteTo.File($#"{logFilePath}debugx.txt", restrictedToMinimumLevel: LogEventLevel.Debug, rollingInterval: RollingInterval.Day)
// 2. for info and warning file sinks below I want only AspNetCore warnings
//.MinimumLevel.Override("Microsoft.AspNetCore", LogEventLevel.Warning)
It's an interesting one
You want to filter log data and want to populate into different file sinks.
For Example /Logs/Error/Errlog.txt and /Logs/Info/InfoLog.txt
You can achieve this by using Serilog.Expressions nuget package. If time permits, I will paste a working example here.
Serilog.Expressions sample from Serilog
https://github.com/serilog/serilog-expressions/blob/dev/example/Sample/Program.cs
In below example it will exclude Name=User line and only print second line on console
using var log = new LoggerConfiguration()
.Filter.ByExcluding("#m like 'Welcome!%' and Name = 'User'")
.WriteTo.Console()
.CreateLogger();
// Logged normally
log.Information("Welcome!, {Name}", "User");
// Excluded by the filter
log.Information("Welcome!, {Name}", "Domain\\UserName");
Here is the filtering example for \Logs\Info\Info-20210720.txt which filters Error, Fatal or Warning levels. More information here
var exprInfo = "#l='Error' or #l='Fatal' or #l='Warning'";
var loggerInfo = new LoggerConfiguration()
.WriteTo.File(
#"C:\Temp\Logs\Info\Info-.txt",
fileSizeLimitBytes: 1_000_000,
outputTemplate: "{Timestamp:yyyy-MM-dd HH:mm:ss.fff} [{Level}] [{SourceContext}] [{EventId}] {Message}{NewLine}{Exception}",
rollingInterval: RollingInterval.Day,
rollOnFileSizeLimit: true,
shared: true,
flushToDiskInterval: TimeSpan.FromSeconds(1))
.MinimumLevel.Override("Microsoft", LogEventLevel.Debug)
.Filter.ByExcluding(exprInfo)
.CreateLogger();
try
{
loggerInfo.Debug("TEST");
SelfLog.Enable(Console.Out);
var sw = System.Diagnostics.Stopwatch.StartNew();
for (var i = 0; i < 100; ++i)
{
loggerInfo.Information("Hello, file logger!>>>>>>{Count}", i);
loggerInfo.Information("Writing to log file with INFORMATION severity level.");
loggerInfo.Debug("Writing to log file with DEBUG severity level.");
loggerInfo.Warning("Writing to log file with WARNING severity level.");
loggerInfo.Error("Writing to log file with ERROR severity level.");
loggerInfo.Fatal("Writing to log file with CRITICAL severity level.");
}
sw.Stop();
Console.WriteLine($"Elapsed: {sw.ElapsedMilliseconds} ms");
Console.WriteLine($"Size: {new FileInfo("log.txt").Length}");
Console.WriteLine("Press any key to delete the temporary log file...");
Console.ReadKey(true);
}
catch (Exception ex)
{
loggerInfo.Fatal(ex, "Application Start-up for Serilog failed");
throw;
}
finally
{
Log.CloseAndFlush();
}
I solved it by using sub loggers and filters as described in here: How can I override Serilog levels differently for different sinks?
Log.Logger = new LoggerConfiguration()
.MinimumLevel.Debug()
.Enrich.FromLogContext()
// Includes Debug from Microsoft.AspNetCore (noisy)
// useful for deep debugging
.WriteTo.File($#"logs/debug.txt", rollingInterval: RollingInterval.Day)
// Info-with-framework (useful for debugging)
.WriteTo.Logger(lc => lc
.MinimumLevel.Information()
.Filter.ByExcluding("RequestPath in ['/health-check', '/health-check-db']")
.WriteTo.File("logs/info-with-framework.txt", rollingInterval: RollingInterval.Day)
.WriteTo.Console()
)
// Info
// framework minimum level is Warning (normal everyday looking at logs)
.WriteTo.Logger(lc => lc
.MinimumLevel.Information()
.Filter.ByExcluding("RequestPath in ['/health-check', '/health-check-db']")
.Filter.ByExcluding("SourceContext = 'Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware'")
.Filter.ByExcluding(logEvent =>
logEvent.Level < LogEventLevel.Warning &&
Matching.FromSource("Microsoft.AspNetCore").Invoke(logEvent))
.WriteTo.File("logs/info.txt", rollingInterval: RollingInterval.Day))
// Warning (bad things - Warnings, Error and Fatal)
.WriteTo.Logger(lc => lc
.MinimumLevel.Warning()
// stopping duplicate stacktraces, see blog 2021/03/10/a11-serilog-logging-in-razor-pages
.Filter.ByExcluding("SourceContext = 'Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware'")
.WriteTo.File("logs/warning.txt", rollingInterval: RollingInterval.Day))
// SignalR - tweak levels by filtering on these namespaces
// Microsoft.AspNetCore.SignalR
// Microsoft.AspNetCore.Http.Connections
.CreateLogger();
Although this works, there may be a better way https://nblumhardt.com/2016/07/serilog-2-write-to-logger/
I feel like you don't need those minium level override calls. The restricted to minimum level parameter in the sinks will take are of filtering.
You do need to set the minimum level to info so the info sink can work.
Related
I'm attempting to format my logs in such a way that Google Cloud will correctly extract the log level. This is running on Cloud Run, with typescript. Cloud Run is grabbing the logs from the container output.
If I do the following, google correctly parses the log line:
console.log(JSON.stringify({
severity: 'ERROR',
message: 'This is testing a structured log error for GCP'
}));
And the log output looks like this:
I've tried a number of different ways to format with winston, ended up with the following:
useFormat = format.combine(
format((info, opts) => {
info['severity'] = info.level;
delete info.level;
return info;
})(),
format.json());
this.winston = winston.createLogger({
level: logLevel,
format: useFormat,
transports: [new winston.transports.Console()]
});
Which looks like it will work (it correctly outputs the json line), I get this in the GCP logs:
Any help appreciated.
Turns out I was close, just needed to .upperCase() the log level (and I'm mapping Verbose -> Debug, I don't really understand why GCP decided to do a totally different log leveling system than everyone else). New code:
useFormat =
format.combine(
format((info, opts) => {
let level = info.level.toUpperCase();
if(level === 'VERBOSE') {
level = 'DEBUG';
}
info['severity'] = level;
delete info.level;
return info;
})(),
format.json());
The last bit of the question is confusing. The problem OP is pointing to is that the json is printed out and the severity is default. The json should not be printed out, only the message, and the severity should be debug. The answer that OP provides does what is wanted.
For others that may be confused in the same way I was.
Is there an equivalent of log.IsDebugEnabled in Winston?
I want to use this to skip expensive logging code in a production environment but have it execute in development.
For example:
if(winston.isDebugEnabled){
// Call to expensive dump routine here
dump();
}
Checking winston.debug just checks whether the method is defined, not whether it is enabled.
Many thanks!
Edit: Added code example.
I've added a method to my logger to achieve just that:
logger.isLevelEnabled = function(level) {
return _.any(this.transports, function(transport) {
return (transport.level && this.levels[transport.level] <= this.levels[level])
|| (!transport.level && this.levels[this.level] <= this.levels[level]);
}, this);
};
This goes through each of your logger's transports and checks whether it 'wants' to log the specified level.
Note _.any is lodash, you can replace with for loop.
I'm sure you'd be able to get that directly from winston, but if you want to have different logging levels for different environments, you should pass these in when you're creating the winston.logger.
For example:
// default log file level is info (which is the lowest by default)
var logFileLevel = 'info';
if (process.env.NODE_ENV == 'production') {
// only write logs with a level of 'error' or above when in production
logFileLevel = 'error';
}
var logger = new (winston.Logger)({
transports: [
new (winston.transports.File)({
filename: '/var/log/node-logger.log',
level: logFileLevel
})
]
});
It's also useful to switch out the transports. You might want to use the Console transport whilst in development, and the File transport when in production for example.
More documentation on all this on the winston readme.
Try
if ( logger.levels[logger.level] >= logger.levels['debug'] ) {
// expensive calculation here
logger.debug(...)
}
Since Winston 3.1.0 (PR), you can use the Logger functions isLevelEnabled(string) & isXXXEnabled() for this.
I am using Winston logging with my Node.js app and have defined a file transport. Throughout my code, I log using either logger.error, logger.warn, or logger.info.
My question is, how do I specify the log level? Is there a config file and value that I can set so that only the appropriate log messages are logged? For example, I'd like the log level to be "info" in my development environment but "error" in production.
If you are using the default logger, you can adjust the log levels like this:
const winston = require('winston');
// ...
winston.level = 'debug';
will set the log level to 'debug'. (Tested with winston 0.7.3, default logger is still around in 3.2.1).
However, the documentation recommends creating a new logger with the appropriate log levels and then using that logger:
const myLogger = winston.createLogger({
level: 'debug'
});
myLogger.debug('hello world');
If you are already using the default logger in your code base this may require you to replace all usages with this new logger that you are using:
const winston = require('winston');
// default logger
winston.log('debug', 'default logger being used');
// custom logger
myLogger.log('debug', 'custom logger being used');
Looks like there is a level option in the options passed covered here
From that doc:
var logger = new (winston.Logger)({
transports: [
new (winston.transports.Console)({ level: 'error' }),
new (winston.transports.File)({ filename: 'somefile.log' })
]
});
Now, those examples show passing level in the option object to the console transport. When you use a file transport, I believe you would pass an options object that not only contains the filepath but also the level.
That should lead to something like:
var logger = new (winston.Logger)({
transports: [
new (winston.transports.File)({ filename: 'somefile.log', level: 'error' })
]
});
Per that doc, note also that as of 2.0, it exposes a setLevel method to change at runtime. Look in the Using Log Levels section of that doc.
There are 6 default levels in winston: silly=0(lowest), debug=1, verbose=2, info=3, warn=4, error=5(highest)
While creating the logger transports, you can specify the log level like:
new (winston.transports.File)({ filename: 'somefile.log', level: 'warn' })
Above code will set log level to warn, which means silly, verbose and info will not be output to somefile.log, while warn, debug and error will.
You can also define your own levels:
var myCustomLevels = {
levels: {
foo: 0,
bar: 1,
baz: 2,
foobar: 3
}
};
var customLevelLogger = new (winston.Logger)({ levels: myCustomLevels.levels });
customLevelLogger.foobar('some foobar level-ed message');
Note that it's better to always include the 6 predefined levels in your own custom levels, in case somewhere used the predefined levels.
You can change the logging level in runtime by modifying the level property of the appropriate transport:
var log = new (winston.Logger)({
transports: [
new (winston.transports.Console)({ level : 'silly' })
]
});
...
// Only messages with level 'info' or higher will be logged after this.
log.transports.Console.level = 'info';
I guess, it works similarly for file but I haven't tried that.
If you want to change the log level on the fly. Like for when you need to trace production issue for short amount of time; then revert to error log level. You can use a dynamic logger provided you can expose a service on the web https://github.com/yannvr/Winston-dynamic-loglevel
apart from this you can cleanly achieve this by imlplementing runtime-node-refresh follow this link for more.
I tried to change NLog's configuration via:
LogManager.Configuration.AddTarget (name, foo);
LogManager.Configuration.LoggingRules.Add (new LoggingRule ("*", LogLevel.Trace, foo));
but it silently ignored my changes. However, when I tried
SimpleConfigurator.ConfigureForTargetLogging (this, level);
it worked right away - but discarded all previous configuration. Why is that, and how do I solve this?
SimpleConfigurator replaces the current configuration, so you cannot use it for aggregated cases.
NLog does not detect and follow changes made to its current configuration unless you notify it via LogManager.Configuration = new_config. So use this instead:
var conf = LogManager.Configuration;
conf.AddTarget (name, foo);
conf.LoggingRules.Add (new LoggingRule ("*", LogLevel.Trace, foo));
LogManager.Configuration = conf;
I started to play with log4net today and so far, I really like it. In order to preserve our current logging functionality, the app needs to create a new log file whenever the application is started. The log file name has the date and time stamp encoded in it. Currently, I've got log4net configured via an XmlConfigurator, which works great, except that the filename for my RollingFileAppender is hardcoded in the configuration XML file.
I'd like to continue to use the XmlConfigurator, but after calling Configure(), I want to get at the RollingFileAppender and, in code, change its file value to be a dynamically-generated string. The sample documentation online seems to be down right now, but I've poked through the SDK reference, and it looks like I could use the Heirarchy and GetAppenders() to do what I need to do. Am I on the right track?
Ok, I took a stab at this and tried the following code, which didn't work:
private static readonly ILog _log = LogManager.GetLogger(typeof(GUI));
// in the config file, I've set the filename to example.log, and it works
XmlConfigurator.Configure(new FileInfo("log_config.xml"));
Hierarchy hierarchy = LogManager.GetRepository() as Hierarchy;
if(hierarchy != null) {
// get the appenders
IAppender[] appenders = hierarchy.GetAppenders();
// change the filename for the RollingFileAppender
foreach( IAppender a in appenders) {
RollingFileAppender rfa = a as RollingFileAppender;
if(rfa == null)
continue;
rfa.File = "newfile.log"; // no runtime error, but doesn't work.
}
}
_log.Info("Application started");
Try this snippet:
XmlConfigurator.Configure();
log4net.Repository.ILoggerRepository repo = LogManager.GetRepository();
foreach (log4net.Appender.IAppender appender in repo.GetAppenders())
{
if (appender.Name.CompareTo("RollingFileAppender") == 0 && appender is log4net.Appender.RollingFileAppender)
{
var appndr = appender as log4net.Appender.RollingFileAppender;
string logPath = "MyApplication.log";
appndr.File = logPath;
appndr.ActivateOptions();
}
I had posted similar article here
Do you in this case need the rolling file appender? If not I would expect that your code would create the desired result if you used the normal file appender.
Edit: Maybe it works with the RollingFile Appender if you call ActivateOptions() on the appender.