I'm using NLog to log in json format, using the targets below (file and network).
When logging to the file target or both file and network, the messages are logged fine.
However, when using only the network target, dbResponseTime and messagesSentTotal will not be included in the message (in fact, any attribute using a layout with "event-properties:item="). All the other properties are present, including the one defined in the end (module).
<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.nlog-project.org/schemas/NLog.xsd NLog.xsd"
autoReload="true"
throwExceptions="false" internalLogLevel="Off">
<targets>
<target name="logHealthCheck" xsi:type="File"
fileName="D:\Logs\HealthCheck.${shortdate}.json"
archiveFileName="D:\Logs\HealthCheck.{#}.json"
archiveNumbering="DateAndSequence"
archiveAboveSize="33554432"
archiveDateFormat="yyyy-MM-dd">
<layout xsi:type="JsonLayout">
<attribute name="timestamp" layout="${date:universalTime=true:format=o}" />
<attribute name="machineName" layout="${machinename}" />
<attribute name="environment" layout="${literal:text=DEV}" />
<attribute name="dbResponseTime" layout="${event-properties:item=dbResponseTime}" encode="false"/>
<attribute name="messagesSentTotal" layout="${event-properties:item=messagesSentTotal}" encode="false"/>
<attribute name="module" layout="${literal:text=core}" />
</layout>
</target>
<target name="logHealthCheckNetwork" xsi:type="Network" address="tcp://127.0.0.1:1514" newLine="true">
<layout xsi:type="JsonLayout">
<attribute name="timestamp" layout="${date:universalTime=true:format=o}" />
<attribute name="machineName" layout="${machinename}" />
<attribute name="environment" layout="${literal:text=DEV}" />
<attribute name="dbResponseTime" layout="${event-properties:item=dbResponseTime}" encode="false"/>
<attribute name="messagesSentTotal" layout="${event-properties:item=messagesSentTotal}" encode="false"/>
<attribute name="module" layout="${literal:text=core}" />
</layout>
</target>
<rules>
<logger name="HealthCheck" minlevel="Debug" writeTo="logHealthCheck,logHealthCheckNetwork">
<filters>
</filters>
</logger>
</rules>
</nlog>
Switched on the internal log, and no error is recorded. Trace below:
2018-02-02 16:43:07.7837 Trace Scanning LoggingRule 'logNamePattern: (HealthCheck:Equals) levels: [ Debug Info Warn Error Fatal ] appendTo: [ logHealthCheckNetwork ]'
2018-02-02 16:43:07.7993 Trace Scanning NetworkTarget 'Network Target[logHealthCheckNetwork]'
2018-02-02 16:43:07.8305 Trace Scanning SimpleLayout ''tcp://127.0.0.1:1514''
2018-02-02 16:43:07.8305 Trace Scanning LiteralLayoutRenderer 'Layout Renderer: ${literal}'
2018-02-02 16:43:07.9085 Trace Scanning JsonAttribute 'NLog.Layouts.JsonAttribute'
2018-02-02 16:43:07.9085 Trace Scanning SimpleLayout ''${date:universalTime=true:format=o}''
2018-02-02 16:43:07.9241 Trace Scanning DateLayoutRenderer 'Layout Renderer: ${date}'
2018-02-02 16:43:07.9397 Trace Scanning JsonAttribute 'NLog.Layouts.JsonAttribute'
2018-02-02 16:43:07.9397 Trace Scanning SimpleLayout ''${machinename}''
2018-02-02 16:43:07.9553 Trace Scanning LiteralLayoutRenderer 'Layout Renderer: ${literal}'
2018-02-02 16:43:07.9553 Trace Scanning JsonAttribute 'NLog.Layouts.JsonAttribute'
2018-02-02 16:43:07.9709 Trace Scanning SimpleLayout ''${literal:text=DEV}''
2018-02-02 16:43:07.9709 Trace Scanning LiteralLayoutRenderer 'Layout Renderer: ${literal}'
2018-02-02 16:43:08.0177 Trace Scanning JsonAttribute 'NLog.Layouts.JsonAttribute'
2018-02-02 16:43:08.0333 Trace Scanning SimpleLayout ''${event-properties:item=dbResponseTime}''
2018-02-02 16:43:08.0333 Trace Scanning EventPropertiesLayoutRenderer 'Layout Renderer: ${event-properties}'
2018-02-02 16:43:08.0801 Trace Scanning JsonAttribute 'NLog.Layouts.JsonAttribute'
2018-02-02 16:43:08.0801 Trace Scanning SimpleLayout ''${event-properties:item=messagesSentTotal}''
2018-02-02 16:43:08.3453 Trace Scanning JsonAttribute 'NLog.Layouts.JsonAttribute'
2018-02-02 16:43:08.3609 Trace Scanning SimpleLayout ''${literal:text=core}''
2018-02-02 16:43:08.3765 Trace Scanning LiteralLayoutRenderer 'Layout Renderer: ${literal}'
NLog 4.5.4 has been released that fixes the issue: https://github.com/NLog/NLog/issues/2573
Related
I am trying to print the exception stack trace as:-
log.error("Error is {}" , ArrayUtils.toString(e.getStackTrace()));
This prints the entire stack trace in a single line.
I tried replacing every terminal "," character with "\n" but that doesnt work.
I also tried:
log.error("Error is \n {}" , ArrayUtils.toString(e.getStackTrace()));
But instead of printing on new line, it just prints the newline character as-is:
Error is \n java.base. ..... ,\njava.something.....
So i want to know how i can include newline in logger... along with the json appender
logback-spring.xml:
<configuration>
<!-- logs to /tmp/spring.log by default -->
<property name="LOG_FILE" value="${LOG_FILE:-${LOG_PATH:-${LOG_TEMP:-${java.io.tmpdir:-/tmp}}/}spring.log}"/>
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
<include resource="org/springframework/boot/logging/logback/console-appender.xml"/>
<include resource="logstash-json-appender.xml"/>
<springProfile name="prod | uat | qa | int">
<root level="INFO">
<appender-ref ref="JSON_CONSOLE"/>
</root>
</springProfile>
<springProfile name="default | dev | test | junit">
<root level="INFO">
<appender-ref ref="CONSOLE"/>
</root>
</springProfile>
</configuration>
logstash-json-appender.xml
<included>
<!--
Defines a JSON formatted appender for both the console and log files.
For configuration details, see:
-->
<springProperty scope="context" name="appName" source="spring.application.name"/>
<appender name="JSON_CONSOLE" class="net.logstash.logback.appender.LoggingEventAsyncDisruptorAppender">
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
<jsonGeneratorDecorator class="net.logstash.logback.decorate.PrettyPrintingJsonGeneratorDecorator"/>
<providers>
<timestamp>
<timeZone>UTC</timeZone>
</timestamp>
<pattern>
<pattern>
{
"level": "%level",
"service": "${appName:-}",
"traceId": "%X{traceId:-}",
"spanId": "%X{spanId:-}",
"parentSpanId": "%X{parentSpanId:-}",
"pid": "${PID:-}",
"thread": "%thread",
"class": "%logger{40}",
"message": "%message"
}
</pattern>
</pattern>
</providers>
</encoder>
</appender>
</appender>
<appender name="JSON_FILE" class="net.logstash.logback.appender.LoggingEventAsyncDisruptorAppender">
<appender class="ch.qos.logback.core.rolling.RollingFileAppender">
<encoder class="net.logstash.logback.encoder.LogstashEncoder">
<jsonGeneratorDecorator class="net.logstash.logback.decorate.PrettyPrintingJsonGeneratorDecorator"/>
<timeZone>UTC</timeZone>
</encoder>
<!-- use the default spring boot conventions here, but leverage a different encoder -->
<file>${LOG_FILE}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>${LOGBACK_ROLLINGPOLICY_FILE_NAME_PATTERN:-${LOG_FILE}.%d{yyyy-MM-dd}.%i.gz}</fileNamePattern>
<cleanHistoryOnStart>${LOGBACK_ROLLINGPOLICY_CLEAN_HISTORY_ON_START:-false}</cleanHistoryOnStart>
<maxFileSize>${LOGBACK_ROLLINGPOLICY_MAX_FILE_SIZE:-10MB}</maxFileSize>
<totalSizeCap>${LOGBACK_ROLLINGPOLICY_TOTAL_SIZE_CAP:-0}</totalSizeCap>
<maxHistory>${LOGBACK_ROLLINGPOLICY_MAX_HISTORY:-7}</maxHistory>
</rollingPolicy>
</appender>
</appender>
</included>
I am trying to log the MDC values in stdout and in logstash.
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%-5level [%thread] %logger{0}: %msg -someName=%X{mdcFieldKey} %n</pattern>
<charset>utf8</charset>
</encoder>
</appender>
<appender name="logstash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
<destination>kibanaurl:port</destination>
<encoder class="net.logstash.logback.encoder.LogstashEncoder">
<customFields>
{
"application-name":"${spring.application.name}",
"someName":"${mdcFieldKey}",
"dummyName":"dummyValue"
}
</customFields>
<provider class="net.logstash.logback.composite.loggingevent.ArgumentsJsonProvider"/>
</encoder>
</appender>
Java code:
MDC.put("mdcFieldKey", "value to be displayed in kibana");
The STDOUT appender is displaying the mdc value correctly on kibana but for the logstash appender I see someName_IS_UNDEFINED. The hardcoded dummyname is also displayed correctly.
Why is the mdc value not populated in logstash custom field?
check here https://github.com/logstash/logstash-logback-encoder#mdc-fields
Add the line <includeMdcKeyName>mdcFieldKey</includeMdcKeyName> as below is one option. The field name will be same as mdc key in this case
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%-5level [%thread] %logger{0}: %msg -someName=%X{mdcFieldKey} %n</pattern>
<charset>utf8</charset>
</encoder>
</appender>
<appender name="logstash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
<destination>kibanaurl:port</destination>
<encoder class="net.logstash.logback.encoder.LogstashEncoder">
<includeMdcKeyName>mdcFieldKey</includeMdcKeyName>
<customFields>
{
"application-name":"${spring.application.name}",
"dummyName":"dummyValue"
}
</customFields>
<provider class="net.logstash.logback.composite.loggingevent.ArgumentsJsonProvider"/>
</encoder>
</appender>
I am using Asynappender and DOMConfigurator to load the xml file.
I am losing half of the logs. My application exits but only few logs appears in log file.
I have found that calling close() method of AsyncAppender class processes pending events before exiting.
Does this means that it flushes all the logs to log file? And if yes then, as I don't have that object as I am simply loading file using DOMConfigurator::configure().
How can I retrieve the Asynappender object to call close()?
Is there any other way to flush the logs before exiting? Using some configuration in xml file?
Below is my code for reference:
#include <log4cxx/logger.h>
#include <log4cxx/xml/domconfigurator.h>
#include<iostream>
using namespace log4cxx;
using namespace log4cxx::xml;
using namespace log4cxx::helpers;
LoggerPtr loggerMyMain(Logger::getLogger( "main"));
int main(int args, char **argv)
{
DOMConfigurator::configure("asynclog4cxxconfig.xml");
LOG4CXX_TRACE(perf, "this is a performance message!!!");
LOG4CXX_DEBUG(loggerMyMain, "this is a debug message.");
LOG4CXX_WARN (loggerMyMain, "this is a warn message, not too bad.");
LOG4CXX_ERROR(loggerMyMain, "this is a error message, something serious is happening.");
LOG4CXX_FATAL(loggerMyMain, "this is a fatal message!!!");
return 0;
}
Below is the XML file:
<?xml version="1.0" encoding="UTF-8" ?>
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
<appender name="appxNormalAppender" class="org.apache.log4j.RollingFileAppender">
<param name="file" value="logfile" />
<param name="MaxFileSize" value="1000KB" />
<param name="MaxBackupIndex" value="3" />
<param name="append" value="true" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%X{process} %d %-5p %C{2} (%F:%L) - %m%n" />
</layout>
</appender>
<appender name="async_appxNormalAppender" class="org.apache.log4cxx.AsyncAppender">
<appender-ref ref="appxNormalAppender"/>
</appender>
<root>
<priority value="debug" />
<appender-ref ref="async_appxNormalAppender"/>
</root>
</log4j:configuration>
The problem got solved, posting it here if it helps someone.
Added the below piece of code at the end.
LoggerPtr root_logger = Logger::getRootLogger();
AppenderPtr app_ptr = root_logger->getAppender("async_appxNormalAppender");
if(app_ptr != NULL)
app_ptr->close();
I am attempting to configure NLog to output error logs in an XML format, however, NLog is not properly loading the configuration.
Here is the LAYOUT portion of my configuration:
<layout xsi:type="XmlLayout" includeAllProperties="true" propertiesElementKeyAttribute="" propertiesElementName="{0}" elementName="errorlog">
<attribute name="time" layout="${longdate}" />
<attribute name="level" layout="${level:upperCase=true}" />
<element name="processname" value="${processname}" />
<element name="logger" value="${logger}" />
<element name="message" value="${message}" />
</layout>
When calling the LogManager.GetCurrentClassLogger() method, the following error is generated:
Parameter name not supported on XmlLayout
Here are the additional details captured in the NLog error log file:
2019-03-04 16:25:39.9645 Trace Scanning Property Renderers 'System.Collections.ObjectModel.ReadOnlyCollection`1[NLog.LayoutRenderers.LayoutRenderer]' System.Collections.ObjectModel
2019-03-04 16:25:39.9645 Trace Scanning LevelLayoutRenderer 'Layout Renderer: ${level}'
2019-03-04 16:25:39.9645 Debug Setting 'XmlLayout.name' to 'processname'
2019-03-04 16:25:39.9835 Warn Error when setting 'processname' on attibute 'name' Exception: System.NotSupportedException: Parameter name not supported on XmlLayout
at NLog.Internal.PropertyHelper.SetPropertyFromString(Object obj, String propertyName, String value, ConfigurationItemFactory configurationItemFactory)
at NLog.Config.LoggingConfigurationParser.ConfigureObjectFromAttributes(Object targetObject, ILoggingConfigurationElement element, Boolean ignoreType)
2019-03-04 16:25:39.9835 Error Parsing configuration from C:\Users\<user>\Documents\Visual Studio 2017\Projects\TestingNlog\TestingNlog\bin\Debug\NLog.config failed. Exception: NLog.NLogConfigurationException: Exception when parsing C:\Users\<user>\Documents\Visual Studio 2017\Projects\TestingNlog\TestingNlog\bin\Debug\NLog.config. ---> System.NotSupportedException: Parameter name not supported on XmlLayout
If I remove the ELEMENT tags from the configuration file, I am able to generate a partial log file with just the single XML element ERRORLOG with the 2 specified attributes. However, I have not found a way to include the ELEMENT tags in my output log files.
NLog.dll - v4.6.0.9068
C# .NET - v4.7.2
Any ideas or assistance would be greatly appreciated!
-Kasey911
Snakefoot is correct,
this has been changed after NLog 4.6 RC 1
In NLog 4.6 RC 1
<layout xsi:type="XmlLayout" elementName='logevent'>
<attribute name="time" layout="${longdate}" />
<attribute name="level" layout="${level:upperCase=true}"/>
<element elementName="message" elementValue="${message}" />
</layout>
In NLog 4.6 RC 2 (not released yet), and probably RTM
<layout xsi:type="XmlLayout" elementName='logevent'>
<attribute name="time" layout="${longdate}" />
<attribute name="level" layout="${level:upperCase=true}"/>
<element name="message" value="${message}" />
</layout>
Can you point me to some log4net tutorial or samples that logs all unhandled exceptions occurring on my MVC Web Api when using OWIN?
I have a Startup class that configures Castle Windsor and OAuth and I'm not sure how application errors are handled for OWIN enabled applications.
I have tried putting the log4net code in the Application_Error event in global.asax but it never gets executed.
public class WebApiApplication : System.Web.HttpApplication {
protected void Application_Start() {
log4net.Config.XmlConfigurator.Configure();
}
protected void Application_Error(object sender,EventArgs e) {
Exception lastException = Server.GetLastError();
var logger = log4net.LogManager.GetLogger(typeof(WebApiApplication));
logger.Fatal(lastException);
}
}
Any ideas why this is?
EDIT: web.config with log4net configuration:
<configSections>
<section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net" />
</configSections>
<log4net debug="true">
<logger name="SiteWorksAPI">
<level value="Debug" />
<appender-ref ref="EventLogAppender" />
</logger>
<appender name="FileAppender" type="log4net.Appender.FileAppender">
<file value="C:\ProgramData\Site\SiteWorksAPILog.txt" />
<appendToFile value="true" />
<lockingModel type="log4net.Appender.FileAppender+MinimalLock" />
<immediateFlush value="true" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="============================================================%nLogger = %logger.%nLocation = %location.%nUnique Token = %property{x-fcc-uniquetoken}.%nDate = %date{dd MMM yyyy - HH:mm:ss}.%nMessage = %message%n" />
</layout>
</appender>
<appender name="EventLogAppender" type="log4net.Appender.EventLogAppender">
<applicationName value="SiteWorks.API" />
<logname value="SiteWorks API Log" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="Date: %date{dd MMM yyyy - HH:mm:ss} %nMessage: %message %nIdentity: %identity %nUnique Token: %property{x-fcc-uniquetoken} %nChange Time Stamp: %property{x-fcc-changedatetime} %nOriginal Time Stamp: %property{x-fcc-originaltimestamp} %nOriginating System: %property{x-fcc-originatingsystem} %nOriginating User: %property{x-fcc-originatingusername} %nRequest Uri: %property{Request} %nResponse Headers: %property{ResponseHeaders}%nLogger: %logger %nThread: %thread %n%n%exception" />
</layout>
</appender>
</log4net>
You're not seeing any log events because you haven't defined a root logger in your config - the only logger you've defined is called SiteWorksAPI so any loggers you create with any other name - for example, the one named after typeof(WebApiApplication) - will not log anything. If you add a breakpoint on the logging statement you'll see that all log levels are disabled.
The appenders defined in a root logger are inherited by all named appenders (unless the child appenders are defined with additivity = false) - but child appenders set their own log level.
<root>
<level value="Debug" />
<appender-ref ref="EventLogAppender" />
</root>
Edit: there's also another thing to consider - your defined logger only uses the EventLogAppender - note that in order to log to the Event Log, the documentation states EventLogAppender will fail if you try to write using an event source that doesn't exist unless it is running with local administrator privileges but the event source needs to be created by an elevated process.
For testing purposes, you might want to add your FileAppender to the logger.