NLog difference between overflowAction Grow and Block - nlog

I am using NLog in my .Net window service and below are the configuration that i am using for setup Nlog.
<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.nlog-project.org/schemas/NLog.xsd NLog.xsd"
autoReload="true"
throwExceptions="false"
internalLogLevel="Debug" internalLogFile="c:\temp\log.txt">
<targets>
<target xsi:type="AsyncWrapper" name="asyncExceptionDB" overflowAction="Grow" >
<target xsi:type="FallbackGroup" name="String" returnToFirstOnSuccess="true">
<target xsi:type="Database" name="ExceptionDB" dbProvider="System.Data.SqlClient" connectionStringName="nlogconnection" commandType="StoredProcedure" commandText="usp_InsertExceptionLog">
<parameter name="#UserId" layout="${event-properties:item=userId}"/>
<parameter name="#Email" layout="${event-properties:item=email}"/>
<parameter name="#ExceptionType" layout="${event-properties:item=exceptionType}" />
<parameter name="#ExceptionMessage" layout="${event-properties:item=exceptionMessage}" />
<parameter name="#InnerException" layout="${event-properties:item=innerException}" />
<parameter name="#CustomMessage" layout="${message}"/>
<parameter name="#ErrorCode" layout="${event-properties:item=errorCode}" />
</target>
</target>
</target>
<target xsi:type="AsyncWrapper" name="asyncLogDB" overflowAction="Block" >
<target xsi:type="FallbackGroup" name="String" returnToFirstOnSuccess="true">
<target xsi:type="Database" name="LogDB" dbProvider="System.Data.SqlClient" connectionStringName="nlogconnection" commandType="StoredProcedure" commandText="usp_InsertLog">
<parameter name="#UserId" layout="${event-properties:item=userId}"/>
<parameter name="#Email" layout="${event-properties:item=email}"/>
<parameter name="#CustomMessage" layout="${message}"/>
<parameter name="#MeetingId" layout="${event-properties:item=meetingId}" />
<parameter name="#ErrorCode" layout="${event-properties:item=errorCode}" />
</target>
</target>
</target>
<rules>
<!-- add your logging rules here -->
<!-- DO NOT CHANGE NAME HERE (Used in Code) -->
<logger name="SOLog" level="Info" writeTo="asyncLogDB" />
<logger name="SOLog" level="Error" writeTo="asyncExceptionDB" />
<logger name="SOLog" level="Info" writeTo="asyncOHLoggerService" />
<logger name="SOLog" level="Error" writeTo="asyncOHLoggerService" />
<!--<logger name="*" level="Error" writeTo="ExceptionDB" />
<logger name="*" level="Fatal" writeTo="ExceptionDB" />-->
</rules>
</targets>
</nlog>
In my configuration i am using overflow Action value as Grow and Block for different targets. An both work fine in my case as they both write data to there desired destination. But i really didn't understood the difference between both the action Grow and Block. Can any one tell me the actual difference between these two (overflow action Block and Grow) actually the Background details that how they work in background.
I am aware about Grow that it will increase the queue size when it reaches its peak point but one problem of using it in my case with above configuration is that it stop my service and throw an error called OutOfMemoryException and stopped my service after that.
after that i used Block rather then using Grow and my memory utilization is fine in this case and all works fine for me. but the only question that arise in my mind that how come Block is different than Grow. I went through the NLOG documentation but that really didn't help me much to understand the concept of these two i.e Grow and Block.

Have updated the Wiki-page for AsyncWrapper-Target:
overflowAction - Action to be taken when in-memory-queue becomes full (Reached queueLimit). This means the background-writer is falling behind, and cannot keep up with the application-threads logging. Default: Discard
Possible values:
Discard - The application-thread will discard logevent to avoid becoming blocked or causing out-of-memory issues
Block - The application-thread will block until the background-writer-thread has taken the next batch. Avoids loosing important logevents, but can block all application-threads.
Grow - The application-thread will ignore the limit, and will just allocate more memory. Can cause the entire application to experience out-of-memory-issues.

Related

NLog Logs not logging

I'm new to NLog and I'm attempting to use it in a new app I'm writing...
Below is my NLog config.
When debugging in VS I get the logs I expect, but when I build and run the program outside of VS I don't see any logs.
Any ideas as to why?
<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.nlog-project.org/schemas/NLog.xsd NLog.xsd"
autoReload="true"
throwExceptions="true"
internalLogLevel="Off" internalLogFile="c:\temp\nlog-internal.log">
<!-- optional, add some variables
https://github.com/nlog/NLog/wiki/Configuration-file#variables
-->
<variable name="myvar" value="myvalue"/>
<!--
See https://github.com/nlog/nlog/wiki/Configuration-file
for information on customizing logging rules and outputs.
-->
<targets>
<!--
add your targets here
See https://github.com/nlog/NLog/wiki/Targets for possible targets.
See https://github.com/nlog/NLog/wiki/Layout-Renderers for the possible layout renderers.
-->
<!--Write events to a file with the date in the filename.-->
<target xsi:type="ColoredConsole" name="ColoredConsole" useDefaultRowHighlightingRules="true"
layout="${message}" />
<target xsi:type="File" name="ConsilioDeployTool_CLI_ErrorLog" fileName="${basedir}/logs/ConsilioDeployTool_CLI_ErrorLog_${shortdate}.log">
<layout xsi:type="CsvLayout" delimiter="Pipe" withHeader="true">
<column name="level" layout="${level:upperCase=true}"/>
<column name="callsite" layout="${callsite:includeSourcePath=true}" />
<column name="stacktrace" layout="${stacktrace:topFrames=10}" />
<column name="exception" layout="${exception:format=ToString}"/>
<column name="logger" layout="${logger:shortName=false}"/>
<column name="message" layout="${message}" />
</layout>
</target>
</targets>
<rules>
<!-- add your logging rules here -->
<!-- Write all events with minimal level of Debug (So Debug, Info, Warn, Error and Fatal, but not Trace) to "f" -->
<logger name="*" level="Info" writeTo="ColoredConsole" />
<!--<logger name="*" level="Error" writeTo="ConsilioDeployTool_CLI_ErrorLog" />-->
<logger name="*" level="Error" writeTo="ConsilioDeployTool_CLI_ErrorLog" />
</rules>
</nlog>
Make sure the user IIS_IUSRS has write permissions on the site folder
You can assign write permission by doing the followin:
In windows explorer right click on the site folder
select Properties
click on the Security tab
Click on Edit...
under Group or user names:, select IIS_IUSRS
Under Permissions for IIS_IUSRS, click on Write checkbox to allow the user write to the site folder
Click Apply and then OK and OK again to close the Properties window
I hope that helps

Asynchronous batch logging using Nlog and Asp.net core 2.0 in database

I am using NLog for logging in asp.net core 2.0. I am storing my log events in database.
I want my messages to save in batches asynchronously.
Nlog.config settings are:
<?xml version="1.0" encoding="UTF-8"?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" autoReload="true"
internalLogLevel="Warn" internalLogFile="c:\temp\internal-nlog.txt">
<extensions>
<add assembly="NLog.Web.AspNetCore" />
</extensions>
<targets>
<target name="file" xsi:type="AsyncWrapper" queueLimit="5000" batchSize="5" overflowAction="Discard">
<target name="db" xsi:type="Database"
connectionString="Data Source=.;Initial Catalog=DatabaseName;User Id=userId;Password=pwd;"
commandType="StoredProcedure" commandText="Sp_name" dbProvider="System.Data.SqlClient">
<parameter name="#TimeStamp" layout="${date}" />
<parameter name="#Level" layout="${uppercase:${level}}" />
<parameter name="#Message" layout="${message}" />
<parameter name="#Exception" layout="${exception}" />
<parameter name="#StackTrace" layout="${exception:format=StackTrace,Data:maxInnerExceptionLevel=10}" />
<parameter name="#eventLookupId" layout ="1" />
</target>
</target>
</targets>
<rules>
<logger name="Microsoft.*" minlevel="Info" writeTo="" final="true" />
<logger name="*" minlevel="Info" writeTo="file" />
</rules>
</nlog>
For the above configuration I am expecting it to log in a batch size of 5 but it is logging immediately at the trigger of event rather than logging in a batch.
Not sure if any other configuration is also required to make it work.
NLog will sent the messages continuously with of max of the batchsize (5 here), and with a time between batches. The default "timeToSleepBetweenBatches" for the asyncWrapper is 1 ms.
e.g. when the batchsize is 100 and the timeToSleepBetweenBatches is 1 sec, a max of 6000 messages/minute are sent.
See the options: https://github.com/NLog/NLog/wiki/AsyncWrapper-target#buffering-options

NLog DB Configuration

From the docs https://github.com/NLog/NLog/wiki/Database-target
It seems settings are shown as attributes on the target element such as:
<target xsi:type="Database"
name="String"
dbUserName="Layout"
dbProvider="String"
and in the example below as separate child nodes:
<target name="database" xsi:type="Database">
<connectionStringName>NLogDb</connectionStringName>
Neither work for me, I just get Invalid configuration exceptions with this message:
NotSupportedException: Parameter connectionStringName not supported on DatabaseTarget
The Config File:
<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
autoReload="true"
internalLogLevel="info"
throwExceptions="true"
internalLogFile="c:\temp\internal-nlog.txt">
<!-- enable asp.net core layout renderers -->
<extensions>
<add assembly="NLog.Web.AspNetCore"/>
</extensions>
<!-- the targets to write to -->
<targets>
<!-- write logs to file -->
<target xsi:type="File" name="allfile" fileName="c:\temp\nlog-all-${shortdate}.log"
layout="${longdate}|${event-properties:item=EventId_Id}|${uppercase:${level}}|${logger}|${message} ${exception:format=tostring}" />
<!-- another file log, only own logs. Uses some ASP.NET core renderers -->
<target xsi:type="File" name="ownFile-web" fileName="c:\temp\nlog-own-${shortdate}.log"
layout="${longdate}|${event-properties:item=EventId_Id}|${uppercase:${level}}|${logger}|${message} ${exception:format=tostring}|url: ${aspnet-request-url}|action: ${aspnet-mvc-action}" />
<target xsi:type="Database"
name="database"
keepConnection="true"
useTransactions="true"
dbProvider="System.Data.SqlClient"
connectionStringName="DefaultConnection"
commandText="INSERT INTO Logs (EventDateTime, EventLevel, UserName, MachineName, EventMessage, ErrorSource, ErrorClass, ErrorMethod, ErrorMessage, InnerErrorMessage) VALUES (#EventDateTime, #EventLevel, #UserName, #MachineName, #EventMessage, #ErrorSource, #ErrorClass, #ErrorMethod, #ErrorMessage, #InnerErrorMessage)">
<parameter name="#EventDateTime" layout="${date:s}" />
<parameter name="#EventLevel" layout="${level}" />
<parameter name="#UserName" layout="${aspnet-user-identity}" />
<parameter name="#MachineName" layout="${machinename}" />
<parameter name="#EventMessage" layout="${message}" />
<parameter name="#ErrorSource" layout="${event-context:item=error-source}" />
<parameter name="#ErrorClass" layout="${event-context:item=error-class}" />
<parameter name="#ErrorMethod" layout="${event-context:item=error-method}" />
<parameter name="#ErrorMessage" layout="${event-context:item=error-message}" />
<parameter name="#InnerErrorMessage" layout="${event-context:item=inner-error-message}" />
</target>
</targets>
<!-- rules to map from logger name to target -->
<rules>
<!--All logs, including from Microsoft-->
<logger name="*" minlevel="Trace" writeTo="allfile,database" />
<!--Skip non-critical Microsoft logs and so log only own logs-->
<logger name="Microsoft.*" maxLevel="Info" final="true
" />
<!-- BlackHole without writeTo -->
<logger name="*" minlevel="Trace" writeTo="ownFile-web" />
</rules>
</nlog>
How it is being called in program.cs
var logger = NLog.Web.NLogBuilder.ConfigureNLog( "nlog.config" ).GetCurrentClassLogger();
(copied from their docs)
Must be missing something obvious, but since there is conflicting info in the docs, and copying other people's configs posted on here, not sure where to go with it
Looks like you are running on NetCore. NLog is not able to read connectionStringName from AppSettings.json as you have found out yourself (Requires extra dependencies to access IConfiguration).
One possible solution is using this extension:
https://www.nuget.org/packages/NLog.Appsettings.Standard/
And use connectionString (Instead of connectionStringName) in NLog.config:
<extensions>
<add assembly="NLog.Web.AspNetCore"/>
<add assembly="NLog.Appsettings.Standard"/>
</extensions>
<target xsi:type="Database" connectionString="${appsettings:name=ConnectionStrings.DefaultConnection}">
Alternative solution is to assign a GDC variable before logging:
NLog.GlobalDiagnosticsContext.Set("DefaultConnection", Configuration.GetConnectionString("DefaultConnection"));
And then use GDC in NLog.config:
<target xsi:type="Database" connectionString="${gdc:item=DefaultConnection}">
See also https://github.com/NLog/NLog/wiki/Gdc-layout-renderer
Update NLog.Extension.Logging ver. 1.4.0
With NLog.Extension.Logging ver. 1.4.0 then you can now use ${configsetting}
See also: https://github.com/NLog/NLog/wiki/ConfigSetting-Layout-Renderer

How do I make aspnet-request available in my NLog config?

I am trying to add the ip address to logging. I installed the NLog.Extended package and ensured that the NLog.Extended.dll is present in my bin next to the base NLog.dll. I have added the variable "${aspnet-request:serverVariable=remote_addr}" to my layout renderer. I get a generic error saying:
An error occurred creating the configuration section handler for nlog: Exception occurred when loading configuration from [my Web.config]
Here is my NLog.config:
<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<extensions>
<add assembly="NLog.Extended" />
</extensions>
<targets async="true">
<target name="fileLog" xsi:type="File" fileName="${basedir}/logs/${shortdate}.log" layout="${aspnet-request:serverVariable=remote_addr} ${longdate} ${callsite} ${level} ${message} ${exception:format=ToString}" />
<target name="dbLog" xsi:type="Database" connectionStringName="db.data" commandText="insert into log ([Date], [Origin], [LogLevel], [Message], [Exception], [StackTrace]) values (#date, #origin, #logLevel, #message, #exception, #stackTrace)">
<parameter name="#date" layout="${date}"/>
<parameter name="#origin" layout="${callsite}"/>
<parameter name="#logLevel" layout="${level}"/>
<parameter name="#message" layout="${message}"/>
<parameter name="#exception" layout="${exception:format=Message,StackTrace}"/>
<parameter name="#stackTrace" layout="${stacktrace}"/>
</target>
</targets>
<rules>
<logger name="*" minlevel="Trace" writeTo="dbLog" />
<logger name="*" minlevel="Trace" writeTo="fileLog" />
</rules>
</nlog>
When I remove the AspNetRequest variable it doesn't complain. I've tried replacing "remote_addr" with "remote_host" with no change. This project is using NLog 2.1.0. Any ideas on what I'm doing wrong here?
If you're using NLog 4.0 then you'll need NLog.Web to use the aspnet=* layout renderers.
It turned out that the NLog.Extended.dll(3.1.0.0) had to match the NLog.dll it was referencing. I upgraded the NLog.dll to 3.1.0.0 and everything is working now.
In my ASP.NET webapi 2.0 solution I had a separate projects for my API's and logging. Using nLog 4.4.12 installed through NuGet and all the helper packages (Config,Extended,Web,Schema) in the logging project.
The "aspnet-request" properties in my config would not work until I added the nLog references to my api project, even though "Copy Local" is set to true for all the nLog DLL's in the logging project. My nLog section from my web.config is below.
Hope this help someone,
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<targets>
<target name="db" xsi:type="Database" connectionStringName="NLogConn" commandType="StoredProcedure" commandText="[dbo].[NLog_AddEntry_p]">
<parameter name="#machineName" layout="${machinename}" />
<parameter name="#siteName" layout="${iis-site-name}" />
<parameter name="#logged" layout="${date}" />
<parameter name="#level" layout="${level}" />
<parameter name="#username" layout="${aspnet-user-identity}" />
<parameter name="#message" layout="${message}" />
<parameter name="#logger" layout="${logger}" />
<parameter name="#properties" layout="${all-event-properties:separator=|}" />
<parameter name="#serverName" layout="${aspnet-request:serverVariable=SERVER_NAME}" />
<parameter name="#port" layout="${aspnet-request:serverVariable=SERVER_PORT}" />
<parameter name="#url" layout="${aspnet-request:serverVariable=HTTP_URL}" />
<parameter name="#https" layout="${when:inner=1:when='${aspnet-request:serverVariable=HTTPS}' == 'on'}${when:inner=0:when='${aspnet-request:serverVariable=HTTPS}' != 'on'}" />
<parameter name="#serverAddress" layout="${aspnet-request:serverVariable=LOCAL_ADDR}" />
<parameter name="#remoteAddress" layout="${aspnet-request:serverVariable=REMOTE_ADDR}:${aspnet-request:serverVariable=REMOTE_PORT}" />
<parameter name="#callSite" layout="${callsite:fileName=true:includeSourcePath=false:skipFrames=1}" />
<parameter name="#exception" layout="${exception:tostring}" />
</target>
</targets>
<rules>
<logger name="*" minlevel="Trace" writeTo="db" />
</rules>

Transform that preserve namespace prefixes

I'm trying to insert an NLog custom config section into my Web.config using this XDT section:
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd" throwExceptions="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xdt:Transform="InsertIfMissing" >
<targets>
<target xsi:type="File" name="logfile" fileName="H:\testLog.txt" layout="${longdate} ${uppercase:${level}} ${message}" />
</targets>
<rules>
<logger name="*" minlevel="Trace" writeTo="logfile" />
</rules>
</nlog>
When I run the XDT transform, my Web.Debug.config contains:
<nlog throwExceptions="true" xmlns="http://www.nlog-project.org/schemas/NLog.xsd">
<targets>
<target d4p1:type="File" name="logfile" fileName="H:\testLog.txt" layout="${longdate} ${uppercase:${level}} ${message}" xmlns:d4p1="http://www.w3.org/2001/XMLSchema-instance" />
</targets>
<rules>
<logger name="*" minlevel="Trace" writeTo="logfile" />
</rules>
</nlog>
Typically, a namespace prefix is arbitrary, so transforming xsi to d4p1 would cause no issues.
However, I get a runtime exception in my application from NLog when I use d4p1. Manually changing the instances of d4p1 to xsi fixes the issue, but it subverts the utility of config transformation for me if the user needs to manually alter the file afterward.
Is there a way to preserve namespace prefixes using XDT?
We had exactly the same issue. I'm not sure why it suddenly started happening with a particular project, but the solution for us was to add the xsi namespace to the top level of the original configuration file (ie the base file the transformations work on). So...
<configuration>
... would become...
<configuration xmlns:xsi="http://www.nlog-project.org/schemas/NLog.xsd">
This did the trick.
An alternative approach that also worked was to apply the transforms on child elements of the nlog element.
Hope that helps someone.
I started to see this problem when I moved my xdt:Transform attribute from the target and rule tags to nlog. Moving them back to the original tags like this solved it:
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd" throwExceptions="true" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<targets xdt:Transform="InsertIfMissing">
<target xsi:type="File" name="logfile" fileName="H:\testLog.txt" layout="${longdate} ${uppercase:${level}} ${message}" />
</targets>
<rules xdt:Transform="InsertIfMissing">
<logger name="*" minlevel="Trace" writeTo="logfile" />
</rules>
</nlog>

Resources