<target xsi:type="File"
async="true"
name="jsonFileTrace"
keepFileOpen="true"
OpenFileCacheTimeout="60"
archiveAboveSize="5242880"
maxArchiveFiles="20"
maxArchiveDays="10"
archiveNumbering="DateAndSequence"
AutoFlush="false"
openFileFlushTimeout="5"
ConcurrentWrites="false"
archiveFileName="${basedir}/logs/trace/archive/${shortdate}/trace.{#}.log"
fileName="${basedir}/logs/trace/${shortdate}.log">
<layout xsi:type="JsonLayout" includeAllProperties="true">
<attribute name="time" layout="${longdate}" />
.......//other attributes
</layout>
</target>
If I configed like above, it can be archived by date. But the folder 2022-09-20 cannot be deleted if current date is 2022-10-01 which already 10 days past since 2022-09-20
Consider using this configuration instead:
<target xsi:type="File" name="jsonFileTrace"
keepFileOpen="true"
archiveAboveSize="5242880"
autoFlush="false"
openFileFlushTimeout="5"
concurrentWrites="false"
fileName="${basedir}/logs/trace/${shortdate}/trace.${shortdate}.log">
<layout xsi:type="JsonLayout" includeAllProperties="true">
<attribute name="time" layout="${longdate}" />
.......//other attributes
</layout>
</target>
And setup a scheduled-task on the machine (or in the application) that runs every day and removes sub-folders that are older than 10 days.
Related
I want to keep last 7 days of files using nlog, but for some reason the following configuration in target=LogToShare is keeping only the latest file:
<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<variable name="WorkflowLoggingDirectory" value="${specialfolder:folder=LocalApplicationData}/Logs" />
<variable name="LogSharedFolder" value="\\myserver\logs" />
<rules>
<logger name="WorkflowLogging" writeTo="LogToShare" />
<logger name="WorkflowLogging" writeTo="WorkflowLogFiles" final="true" />
<logger minLevel="Info" writeTo="EventLog" />
</rules>
<targets>
<target type="File"
name="LogToShare"
layout="${time} ${level} ${message}"
fileName="${LogSharedFolder}\${machinename}_${shortdate}.log"
archiveFileName="${LogSharedFolder}\${machinename}_${shortdate}.{#}.log"
archiveEvery="Day"
archiveNumbering="Rolling"
maxArchiveFiles="7"
keepFileOpen="true"
openFileCacheTimeout="5"
concurrentWrites="true"
encoding="utf-8"
writeBom="true" />
<target type="File" name="WorkflowLogFiles" fileName="${WorkflowLoggingDirectory}/${shortdate}_Execution.log" layout="${time} ${level} ${message}" keepFileOpen="true" openFileCacheTimeout="5" concurrentWrites="true" encoding="utf-8" writeBom="true" />
<target type="EventLog" name="EventLog" layout="${processname} ${assembly-version} ${newline}${message}" source="UiPath" log="Application" />
</targets>
</nlog>
The files in \\myserver\logs that are kept are only the files from the current day:
machine1_2020-08-09.log
machine2_2020-08-09.log
machine3_2020-08-09.log
machine4_2020-08-09.log
Tomorrow, the files that I will get are:
machine1_2020-08-10.log
machine2_2020-08-10.log
machine3_2020-08-10.log
machine4_2020-08-10.log
And the files from 2020-08-09 will be removed.
I'm using NLog v4.3.9. How can I keep the last 7 days of log?
When using NLog 4.5 (or newer) then you can do this:
<target type="File"
name="LogToShare"
layout="${time} ${level} ${message}"
fileName="${LogSharedFolder}/${machinename}_${shortdate}.log"
maxArchiveFiles="7"
keepFileOpen="true"
openFileCacheTimeout="5"
concurrentWrites="true"
encoding="utf-8"
writeBom="true" />
When using older NLog versions, then you can try this:
<target type="File"
name="LogToShare"
layout="${time} ${level} ${message}"
fileName="${LogSharedFolder}/${machinename}_${date:format=yyyy-MM-dd}.log"
archiveFileName="${LogSharedFolder}/${machinename}_{#}.log"
archiveDateFormat="yyyy-MM-dd"
archiveNumbering="Date"
archiveEvery="Year"
maxArchiveFiles="7"
keepFileOpen="true"
openFileCacheTimeout="5"
concurrentWrites="true"
encoding="utf-8"
writeBom="true" />
Explanation of why the extra parameters are needed:
archiveFileName - Using {#} allows the archive cleanup to generate proper file wildcard.
archiveDateFormat - Must match the ${date:format=} of the fileName (So remember to correct both date-formats, if change is needed)
archiveNumbering=Date - Configures the archive cleanup to support parsing of filenames as dates.
archiveEvery=Year - Activates the archive cleanup, but also the archive file operation. Because the configured fileName automatically ensures the archive file operation, then we don't want any additional archive operations (Ex. avoiding generating extra empty files at midnight).
maxArchiveFiles - How many archive files to keep around.
I have the setting of NLog below:
<target name="asyncFile" xsi:type="AsyncWrapper" queueLimit="10000" overflowAction="Block">
<target xsi:type="File"
fileName="/storage/sdcard0/com.my.app/Logs/${shortdate}.log"
archiveFileName="/storage/sdcard0/com.my.app/Logs/archived.{#}.log"
maxArchiveFiles="5"
archiveDateFormat="yyyy-MM-dd"
archiveAboveSize="1048576"
archiveEvery="Day"
archiveNumbering="DateAndSequence"
concurrentWrites="true"
keepFileOpen="false">
<layout xsi:type="JsonLayout" includeAllProperties="true" excludeProperties="Comma-separated list (string)">
<attribute name="time" layout="${longdate}" />
<attribute name="level" layout="${level:upperCase=true}"/>
<attribute name="message" layout="${message}" />
<attribute name="exception" layout="${exception}" />
<attribute name="threadid" layout="${threadid}" />
</layout>
</target>
</target>
Expected behaviours:
No more than 5 archived files are generated, and each has a maximum size of 1MB.
archived.1.log
archived.2.log
archived.3.log
archived.4.log
archived.5.log
Actual behaviours:
One archived file has more 40 MB
Any idea?
Using ${shortdate} in fileName is not supported together with archiveFileName.
fileName="/storage/sdcard0/com.my.app/Logs/${shortdate}.log"
Solution 1 (New style that works with NLog ver. 4.5 and newer)
<target xsi:type="File"
fileName="/storage/sdcard0/com.my.app/Logs/App.${shortdate}.log"
maxArchiveFiles="5"
archiveAboveSize="1048576"
concurrentWrites="true"
keepFileOpen="false">
Solution 2: (Remove ${shortdate} from NLog so it works as intended):
<target xsi:type="File"
fileName="/storage/sdcard0/com.my.app/Logs/ActiveLog.log"
archiveFileName="/storage/sdcard0/com.my.app/Logs/archived.{#}.log"
maxArchiveFiles="5"
archiveDateFormat="yyyy-MM-dd"
archiveAboveSize="1048576"
archiveEvery="Day"
archiveNumbering="DateAndSequence"
concurrentWrites="true"
keepFileOpen="false">
See also: https://github.com/NLog/NLog/wiki/File-target#archive-old-log-files
See also: https://github.com/NLog/NLog/wiki/FileTarget-Archive-Examples
I am using NLog for logging in asp.net core 2.0. I am storing my log events in database.
I want my messages to save in batches asynchronously.
Nlog.config settings are:
<?xml version="1.0" encoding="UTF-8"?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" autoReload="true"
internalLogLevel="Warn" internalLogFile="c:\temp\internal-nlog.txt">
<extensions>
<add assembly="NLog.Web.AspNetCore" />
</extensions>
<targets>
<target name="file" xsi:type="AsyncWrapper" queueLimit="5000" batchSize="5" overflowAction="Discard">
<target name="db" xsi:type="Database"
connectionString="Data Source=.;Initial Catalog=DatabaseName;User Id=userId;Password=pwd;"
commandType="StoredProcedure" commandText="Sp_name" dbProvider="System.Data.SqlClient">
<parameter name="#TimeStamp" layout="${date}" />
<parameter name="#Level" layout="${uppercase:${level}}" />
<parameter name="#Message" layout="${message}" />
<parameter name="#Exception" layout="${exception}" />
<parameter name="#StackTrace" layout="${exception:format=StackTrace,Data:maxInnerExceptionLevel=10}" />
<parameter name="#eventLookupId" layout ="1" />
</target>
</target>
</targets>
<rules>
<logger name="Microsoft.*" minlevel="Info" writeTo="" final="true" />
<logger name="*" minlevel="Info" writeTo="file" />
</rules>
</nlog>
For the above configuration I am expecting it to log in a batch size of 5 but it is logging immediately at the trigger of event rather than logging in a batch.
Not sure if any other configuration is also required to make it work.
NLog will sent the messages continuously with of max of the batchsize (5 here), and with a time between batches. The default "timeToSleepBetweenBatches" for the asyncWrapper is 1 ms.
e.g. when the batchsize is 100 and the timeToSleepBetweenBatches is 1 sec, a max of 6000 messages/minute are sent.
See the options: https://github.com/NLog/NLog/wiki/AsyncWrapper-target#buffering-options
From the docs https://github.com/NLog/NLog/wiki/Database-target
It seems settings are shown as attributes on the target element such as:
<target xsi:type="Database"
name="String"
dbUserName="Layout"
dbProvider="String"
and in the example below as separate child nodes:
<target name="database" xsi:type="Database">
<connectionStringName>NLogDb</connectionStringName>
Neither work for me, I just get Invalid configuration exceptions with this message:
NotSupportedException: Parameter connectionStringName not supported on DatabaseTarget
The Config File:
<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
autoReload="true"
internalLogLevel="info"
throwExceptions="true"
internalLogFile="c:\temp\internal-nlog.txt">
<!-- enable asp.net core layout renderers -->
<extensions>
<add assembly="NLog.Web.AspNetCore"/>
</extensions>
<!-- the targets to write to -->
<targets>
<!-- write logs to file -->
<target xsi:type="File" name="allfile" fileName="c:\temp\nlog-all-${shortdate}.log"
layout="${longdate}|${event-properties:item=EventId_Id}|${uppercase:${level}}|${logger}|${message} ${exception:format=tostring}" />
<!-- another file log, only own logs. Uses some ASP.NET core renderers -->
<target xsi:type="File" name="ownFile-web" fileName="c:\temp\nlog-own-${shortdate}.log"
layout="${longdate}|${event-properties:item=EventId_Id}|${uppercase:${level}}|${logger}|${message} ${exception:format=tostring}|url: ${aspnet-request-url}|action: ${aspnet-mvc-action}" />
<target xsi:type="Database"
name="database"
keepConnection="true"
useTransactions="true"
dbProvider="System.Data.SqlClient"
connectionStringName="DefaultConnection"
commandText="INSERT INTO Logs (EventDateTime, EventLevel, UserName, MachineName, EventMessage, ErrorSource, ErrorClass, ErrorMethod, ErrorMessage, InnerErrorMessage) VALUES (#EventDateTime, #EventLevel, #UserName, #MachineName, #EventMessage, #ErrorSource, #ErrorClass, #ErrorMethod, #ErrorMessage, #InnerErrorMessage)">
<parameter name="#EventDateTime" layout="${date:s}" />
<parameter name="#EventLevel" layout="${level}" />
<parameter name="#UserName" layout="${aspnet-user-identity}" />
<parameter name="#MachineName" layout="${machinename}" />
<parameter name="#EventMessage" layout="${message}" />
<parameter name="#ErrorSource" layout="${event-context:item=error-source}" />
<parameter name="#ErrorClass" layout="${event-context:item=error-class}" />
<parameter name="#ErrorMethod" layout="${event-context:item=error-method}" />
<parameter name="#ErrorMessage" layout="${event-context:item=error-message}" />
<parameter name="#InnerErrorMessage" layout="${event-context:item=inner-error-message}" />
</target>
</targets>
<!-- rules to map from logger name to target -->
<rules>
<!--All logs, including from Microsoft-->
<logger name="*" minlevel="Trace" writeTo="allfile,database" />
<!--Skip non-critical Microsoft logs and so log only own logs-->
<logger name="Microsoft.*" maxLevel="Info" final="true
" />
<!-- BlackHole without writeTo -->
<logger name="*" minlevel="Trace" writeTo="ownFile-web" />
</rules>
</nlog>
How it is being called in program.cs
var logger = NLog.Web.NLogBuilder.ConfigureNLog( "nlog.config" ).GetCurrentClassLogger();
(copied from their docs)
Must be missing something obvious, but since there is conflicting info in the docs, and copying other people's configs posted on here, not sure where to go with it
Looks like you are running on NetCore. NLog is not able to read connectionStringName from AppSettings.json as you have found out yourself (Requires extra dependencies to access IConfiguration).
One possible solution is using this extension:
https://www.nuget.org/packages/NLog.Appsettings.Standard/
And use connectionString (Instead of connectionStringName) in NLog.config:
<extensions>
<add assembly="NLog.Web.AspNetCore"/>
<add assembly="NLog.Appsettings.Standard"/>
</extensions>
<target xsi:type="Database" connectionString="${appsettings:name=ConnectionStrings.DefaultConnection}">
Alternative solution is to assign a GDC variable before logging:
NLog.GlobalDiagnosticsContext.Set("DefaultConnection", Configuration.GetConnectionString("DefaultConnection"));
And then use GDC in NLog.config:
<target xsi:type="Database" connectionString="${gdc:item=DefaultConnection}">
See also https://github.com/NLog/NLog/wiki/Gdc-layout-renderer
Update NLog.Extension.Logging ver. 1.4.0
With NLog.Extension.Logging ver. 1.4.0 then you can now use ${configsetting}
See also: https://github.com/NLog/NLog/wiki/ConfigSetting-Layout-Renderer
I have my nlog confiiguration as below
<?xml version="1.0" encoding="UTF-8"?>
<nlog throwExceptions="true">
<targets>
<target name="file" type="AsyncWrapper" queueLimit="5000" overflowAction="Discard">
<target type="BufferingWrapper" name="file" bufferSize="120">
<target type="File" fileName="${basedir}/logs/MyApplicationLog.log" layout="${longdate} ${exception:format=tostring} ${message} ${newline}" archiveFileName="${basedir}/archives/MyApplicationLogArchive/MyApplicationLog_{##}.log" archiveEvery="Day" archiveNumbering="Rolling" maxArchiveFiles="7" keepFileOpen="true" openFileCacheTimeout="30" />
</target>
</target>
<target name="MyApplicationHandlerLog" type="AsyncWrapper" queueLimit="5000" overflowAction="Discard">
<target type="BufferingWrapper" name="file" bufferSize="120">
<target type="File" fileName="${basedir}/logs/MyApplicationHandlerLog.log" layout="${longdate} ${exception:format=tostring} ${message} ${newline}" archiveFileName="${basedir}/archives/MyApplicationHandlerLogArchive/MyApplicationHandlerLog_{##}.log" archiveEvery="Day" archiveNumbering="Rolling" maxArchiveFiles="7" keepFileOpen="true" openFileCacheTimeout="30" />
</target>
</target>
<target name="MyApplicationHandlerMetrics" type="AsyncWrapper" queueLimit="5000" overflowAction="Discard">
<target type="BufferingWrapper" name="file" bufferSize="120">
<target type="File" fileName="${basedir}/logs/MyApplicationHandlerMetrics.log" layout="${longdate} ${exception:format=tostring} ${message} ${newline}" archiveFileName="${basedir}/archives/MyApplicationHandlerMetricsArchive/MyApplicationHandlerMetrics_{##}.log" archiveEvery="Day" archiveNumbering="Rolling" maxArchiveFiles="7" keepFileOpen="true" openFileCacheTimeout="30" />
</target>
</target>
</targets>
<rules>
<logger name="*" minlevel="Info" writeTo="file" />
<logger name="MyApplicationHandlerLog" minlevel="Info" writeTo="MyApplicationHandlerLog" />
<logger name="MyApplicationHandlerMetrics" minlevel="Info" writeTo="MyApplicationHandlerMetrics" />
</rules>
</nlog>
I am facing multiple issues listed below - need your help to sort them.
Perfomance issues - I have enabled KeepFileOpen in the targets which improved the performance drastically.But have read somewhere that we need to set openFileCacheTimeout as well to make the performance optimistic. Is this true? or will it screw the performance if both are set?
Not Writing the logs to multiple files as in the rules - But is not working as expected.All logs getting written to MyApplicationHandlerMetrics
Archiving works as expected - "No issues I am facing at the moment"
Greatly appreciated if any suggestion can be given to improve the performance as my system deals with 140 messages/second
Thanks,
Vinod
Performance issues. Yes performance will be better using KeepFileOpen=true. openFileCacheTimeout will help NLog versions older than 4.4.2 to recover from file-problems. Use a timeout of 10 mins and no performance hit will be seen.
Pretty sure that it is a problem at your end, and not an issue in NLog. Have you checked the Internal Log ? Maybe start with a simple configuration that works, and extend from there?
Great
Well NLog ver. 4.4.2 can handle over 200.000 messsages/second, when using a SSD-disk.