I'm developing API where I use NLog for logging incoming requests and their responses.
Occasionally I'm getting following error on my server
Application: w3wp.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.IO.FileLoadException
at System.Runtime.InteropServices.Marshal.ThrowExceptionForHRInternal(Int32, IntPtr)
at NLog.Internal.FileAppenders.BaseFileAppender.WindowsCreateFile(System.String, Boolean)
at NLog.Internal.FileAppenders.BaseFileAppender.TryCreateFileStream(Boolean)
at NLog.Internal.FileAppenders.BaseFileAppender.CreateFileStream(Boolean)
at NLog.Internal.FileAppenders.RetryingMultiProcessFileAppender.Write(Byte[])
at NLog.Targets.FileTarget.WriteToFile(System.String, NLog.LogEventInfo, Byte[], Boolean)
at NLog.Targets.FileTarget.ProcessLogEvent(NLog.LogEventInfo, System.String, Byte[])
at NLog.Targets.FileTarget.FlushCurrentFileWrites(System.String, NLog.LogEventInfo, System.IO.MemoryStream, System.Collections.Generic.List`1<NLog.Common.AsyncContinuation>)
at NLog.Targets.FileTarget.Write(NLog.Common.AsyncLogEventInfo[])
at NLog.Targets.Target.WriteAsyncLogEvents(NLog.Common.AsyncLogEventInfo[])
at NLog.Targets.Wrappers.AsyncTargetWrapper.ProcessPendingEvents(System.Object)
at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
at System.Threading.TimerQueueTimer.CallCallback()
at System.Threading.TimerQueueTimer.Fire()
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
My nlog.config looks like this
<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" autoReload="true" throwExceptions="true">
<variable name="logPath" value="${basedir}/App_Data/Logs"/>
<variable name="rowFormatInfo" value="${longdate} | ${level:uppercase=true} | ${message} | Thread: ${threadid}" />
<targets async="true">
<target name="traceFile" xsi:type="AsyncWrapper" overflowAction="Grow">
<target xsi:type="File" fileName="${logPath}/${shortdate}.trace.log" layout="${rowFormatInfo}" />
</target>
</targets>
<rules>
<logger name="*" minlevel="Trace" maxlevel="Debug" writeTo="traceFile" />
</rules>
</nlog>
In my opinion the problem lies with the fact that I'm using async wrapper which can result into mix of several threads trying to access the log file.
Is there any workaround to prevent such errors occurring in the peak times where my API has to deal with tens of requests per second? I don't think that switching to sync logging helps as the API runs in several concurrent threads itself.
Thanks for suggestions
Multiple threads within the same application should not give any issues, as they will just write to the async-queue (and not the file). But if you have multiple applications writing to the same file, then you can get issues.
You can also get this issue if you have an external application can monitors the log-file and tries to copy the contents somewhere else (Ex. SPLUNK).
If you don't have multiple applications writing to the same file, then just add these options to the file-target:
<target xsi:type="File"
fileName="${logPath}/${shortdate}.trace.log"
layout="${rowFormatInfo}"
keepFileOpen="true"
concurrentWrites="false" />
If you do have multiple Applications/AppDomains writing to the same file, then upgrade to latest NLog (currently 4.4.10) and use these options on the file-target:
<target xsi:type="File"
fileName="${logPath}/${shortdate}.trace.log"
layout="${rowFormatInfo}"
keepFileOpen="true"
concurrentWrites="true" />
Related
I have an example dialplan in a form of:
<include>
<context name="application">
<extension name="play tone">
<condition>
<action application="log" data="INFO *****Playing tone1***** "/>
<action application="answer"/>
<action application="sleep" data="1000"/>
<action application="playback" data="tone_stream://%(1850,4150,475,425);loops=-1" />
<action application="hangup"/>
</condition>
</extension>
</context>
</include>
This bit works with Freeswitch 1.4 but is not working with Freeswitch 1.8, I am getting debug output like this:
2019-05-29 11:14:59.971372 [DEBUG] sofia.c:7291 Channel sofia/sip-profile/447872019798#127.0.0.1:5062 entering state [ready][200]
EXECUTE sofia/sip-profile/447872019798#127.0.0.1:5062 sleep(1000)
EXECUTE sofia/sip-profile/447872019798#127.0.0.1:5062 playback(tone_stream://%(1850,4150,475,425);loops=-1)
2019-05-29 11:15:00.971418 [ERR] switch_core_file.c:304 Invalid file format [tone_stream] for [%(1850,4150,475,425);loops=-1]!
EXECUTE sofia/sip-profile/447872019798#127.0.0.1:5062 hangup()
I have consulted documentation:
https://freeswitch.org/confluence/display/FREESWITCH/mod_tone_stream
https://freeswitch.org/confluence/display/FREESWITCH/Tone_stream
Not sure what to think and not sure where to look for some answers. Is it my configuration somewhere that disables tone_streams? Is it a bug in the new version?
For any hint or outright solution I would be grateful.
I am unable to use the NLog MessageBox target in a WinForm application. The config file indicates the xsi:type is invalid. When I run the program I get this error:
System.TypeInitializationException
HResult=0x80131534
Message=The type initializer for 'Catalog.Form2' threw an exception.
Source=Catalog
StackTrace:
at Catalog.Form2..ctor() in L:\SourceCode\Catalog\Catalog\Form2.cs:line 29
at Catalog.Program.Main() in L:\SourceCode\Catalog\Catalog\Program.cs:line 16
Inner Exception 1:
NLogConfigurationException: Exception when parsing L:\SourceCode\Catalog\Catalog\bin\Debug\NLog.config.
Inner Exception 2:
ArgumentException: Target cannot be found: 'MessageBox'
My config file contains this:
<targets>
<target name="errors" xsi:type="MessageBox" layout="${message}" />
</targets>
<rules>
<logger name="*" minlevel="Debug" writeTo="errors" />
</rules>
If I change the target to "Console" everything works. Why does MessageBox not work?
NLog 4.5.11,
NLog.Schema 4.5.11,
.NET Framwork 4.7.2,
VS 2017 Pro (15.9.7)
For the MessageBox target you need to install the NLog.Windows.Forms NuGet package.
See also: https://nlog-project.org/config/?tab=targets&search=messagebox
Running: Cassandra 2.1.8.689 | DSE 4.7.3 on Linux 2.6
Configured logback.xml. Cassandra stops writing to log file at midnight each day. Last entry in log file:
INFO [CompactionExecutor:6550] 2017-01-05 23:59:58,928INFO ...
After changing filename in logback.xml (ie. system1.log to system2.log), writing resumes. The same behavior is consistent across 6 nodes.
Thank you Aaron. Logback details below.
<configuration scan="true">
<appender name="FIX_WINDOW_BASED_FILE"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${cassandra.logdir}/system.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${cassandra.logdir}/system%i.log</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>31</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>200MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%-5level [%thread] %date{ISO8601} %marker %F:%L - %msg%n</pattern>
<!-- old-style log format
<pattern>%5level [%thread] %date{ISO8601} %F (line %L) %msg%n</pattern>
-->
</encoder>
Problem resolved. After upgrading DSE from 4.7.3 to 4.8.11, output to the system.log is working fine.
You should be able to control this behavior in your logback.xml file. In that file should be an <appender> section, which should look something like this:
<appender name="stash" class="ch.qos.logback.core.rolling.RollingFileAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
<file>/var/log/cassandra/system.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>/var/log/cassandra/system.log.%d{yyyy-MM-dd}-%i</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<!-- or whenever the file size reaches 50MB -->
<maxFileSize>50MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<!-- keep 5 days' worth of history -->
<maxHistory>5</maxHistory>
</rollingPolicy>
<encoder class="net.logstash.logback.encoder.LogstashEncoder" />
</appender>
My guess is that the file rollover is triggered at midnight, but there must be something in your <fileNamePattern> preventing it from creating a new file.
In my Azure Web Site's Event Log I see same error every 1-5 minutes:
1 [varies]
5
50000780
What does this error message mean? Thank you in advance!
I found a similar issue, albeit for Sharepoint and Lightswitch. It looks like they had a similar error number show up in their event logs, breaking it down into the XML output:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="IIS Express" />
<EventID Qualifiers="32768">2284</EventID> <!-- Searched on this -->
<Level>3</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2013-09-14T03:48:18.000000000Z" />
<EventRecordID>8608</EventRecordID>
<Channel>Application</Channel>
<Computer>PSN-W12S-720</Computer>
<Security />
</System>
<EventData>
<Data>1</Data>
<Data>5</Data>
<Binary>50000780</Binary>
</EventData>
</Event>
Then I did a search on the EventID from above and found that there is an TechNet Article about Event ID 2284 which essentially states that FREB wasn't able to write to the logs.
FailedRequestTracing module failed to write buffered events to log
file for the request that matched failure definition. No logs will be
generated until this condition is corrected. The problem happened at
least %1 times in the last %2 minutes. The data is the error.
Do you happen to have Failed Request Tracing enabled for your site? If not you might want to, apparently it's trying to write something every 1-5 minutes.
I work with Mule and I have some problem.
My mainflow.xml:
<?xml version="1.0" encoding="UTF-8"?><mule xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:vm="http://www.mulesoft.org/schema/mule/vm"
xmlns:file="http://www.mulesoft.org/schema/mule/file" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" xmlns:core="http://www.mulesoft.org/schema/mule/core"
xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:management="http://www.mulesoft.org/schema/mule/management"
version="CE-3.3.0"
xsi:schemaLocation="http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/current/mule-vm.xsd http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd http://www.mulesoft.org/schema/mule/management http://www.mulesoft.org/schema/mule/management/current/mule-management.xsd">
<!-- For show MX4J -->
<management:jmx-default-config port="1098"
registerMx4jAdapter="true">
</management:jmx-default-config>
<management:jmx-mx4j-adaptor jmxAdaptorUrl="http://localhost:9999"
login="admin" password="admin">
</management:jmx-mx4j-adaptor>
<management:jmx-server enableStatistics="true">
<management:connector-server
url="service:jmx:rmi:///jndi/rmi://localhost:1099/server" rebind="false" />
</management:jmx-server>
<management:rmi-server createRegistry="true" />
<!-- For get statistics -->
<notifications>
<notification event="EXCEPTION" />
<notification event="ENDPOINT-MESSAGE" />
<notification-listener ref="endpointNotificationLogger" />
<notification-listener ref="statesModuleExceptionNotificationListener" />
</notifications>
<spring:beans>
<!-- For show MX4J -->
<spring:bean id="exporter"
class="org.springframework.jmx.export.MBeanExporter" lazy-init="false">
<spring:property name="beans">
<spring:map>
<spring:entry key="bean:name=MBean1"
value-ref="MBean1" />
<spring:entry key="bean:name=MBean2"
value-ref="MBean2" />
</spring:map>
</spring:property>
</spring:bean>
<spring:bean id="MBean1" name="MBean1"
class="MBean1"></spring:bean>
<spring:bean id="MBean2" name="MBean2"
class="MBean2"></spring:bean>
<!-- For get statistics -->
<spring:bean name="endpointNotificationLogger"
class="MyEndpointNotificationListener" />
<spring:bean name="myExceptionNotificationListener"
class="MyExceptionNotificationListener" />
<spring:bean id="MyBean" name="MyBean"
class="MyBean">
</spring:bean>
</spring:beans>
<!-- <file:connector name="input" autoDelete="true" doc:name="File" />-->
<file:connector name="inputFileConnector" autoDelete="true" doc:name="File" >
<service-overrides messageReceiver="myFileMessageReceiver"/>
</file:connector>
<flow name="UnpackHandler" doc:name="UnpackHandler">
...
</flow>
<flow name="XmlToCsvTransformHandler" doc:name="XmlToCsvTransformHandler">
...
</flow>
Everything works OK in Tomcat. But after that I commit changes to SVN, checkout everything, package without changes and deploy app in Tomcat my application do not work!
The latest text in logs of Tomcat for my application:
DEBUG 18.12.2014 15:12:08 997 - DisposableBeanAdapter.invokeCustomDestroyMethod - Invoking destroy method 'dispose' on bean with name '_muleSystemModel'
INFO 18.12.2014 15:12:08 997 - ModelLifecycleManager.fireDisposePhase - Disposing model: _muleSystemModel
And there are no any logs about errors...
When application works OK, after this lines I see
DEBUG 18.12.2014 10:59:53 765 - MuleXmlBuilderContextListener.initialize - Mule config file(s): WEB-INF/classes/mainflow.xml
If I have this error and delete from mainflow.xml parts for mx4j and notifications everything works again:
<?xml version="1.0" encoding="UTF-8"?><mule xmlns="http://www.mulesoft.org/schema/mule/core" mlns:vm="http://www.mulesoft.org/schema/mule/vm" xmlns:file="http://www.mulesoft.org/schema/mule/file" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:spring="http://www.springframework.org/schema/beans" xmlns:core="http://www.mulesoft.org/schema/mule/core" xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="CE-3.3.0" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/current/mule-vm.xsd http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd ">
</spring:beans>
<file:connector name="input" autoDelete="true" doc:name="File" />
<flow name="UnpackHandler" doc:name="UnpackHandler">
...
</flow>
<flow name="XmlToCsvTransformHandler" doc:name="XmlToCsvTransformHandler">
...
</flow>
After I write parts for mx4j and notifications step by step, do not paste all text of file what I take from SVN, it works again! So there is no difference between mainflow.xml when application works and doesn't work.
This behavior is in Linux server. In the Windows server and in my local machine everything is OK. I thought about differences between slashes, but I don't write slashes...
Why application behaves so strangely?
With the information you provide I can only guess that you have some logic in the notifications that hangs them if something on which they are dependedant is not already initializated somehow.
The fact that later it does work would somehow indicate this. I can suggest you to use in the beans the attribute depends-on to make sure it's initialized after your bean dependencies in case there are.