Issue with no log4net Logger.Appenders even though log shows config successfully loaded - log4net

I would appreciate any help you can provide? Not sure what I am doing wrong...
I am creating a log4net compatibility library for my new Visual Studio tool ErrorUnit at https://github.com/JohnGoldInc/ErrorUnit.Logger_log4net
Line 37 of https://github.com/JohnGoldInc/ErrorUnit.Logger_log4net/blob/master/ErrorUnitLogger.cs has Count=0 Appenders even though my log shows that appenders were loaded:
16 namespace ErrorUnit.Logger_log4net
17 {
18 public class ErrorUnitLogger : ILogger
19 {
20 private static ILog log = LogManager.GetLogger(typeof(ErrorUnitLogger));
21
22 public IEnumerable<string> GetErrorUnitJson(DateTime afterdate)
23 {
24 var ErrorUnitJson = new ConcurrentBag<string>();
25 log4net.Util.LogLog.InternalDebugging = true; //todo remove
26 System.Diagnostics.Trace.Listeners.Add(new System.Diagnostics.TextWriterTraceListener(#"f:\Temp\Logger_log4net.log", "myListener"));
27 System.Diagnostics.Trace.AutoFlush = true;
28 System.Diagnostics.Trace.TraceInformation("Test Logger_log4net message.");
29
30 var config = log4net.Config.XmlConfigurator.Configure();
31 log = LogManager.GetLogger(typeof(ErrorUnitLogger));
32
33 // Parallel.ForEach(logs, log => {
34 var log4net_Logger = log.Logger as log4net.Repository.Hierarchy.Logger;
35 if (log4net_Logger != null)
36 {
37 Parallel.ForEach(log4net_Logger.Appenders.Cast<log4net.Appender.IAppender>(), appender =>
For the log ( the log without ellipsis is at https://github.com/JohnGoldInc/ErrorUnit.Logger_log4net/blob/master/README.md ):
devenv.exe Information: 0 : Test Logger_log4net message.
log4net: configuring repository [log4net-default-repository] using .config file section
log4net: Application config file is [F:\Documents\Visual Studio 2015\Projects\WebApplication1\WebApplication1.Tests\obj\Release]
log4net: Configuring Repository [log4net-default-repository]
log4net: Configuration update mode [Merge].
...
log4net: Created Appender [AdoNetAppender]
log4net: Adding appender named [AdoNetAppender] to logger [root].
log4net: Hierarchy Threshold []
...
Thanks!

Was not looking at top level Logger, to get to top level logger refer to .Parent property:
var log4net_Logger = log.Logger as log4net.Repository.Hierarchy.Logger;
while (log4net_Logger != null && log4net_Logger.Appenders.Count == 0)
log4net_Logger = log4net_Logger.Parent;

I do not have enough reputation to add a comment to your answer... so adding a new answer here but this is more so a comment
From your log, it seems you are adding AdoNetAppender to the 'root' Logger, so if you wish to get all the appenders for 'root' you may need to access the 'root' logger (which sits at the top of the hierarchy). Accessing the parent Logger, in this case, worked for you because 'root' happens to be parent of ErrorUnitLogger you have...but that may not always be true in general
If you are not already aware, you can access 'root' logger like this
Hierarchy hierarchy = (Hierarchy)log4net.LogManager.GetRepository();
Logger rootLogger = hierarchy.Root;

You are sending your test message before you configure log4net. First configure log4net and than send the test trace message:
System.Diagnostics.Trace.TraceInformation("Test Logger_log4net message.");

Related

Hazelcast-jet: got error when enriching stream using direct lookup

I am following Doc to try out how to enrich an unbounded stream by directly looking up from a IMap. I have two Maps:
Product: Map<String, Product> (ProductId as key)
Seller: Map<String, Seller> (SellerId as key)
Both Product and Seller are very simple classes:
public class Product implements DataSerializable {
String productId;
String sellerId;
int price;
...
public class Seller implements DataSerializable {
String sellerId;
int revenue;
...
I have two data generators keep pushing data to the two maps. The event-journal are enabled for both maps. I have verified the event-journal works fine.
I want to enrich the stream event of Product map with Seller map. Here is a snippet of my code:
IMap<String, Seller> sellerIMap = jetClient.getMap(SellerDataGenerator.SELLER_MAP);
StreamSource<Product> productStreamSource = Sources.mapJournal(ProductDataGenerator.PRODUCT_MAP, Util.mapPutEvents(), Util.mapEventNewValue(), START_FROM_CURRENT);
p.drawFrom(productStreamSource)
.withoutTimestamps()
.groupingKey(Product::getSellerId)
.mapUsingIMap(sellerIMap, (product, seller) -> new EnrichedProduct(product, seller))
.drainTo(getSink());
try {
JobConfig jobConfig = new JobConfig();
jobConfig.addClass(TaskSubmitter.class).addClass(Seller.class).addClass(Product.class).addClass(ExtendedProduct.class);
jobConfig.setName(Constants.BASIC_TASK);
Job job = jetClient.newJob(p, jobConfig);
} finally {
jetClient.shutdown();
}
When job was submitted, I got following error:
com.hazelcast.spi.impl.operationservice.impl.Invocation - [172.31.33.212]:80 [jet] [3.1] Failed asynchronous execution of execution callback: com.hazelcast.util.executor.DelegatingFuture$DelegatingExecutionCallback#77ac0407for call Invocation{op=com.hazelcast.map.impl.operation.GetOperation{serviceName='hz:impl:mapService', identityHash=1939050026, partitionId=70, replicaIndex=0, callId=-37944, invocationTime=1570410704479 (2019-10-07 01:11:44.479), waitTimeout=-1, callTimeout=60000, name=sellerMap}, tryCount=250, tryPauseMillis=500, invokeCount=1, callTimeoutMillis=60000, firstInvocationTimeMs=1570410704479, firstInvocationTime='2019-10-07 01:11:44.479', lastHeartbeatMillis=0, lastHeartbeatTime='1970-01-01 00:00:00.000', target=[172.31.33.212]:80, pendingResponse={VOID}, backupsAcksExpected=0, backupsAcksReceived=0, connection=null}
I tried to put one and two instances in my cluster and got the same error message. I couldn't figure out what was the root cause.
It seems that your problem is a ClassNotFoundException, even though you added the appropriate classes to the job. The objects you store in the IMap exist independent of your Jet job and when the event journal source asks for them, Jet's IMap code tries to deserialize them and fails because Jet doesn't have your domain model classes on its classpath.
To move on, add a JAR with the classes you use in the IMap to Jet's classpath. We are looking for a solution that will remove this requirement.
The reason you haven't got the exception stacktrace in the log output is due to the default java.util.logging setup you end up with when you don't explicitly add a more flexible logging module, such as Log4j.
The next version of Jet's packaging will improve this aspect. Until that time you can follow these steps:
Go to the lib directory of Jet's distribution package and download Log4j into it:
$ cd lib
$ wget https://repo1.maven.org/maven2/log4j/log4j/1.2.17/log4j-1.2.17.jar
Edit bin/common.sh to add the module to the classpath. Towards the end of the file there is a line
CLASSPATH="$JET_HOME/lib/hazelcast-jet-3.1.jar:$CLASSPATH"
You can duplicate this line and replace hazelcast-jet-3.1 with log4j-1.2.17.
At the end of commons.sh there is a multi-line command that constructs the JAVA_OPTS variable. Add "-Dhazelcast.logging.type=log4j" and "-Dlog4j.configuration=file:$JET_HOME/config/log4j.properties" to the list.
Create a file log4j.properties in the config directory:
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d %5p [%c{1}] [%t] - %m%n
# Change this level to debug to diagnose failed cluster formation:
log4j.logger.com.hazelcast.internal.cluster=info
log4j.logger.com.hazelcast.jet=info
log4j.rootLogger=info, stdout

When logging with NLog, why does output from a library DLL come out in a different format?

I have some code that uses NLog and I like to capture this output in my unit tests so I can see what's going on. I use MSpec as my test framework and NCrunch as the test runner. I configre logging once per test assembly, like this:
public class LogSetup : IAssemblyContext
{
static Logger log;
public void OnAssemblyStart()
{
var configuration = new LoggingConfiguration();
var unitTestRunnerTarget = new TraceTarget();
configuration.AddTarget("Unit test runner", unitTestRunnerTarget);
unitTestRunnerTarget.Layout =
"${time}|${pad:padding=-5:inner=${uppercase:${level}}}|${pad:padding=-16:inner=${callsite:className=true:fileName=false:includeSourcePath=false:methodName=false:includeNamespace=false}}|${message}";
var logEverything = new LoggingRule("*", LogLevel.Trace, unitTestRunnerTarget);
configuration.LoggingRules.Add(logEverything);
LogManager.Configuration = configuration;
log = LogManager.GetCurrentClassLogger();
log.Info("Logging initialized");
}
public void OnAssemblyComplete() { }
}
There are no XML configuration files anywhere and this is the only place the logging configuration is set.
When I run the unit tests, I get output like this example:
22:37:51.0253|DEBUG|SimulatorState |Transitioning from => TA.DigitalDomeworks.HardwareSimulator.StateStartup
22:37:51.1263|DEBUG|SimulatorState |Transitioning from TA.DigitalDomeworks.HardwareSimulator.StateStartup => TA.DigitalDomeworks.HardwareSimulator.StateReceivingCommand
nCrunch.TestHost462.x86.exe Information: 0 : 22:37:51.5923|INFO |TransactionObserver|Transaction pipeline connected to channel with endpoint Simulator:Fast
nCrunch.TestHost462.x86.exe Information: 0 : 22:37:51.7393|INFO |TransactionObserver|Committing transaction TID=1 [GINF] [{no value}] 00:00:10 Created
22:37:51.7623|DEBUG|c__DisplayClass0_1`1|StatusTransaction[1]: Subscribe()
nCrunch.TestHost462.x86.exe Information: 0 : 22:37:51.7932|INFO |SimulatorState |State [ReceiveCommand] received stimulus 'G'
nCrunch.TestHost462.x86.exe Information: 0 : 22:37:51.7932|INFO |SimulatorState |State [ReceiveCommand] received stimulus 'I'
nCrunch.TestHost462.x86.exe Information: 0 : 22:37:51.7943|INFO |SimulatorState |State [ReceiveCommand] received stimulus 'N'
nCrunch.TestHost462.x86.exe Information: 0 : 22:37:51.7943|INFO |SimulatorState |State [ReceiveCommand] received stimulus 'F'
22:37:51.7943|DEBUG|SimulatorState |Transitioning from TA.DigitalDomeworks.HardwareSimulator.StateReceivingCommand => TA.DigitalDomeworks.HardwareSimulator.StateExecutingCommand
22:37:51.7943|DEBUG|StateExecutingCommand|Processing command [GINF]
22:37:51.7943|DEBUG|SimulatorState |Transitioning from TA.DigitalDomeworks.HardwareSimulator.StateExecutingCommand => TA.DigitalDomeworks.HardwareSimulator.StateSendStatus
As you can see, the log output gets written using two different layouts. The first couple of lines are as expected, then there's this monstrosity:
nCrunch.TestHost462.x86.exe Information: 0 : 22:37:51.5923|INFO |TransactionObserver|Transaction pipeline connected to channel with endpoint Simulator:Fast
It's almost like the output got formatted twice, once using a default format and then again using the format I want. There doesn't seem to be a pattern as to when the "odd" output happens. In line 1, the ouput came from class SimulatorState, then in line 6 there is more output from SimulatorState but this time it is in the mutant format.
How can this be?

Get class name along with package using log4j in Java

I have various test Class which contains logger steps for low level debugging across code.
Now I want to implement logger in such a way that current class name is displayed along with my logger info
Suppose if my AutTest.java code has logger.info("This is logger") then console should display my current class name along with logger.
Just declare the below line of code in Base class which is extended by child class
For only class name
public final Logger logger = Logger.getLogger(this.getClass().getSimpleName());
OR
Package info will be displayed
public final Logger logger = Logger.getLogger(this.getClass().getName());
Eg:
In my com.aut.AppTest.java
logger.info("This is logger");
OUTPUT 1:
2015-12-29 15:15:15 INFO AutTest - This is logger
OUTPUT 2:
2015-12-29 16:16:16 INFO com.aut.AutTest - This is logger

How can I include xml configuration in logback.groovy

I'm writing a Spring Boot app and need the flexibility of controlling my logback configuration using Groovy. In Spring Boot all I have to do is create src/main/resources/logback.groovy and it is automatically used for configuration.
What I would like to do though is start with Spring Boot's default logback configuration, and just override or modify settings as needed.
If I were using logback.xml instead of logback.groovy I could do something like the following.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/base.xml"/>
<logger name="org.springframework.web" level="DEBUG"/>
</configuration>
Is there something similar to the include line above that I can use in logback.groovy? I can look at the contents of base.xml and it's other included files to see how to replicate this manually, but it would add a bit of boilerplate code I'd like to avoid.
Thanks for any help you can provide.
There's an online tool that translates given logback.xml file to equivalent logback.groovy. In your case it resulted in:
//
// Built on Thu Jul 16 09:35:34 CEST 2015 by logback-translator
// For more information on configuration files in Groovy
// please see http://logback.qos.ch/manual/groovy.html
// For assistance related to this tool or configuration files
// in general, please contact the logback user mailing list at
// http://qos.ch/mailman/listinfo/logback-user
// For professional support please see
// http://www.qos.ch/shop/products/professionalSupport
import static ch.qos.logback.classic.Level.DEBUG
logger("org.springframework.web", DEBUG)
When it comes to <include> it's not supported for groovy configurations.
How do you feel about instead of adding/overriding your configuration, you reload it again?
You can create a Spring Bean that will see if a logback file is in a location you specify, and if it is, reload using that file
Example
#Component
public class LoggingHelper {
public static final String LOGBACK_GROOVY = "logback.groovy";
#PostConstruct
public void resetLogging() {
String configFolder = System.getProperty("config.folder");
Path loggingConfigFile = Paths.get(configFolder, LOGBACK_GROOVY);
if (Files.exists(loggingConfigFile) && Files.isReadable(loggingConfigFile)) {
LoggerContext loggerContext = (LoggerContext) LoggerFactory.getILoggerFactory();
ContextInitializer ci = new ContextInitializer(loggerContext);
loggerContext.reset();
try {
ci.configureByResource(loggingConfigFile.toUri().toURL());
} catch (JoranException e) {
// StatusPrinter will handle this
} catch (MalformedURLException e) {
System.err.println("Unable to configure logger " + loggingConfigFile);
}
StatusPrinter.printInCaseOfErrorsOrWarnings(loggerContext);
}
}
}
I am using this snippet to start my logback.groovy file
import ch.qos.logback.classic.joran.JoranConfigurator
import org.xml.sax.InputSource
def configurator = new JoranConfigurator()
configurator.context = context
def xmlString = '<?xml version="1.0" encoding="UTF-8"?>\n<configuration>\n <include resource="org/springframework/boot/logging/logback/base.xml"/>\n</configuration>'
configurator.doConfigure(new InputSource(new StringReader(xmlString)))
Contrary to the documentation stating that:
Everything you can do using XML in configuration files, you can do in
Groovy with a much shorter syntax.
include is not possible with Groovy out-of-the-box. However, thanks to a bug ticket that was opened in 2014, there are a couple of workarounds. I am including them here (slightly edited), but all credit goes to "Yih Tsern" from the original JIRA bug:
logback.groovy
include(new File('logback-fragment.groovy'))
root(DEBUG, ["CONSOLE"])
def include(File fragmentFile) {
GroovyShell shell = new GroovyShell(
getClass().classLoader,
binding,
new org.codehaus.groovy.control.CompilerConfiguration(scriptBaseClass: groovy.util.DelegatingScript.name))
Script fragment = shell.parse(fragmentFile.text)
fragment.setDelegate(this)
fragment.run()
}
logback-fragment.groovy:
// NOTE: No auto-import
import ch.qos.logback.core.*
import ch.qos.logback.classic.encoder.*
appender("CONSOLE", ConsoleAppender) {
encoder(PatternLayoutEncoder) {
pattern = "%d [%thread] %level %mdc %logger{35} - %msg%n"
}
}
Given the workaround and a pull-request to add the feature, I'm not sure why the functionality hasn't been added to Logback core yet.

Log4net EventLogAppender Log Event ID

Is there a way to log an event into the windows event log with a specified eventid per message? I am using log4net v 1.2.10.
Based on what I see in the EventLogAppender source code the following should do the trick:
log4net.ThreadContext.Properties["EventID"] = 5;
Just call this before you write your log messages (if you do not set it for all messages you should remove the "EventID" again from the Properties.
N.B the property key is case sensitive.
When one uses the native .net Event Log APIs in System.Diagnostics, the WriteEntry methods allow setting the eventID and category. In these APIs:
eventID is a 32 bit int, but its value must be between 0 and 65535
category is a 16 bit int, but its value must be positive. If the
event source includes a category resource file, the event viewer will
use the integer category value to lookup a localized “Task category”
string. Otherwise, the integer value is displayed.
The categories must be numbered consecutively, beginning with the
number 1
Log4net supports writing an EventID and a Category, but it isn’t straight forward. When log4net’s EventLogAppender logs an event, it looks at a dictionary of properties. The named properties "EventID" and "Category" are automatically mapped by the EventLogAppender to the corresponding values in the event log. I’ve seen a few good suggested ways to use log4net’s EventLogAppender and set the EventID and Category in the Windows event log.
a. Using log4net’s appender filtering, a filter may be registered that can add the EventID and Category properties. This method has a nice benefit that the standard log4net wrappers are used and so this can be implemented without changing existing logging code. The difficulty in this method is some mechanism has to be created to calculate the EventID and Category from the logged information. For instance, the filter could look at the exception source and map that source to a Category value.
b. Log4net may be extended so custom logging wrappers can be used that can include EventID and Category parameters. Adding EventID is demonstrated in the log4net sample “Extensibility – EventIDLogApp” which is included in the log4net source. In the extension sample a new interface (IEventIDLog) is used that extends the standard ILog interface used by applications to log. This provides new logging methods that include an eventId parameter. The new logging methods add the eventId to the Properties dictionary before logging the event.
public void Info(int eventId, object message, System.Exception t)
{
if (this.IsInfoEnabled)
{
LoggingEvent loggingEvent = new LoggingEvent(ThisDeclaringType, Logger.Repository, Logger.Name, Level.Info, message, t);
loggingEvent.Properties["EventID"] = eventId;
Logger.Log(loggingEvent);
}
}
c. Log4net supports a ThreadContext object that contains a Properties dictionary. An application could set the EventID and Category properties in this dictionary and then when the thread calls a logging method, the values will be used by the EventLogAppender.
log4net.ThreadContext.Properties["EventID"] = 5;
Some helpful references:
Log4net home page
Log4net SDK reference
Log4net samples
Log4net source
Enhancing log4net exception logging
Log4Net Tutorial pt 6: Log Event Context
Customizing Event Log Categories
EventSourceCreationData.CategoryResourceFile Property
Event Logging Elements
EventLog.WriteEntry Method
Well, the solution was to build the extension project "log4net.Ext.EventID" and to use its types: IEventIDLog, EventIDLogImpl and EventIDLogManager.
Another solution is to add a custom Filter as described here: Enhancing log4net exception logging (direct link to the Gist just in case).
As the author points out:
... EventLogAppender uses inline consts to check them. Once they are added they will be used by the mentioned EventLogAppender to mark the given entries with EventId and Category.
The filter implementation will look like the code below (stripped down gist) with the added benefit that if you make GetEventId method public, you can write some tests against it
public class ExceptionBasedLogEnhancer : FilterSkeleton
{
private const string EventLogKeyEventId = "EventID";
public override FilterDecision Decide(LoggingEvent loggingEvent)
{
var ex = loggingEvent.ExceptionObject;
if (ex != null)
{
loggingEvent.Properties[EventLogKeyEventId] = GetEventId(ex);
}
return FilterDecision.Neutral;
}
private static short GetEventId(Exception ex)
{
// more fancy implementation, like getting hash of ex properties
// can be provided, or mapping types of exceptions to eventids
// return no more than short.MaxValue, otherwise the EventLog will throw
return 0;
}
}
Extend ILog.Info() to take an event ID:
public static class LogUtils
{
public static void Info(this ILog logger, int eventId, object message)
{
log4net.ThreadContext.Properties["EventID"] = eventId;
logger.Info(message);
log4net.ThreadContext.Properties["EventID"] = 0; // back to default
}
}
Then call it like this:
using LogUtils;
private static readonly log4net.ILog _logger = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
_logger.Info(3, "First shalt thou take out the Holy Pin, then shalt thou count to three.");

Resources