MapUtils with Logger - log4j

I am using MapUtils.verbosePrint(System.out, "", map) to dump the contents of a map in Java. They (management) do not like us using System.out.println().
We are using log4j. They made the logger into a variable "l" so we can say something like l.debug("This is going to the logfile in debug mode).
I would like to get the output buffer(s) from l so I could pass it into verbosePrint() instead of System.out. I looked at all the methods and members of the logger and did things like getAppenders() and tried all those elements but I could not find anything that helped.
Has anyone else done this? I know the logger may write to > 1 output.

You can use Log4j IOStreams to create PrintStreams that will send everything to a logger. This is mostly useful to log debug output from legacy APIs like JDBC or Java Mail that do not have a proper logging system. I wouldn't advise it in other cases, since your messages might be merged or split into several log messages.
I would rather use one of these approaches:
simply log the map using Logger#debug(Object). This will lazily create an ObjectMessage (only if debug is enabled), which is usually formatted using the map's toString() method. Some layouts might format it differently (like the JSON Template Layout).
eagerly create a MapMessage or StringMapMessage:
if (l.isDebugEnabled()) {
l.debug(new MapMessage(map));
}
This gives you more formatting options. For example the layout pattern %m{JSON} will format your message as JSON.
if your are set on the format provided by MapUtils#verbosePrint, you can extend ObjectMessage and overwrite its getFormattedMessage() and formatTo() methods.
public String getFormattedMessage() {
final ByteArrayOutputStream os = new ByteArrayOutputStream();
MapUtils.verbosePrint(new PrintStream(os), "", );
return new String(os.toByteArray());
}

Related

switching FileAppenders on the fly

I've got a legacy logging class. Its got a static Logger reference (named logger) and a bunch of static methods.
Each method takes a String input and writes it to System.out.println and to logger if logger is not null.
Its got a constructor that initializes logger. But this constructor only has package scope and I'm pretty sure its not being called anywhere. Therefore logger is always null and the class essentially only ever executes System.out.println
I want to change this so it can be used in a multi threaded application where each thread writes to its own unique FileAppender.
And that's where I'm stuck.
Basically, what I want to do is have this static class associated with a bunch of different log4j FileAppenders. Each FileAppender can be created by the Thread, and the file name can be derived from unique information known to the Thread.
What I can't figure out how to do is magically use Log4j to communicate that Thread's unique FileAppender to this legacy logging class.
Ideas? Hints? Suggestions?
Mark
It is possible to change the target log file name dynamically using a RoutingAppender and the ThreadContext map.
This can all be done with configuration (no need for custom code where threads create FileAppenders). In the RoutingAppender configuration you can specify a ThreadContext key you want to use to switch the target FileAppender. For example, you can use the string "ROUTINGKEY". Each thread puts a unique value in the ThreadContext map for key "ROUTINGKEY", and this value is used to select the Appender that the log event is routed to. You can even set it up to dynamically create log files that have the ROUTINGKEY value in the file name, so not all target log files need to be known in advance.
The FAQ page has a good example: http://logging.apache.org/log4j/2.x/faq.html#separate_log_files

Log4j, dynamic file name based on .getLogger("") parameter

My application controls an arbitrary number of devices. I want to have log files for each of the devices. Basically, i want to be able to call LogManager.getLogger(deviceId) and log the corresponding thing.
I looked through a number of topics here and on other sites, still i am not quite sure if it is possible or not. I am not quite sure whether i should extend an appender or a logger.
How does one go about solving this task?
One solution, though it's not optimal, would be to add a fileappender in the constructor of your device object :
String device = "thing"
Logger log1 = Logger.getLogger("org.path."+device);
log1.setAdditivity(false);
log1.addAppender(new FileAppender(new SimpleLayout(), "org.path."+device ));

Custom Logback Appender - Prepending file header and making it rollover

The functionality that I need is writing a header line at the beginning of the configured log file. The log file should, in addition, get rolled over based on a time pattern (I'm talking logback 1.0.7).
So, I'm thinking of writing an Appender - although I'm not sure whether it's not a custom Layout that I actually need.
1) Appender
Per logback's documentation, the right approach is to extend AppenderSkeleton, but then how would I combine this with the RollingFileAppender (to make the file rollover?)
On the other hand, if I extend RollingFileAppender, what method do I override to just decorate the existing functionality? How do I tell it to write that particular String only at the beginning of the file?
2) Layout
Analogously, the approach seems to be extending LayoutBase, and providing an implementation for doLayout(ILoggingEvent event).
But again, I don't know how to just decorate the behaviour - just adding a new line in the file, rather than disrupting its functionality (because I still want the rest of the logs to show up properly).
The getFileHeader() in LayoutBase looks promising, but how do I use it? Is it even intended to be overridden by custom layouts? (probably yes, since it's part of the Layout interface, but then how?)
Thank you!
Here I am answering my own question, just in case someone else comes across the same problem.
This is how I eventually did it (don't know however if it's the orthodox way):
Instead of extending AppenderSkeleton, I extended RollingFileAppender (to keep the rollover functionality), and overrode its openFile() method. In here I could manipulate the log file and write the header in it, after letting it do whatever it needed to do by default. Like this:
public void openFile(String fileName) throws IOException {
super.openFile(fileName);
File activeFile = new File(getFile());
if (activeFile.exists() && activeFile.isFile() && activeFile.length() == 0) {
FileUtils.writeStringToFile(activeFile, header);
}
}
I configured the header in logback.xml, as simple as this: <header> value </header>. This injects it in the header field of my new appender.
Seems to work without problems, but please do post if you know a better way!
Your solution has a problem: it deletes the first line of log of each new file. I think this is because you write the header whereas the file is open by logback.
I've found another solution that does not have this problem:
public void openFile(String fileName) throws IOException
{
super.openFile(fileName);
File activeFile = new File(getFile());
if (activeFile.exists() && activeFile.isFile() && activeFile.length() == 0)
{
lock.lock();
try
{
new PrintWriter(new OutputStreamWriter(getOutputStream(), StandardCharsets.UTF_8), true).println("your header");
}
finally
{
lock.unlock();
}
}
}

How to log Process id using Log4cxx or log4j

I am using log4cxx my project and i can able to log current thread id using [%t] marker, how to log process id in it or log4j?.
I am using ConversionPattern & xml based configuration file.
Thanks,
Based on the above answers, I'm going to do this in log4j as follows:
import java.lang.management.*;
import org.apache.log4j.MDC;
private String getPID() {
RuntimeMXBean rt = ManagementFactory.getRuntimeMXBean();
return rt.getName();
}
private void configLog4j() {
// call this from somewhere before you start logging
MDC.put("PID", getPID());
}
Then in my log4j.properties:
log4j.appender.FILE.layout.ConversionPattern=%d %X{PID} [%t] %p %c{1} %x - %m%n
This will actually yield a PID that consists of the ID number and the hostname, at least on my implementation of Java, and from what I read that could be implementation specific. You could go further and split out just the PID.
I've grepped through log4j's and log4cxx's documentation and nowhere did I find anything about logging process id.
So, to be short: no, you can't log process id, at least not directly.
Since you're using C++, you can get your program's PID. Here is how to do it in Linux (Ubuntu in this link). Here is how do do it in Windows.
Get that PID at your program start, use an MDC and put your PID in it.
I don't think there's a better way.
Doing this in log4j would be even trickier, since I know of no way for a running program to get it's PID using standard Java classes.
This doesnt exist in any of the log4xxx, but with a litle effort you can make it yourself. Actually it's a very simple task if you don't mind a little coding. This is basically what I did few times - override actual appender, or it's layout, make sure your class sticks the process ID into event properties map. Then use this property by name as if it was an MDC property. Using MDC directly like suggested above is not the best choice because they are thread bound and you will have to make sure every thread puts the PID when it starts. But if you can't or don't want to override the appender or layout, then this would probably be the only option.
The answer by #skiphoppy works very well for Log4j1.x, but I thought it could be updated to show how it works in the new Log4j2.
(NOTE: I tried to submit this as an edit of the posting above as it is only a minor revision of the answer code, but I'm posting it as a separate answer since my revision was rejected.)
import java.lang.management.ManagementFactory;
import java.lang.management.RuntimeMXBean;
import org.apache.logging.log4j.ThreadContext;
private String getPID() {
RuntimeMXBean rt = ManagementFactory.getRuntimeMXBean();
return rt.getName();
}
private void configLog4j() {
// Here is the Log4j2 way
ThreadContext.put("PID", rtmx.getName());
}
As skiphoppy's answer states, it outputs a little more than just the process ID. For instance, on my machine (Fedora 20):
16237#localhost.localdomain
You can extract just the process id with the following code, placed in your XML configuration file: %replace{%X{PID}}{[A-Za-z#\.]*}{}
Given the output above for the process id:
16237#localhost.localdomain
the regex will produce
16237
There is no feature in Log4J to achieve this, however you could pass the process id in and use that.
This blog post shows one way to go about it: http://blackbeanbag.net/wp/2009/01/14/log-file-management-in-coherence-using-log4j/
Basically, pass in the process id as a system property and then use that in the Log4j pattern.
Apparently, this is due to the JVM not providing an easy method to access the process id. From JDK1.5+, this may work.
(Archived from dead link http://www.theresearchkitchen.com/archives/100 )

Using log4net as a logging mechanism for SSIS?

Does anyone know if it is possible to do logging in SSIS (SQL Server Integration Services) via log4net? If so, any pointers and pitfalls to be aware of? How's the deployment story?
I know the best solution to my problem is to not use SSIS. The reality is that as much as I hate this POS technology, the company I work with encourages the use of these apps instead of writing code. Meh.
So to answer my own question: it is possible. I'm not sure how our deployment story will be since this will be done in a few weeks from now.
I pretty much took the information from these sources and made it work. This one explains how to make referencing assemblies work with SSIS, click here. TLDR version: place it in the GAC and also copy the dll to the folder of your targetted framework. In my case, C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727. To programmatically configure log4net I ended up using this link as reference.
This is how my logger configuration code looks like for creating a file with the timestamp on it:
using log4net;
using log4net.Config;
using log4net.Layout;
using log4net.Appender;
public class whatever
{
private ILog logger;
public void InitLogger()
{
PatternLayout layout = new PatternLayout("%date [%level] - %message%newline");
FileAppender fileAppenderTrace = new FileAppender();
fileAppenderTrace.Layout = layout;
fileAppenderTrace.AppendToFile = false;
// Insert current date and time to file name
String dateTimeStr = DateTime.Now.ToString("yyyyddMM_hhmm");
fileAppenderTrace.File = string.Format("c:\\{0}{1}", dateTimeStr.Trim() ,".log");
// Configure filter to accept log messages of any level.
log4net.Filter.LevelMatchFilter traceFilter = new log4net.Filter.LevelMatchFilter();
traceFilter.LevelToMatch = log4net.Core.Level.All;
fileAppenderTrace.ClearFilters();
fileAppenderTrace.AddFilter(traceFilter);
fileAppenderTrace.ImmediateFlush = true;
fileAppenderTrace.ActivateOptions();
// Attach appender into hierarchy
log4net.Repository.Hierarchy.Logger root = ((log4net.Repository.Hierarchy.Hierarchy)LogManager.GetRepository()).Root;
root.AddAppender(fileAppenderTrace);
root.Repository.Configured = true;
logger = log4net.LogManager.GetLogger("root");
}
}
Hopefully this might help someone in the future or at least serve as a reference if I ever need to do this again.
Sorry, you didn't dig deep enough. There are 5 different destinations that you can log to, and 7 columns you can choose to include or not include in your logging as well as between 18 to 50 different events that you can capture logging on. You appear to have chosen the default logging, and dismissed it because it didn't work for you out of the box.
Check these two blogs for more information on what can be done with SSIS logging:
http://consultingblogs.emc.com/jamiethomson/archive/2005/06/11/SSIS_3A00_-Custom-Logging-Using-Event-Handlers.aspx
http://www.sqlservercentral.com/blogs/michael_coles/archive/2007/10/09/3012.aspx

Resources