I'm using a nlog's CSV layout file for data logging.
<target xsi:type="File" [...]>
<layout xsi:type="CSVLayout">
<column name="Value1" layout="${event-context:item=Value1}"/>
[...]
I'm getting column headers appended to the file every time I run the logging application. Is there a way to only get the column headers created when a new log file is created?
Thanks
Related
I'm trying to figure out how a custom appender should be configured in a Databricks environment but I cannot figure it out.
When cluster is running, in driver logs, time is displayed as 'unknown' for my custom log file and when cluster is stopped, custom log file is not displayed at all in the log files list
#appender configuration
log4j.appender.bplm=com.databricks.logging.RedactionRollingFileAppender
log4j.appender.bplm.layout=org.apache.log4j.PatternLayout
log4j.appender.bplm.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n
log4j.appender.bplm.rollingPolicy=org.apache.log4j.rolling.TimeBasedRollingPolicy
log4j.appender.bplm.rollingPolicy.FileNamePattern=logs/log4j-%d{yyyy-MM-dd-HH}-bplm.log.gz
log4j.appender.bplm.rollingPolicy.ActiveFileName=logs/log4j-bplm.log
log4j.logger.com.myPackage=INFO,bplm
Above configuration was added to following files
"/databricks/spark/dbconf/log4j/executor/log4j.properties"
"/databricks/spark/dbconf/log4j/driver/log4j.properties"
"/databricks/spark/dbconf/log4j/master-worker/log4j.properties"
After above configuration was added to above mentioned files, there are two issues which i cannot figure out.
1 - When cluster is running, if I go to driver logs in the list of log files, I can see my custom logfile generated, correctly populated, but time column is displayed as 'unknown'.
2 - When cluster is stopped, if I go to driver logs in the list of log files, my custom appender are not displayed. ( stdout, stderr, and log4j-active are displayed )
I also used different FileNamePatterns, but issues mentioned above seems to happens for any configuration I tried
log4j.appender.bplm.rollingPolicy.FileNamePattern=logs/log4j-%d{yyyy-MM-dd-HH}.bplm.log.gz - appender1
log4j.appender.bplm.rollingPolicy.FileNamePattern=logs/log4j.bplm-%d{yyyy-MM-dd-HH}.log.gz - appender2
log4j.appender.bplm.rollingPolicy.FileNamePattern=logs/bplm-log4j-%d{yyyy-MM-dd-HH}.log.gz - appender3
log4j.appender.bplm.rollingPolicy.FileNamePattern=logs/bplm.log4j-%d{yyyy-MM-dd-HH}.log.gz - appender4
log4j.appender.bplm.rollingPolicy.FileNamePattern=logs/log4j-%d{yyyy-MM-dd-HH}.log.bplm.gz - appender5
log4j.appender.bplm7.rollingPolicy.FileNamePattern=logs/log4j-bplm-%d{yyyy-MM-dd-HH}.log.gz - appender7
log4j.appender.bplm8.rollingPolicy.FileNamePattern=logs/log4j-%d{yyyy-MM-dd-HH}-bplm.log.gz - appender8
I also tried to put *-active in ActiveFileName, but result was the same
log4j.appender.custom.rollingPolicy.FileNamePattern=/tmp/custom/logs/log4j-bplm-%d{yyyy-MM-dd-HH}.log.gz
log4j.appender.custom.rollingPolicy.ActiveFileName=/tmp/custom/logs/log4j-bplm-active.log
I am trying to run a testng.suite but it is not running all methods from each class. It just ran first few methods from each class.I broke my .xml file into two .xml files. In one file I added first four classes and ran it successfully, testng.xml file ran every method from each class and did not miss even one method. Then I created a second .xml and ran my last class,
""
it was also successful and ran every method from this class. But when I run all five test classes in one .xml file it only run few methods form each class. Is there any limit of class/methods that we can run in one .xml file. How many classes/method we can run in one .xml file?
I did not understand this, I will really appreciate if some one help me here.
Below is my testng.xml file.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="finalSuite">
<test name="final Test">
<classes>
<class name="com.qa.MurcryTour.Tests.HomePageTest"/>
<class name="com.qa.MurcryTour.Tests.LoginPageTest"/>
<class name="com.qa.MurcryTour.Tests.RegisterPageTest"/>
<class name="com.qa.MurcryTour.Tests.FlightsPageTest"/>
<class name="com.qa.MurcryTour.Tests.DataDriven_RegisterPageTest"/>
</classes>
</test> <!-- final Test -->
</suite> <!-- finalSuite -->
I'm using AWS Glue to crawl XML files and add them to a Glue database table. The DynamicFrame I'm using identifies several choices in the XML schema. I can resolve most of them, but there's one case that I can't figure out.
The relevant part of the XML structure is:
<root>
<order>
<lineitems>
<lineitem>
...
</lineitem>
</lineitems>
</order>
</root>
The DynamicFrame shows lineitems as a struct and lineitems/lineitem as a choice between array or struct, I suspect because some orders have one lineitem, whereas other orders have multiple lineitems. I've tried calling resolveChoice with project:array, but that results in element:unknown so I can no longer see the structure of the lineitem. I'm not sure what else to try here, any ideas?
I have make my properties file ok,but what should I do if I want to put the log file in a folder relate to the date?
For example,today is 12/29 2015,at 10:30,I started my java project,the log4j.propertites about the log like the following ones:
log4j.appender.inforlog=org.apache.log4j.DailyRollingFileAppender
log4j.appender.inforlog.DatePattern='.'yyyy-MM-dd-HH
log4j.appender.inforlog.File=D:/inforLogs/2015/12/searchrecord
when it comes to 11:00,there will be a log file named searchrecord.2015-12-29-10 in "D:/inforLogs/2015/12/", when it comes to 01/01 2016,the log file will alse in file "D:/inforLogs/2015/12/",but I want to make it in file "D:/inforLogs/2016/01/" by write the properties file properly,what should I do?
I have resolve the problem myself,here is the properties file
log4j.appender.inforlog.DatePattern='s/'yyyy'/'MM'/searchrecord-'dd'_'HH'.log'
log4j.appender.inforlog.File=D:/inforLog
Documentation says that 'if file names haven't changed, no rollover'.fileName is derived with the fileName pattern String.
I have two observations :
1)If today appender doesn't have message to write then it wont roll the file even if triggring time has passed (i.e. we have a log file which was modified yesterday ).
2)If the yesterdays log file is 0KB(means yesterday no log was written in file) & today appender have some messages to write then it rolls the 0kb file and writes data to newly created log file
I wanted to discuss whether above both cases are correctly implemented by TimeBasedRollingPolicy class OR should the implementation be changed ?
My implementation strategy for 1st Scenario would be 'if FileNamePattern is set to %d{dd-MM-yyyy} then at the midnight file should be rolled irrespective whether appender has data to write if yesterday's file is non empty.'
In case of 2nd case if yesterday's file is 0kb means no message was logged yesterday then it should write data into same file . because main purpose of rolling is to take backup of the logs and if file is empty is it worth to roll it ?
take below configuration of log4j.properties file as referenec for above two scenario's discussion
Sample log4j.properties
####### Root Logger ########################################
log4j.rootLogger=ERROR,CA,FA
############################################################
################### APPENDERS ##############################
############################################################
# CA is set to be a ConsoleAppender
log4j.appender.CA=org.apache.log4j.ConsoleAppender
log4j.appender.CA.layout=org.apache.log4j.PatternLayout
log4j.appender.CA.layout.ConversionPattern=%d %p %t %c: %m%n
# FA is set to be a FileAppender
log4j.appender.FA=org.apache.log4j.rolling.RollingFileAppender
log4j.appender.FA.RollingPolicy=org.apache.log4j.rolling.TimeBasedRollingPolicy
log4j.appender.FA.RollingPolicy.FileNamePattern=.\\logs\\application.log-%d{dd-MM-yyyy}
log4j.appender.FA.File=.\\logs\\application.log
log4j.appender.FA.layout=org.apache.log4j.EnhancedPatternLayout
log4j.appender.FA.layout.ConversionPattern=%d %p %t %c: %m%n
log4j.appender.FA.Append=true