I'm reading up on Log4perl and want to try and use it for simple log management of my Perl scripts running on a Linux box. I've also read up on newsyslog and logrotate but want to use Log4perl if at all possible.
I'm trying to configure the /etc/log4perl.conf file so that it:
Defines a widget logger (INFO level) that will write all output to /opt/myapp/logs/myapp-<datetime>.log, where <datetime> is a date/time formatted string like 2012-12-20
This myapp-<datetime>.log file needs to be rotated daily (preferably at midnight), where the old file is deleted, and a new file is created with <datetime> + 1. For instance, myapp-2012-12-20.log would be replaced with myapp-2012-12-21.log, etc.
Here's my best attempt which I believe is close, but is still missing some configuration:
#####/etc/log4perl.conf############################################################
log4perl.logger.widget = INFO, MyAppLogAppender
log4perl.appender.MyAppLogAppender = Log::Log4perl::Appender::File
log4perl.appender.MyAppLogAppender.filename = /opt/myapp/logs/myapp-???.log
log4perl.appender.MyAppLogAppender.layout = Log::Log4perl::Layout::SimpleLayout
###################################################################################
How do I configure log4perl.appender.MyAppLogAppender to rotate once a day, delete the old file, and create a new one with a correct timestamp? Thanks in advance.
Here's an example of a Log::Log4perl configuration file, defining a daily rollover at midnight (date pattern yyyy-MM-dd), keeping a maximum of 5 saved logfiles around, at WARN level, and dumping everything to screen:
log4perl.logger = TRACE, Screen, Logfile
log4perl.appender.Logfile = Log::Dispatch::FileRotate
log4perl.appender.Logfile.Threshold = WARN
log4perl.appender.Logfile.filename = test.log
log4perl.appender.Logfile.max = 5
log4perl.appender.Logfile.DatePattern = yyyy-MM-dd
log4perl.appender.Logfile.TZ = PST
log4perl.appender.Logfile.layout = Log::Log4perl::Layout::PatternLayout
log4perl.appender.Logfile.layout.ConversionPattern = %d %m %n
log4perl.appender.Screen = Log::Log4perl::Appender::Screen
log4perl.appender.Screen.stderr = 0
log4perl.appender.Screen.utf8 = 1
log4perl.appender.Screen.layout = Log::Log4perl::Layout::PatternLayout::Multiline
log4perl.appender.Screen.layout.ConversionPattern = [%p] %m %n
(reference: https://metacpan.org/module/Log::Log4perl::FAQ#How-can-I-roll-over-my-logfiles-automatically-at-midnight-)
There is a gotcha in Log::Dispatch::FileRotate, if your daily job is run later in the day (say 23:00) and takes 2h (so ends at about 01:00), the log rotation will never happens with a "day" pattern like :
log4perl.appender.Logfile.DatePattern = yyyy-MM-dd
A simple workaround is to use an "hourly" pattern like this :
log4perl.appender.Logfile.DatePattern = yyyy-MM-dd-HH
So when the next run starts at 23:00, the log file get rotated as more than one hour has passed.
Related
I am trying to change the code in a phyton script for OBS studio to show the dates of coming events from a google calendar. But the output to OBS Studio only shows the same (last) date on every event. The script log shows it as it should be thou...
After struggling to find a way to convert the dictionary items to print in a way that I wanted to show it, I finally thought I had made it work the way I wanted.
I am new to python and have basically just searched for answers to how to solve what I needed to change in the code.
It took me days to find out about datetime.datetime and how strftime could work together, and that I needed to upgrade Dateutil to a more recent version to not get some of the errors I got.
Anyway, since I am new to coding and most of this script has been written by someone else it is somewhat hard for me to see where this problem lies.
it works as it should in the script log but the date in "stime" becomes the same for every event when I send it to "text" in OBS Studio.
If anyone could help me with a solution to this, I would be very happy.
# Time objects using datetime
dt_now = dt.utcnow()
now = dt.utcnow().isoformat() + 'Z' # 'Z' indicates UTC time
#Timeformat
locale.setlocale(locale.LC_TIME, "sv_SE") # swedish
tmfmt = '%d %B, %H:%M '
# Gets events currently happending by setting bounds to events happening within a second of current datetime
events = service.events().list(calendarId=cal_url, timeMin=now, timeMax=(dt_now+datetime.timedelta(7,1)).isoformat() +'Z',
maxResults=max_events, singleEvents=True, orderBy='startTime').execute()
# Logs the events to console
for event in events['items']:
mystart = (event['start']['dateTime'])
stime = dt.strftime(dtparse(mystart), format=tmfmt)
print(stime)
#print(datetime.datetime.utcnow().date())
#print (event['start']['dateTime'])
print(event['summary'])
#print(dt_now("%d %b, %Y"))
# Updates the text for each event
count = 0
stream_event_happening = False
record_event_happening = False
for event in events['items']:
if(count >= max_events):
break
text = stime + "\n" + event['summary']
settings = obs.obs_data_create()
obs.obs_data_set_string(settings, "text", text)
source = obs.obs_get_source_by_name(source_names[count])
obs.obs_source_update(source, settings)
obs.obs_data_release(settings)
obs.obs_source_release(source)
settings2 = obs.obs_data_create()
obs.obs_data_set_string(settings2, "file", "{}/{}.jpg".format(images_path, text))
source2 = obs.obs_get_source_by_name(image_sources[count])
obs.obs_source_update(source2, settings2)
obs.obs_data_release(settings2)
obs.obs_source_release(source2)
count += 1
text = stime + "\n" + event['summary']
shows only the same date but different events...
Wow, just a few minutes later I found a solution on my own... I added:
mystart = (event['start']['dateTime'])
stime = dt.strftime(dtparse(mystart), format=tmfmt)
just before:
text = stime + "\n" + event['summary']
and now it works as it should :)
I am using the below configuration to generate the log rolling based on size. The logs are created, however the rolling stops after creating the max files (10). The default file is keep on overwriting and all the rolled over files never gets updated.
appender.sample.type = RollingFile
appender.sample.name = SampleRollingFile
appender.sample.fileName = ${basedir}/logs/${machine}_My_Sample.0.log
appender.sample.filePattern = ${basedir}/logs/${machine}_My_Sample.%i.log
appender.sample.layout.type = PatternLayout
appender.sample.layout.pattern = %d{yyyy-MM-dd HH:mm:ss} %-5p [T:%T] %c{1}:%L - %m%n
appender.sample.policies.type = Policies
appender.sample.policies.size.type = SizeBasedTriggeringPolicy
appender.sample.policies.size.size = 100MB
appender.sample.strategy.type = DefaultRolloverStrategy
appender.sample.strategy.max = 10
But when I changed my default initial file name to My_Sample.log (removing the .0 index) it works perfectly fine. My application requirement is initial file with .0 and the rolling the logs from 1 to 10.
I added the below properties to get the desired result. By enabling tracing (status = TRACE) in the log4j2 they use the index 0 to play around with the renaming and purge. To avoid the collision, I started the index from 1 and the continuous rolling is happening !!
appender.sample.strategy.fileIndex = 1
appender.sample.strategy.min = 1
I'm looking for a little help on a Lua script. Essentially I'm looking to match an approaching date X number of minutes prior to today. In the example below I've used 9000 minutes.
alarm.get ()
message = "Certificate Expiry Warning - Do something"
SUPPKEY = "Certificate Expiry"
SUBSYS = "1.1"
SOURCE = "SERVERNAME"
--local pattern = "(%d-%m-%Y)"
local t = os.date('*t'); -- get current date and time
print(os.date("%d-%m-%Y")); --Prints todays date
t.min = t.min - 9000; -- subtract 9000 minutes
--print(os.date("%Y-%m-%d %H:%m:%S", os.time(t))); --Original Script
print(os.date("%d-%m-%Y", os.time(t))); --Prints alerting date
if string.match ~=t.min --Match string
--if string.match(a.message, pattern)
--then print (al.message)
then print ("We have a match")
--then nimbus.alarm (1, message , SUPPKEY , SUBSYS , SOURCE) --Sends alert
else print ("Everything is fine") --Postive, no alert
--else print (al.message)
end
The alarm.get grabs a line of text that looks like this:
DOMAIN\USERNAME,Web Server (WebServer),13/01/2017 09:13,13/01/2019,COMPANY_NAME,HOSTNAME_FQDN,SITE
So the line shown above is passed as an a.message variable and I'm looking to match the date highlighted in bold to today's date with 9000 minutes taken off it.
The commented out parts are just me testing different things.
I'm not sure if I understood the question well, but from my perspective it seems you are trying to do two things:
Retrieve current time minus 9000 minutes in format DD/MM/YYYY.
Compare this time to the one your program reads from file and do something, when the two dates are equal.
Here goes my sample code:
-- Settings
local ALLOWED_AGE = 9000 -- In minutes
-- Input line (for testing only)
local inputstr = "DOMAIN\\USERNAME,Web Server (WebServer),13/01/2017 09:13,13/01/2019,COMPANY_NAME,HOSTNAME_FQDN,SITE"
-- Separate line into 7 variables by token ","
local path, server, time, date, company_name, hostname, site = string.match(inputstr, "([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+)")
-- Check, if the line is ok (not necessary, but should be here to handle possible errors)
-- Also note, some additional checks should be here (eg. regex to match DD/MM/YYYY format)
if date == nil then
print("Error reading line: "..inputstr)
end
-- Get current time minus 9000 minutes (in format DD/MM/YYYY)
local target_date = os.date("%d/%m/%Y", os.time() - ALLOWED_AGE * 60)
-- Printing what we got (for testing purposes)
print("Target date: "..target_date..", Input date: "..date)
-- Testing the match
if target_date == date then
print("Dates are matched!")
else
print("Dates are not matched!")
end
Although I'm not sure, whether you shouldn't be instead checking for "one date is bigger/smaller then the other" in your case.
Then the code above should be modified to something like this:
-- Extract day, month and year from date in format DD/MM/YYYY
local d, m, y = string.match(date, "([^/]+)/([^/]+)/([^/]+)")
-- Note I'm adding one day, so the certificate will actually expire day after it's "valid until" date.
local valid_until = os.time({year = y, month = m, day = d + 1})
local expire_time = os.time() - ALLOWED_AGE * 60 -- All certificates older than this should expire.
-- Printing what we got (for testing purposes)
print("Expire time: "..expire_time..", Cert valid until: "..valid_until)
-- Is expired?
if valid_until <= expire_time then
print("Oops! Certificate expired.")
else
print("Certificate date is valid.")
end
I need to append date as my logfile name contains date at the end.
e.g :
access_log.2013-12-11
access_log.2013-12-10
access_log.2013-12-09
access_log.2013-12-08
.
.
.
access_log.2013-09-08
AsI need to set logpath name under Fail2ban conf file (i.e jail.local)
I am aware that I can use '*' while mentioning log file name but as our log files are large and we also store 30 days worth of log files, so I thought it is not a good practice and will also performance related effects.
logpath = /opt/atlassian/jira/logs/access_log.*
Tested the below one's :
logpath = /opt/atlassian/jira/logs/access_log.%Y-%m-%d
logpath = "/opt/atlassian/jira/logs/access_log.%Y-%m-%d"
logpath = "/opt/atlassian/jira/logs/access_log.'%Y-%m-%d'"
but none have worked
Can anyone please help me in appending a variable date at the end of logpath to cover the above mentioned log files
I am not sure how you are getting your dates, but this will create a logpath in the format you desire
import datetime
d = datetime.date.today().strftime('%Y-%m-%d')
logpath = "/opt/atlassian/jira/logs/access_log/%s" %d
print logpath
I'm trying to make a system which backs up and restores points for a gameserver, so it can safely restart without loosing anything.
I have made a script to do just this and the actual backing up part works fine, but the restore part does not.
This is the script that runs if 'Backup(read)' is used (Backup(write) works perfectly as it is designed to do):
if (source and read) then
System.LogAlways("[System] Restoring serverdata from file 'backup.CHK'");
for line in source:lines() do
Backup = {};
Backup.Date = (Date or line:match("File Last Modified: (.-)"));
Backup.Time = (Time or line:match("time: (.-)"));
US = tonumber((US or line:match("us: (.-)")));
NK = tonumber((NK or line:match("nk: (.-)")));
local params = {class = "Player";
position = {x = 1, y = 1, z = -1000};
Respawn = { bRespawn = 0; nTimer =0; bUnique = 1; };
bUsable = 0;
orientation = {0, 90, 135};
name = "BackupEntity"; };
local ent = System.SpawnEntity(params);
g_gameRules.game:SetTeam(1, ent.id);
g_gameRules.game:SetSynchedEntityValue(playerId, 100, (NK/3));
g_gameRules.game:SetTeam(2, ent.id);
g_gameRules.game:SetSynchedEntityValue(playerId, 100, (US/3));
System.RemoveEntity(params);
end
source:close();
return;
end
I'm not sure what I'm doing wrong,and most sites that I have looked at don't help that much. The problem is that it's not reading any values from the file.
Any help will be appreciated :).
Edit:
The reason that we have to divide the score by 3 is because the server multiplies all scores by 3. If we were not to divide it by 3, then the score will always be 3 times larger on each restore.
Example contents of the backup.CHK file:
The server is dependent on this file, and writes to it every hour. Please do not edit.
File Last Modified: 11/07/2013
This file was generated by the servers' autobackup system.
--------------------------
time: 22:51
us: 453445
nk: 454567
A couple of ideas of what might be causing the problem:
Use of (.-) lazy matching which matches the shortest pattern possible -- this can include an empty string. Usually, you want to make the pattern as specific as possible while still matching the required possible inputs. eg. It looks like (%d+) for us and nk is an appropriate fit.
The for line in source:lines() do reads one line at a time. That necessarily means not all the variables are going to be set inside the loop. Yet everything starting at local params and down uses those variables as if they were. It seems to me that section of code shouldn't even be in the loop.
Lastly, have you considered saving the Backup file as just another lua file? Doing so means you can let lua do the heavy lifting for you and you won't have to bother parsing it yourself. That also minimizes the risk for error.