I need to append date as my logfile name contains date at the end.
e.g :
access_log.2013-12-11
access_log.2013-12-10
access_log.2013-12-09
access_log.2013-12-08
.
.
.
access_log.2013-09-08
AsI need to set logpath name under Fail2ban conf file (i.e jail.local)
I am aware that I can use '*' while mentioning log file name but as our log files are large and we also store 30 days worth of log files, so I thought it is not a good practice and will also performance related effects.
logpath = /opt/atlassian/jira/logs/access_log.*
Tested the below one's :
logpath = /opt/atlassian/jira/logs/access_log.%Y-%m-%d
logpath = "/opt/atlassian/jira/logs/access_log.%Y-%m-%d"
logpath = "/opt/atlassian/jira/logs/access_log.'%Y-%m-%d'"
but none have worked
Can anyone please help me in appending a variable date at the end of logpath to cover the above mentioned log files
I am not sure how you are getting your dates, but this will create a logpath in the format you desire
import datetime
d = datetime.date.today().strftime('%Y-%m-%d')
logpath = "/opt/atlassian/jira/logs/access_log/%s" %d
print logpath
Related
In an application, I can get the path to a file which resides in a directory as a string:
"/path/to/the/file.txt"
In order to write another another file into that same directory, I want to change the string "/path/to/the/file.txt" and remove the part "file.txt" to finally only get
"/path/to/the/"
as a string
I could use
string = "/path/to/the/file.txt"
string.split('/')
and then glue all the term (except the last one) together with a loop
Is there an easy way to do it?
You can use os.path.basename for getting last part of path and delete it with using replace.
import os
path = "/path/to/the/file.txt"
delete = os.path.basename(os.path.normpath(path))
print(delete) # will return file.txt
#Remove file.txt in path
path = path.replace(delete,'')
print(path)
OUTPUT :
file.txt
/path/to/the/
Let say you have an array include txt files . you can get all path like
new_path = ['file2.txt','file3.txt','file4.txt']
for get_new_path in new_path:
print(path + get_new_path)
OUTPUT :
/path/to/the/file2.txt
/path/to/the/file3.txt
/path/to/the/file4.txt
Here is what I finally used
iter = len(string.split('/'))-1
directory_path_str = ""
for i in range(0,iter):
directory_path_str = directory_path_str + srtr.split('/')[i] + "/"
I need to pass the word true or false to a data template file in terraform. However, if I try to provide the value, it comes out 0 or 1 due to interpolation syntax. I tried doing \\true\\ as recommended in https://www.terraform.io/docs/configuration/interpolation.html, however that results in \true\, which obviously isn't right. Same with \\false\\ = \false\
To complicate matters, I also have a scenario where I need to pass it the value of a variable, which can either equal true or false.
Any ideas?
# control whether to enable REST API and set other port defaults
data "template_file" "master_spark_defaults" {
template = "${file("${path.module}/templates/spark/spark- defaults.conf")}"
vars = {
spark_server_port = "${var.application_port}"
spark_driver_port = "${var.spark_driver_port}"
rest_port = "${var.spark_master_rest_port}"
history_server_port = "${var.history_server_port}"
enable_rest = "${var.spark_master_enable_rest}"
}
}
var.spark_master_enable_rest can be either true or false. I tried setting the variable as "\\${var.spark_master_enable_rest}\\" but again this resulted in either \true\ or \false\
Edit 1:
Here is the relevant portion of conf file in question:
spark.ui.port ${spark_server_port}
# set to default worker random number.
spark.driver.port ${spark_driver_port}
spark.history.fs.logDirectory /var/log/spark
spark.history.ui.port ${history_server_port}
spark.worker.cleanup.enabled true
spark.worker.cleanup.appDataTtl 86400
spark.master.rest.enabled ${enable_rest}
spark.master.rest.port ${rest_port}
I think you must be overthinking,
if i set my var value as
spark_master_enable_rest="true"
Then i get :
spark.worker.cleanup.enabled true
spark.worker.cleanup.appDataTtl 86400
spark.master.rest.enabled true
in my result when i apply.
I ended up creating a cloud-config script to find/replace the 0/1 in the file:
part {
content_type = "text/x-shellscript"
content = <<SCRIPT
#!/bin/sh
sed -i.bak -e '/spark.master.rest.enabled/s/0/false/' -e '/spark.master.rest.enabled/s/1/true/' /opt/spark/conf/spark-defaults.conf
SCRIPT
}
I have N files in the same folder with different index numbers like
Fe_1Sec_1_.txt
Fe_1Sec_2_.txt
Fe_1Sec_3_.txt
Fe_2Sec_1_.txt
Fe_2Sec_2_.txt
Fe_2Sec_3_.txt
.
.
.
and so on
Ex: If I need to run my code with only the files with time = 1 Sec, I can make it manually as follow:
path = "input/*_1Sec_*.txt"
files = glob.glob(path)
print(files)
which gave me:
Out[103]: ['input\\Fe_1Sec_1_.txt', 'input\\Fe_1Sec_2_.txt', 'input\\Fe_1Sec_3_.txt']
In case of I need to run my code for all files separately (depending on the measurement time in seconds, i.e. the name of file)
I tried this code to get the path for each time of measurement:
time = 0
while time < 4:
time += 1
t = str(time)
path = ('"input/*_'+t+'Sec_*.txt"')
which gives me:
"input/*_1Sec_*.txt"
"input/*_2Sec_*.txt"
"input/*_3Sec_*.txt"
"input/*_4Sec_*.txt"
After that I tried to use this path as follow:
files = glob.glob(path)
print(files)
But it doesn't import the wanted files and give me :
"input/*_1Sec_*.txt"
[]
"input/*_2Sec_*.txt"
[]
"input/*_3Sec_*.txt"
[]
"input/*_4Sec_*.txt"
[]
Any suggestions, please??
I think the best way would be to simply do
for time in range(1, 5): # 1,2,3,4
glob_path = 'input/*_{}Sec_*.txt'.format(time)
for file_path in glob.glob(glob_path):
do_something(file_path, measurement) # or whatever
I require the log4net to create date wise with log file name as "Application.20130125.txt", format of "Application.yyyyMMdd.txt" at the time of creating log file instead at the time of date changing at night 12 AM.
And when the date changes, it should create new logfile as "Application.20130126.txt"
Could you please suggest what changes need to be done to the
below code so that the date is created first and at the time of rolling,
creates a new file with the date "Application.20130126.txt"
LogPath = "C:\Logs\"
fileName = "ApplicationName" & "..txt"
hierarchy = DirectCast(LogManager.GetRepository(), Hierarchy)
patternLayout.ConversionPattern = "%m%n"
patternLayout.ActivateOptions()
roller.Layout = patternLayout
roller.RollingStyle = RollingFileAppender.RollingMode.Date
roller.DatePattern = "yyyyMMdd"
roller.AppendToFile = True
roller.StaticLogFileName = True
roller.File = LogPath & fileName
roller.PreserveLogFileNameExtension = True
roller.ActivateOptions()
hierarchy.Root.AddAppender(roller)
hierarchy.Root.Level = Level.Debug
hierarchy.Configured = True
log = LogManager.GetLogger("RollingFileAppender")
Please find my sample below that i have used that would write to the file as you need:
Dim fileappender = New log4net.Appender.RollingFileAppender()
fileappender.AppendToFile = True
fileappender.Threshold = log4net.Core.Level.Debug
fileappender.File = "MyLogFile_"
fileappender.DatePattern = "yyyyMMdd"
fileappender.StaticLogFileName = False
fileappender.Layout = New log4net.Layout.SimpleLayout()
fileappender.RollingStyle = log4net.Appender.RollingFileAppender.RollingMode.Date
fileappender.ActivateOptions()
DirectCast(log4net.LogManager.GetRepository(), log4net.Repository.Hierarchy.Hierarchy).Root.AddAppender(fileappender)
log4net.Config.BasicConfigurator.Configure(fileappender)
The above would create a log file with its filename as "MyLogFile_20130125" today
and once the date time changes it would write a new file as "MyLogFile_20130126"
You can alternatively test it by setting the datePattern to "yyyyMMddhhmm" and that would write a new log file each minute.
I'm reading up on Log4perl and want to try and use it for simple log management of my Perl scripts running on a Linux box. I've also read up on newsyslog and logrotate but want to use Log4perl if at all possible.
I'm trying to configure the /etc/log4perl.conf file so that it:
Defines a widget logger (INFO level) that will write all output to /opt/myapp/logs/myapp-<datetime>.log, where <datetime> is a date/time formatted string like 2012-12-20
This myapp-<datetime>.log file needs to be rotated daily (preferably at midnight), where the old file is deleted, and a new file is created with <datetime> + 1. For instance, myapp-2012-12-20.log would be replaced with myapp-2012-12-21.log, etc.
Here's my best attempt which I believe is close, but is still missing some configuration:
#####/etc/log4perl.conf############################################################
log4perl.logger.widget = INFO, MyAppLogAppender
log4perl.appender.MyAppLogAppender = Log::Log4perl::Appender::File
log4perl.appender.MyAppLogAppender.filename = /opt/myapp/logs/myapp-???.log
log4perl.appender.MyAppLogAppender.layout = Log::Log4perl::Layout::SimpleLayout
###################################################################################
How do I configure log4perl.appender.MyAppLogAppender to rotate once a day, delete the old file, and create a new one with a correct timestamp? Thanks in advance.
Here's an example of a Log::Log4perl configuration file, defining a daily rollover at midnight (date pattern yyyy-MM-dd), keeping a maximum of 5 saved logfiles around, at WARN level, and dumping everything to screen:
log4perl.logger = TRACE, Screen, Logfile
log4perl.appender.Logfile = Log::Dispatch::FileRotate
log4perl.appender.Logfile.Threshold = WARN
log4perl.appender.Logfile.filename = test.log
log4perl.appender.Logfile.max = 5
log4perl.appender.Logfile.DatePattern = yyyy-MM-dd
log4perl.appender.Logfile.TZ = PST
log4perl.appender.Logfile.layout = Log::Log4perl::Layout::PatternLayout
log4perl.appender.Logfile.layout.ConversionPattern = %d %m %n
log4perl.appender.Screen = Log::Log4perl::Appender::Screen
log4perl.appender.Screen.stderr = 0
log4perl.appender.Screen.utf8 = 1
log4perl.appender.Screen.layout = Log::Log4perl::Layout::PatternLayout::Multiline
log4perl.appender.Screen.layout.ConversionPattern = [%p] %m %n
(reference: https://metacpan.org/module/Log::Log4perl::FAQ#How-can-I-roll-over-my-logfiles-automatically-at-midnight-)
There is a gotcha in Log::Dispatch::FileRotate, if your daily job is run later in the day (say 23:00) and takes 2h (so ends at about 01:00), the log rotation will never happens with a "day" pattern like :
log4perl.appender.Logfile.DatePattern = yyyy-MM-dd
A simple workaround is to use an "hourly" pattern like this :
log4perl.appender.Logfile.DatePattern = yyyy-MM-dd-HH
So when the next run starts at 23:00, the log file get rotated as more than one hour has passed.