I am trying to set up a logger that will write a new timestamped log file every time the application is run in a specific directory.
for example what I am trying to do is
timestampFilename = time.strftime("runtimelog%b_%d_%Y_%H:%M:%S.txt")
fh = logging.FileHandler(r'C:\my\folder\logs\'+timestampFilename, mode='w')
An example tweeked from the Logging Cookbook:
import logging
import os
from datetime import datetime
# create logger with 'spam_application'
logger = logging.getLogger('MYAPP')
logger.setLevel(logging.DEBUG)
# create file handler which logs in a specific directory
logdir = '.'
if 'APP_LOG_DIR' in os.environ:
logdir = os.environ['APP_LOG_DIR']
logfile = datetime.now().strftime("run_%b_%d_%Y_%H_%M_%S.log")
fh = logging.FileHandler(os.path.join(logdir, logfile))
fh.setLevel(logging.DEBUG)
# create formatter and add it to the handlers
formatter = logging.Formatter('[%(asctime)s][%(name)s][%(levelname)s] %(message)s')
fh.setFormatter(formatter)
# add the handlers to the logger
logger.addHandler(fh)
logger.debug("my first log line")
The log directory may be configured with the environment variable APP_LOG_DIR and path names are built in a platform independent way thanks to os.path.
Related
I need to disable Tornado from logging to STDOUT. I am using Python 3.8 and is running on Ubuntu 18.04. I want my log statements to be handled by a rotating file logger only. The issue is that logged statements are logged into the file and also to console:
import logging
from logging.handlers import RotatingFileHandler
logger = logging.getLogger("ex_logger")
nh = logging.NullHandler()
rfh = RotatingFileHandler(filename="./logs/process.log", mode='a', maxBytes=50000000, backupCount=25, encoding=None, delay=False)
rfh.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
rfh.setFormatter(formatter)
logger.handlers = []
logger.propagete = False
logger.addHandler(rfh)
logging.getLogger("tornado.access").handlers = []
logging.getLogger("tornado.application").handlers = []
logging.getLogger("tornado.general").handlers = []
logging.getLogger("tornado.access").addHandler(nh)
logging.getLogger("tornado.application").addHandler(nh)
logging.getLogger("tornado.general").addHandler(nh)
logging.getLogger("tornado.access").propagate = False
logging.getLogger("tornado.application").propagate = False
logging.getLogger("tornado.general").propagate = False
....
def main():
######
# this message eppears in both the output log file and stdout
######
logger.info(" application init ... ")
asyncio.set_event_loop_policy(tornado.platform.asyncio.AnyThreadEventLoopPolicy())
tornado.options.parse_command_line()
app = Application()
app.listen(options.port)
tornado.ioloop.IOLoop.current().start()
if __name__ == "__main__":
main()
The problem is that from the moment you start up your IDE, logging.getLogger("tornado") may have a StreamHandler attached. This doesn't happen with every IDE but it does happen with Spyder. So that is the one you have to replace by a NullHandler:
import logging
nh = logging.NullHandler()
tornado_logger = logging.getLogger("tornado")
tornado_logger.handlers.clear()
# tornado_logger.handlers = [] # would work instead of .clear() but it's best practice to change the list and not replace it, just in case some other variable name somewhere else still refers to it.
tornado_logger.addHandler(nh)
You don't need to do anything with the children of the "tornado" logger, e.g. "tornado.access", et cetera.
You also need to define a logging policy for the root handler (logging.getLogger("")). Tornado looks at the root handler to decide whether logging has already been configured or needs a default setup.
I created a small function to setup logging, with a filehandler for 'everything', and smtphandler for error and above. Error logs write to the log file and send correctly to email, but debug, info, notset don't, even though setlevel is set to 0 for filehandler. Why's that? Code below
#logsetup.py
import logging
import logging.handlers
def _setup_logger(name, log_file):
"""Function to setup logger"""
logger = logging.getLogger(name)
#Create Formatters
file_formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
mail_formatter = logging.Formatter('%(name)s - %(message)s')
#Create handler, set formatting, set log level
file_handler_obj = logging.FileHandler(log_file)
file_handler_obj.setFormatter(file_formatter)
file_handler_obj.setLevel(0)
#Create handler, set formatting, set log level
smtp_handler_obj = logging.handlers.SMTPHandler(mailhost=('smtp.gmail.com', 587),
fromaddr='mymail#example.com',
toaddrs='mymail#example.com',
subject='Error in Script',
credentials=('mymail#example.com', 'pwexample'), #username, password
secure=())
smtp_handler_obj.setFormatter(mail_formatter)
smtp_handler_obj.setLevel(logging.ERROR)
# add the handlers to logger
logger.addHandler(smtp_handler_obj)
logger.addHandler(file_handler_obj)
return logger
#mytest.py
import time
import logsetup
if __name__ == '__main__':
TEST_SETTINGS = config_funcs._get_config('TEST_SETTINGS')
logtime = time.strftime('%Y%m%d') # -%H%M%S")
log = logsetup._setup_logger('TEST', TEST_SETTINGS['logging_dir'] + 'Py_Log_%s.log' % logtime)
log.error('Writes to log file and sends email')
log.debug('Supposed to write to log file, does nothing.')
Apparently, logging needs it's own logging level aside from the handlers. Setting logger.setLevel(logging.DEBUG) right before returning logger causes it to work correctly. Documentation says
When a logger is created, the level is set to NOTSET (which causes all
messages to be processed when the logger is the root logger, or
delegation to the parent when the logger is a non-root logger). Note
that the root logger is created with level WARNING.
Which means that if the handlers are lower level than the root logger (which ERROR is not, but DEBUG is) then the handlers which I guess is a child because I'm getting a named logger? Not quite sure on the why of it, but that does 'fix' it, in case anyone comes to this later.
Hello folks!
#this is values from xml file.
clientID = logObject['meta']['clientID']
authToken = logObject['meta']['authToken']
logType = logObject['logType']
FORMAT = '%(asctime)-15s %(logType)s %(process)d %(user)-8s %(message)s'
d = {'logType': logType ,'user': getpass.getuser()}
#line creating two log files(access.log and error.log)
logging.basicConfig(filename = 'access.log', filemode = 'w', format=FORMAT)
logging.basicConfig(filename = 'error.log', filemode = 'w', format=FORMAT)
if(clientID == ""):
# logger = setup_logger('first_logger', 'access.log',logType)
logger.warning('Please Enter clientID', extra=d)
This is my sample code.
what i need is to create two files which i have mentioned. but problem is it only creates single file everytime and messages goes to that file only.
So i want that if i mention logger.error("msg") or logger.warning("msg") then it should go to that log file.
When you use just logging you actually use single root logger created during import logging. You may use it with several handlers. For example:
# 1. logging to file
filename = (
'log_file{}.log'
.format(
dt.datetime.today().strftime("_date_%Y-%m-%d_time_%H-%M-%S")))
# path to log folder.
path = os.path.join(os.getcwd(), 'logs')
# create log folder if it does not exist.
if not os.path.isdir(path):
os.makedirs(path, exist_ok=True)
to_file = logging.FileHandler(os.path.join(path, filename))
to_file.addFilter(lambda x: x.levelno in [20, 40])
# 2. logging to console
to_console = logging.StreamHandler()
to_console.addFilter(lambda x: x.levelno in [20, 21, 40])
# 3. root logger configuration
logging.basicConfig(
level=10,
datefmt='%Y-%m-%d %H:%M:%S',
format='[%(asctime)s]:%(threadName)s:%(levelname)s:%(message)s',
handlers=[to_console, to_file])
If you want to log into 2 files then just create 2 handlers logging.FileHandler(...), register them, and use newly configured root logger as usual:
logging.info('Some info')
Another option is to create 2 loggers. Usually you need to do so if you want to separate several sources of log messages.
I am using the logging module in Python 3.5, I feel like I have done this before, but I would like like to change the logging level with a constant supplied by my config file.
Currently I am using:
logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)
logger.setLevel(logging.INFO)
I would like to declare INFO, DEBUG etc in the config file and replace using the literal.
my_script_config:
LOG_LEVEL = 'DEBUG' #this value to be modified by user.
script:
import my_script_conf as msc
SET_LEVEL = 'logging.' + msc.LOG_LEVEL
BASIC_LEVEL = 'level=' + SET_LEVEL
logger = logging.getLogger(__name__)
logging.basicConfig(BASIC_LEVEL)
logger.setLevel(SET_LEVEL)
Python gets upset by this, any help on my terrible coding would be much appreciated. Any route to achieve the same result, I basically want it readable and easy to use, to have the logging modules level set in the external config file seems like the most sensible option. You pythonista's may have other ideas!
Thanks, Frank
I concatenated the string in a another variable then ran an exec on the variable.
my_script_config:
LOG_LEVEL = 'DEBUG' #this value to be modified by user.
script:
import my_script_conf as msc
SET_LEVEL = 'logger.setLevel(' + msc.LOG_LEVEL + ')
BASIC_LEVEL = 'logging.basicConfig(level=' + msc.LOG_LEVEL + ')'
logger = logging.getLogger(__name__)
exec(SET_LEVEL)
exec(BASIC_LEVEL)
First of all, sorry for my English.
i'm using Python3.x. I want to create a handler that storage the current logging
information into a table in SQLite3.
i've a handler that show by console de current logging and another that write the
same into a txt file.
I leave you my example:
import logging
class Logger(object):
'''test logger'''
def __init__(self):
'''constructor'''
self.logger = logging.getLogger('CIRE')
self.logger.setLevel(logging.DEBUG)
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
formatter = '%(asctime)s. %(name)s|%(levelname)s| %(message)s'
dateFormat = '%m/%d/%Y %I:%M:%S %p'
logging.basicConfig(format=formatter, datefmt=dateFormat, filename='Logger.log')
formatter = logging.Formatter(formatter, dateFormat)
ch.setFormatter(formatter)
self.logger.addHandler(ch)
l = Logger()
l.logger.debug("test")
OUTPUT:
02/07/2017 12:54:13 PM. CIRE|DEBUG| test
thanks!!