I am trying to change the default behaviour of the logging module so that it adds a prefix string to each log level.
logging.basicConfig(level=logging.INFO, format='%(message)s', filename='output.log', filemode='w')
console = logging.StreamHandler()
logging.getLogger('').addHandler(console) # add the handler to the root logger
log = logging.getLogger(__name__)
log.info("this is an info")
log.error("this is an error")
log.warning("this is a warning")
I want
log.error("this is an error")
to print:
Error: this is an error
Or, the following line,
log.warning("this is a warning")
to print:
Warning: this is a warning
Error prefix in red and warning prefix in yellow.
Is there any non-hacky way to do this?
Related
I have configured my logging to log both File & Stream.
I just want to see the print statements in my console & see other info on my log file (except unhandled exceptions as python shows them on the console by default)
I set the StreamLogging.setLevel(level=logging.CRITICAL) but it still shows API connection messages.
I want to get the Python's default stderr like before configuring the logging, but also change the output, like adding %(asctime)s before printing.
My Logging
import logging
FileLoggingFormat = f"%(message)s"
FileLoggingDate = r"%m/%d/%Y %H:%M:%S"
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()
# Add StreamHandler
StreamLogging = logging.StreamHandler()
logger.addHandler(StreamLogging)
StreamLogging.setFormatter(logging.Formatter("%(message)s"))
StreamLogging.setLevel(level=logging.CRITICAL)
# Add FileHandler
FileLogging = logging.FileHandler(
filename="Log.log",
mode="w",
encoding="utf-8"
)
logger.addHandler(FileLogging)
FileLogging.setFormatter(logging.Formatter(FileLoggingFormat, FileLoggingDate))
❌ Not suitable Result in Console
(Printed statements showed by 📝)
📝 Port: 1523
📝 NotFound.
INFO:library.connection.connection:Connecting...
Traceback (most recent call last):
File "c:\Users\test.py", line 20, in <module>
f.write(message)
TypeError: write() argument must be str
INFO:library.session.session:Session stopped
Session stopped
✅ Suitable Result in Console
(Only display prints & unhandled exceptions)
📝 Port: 1523
📝 NotFound.
Traceback (most recent call last):
File "c:\Users\test.py", line 20, in <module>
f.write(message)
TypeError: write() argument must be str
✅ Suitable Result in Console
(Only display prints & unhandled exceptions with date, added %(asctime)s to StreamHandler)
2022/10/22 10:16:03.678
📝 Port: 1523
2022/10/22 10:16:03.700
📝 NotFound.
2022/10/22 10:16:03.706
Traceback (most recent call last):
File "c:\Users\test.py", line 20, in <module>
f.write(message)
TypeError: write() argument must be str
There are several things that need explaining:
Using logging.basicConfig
You use logging.basicConfig(level=logging.INFO). That has for effect to create a StreamHandler and attach it to the root logger (the one you are using since logger = getLogger() and getLogger() with no argument returns the "root" logger).
At the end of your logging setup, if you do:
print(logger, logger.handlers)
It will give:
<RootLogger root (INFO)> [<StreamHandler <stderr> (NOTSET)>, <StreamHandler <stderr> (CRITICAL)>, <FileHandler /home/vvvvv/Log.log (NOTSET)>]
You have 2 StreamHandler instead of just 1. Moreover, the superfluous StreamHandler has a logging level of NOTSET meaning it will log everything. This is the cause of your bug.
You could solve it by removing this line:
logging.basicConfig(level=logging.INFO)
Doing so, gives
<RootLogger root (INFO)> [<StreamHandler <stderr> (CRITICAL)>, <FileHandler /home/vvvvv/Log.log (NOTSET)>]
Using a child logger instead of the root logger
As previously said, you are using the root logger (logger = logging.getLogger()). You should refrain from doing so and instead use a child logger, like this:
# logger = logging.getLogger()
logger = logging.getLogger("foobar")
logger.propagate = False
Set propagate to False to avoid sending the logs upwards to the root logger.
Piecing everything together gives the correct behavior:
import logging
FileLoggingFormat = f"%(message)s"
FileLoggingDate = r"%m/%d/%Y %H:%M:%S"
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("foobar")
logger.propagate = False
# Add StreamHandler
StreamLogging = logging.StreamHandler()
StreamLogging.setFormatter(logging.Formatter("%(message)s"))
StreamLogging.setLevel(level=logging.CRITICAL)
logger.addHandler(StreamLogging)
# Add FileHandler
FileLogging = logging.FileHandler(
filename="Log.log",
mode="w",
encoding="utf-8"
)
logger.addHandler(FileLogging)
FileLogging.setFormatter(
logging.Formatter(FileLoggingFormat, FileLoggingDate)
)
I'd like to set different logging levels per handler and logger in python. I'd like debug level logs for all loggers to be sent to their filehandlers, but be able to individually select the log level for each logger for what's printed to the console.
import logging
loggerA = logging.getLogger('a')
fhA = logging.FileHandler(filename='a.log', mode='w')
fhA.setLevel(logging.DEBUG)
loggerA.addHandler(fhA)
loggerB = logging.getLogger('b')
fhB = logging.FileHandler(filename='b.log', mode='w')
fhB.setLevel(logging.DEBUG)
loggerB.addHandler(fhB)
logging.basicConfig(level=logging.INFO)
logging.getLogger('a').setLevel(logging.INFO)
logging.getLogger('b').setLevel(logging.WARN)
loggerA.info("TEST a")
loggerB.info("TEST b")
I'd expect all logs to be sent to the files, but only show WARNING and above from 'b' in the console, and INFO and above from 'a' in the console. With the above code, 'b' doesn't have any logs in the file.
I created a small function to setup logging, with a filehandler for 'everything', and smtphandler for error and above. Error logs write to the log file and send correctly to email, but debug, info, notset don't, even though setlevel is set to 0 for filehandler. Why's that? Code below
#logsetup.py
import logging
import logging.handlers
def _setup_logger(name, log_file):
"""Function to setup logger"""
logger = logging.getLogger(name)
#Create Formatters
file_formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
mail_formatter = logging.Formatter('%(name)s - %(message)s')
#Create handler, set formatting, set log level
file_handler_obj = logging.FileHandler(log_file)
file_handler_obj.setFormatter(file_formatter)
file_handler_obj.setLevel(0)
#Create handler, set formatting, set log level
smtp_handler_obj = logging.handlers.SMTPHandler(mailhost=('smtp.gmail.com', 587),
fromaddr='mymail#example.com',
toaddrs='mymail#example.com',
subject='Error in Script',
credentials=('mymail#example.com', 'pwexample'), #username, password
secure=())
smtp_handler_obj.setFormatter(mail_formatter)
smtp_handler_obj.setLevel(logging.ERROR)
# add the handlers to logger
logger.addHandler(smtp_handler_obj)
logger.addHandler(file_handler_obj)
return logger
#mytest.py
import time
import logsetup
if __name__ == '__main__':
TEST_SETTINGS = config_funcs._get_config('TEST_SETTINGS')
logtime = time.strftime('%Y%m%d') # -%H%M%S")
log = logsetup._setup_logger('TEST', TEST_SETTINGS['logging_dir'] + 'Py_Log_%s.log' % logtime)
log.error('Writes to log file and sends email')
log.debug('Supposed to write to log file, does nothing.')
Apparently, logging needs it's own logging level aside from the handlers. Setting logger.setLevel(logging.DEBUG) right before returning logger causes it to work correctly. Documentation says
When a logger is created, the level is set to NOTSET (which causes all
messages to be processed when the logger is the root logger, or
delegation to the parent when the logger is a non-root logger). Note
that the root logger is created with level WARNING.
Which means that if the handlers are lower level than the root logger (which ERROR is not, but DEBUG is) then the handlers which I guess is a child because I'm getting a named logger? Not quite sure on the why of it, but that does 'fix' it, in case anyone comes to this later.
I would like to create log file for each iteration of my script. I am creating logger for each iteration and i need close this opened file at the end of iteration because if I am not, I will recieve error Too many open files.
formatter = logging.Formatter('%(asctime)s %(message)s')
handler = logging.FileHandler(logging_file)
handler.setFormatter(formatter)
logger = logging.getLogger(name)
logger.setLevel(logging.INFO)
logger.addHandler(handler)
handler.close()
This should work but I recieved SyntaxError:
File "start.py", line 73
handler.close()
^
SyntaxError: invalid syntax
How should I close file handlers at the end of iteration? I tried to create handler and close it immediately to be sure there is no problem that I added handler to logger before, but the problem persist:
handler = logging.FileHandler(logging_file)
handler.close()
I'm trying to add some logging to my application however I can't seem to get the correct log level statements to display. I have set my logger to INFO yet it only displays warnings and errors in both the console and log file
Am I missing anything?
import logging
logger = logging.getLogger("mo_test")
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
fh = logging.FileHandler('test.log')
ch = logging.StreamHandler()
ch.setLevel(logging.INFO)
fh.setLevel(logging.INFO)
fh.setFormatter(formatter)
ch.setFormatter(formatter)
logger.addHandler(fh)
logger.addHandler(ch)
logger.info("This is an Info")
logger.debug("this is a debug")
logger.warning("This is a warning")
logger.error("Oh error")
The content of the log file and console is then only:
2015-03-09 15:32:44,601 - mo_test - WARNING - This is a warning
2015-03-09 15:32:44,601 - mo_test - ERROR - Oh error
Thanks
Set the logging level on the logger. If you don't set it, the default logging level is 30, i.e. logging.WARNING:
logger = logging.getLogger("mo_test")
logger.setLevel(logging.INFO)
The logging flow chart shows how the logger itself filters by logging level ("Logger enabled for level of call?") even before the handlers get a chance to handle the record ("Handler enabled for level of LogRecord?"):
You seem to be missing a call to ``logging.basicConfig(). Without this call,logging` is in an unconfigured state and anything can happen.