Python logging not outputing correct levels? - python-3.x

I'm trying to add some logging to my application however I can't seem to get the correct log level statements to display. I have set my logger to INFO yet it only displays warnings and errors in both the console and log file
Am I missing anything?
import logging
logger = logging.getLogger("mo_test")
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
fh = logging.FileHandler('test.log')
ch = logging.StreamHandler()
ch.setLevel(logging.INFO)
fh.setLevel(logging.INFO)
fh.setFormatter(formatter)
ch.setFormatter(formatter)
logger.addHandler(fh)
logger.addHandler(ch)
logger.info("This is an Info")
logger.debug("this is a debug")
logger.warning("This is a warning")
logger.error("Oh error")
The content of the log file and console is then only:
2015-03-09 15:32:44,601 - mo_test - WARNING - This is a warning
2015-03-09 15:32:44,601 - mo_test - ERROR - Oh error
Thanks

Set the logging level on the logger. If you don't set it, the default logging level is 30, i.e. logging.WARNING:
logger = logging.getLogger("mo_test")
logger.setLevel(logging.INFO)
The logging flow chart shows how the logger itself filters by logging level ("Logger enabled for level of call?") even before the handlers get a chance to handle the record ("Handler enabled for level of LogRecord?"):

You seem to be missing a call to ``logging.basicConfig(). Without this call,logging` is in an unconfigured state and anything can happen.

Related

Adding a prefix to different log levels

I am trying to change the default behaviour of the logging module so that it adds a prefix string to each log level.
logging.basicConfig(level=logging.INFO, format='%(message)s', filename='output.log', filemode='w')
console = logging.StreamHandler()
logging.getLogger('').addHandler(console) # add the handler to the root logger
log = logging.getLogger(__name__)
log.info("this is an info")
log.error("this is an error")
log.warning("this is a warning")
I want
log.error("this is an error")
to print:
Error: this is an error
Or, the following line,
log.warning("this is a warning")
to print:
Warning: this is a warning
Error prefix in red and warning prefix in yellow.
Is there any non-hacky way to do this?

How to set a logging level per handler and per module using python's logging module

I'd like to set different logging levels per handler and logger in python. I'd like debug level logs for all loggers to be sent to their filehandlers, but be able to individually select the log level for each logger for what's printed to the console.
import logging
loggerA = logging.getLogger('a')
fhA = logging.FileHandler(filename='a.log', mode='w')
fhA.setLevel(logging.DEBUG)
loggerA.addHandler(fhA)
loggerB = logging.getLogger('b')
fhB = logging.FileHandler(filename='b.log', mode='w')
fhB.setLevel(logging.DEBUG)
loggerB.addHandler(fhB)
logging.basicConfig(level=logging.INFO)
logging.getLogger('a').setLevel(logging.INFO)
logging.getLogger('b').setLevel(logging.WARN)
loggerA.info("TEST a")
loggerB.info("TEST b")
I'd expect all logs to be sent to the files, but only show WARNING and above from 'b' in the console, and INFO and above from 'a' in the console. With the above code, 'b' doesn't have any logs in the file.

Connecting to gremlin server from python API duplicates log messages

After connecting to a Gremlin Server, all of my log messages are duplicated. I am using the following code to connect.
graph = anonymous_traversal.traversal().withRemote(DriverRemoteConnection("ws://localhost:8182/gremlin", "g"))
I guess that the gremlin-python API somehow enable the root logger, but I can not find it where. Maybe I overlooked some settings. Any bits of advice to overcome this issue is very welcome.
Here is the whole sample code which I used to replicate this problem.
import logging
from gremlin_python.driver.driver_remote_connection import DriverRemoteConnection
from gremlin_python.process import anonymous_traversal
I used to test this as the main entry point of the script.
if __name__ == '__main__':
Setting up my local logger instance
logger = logging.getLogger(__name__)
logger.setLevel(logging.getLevelName('DEBUG'))
log_format = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
log_file_handler = logging.FileHandler('foo.log', mode='w')
log_file_handler.setFormatter(log_format)
log_file_handler.setLevel(logging.DEBUG)
logger.addHandler(log_file_handler)
console_log_handler = logging.StreamHandler()
console_log_handler.setLevel(logging.getLevelName('INFO'))
console_log_handler.setFormatter(log_format)
logger.addHandler(console_log_handler)
And sending some messages to log before the connection estabilished.
logger.debug("foo")
logger.info("foo")
logger.warning("and")
logger.error("foo again")
Comment/uncomment the following line to test without gremlin connection
graph = anonymous_traversal.traversal().withRemote(DriverRemoteConnection("ws://localhost:8182/gremlin", "g"))
Finally, sending some more message afte the connection.
logger.debug("after foo")
logger.info("after foo")
logger.warning("after and")
logger.error("after foo again")
The result is something like this.
2019-04-18 16:09:57,746 - INFO - foo
2019-04-18 16:09:57,746 - WARNING - and
2019-04-18 16:09:57,746 - ERROR - foo again
DEBUG:__main__:after foo
2019-04-18 16:09:59,107 - INFO - after foo
INFO:__main__:after foo
2019-04-18 16:09:59,107 - WARNING - after and
WARNING:__main__:after and
2019-04-18 16:09:59,108 - ERROR - after foo again
ERROR:__main__:after foo again
I think gremlin should use its own logger and should not cause any side effects in the working of other logger instances.
I found a very simple workaround. It is not pretty.
def non_root_logger():
for handler in logging.root.handlers:
logging.root.removeHandler(handler)
return logging.getLogger(__name__)

Why don't my lower level logs write to the file, but the error and above do?

I created a small function to setup logging, with a filehandler for 'everything', and smtphandler for error and above. Error logs write to the log file and send correctly to email, but debug, info, notset don't, even though setlevel is set to 0 for filehandler. Why's that? Code below
#logsetup.py
import logging
import logging.handlers
def _setup_logger(name, log_file):
"""Function to setup logger"""
logger = logging.getLogger(name)
#Create Formatters
file_formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
mail_formatter = logging.Formatter('%(name)s - %(message)s')
#Create handler, set formatting, set log level
file_handler_obj = logging.FileHandler(log_file)
file_handler_obj.setFormatter(file_formatter)
file_handler_obj.setLevel(0)
#Create handler, set formatting, set log level
smtp_handler_obj = logging.handlers.SMTPHandler(mailhost=('smtp.gmail.com', 587),
fromaddr='mymail#example.com',
toaddrs='mymail#example.com',
subject='Error in Script',
credentials=('mymail#example.com', 'pwexample'), #username, password
secure=())
smtp_handler_obj.setFormatter(mail_formatter)
smtp_handler_obj.setLevel(logging.ERROR)
# add the handlers to logger
logger.addHandler(smtp_handler_obj)
logger.addHandler(file_handler_obj)
return logger
#mytest.py
import time
import logsetup
if __name__ == '__main__':
TEST_SETTINGS = config_funcs._get_config('TEST_SETTINGS')
logtime = time.strftime('%Y%m%d') # -%H%M%S")
log = logsetup._setup_logger('TEST', TEST_SETTINGS['logging_dir'] + 'Py_Log_%s.log' % logtime)
log.error('Writes to log file and sends email')
log.debug('Supposed to write to log file, does nothing.')
Apparently, logging needs it's own logging level aside from the handlers. Setting logger.setLevel(logging.DEBUG) right before returning logger causes it to work correctly. Documentation says
When a logger is created, the level is set to NOTSET (which causes all
messages to be processed when the logger is the root logger, or
delegation to the parent when the logger is a non-root logger). Note
that the root logger is created with level WARNING.
Which means that if the handlers are lower level than the root logger (which ERROR is not, but DEBUG is) then the handlers which I guess is a child because I'm getting a named logger? Not quite sure on the why of it, but that does 'fix' it, in case anyone comes to this later.

Python logging printing double to console

It seems that on a couple machines I'm getting double output like this:
INFO LED NOTIFICATION STARTED
INFO:output_logger:LED NOTIFICATION STARTED
This is the function I'm using:
def setup_logger(name, log_file, level=logging.INFO, ContentFormat='%(asctime)s %(levelname)s %(message)s', DateTimeFormat="%Y-%m-%d %H:%M:%S", CreateConsoleLogger=False):
"""Function setup as many loggers as you want"""
logger = logging.getLogger(name)
logger.setLevel(level)
if CreateConsoleLogger:
# create console handler
handler = logging.StreamHandler()
handler.setLevel(level)
formatter = logging.Formatter("%(levelname)s %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)
# create a file handler
handler = RotatingFileHandler(log_file, maxBytes=2000000, backupCount=5)
handler.setLevel(level)
formatter = logging.Formatter(ContentFormat, DateTimeFormat)
handler.setFormatter(formatter)
logger.addHandler(handler)
return logger
This is how I'm creating the logger:
output_logger = setup_logger('output_logger', 'log/autofy.log', level=logging.DEBUG, CreateConsoleLogger=True)
And this is how I call it:
output_logger.info("LED NOTIFICATION STARTED")
On a most of computers I just see the same message printed to the console that's saved to the file as expected ("INFO LED NOTIFICATION STARTED"), but on other computers it's doing this weird double output thing. My code is exactly the same from one computer to another, so any ideas what could be causing this on some computers and not others?
EDIT
I'm writing the script using notepad++ and running it in a terminal window on an Ubuntu 16.04 machine. I'm using python3.
Try adding this to your code:
logging._get_logger().propagate = False

Resources