I'm trying to duplicate everything that could display on console to a log file. This includes unhandled exceptions and test results from unittest.
Curranty following is not being done,
unhandled exceptions is NOT being logged into the log file.
test results from unittest is NOT being logged into the log file.
#2 is more important for me.
I work in Lunux/Unix but it would be great if the solution will work on Windows as well.
Any help will be appreciated. Below is my code,
logger = logging.getLogger(__name__)
logfile = datetime.now().strftime(pathlib.PurePath(__file__).stem + '_%H_%M_%d_%m_%Y')
class TestClass(unittest.TestCase):
logFormatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
rootLogger = logging.getLogger()
fileHandler = logging.FileHandler("{0}.log".format(logfile))
fileHandler.setFormatter(logFormatter)
rootLogger.addHandler(fileHandler)
rootLogger.setLevel(logging.DEBUG)
consoleHandler = logging.StreamHandler(sys.stderr)
consoleHandler.setFormatter(logFormatter)
rootLogger.addHandler(consoleHandler)
def test_logging(self):
self.fail("Why is this not being logged into the log file?")
Related
In a library, I have declared a custom logger like this in the file log.py:
import os
import sys
import logging
LOG_LEVEL = os.getenv('LOG_LEVEL', 'INFO')
# disable root logger
root_logger = logging.getLogger()
root_logger.disabled = True
# create custom logger
logger = logging.getLogger('my-handle')
logger.removeHandler(sys.stdout)
logger.setLevel(logging.getLevelName(LOG_LEVEL))
formatter = logging.Formatter(
'{"timestamp": "%(asctime)s", "level": "%(levelname)s", "logger": "%(name)s", '
'"filename": "%(filename)s", "message": "%(message)s"}',
'%Y-%m-%dT%H:%M:%S%z')
handler = logging.FileHandler('file.log', encoding='utf-8')
handler.setFormatter(formatter)
handler.setLevel(logging.getLevelName(LOG_LEVEL))
logger.addHandler(handler)
I then call this logger from other files doing:
from log import logger
logger.debug('something')
When running the code above on my computer, it only sends the logs to the file.log file. But when running the exact same code on a Lambda function, all the logs appear in CloudWatch. What is needed to disable the CloudWatch logs? it costs a lot for stuff I don't want and already log somewhere else (in s3 for that matter - much cheaper)
I am able to store error logs to a file... but not info() or any other Logging Levels.
What am I doing wrong?
How can I store any level of logs to FileHandler?
code.py
import sys
import logging
def setup_logging():
global logger
logger = logging.getLogger()
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
open('data_module.log', 'w').close() # empty logs
global fileHandler
fileHandler = logging.FileHandler('data_module.log')
fileHandler.setFormatter(formatter)
fileHandler.setLevel(logging.DEBUG)
logger.addHandler(fileHandler)
logger.error('Started') # info
logger.info('information') # info
test.py:
import code as c
c.setup_logging()
with open('data_module.log', 'r') as fileHandler:
logs = [l.rstrip() for l in fileHandler.readlines()]
open('data_module.log', 'w').close() # empty logs
assert len(logs) == 2
Error:
AssertionError: assert 1 == 2
Please let me know if there's anything else I should add to post.
You need to set the level for the logger itself:
logger.setLevel(logging.DEBUG)
The default log level is WARN: when you write a DEBUG-level message, the logger does not handle it (ie send it to a handler). The handler you added is never invoked.
The handler can have its own level, but that is consulted only after the handler is invoked. If a logger sends a DEBUG message to a handler that is only interested in INFO+ messages, it does nothing.
I have usage and application logs.
I want to write application logs to app.log file and usage logs to usage.log file.
Here is how I have tried to do this:
# Application Log:
logging.basicConfig(filename = "app.log", level = logging.DEBUG, format = "%(asctime)s - %(name)s - %(levelname)s - %(message)s")
# Usage Log:
logging.basicConfig(filename = "usage.log", level = logging.DEBUG, format = "%(asctime)s - %(name)s - %(levelname)s - %(message)s")
For some reason I don't see the second file created.
Please advise what am I missing here?
This is what solved my issue:
import logging
formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
def setup_logger(name, log_file, level=logging.INFO):
"""To setup as many loggers as you want"""
handler = logging.FileHandler(log_file)
handler.setFormatter(formatter)
logger = logging.getLogger(name)
logger.setLevel(level)
logger.addHandler(handler)
return logger
# first file logger
logger = setup_logger('first_logger', 'first_logfile.log')
logger.info('This is just info message')
# second file logger
super_logger = setup_logger('second_logger', 'second_logfile.log')
super_logger.error('This is an error message')
def another_method():
# using logger defined above also works here
logger.info('Inside method')
Source
add different Handlers to the default logger:
import logging
def init_logging():
logger = logging.getLogger()
file_handler = logging.FileHandler('info.log')
error_handler = logging.FileHandler('error.log')
console_handler = logging.StreamHandler()
formatter = logging.Formatter(
'%(asctime)s - %(name)s - %(levelname)s - %(message)s')
file_handler.setFormatter(formatter)
console_handler.setFormatter(formatter)
error_handler.setFormatter(formatter)
error_handler.setLevel(logging.ERROR)
logger.addHandler(file_handler)
logger.addHandler(console_handler)
logger.addHandler(error_handler)
logger.setLevel(logging.DEBUG)
# use
init_logging()
LOGGER = logging.getLogger()
LOGGER.info('abc')
I am trying to setup a logging mechanism for a python module.
Following is the example code that I have written to setup logging
import logging
def init_logger(logger):
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(filename)s - %(funcName)s - %(message)s')
ch = logging.StreamHandler()
ch.setFormatter(formatter)
ch.setLevel(logging.INFO)
logger.addHandler(ch)
file_handler = logging.FileHandler('test_logging.log')
file_handler.setFormatter(formatter)
file_handler.setLevel(logging.DEBUG)
logger.addHandler(file_handler)
def foo1():
logger = logging.getLogger(__name__)
logger.info('Test Info')
logger.debug('Test Debug')
logger.error('Test Error')
def foo2():
logger = logging.getLogger(__name__)
logger.info('Test Info')
logger.debug('Test Debug')
logger.error('Test Error')
if __name__ == '__main__':
logger = logging.getLogger(__name__)
init_logger(logger)
foo1()
foo2()
I expect the logging to print info level and above to stdout and debug level and above to be written to the log file. But what I see is that only error level is outputted to both stdout and log file.
2019-08-13 11:20:07,775 - ERROR - test_logger.py - foo1 - Test Error
2019-08-13 11:20:07,776 - ERROR - test_logger.py - foo2 - Test Error
As per the documentation getLogger should return the same instance of logger. I even tried to create a new instance for the first time like logger = logging.Logger(__name__) but no luck with that. I am not understanding what am I missing here.
Short answer: you must use logging.basicConfig(level=...) or logger.setLevel in your code.
When you use logging.getLogger('some_name') for the first time you create a new logger with level = NOTSET = 0.
# logging module source code
class Logger(Filterer):
def __init__(self, name, level=NOTSET):
...
logging.NOTSET seems to be a valid level value but it is not. Actually it is illegal value that says that logger is not enabled to log anything and forces logger to use level from parent logger (root logger). This logic is defined in Looger.getEffectiveLevel method:
# logging module source code
def getEffectiveLevel(self):
logger = self
while logger:
if logger.level: # 0 gives False here
return logger.level
logger = logger.parent # 0 makes this line reachable
return NOTSET
Root logger has level=WARNING so newly created loggers inherit this level:
# logging module source code
root = RootLogger(WARNING)
logging.getLogger does not allow you to specify logging level. So you have to use logging.basicConfig to modify root logger or logger.setLevel to modify newly created logger somewhere in the very beginning of the script.
I guess the feature should be documented in logging module guides/documentation.
I'm setting up logging on a flask service. I tried to write logs to an azure blob storage with the following code
import logging
import sys
from azure_storage_logging.handlers import BlobStorageRotatingFileHandler
logger = logging.getLogger('service_logger')
logger.setLevel(logging.DEBUG)
log_formater = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(process)d - %(message)s')
azure_blob_handler = BlobStorageRotatingFileHandler(filename = 'service.log',
account_name='servicestorage',
account_key='',
maxBytes= maxBytes,
container='service-log')
azure_blob_handler.setLevel(logging.INFO)
azure_blob_handler.setFormatter(log_formater)
logger.addHandler(azure_blob_handler)
Then, I tried with logger.warning('test warning'), but there's not log file created on the azure blob container.
Can anyone help me figure out where I did wrong?
Best,
Eigen
I tried your code in my flask but did't reproduce your issue.
code:
#app.route('/log')
def log():
import logging
import sys
from azure_storage_logging.handlers import BlobStorageRotatingFileHandler
mystorageaccountname='***'
mystorageaccountkey='***'
logger = logging.getLogger('service_logger')
logger.setLevel(logging.DEBUG)
log_formater = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(process)d - %(message)s')
azure_blob_handler = BlobStorageRotatingFileHandler(filename = 'service.log',
account_name=mystorageaccountname,
account_key=mystorageaccountkey,
maxBytes= 5,
container='service-log')
azure_blob_handler.setLevel(logging.INFO)
azure_blob_handler.setFormatter(log_formater)
logger.addHandler(azure_blob_handler)
logger.warning('warning message')
output:
More details, you could refer to the doc and source code.
Hope it helps you. Any concern,please let me know.