So we have gone from using a single thread application to expanding to processes in google app engine flexible. Problem is that logging is not working when we run the GAE app, but it works if I log from my local machine. Getting the logger is using the following function:
def getLogger(self, name: str):
if not self.instance.logging_setup:
if self.execution_in_cloud():
client = google.cloud.logging_v2.Client()
client.get_default_handler()
client.setup_logging(log_level=self.get_log_level())
logging.StreamHandler().setFormatter(StackdriverFormatter())
else:
logging.basicConfig(format='%(asctime)s %(levelname)-8s [%(threadName)s] %(name)-15s %(message)s',
stream=sys.stdout,
level=self.get_log_level())
self.instance.logging_setup = True
return logging.getLogger(name)
So this works perfectly, all loggers fetch this function from this context class. If a new process is spawned, the logging will be setup for that handler. Logging from the processes works locally, but not when running in the GAE App. Is this a known issue, or is it another setup of the logger that is required?
Related
I have some code that is using logger package to log to a file on a Lambda function. The problem is that it also sends everything to CloudWatch, and since there are a lot of logs, is very expensive.
When I was using Python 3.7, this was working and logging only to the file:
import os
import sys
import logging
LOG_LEVEL = os.getenv('LOG_LEVEL', 'INFO')
root_logger = logging.getLogger()
root_logger.disabled = True
# create custom logger
logger = logging.getLogger('my_logger')
logger.removeHandler(sys.stdout)
logger.setLevel(logging.getLevelName(LOG_LEVEL))
for handler in logger.handlers:
logger.removeHandler(handler)
handler = logging.FileHandler('file.log', encoding='utf-8')
handler.setLevel(logging.getLevelName(LOG_LEVEL))
logger.addHandler(handler)
sys.stdout = open(os.devnull, 'w')
run_code_that_logs_stuff()
But after upgrading to Python 3.9, the logs started to show in CloudWatch again.
I have tried changing the last 2 lines with:
with open(os.devnull, 'w') as f, contextlib.redirect_stdout(f):
run_code_that_logs_stuff()
but same result:
START RequestId: 6ede89b6-26b0-4fac-872a-48ecf64c41d1 Version: $LATEST
2022-05-29T05:46:50.949+02:00 [INFO] 2022-05-29T03:46:50.949Z 6ede89b6-26b0-4fac-872a-48ecf64c41d1 Printing stuff I don't want to appear
The easiest way to prevent Lambda from writing to CloudWatch Logs is to remove the IAM policy that allows it to do so.
If you look at your Lambda's configuration, you'll see an execution role. If you look at that role's configuration, you'll see that it references the managed policy AWSLambdaBasicExecutionRole (or, if it's running inside a VPC, AWSLambdaVPCAccessExecutionRole). If you look at those managed policies, you'll see that they grant permissions logs:CreateLogGroup, logs:CreateLogStream, and logs:PutLogEvents.
Replace those managed policies with your own managed policies that grant everything except those three permissions.
Or, alternatively, create a managed policy that explicitly denies those three permissions (and nothing else), and attach it to the Lambda's execution role.
Update:
I noted in the comments that you can remove all handlers from the logging module at the start of your Lambda code. However, I think that's an incredibly bad idea, as it means that you will have no visibility into your function if things go wrong.
A much better approach would be to leverage the capabilities of the Python logging framework, and log your output at different levels. At the top of your function, use this code to set the level of the logger:
LOGGER = logging.getLogger(__name__)
LOGGER.setLevel(os.environ.get('LOG_LEVEL', 'WARNING'))
Then, in your code, use the appropriate level when logging:
LOGGER.debug("this message won't normally appear")
LOGGER.warning("this message will", exc_info=True)
This should trim your log messages considerably, while still giving you the ability to see what's happening in your function when things go wrong.
Docs: https://docs.python.org/3/library/logging.html
Update 2:
If you're determined to disable logging entirely (imo a bad idea), then I recommend doing it within the logging framework, rather than hacking streams:
import logging
logging.root.handlers = [logging.NullHandler()]
def lambda_handler(event, context):
logging.warn("this message won't appear")
I have a small program running on google app engine, I am using this function to write logs, in python 3.7:
def write_entry(text="", severity="", log_dict={}):
"""Writes log entries to the given logger."""
logging_client = logging.Client()
auth_data = auth().data_values
logger = logging_client.logger(auth_data['LOGGER_NAME'])
if not log_dict:
# Simple text log with severity.
logger.log_text(text, severity=severity)
if log_dict:
logger.log_struct(log_dict)
I've tested the program using this function from localhost with ngrok and the logs were collected.
After I've uploaded the code to app engine it only writes the first log, no other logs is collected. Does it need some permission in google app engine to work properly?
I am trying to write logs to Logging from Python applications by using Cloud Logging API Cloud client library with "execution ID" that as same as google's default value.
logger setup:
from google.cloud import logging
from google.cloud.logging.resource import Resource
log_client = logging.Client()
# This is the resource type of the log
log_name = 'cloudfunctions.googleapis.com%2Fcloud-functions'
# Inside the resource, nest the required labels specific to the resource type
res = Resource(type="cloud_function",
labels={
"function_name": "my-function",
"region": "asia-east2"
})
logger = log_client.logger(log_name.format("my-project"))
write log:
logger.log_struct({"message": request.remote_addr}, resource=res, severity='INFO')
It's currently not possible to do this using the purely the Cloud Function Framework itself, but you can try to extract the executionId from the request itself by using the following:
request.headers.get('function-execution-id')
I found an issue in Cloud Functions Github tracking the implementation of a native way to get those values, you can follow this thread for updates, if you'd like.
I had the same issue using an older version of google-cloud-logging. I was able to get this functional using the default python logging module. In a cloud function running python 3.8 and google-cloud-logging==2.5.0, the executionId is correctly logged with logs, as well as the severity within stackdriver.
main.py:
# Imports the Cloud Logging client library
import google.cloud.logging
# Instantiates a client
client = google.cloud.logging.Client()
# Retrieves a Cloud Logging handler based on the environment
# you're running in and integrates the handler with the
# Python logging module. By default this captures all logs
# at INFO level and higher
client.get_default_handler()
client.setup_logging()
# Imports Python standard library logging
import logging
def hello_world(req):
# Emits the data using the standard logging module
logging.info('info')
logging.warning('warn')
logging.error('error')
return ""
requirements.txt:
google-cloud-logging==2.5.0
Triggering this cloud function results in the following in stackdriver:
I'm developing a web application in Flask, using GAE.
My issue here is: Every time that my application tries to log, i got multiple entries on log file:
log viewer
.
My dbconnection class only imports a default logger class that i created and calls unexpected_error_log() to write whenever it needed.
My logger class:
import logging
from google.cloud import logging as cloudlogging
class LoggerDB:
def __init__(self):
log_client = cloudlogging.Client()
log_handler = log_client.get_default_handler()
self.cloud_logger = logging.getLogger("cloudLogger")
self.cloud_logger.setLevel(logging.INFO)
self.cloud_logger.addHandler(log_handler)
def unexpected_error_log(self, name, error="Unhandled Exception"):
self.cloud_logger.error("Unexpected Error on %s: %s", name, error)
Code when executed:
def insertVenda(self, venda):
try:
query = "xxxxx"
self.cursor.execute(query)
self.connection.commit()
return "Success"
except Exception as error:
self.logger.unexpected_error_log(__name__, error)
self.connection.rollback()
return "Error"
I suspect that gunicorn/app logging is duplicating my logs, but i don't know how to handle this case.
Did someone had the same problem?
I am struggling with this at the moment, my suspicion at the moment if you include something like this:
# Imports Python standard library logging
import logging
import google.cloud.logging
# Instantiates a client
client = google.cloud.logging.Client()
# Retrieves a Cloud Logging handler based on the environment
# you're running in and integrates the handler with the
# Python logging module. By default this captures all logs
# at INFO level and higher
client.get_default_handler()
client.setup_logging()
I get logs OK, but multiple duplicates.
If I omit I just get single stdout print statements going to the stackdriver logs.
I am using the websockets package to create a websocket server in python; at the same time I am heavily using the logging in different log-levels.
The websockets documentation mentions the logger configuration and that it could be changed to the log-level ERROR using
logger = logging.getLogger('websockets.server')
logger.setLevel(logging.ERROR)
logger.addHandler(logging.StreamHandler())
This, however, does not have any effect in my code, wherever I place it (below the import, before the websockets import, within __main__). I'd like to have two configurations - one global logging configuration and the logger-configuration of the websockets server.
Another option would be to disable the logging completely for the websockets module but I can't figure out how to do that. Any ideas?
It turns out it is not only the websockets.server that is emitting lots of debug messages, there's other ones too. I've ended up hunting down those packages by using the %(name)s field in the root logger configuration and could find websockets.protocol being responsible for the additional messages.
logging.basicConfig(
level=logging.DEBUG,
format='%(asctime)s %(name)s %(levelname)-8s %(message)s',
datefmt='(%H:%M:%S)')
# disable all loggers from different files
logging.getLogger('asyncio').setLevel(logging.ERROR)
logging.getLogger('asyncio.coroutines').setLevel(logging.ERROR)
logging.getLogger('websockets.server').setLevel(logging.ERROR)
logging.getLogger('websockets.protocol').setLevel(logging.ERROR)
When you start a websocket, set:
websocket.enableTrace(False)
No DEBUG and hart beat messages will be logged or shown in the console.
My example code:
def start_websocket(url, listen_method):
"""
Start a websocket connection.
:param url: string
:param listen_method: method
"""
# Enable or Disable logging
websocket.enableTrace(False)
ws = websocket.WebSocketApp(
url,
on_message=listen_method,
on_error=on_error,
on_close=on_close
)
ws.on_open = on_open
ws.run_forever(ping_interval=3, ping_timeout=2)