How do I integrate custom exception handling with the FastAPI exception handling? - python-3.x

Python version 3.9, FastAPI version 0.78.0
I have a custom function that I use for application exception handling. When requests run into internal logic problems, i.e I want to send an HTTP response of 400 for some reason, I call a utility function.
#staticmethod
def raise_error(error: str, code: int) -> None:
logger.error(error)
raise HTTPException(status_code=code, detail=error)
Not a fan of this approach. So I look at
from fastapi import FastAPI, HTTPException, status
from fastapi.respones import JSONResponse
class ExceptionCustom(HTTPException):
pass
def exception_404_handler(request: Request, exc: HTTPException):
return JSONResponse(status_code=status.HTTP_404_NOT_FOUND, content={"message": "404"})
app.add_exception_handler(ExceptionCustom, exception_404_handler)
The problem I run into with the above approach is the inability to pass in the message as an argument.
Any thoughts on the whole topic?

Your custom exception can have any custom attributes that you want. Let's say you write it this way:
class ExceptionCustom(HTTPException):
pass
in your custom handler, you can do something like
def exception_404_handler(request: Request, exc: HTTPException):
return JSONResponse(status_code=status.HTTP_404_NOT_FOUND, content={"message": exc.detail})
Then, all you need to do is to raise the exception this way:
raise ExceptionCustom(status_code=404, detail='error message')
Note that you are creating a handler for this specific ExceptionCustom. If all you need is the message, you can write something more generic:
class MyHTTPException(HTTPException):
pass
def my_http_exception_handler(request: Request, exc: HTTPException):
return JSONResponse(status_code=exc.status_code, content={"message": exc.detail})
app.add_exception_handler(MyHTTPException, my_http_exception_handler)
This way you can raise any exception, with any status code and any message and have the message in your JSON response.
There's a detailed explanation on FastAPI docs

You can add custom exception handlers, and use attributes in your Exception class (i.e., class MyException(Exception) in the example below) to pass any message/variables that you would like to do so. The exception handler (i.e., #app.exception_handler(MyException) in the case below) will handle the exception as you wish and return your custom message. For more options, please have a look at this related answer as well.
Working Example
To trigger the exception in the example below, access the following URL from your browser: http://localhost:8000/something
from fastapi import FastAPI, Request, status
from fastapi.responses import JSONResponse
class MyException(Exception):
def __init__(self, name: str):
self.name = name
app = FastAPI()
#app.exception_handler(MyException)
async def my_exception_handler(request: Request, exc: MyException):
return JSONResponse(status_code=status.HTTP_404_NOT_FOUND,
content={"message": f"{exc.name} cannot be found." })
#app.get("/{name}")
def read_name(name: str):
if name == "something":
raise MyException(name=name)
return {"name": name}

Related

Django Middleware: RecursionError when accessing `self.request.user` in database query wrapper

I'm testing my database query middleware (Django docs here) on a sample django app with a Postgres db. The app is the cookiecutter boilerplate. My goal with the middleware is simply to log the user ID for all database queries. Using Python3.9 and Django3.2.13. My middleware code is below:
# Middleware code
import logging
import django
from django.db import connection
django_version = django.get_version()
logger = logging.getLogger(__name__)
class Logger:
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
with connection.execute_wrapper(QueryWrapper(request)):
return self.get_response(request)
class QueryWrapper:
def __init__(self, request):
self.request = request
def __call__(self, execute, sql, params, many, context):
# print(self.request.user)
return execute(sql, params, many, context)
If print(self.request.user.id) is commented out, everything works fine. However, I've found that uncommenting it, or any type of interaction with the user field in the self.request object, causes a Recursion Error:
RecursionError at /about/
maximum recursion depth exceeded
Request Method: GET
Request URL: http://127.0.0.1:8000/about/
Django Version: 3.2.13
Exception Type: RecursionError
Exception Value:
maximum recursion depth exceeded
Exception Location: /opt/homebrew/lib/python3.9/site-packages/django/db/models/sql/query.py, line 192, in __init__
Python Executable: /opt/homebrew/opt/python#3.9/bin/python3.9
Python Version: 3.9.13
In the error page, that is followed by many repetitions of the below error:
During handling of the above exception ('SessionStore' object has no attribute '_session_cache'), another exception occurred:
/opt/homebrew/lib/python3.9/site-packages/django/contrib/sessions/backends/base.py, line 233, in _get_session
return self._session_cache …
During handling of the above exception ('SessionStore' object has no attribute '_session_cache'), another exception occurred:
/opt/homebrew/lib/python3.9/site-packages/django/contrib/sessions/backends/base.py, line 233, in _get_session
return self._session_cache …
From consulting other SO posts, it seems accessing the user field should work fine. I've checked that the django_session table exists, and my middleware is also located at the very bottom of my middlewares (that include "django.contrib.sessions.middleware.SessionMiddleware" and "django.contrib.auth.middleware.AuthenticationMiddleware")
What's wrong here?
The following things happen when you write self.request.user:
The request is checked for an attribute _cached_user, if present the cached user is returned if not auth.get_user is called with the request.
Using the session key, Django checks the session to get the authentication backend used for the current user. Here if you are using database based sessions a database query is fired.
Using the above authentication backend Django makes a database query to get the current user using their ID.
As noted from the above points, unless there is a cache hit this process is going to cause a database query.
Using database instrumentation you have installed a wrapper around database queries, the problem is that this wrapper itself is trying to make more queries (trying to get the current user), causing it to call itself. One solution would be to get the current user before installing your wrapper function:
class Logger:
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
_user_authenticated = request.user.is_authenticated # Making sure user is fetched (Lazy object)
with connection.execute_wrapper(QueryWrapper(request)):
return self.get_response(request)
class QueryWrapper:
def __init__(self, request):
self.request = request
def __call__(self, execute, sql, params, many, context):
print(self.request.user)
return execute(sql, params, many, context)

How to mock a (PyActiveResource) pyactiveresource.connection UnauthorizedAccess response return, using Django TestCase?

I need to create a unit test that mock a REST API failure call, with a side effect of returning an UnauthorizedAccess exception, from the PyActiveResource project (https://github.com/Shopify/pyactiveresource) and store it in the DB. What I've create so far worked and I've got the desired returned side effect. Then, I catch it on the function foo.function_that_call_myfuncion() which looks like this
my_func.py:
from pyactiveresource.connection import UnauthorizedAccess
class MyFuncAnyThing:
...
def function_that_call_myfuncion(self, attr=None):
try:
module.MyTestClass.myfunction(attr)
except UnauthorizedAccess as error:
#Catch the error response object and store it on DB
resp_body = error.response.body
resp_code = error.response.code
#store it on DB
...
And my test file looks like this
unit_test.py:
from pyactiveresource.connection import UnauthorizedAccess
class TestFoo(TestCase):
def test_exception_unauthorized_access(self):
foo = SomeThingFactory()
with patch('module.MyTestClass.myfunction', side_effect=UnauthorizedAccess()):
foo.function_that_call_myfuncion()
#assertions goes below here
...
So, when the execution reached the try block on function_that_call_myfuncion from my_func.py module, the mock function return the desired exception (UnauthorizedAccess) and the object returned looks like this:
error
UnauthorizedAccess('Response(code=None, body="", headers={}, msg="")')
My problems begins when I try to mock the Response body returned on the UnauthorizedAccess exception. This is what I'm doing:
unit_test.py:
from pyactiveresource.connection import UnauthorizedAccess
class TestFoo(TestCase):
def test_exception_unauthorized_access(self):
foo = SomeThingFactory()
bar = MagicMock()
bar.code = 401
bar.body = '{"errors": "Login or wrong password"}'
with patch('module.MyTestClass.myfunction', side_effect=UnauthorizedAccess(bar)):
foo.function_that_call_myfuncion()
#assertions goes below here
...
And that's is how the mocked object looks like:
error
UnauthorizedAccess('Response(code=401, body="<MagicMock name=\'mock.read()\' id=\'2243840764512\'>", headers={}, msg="<MagicMock name=\'mock.msg\' id=\'2243840808464\'>")')
Note that the code attribute on Response is 401, but the body is empty, even though I've set it here bar.body = '{"errors": "Login or wrong password"}'. I also tried to create a Response object and pass it as parameter on the constructor for UnauthorizedAccess class, which is a subclass of
class ConnectionError(Error): of the pyactiveresource.connection lib code (https://github.com/Shopify/pyactiveresource/blob/e609d844ebace603f74bc5f0a67e9eafe7fb25e1/pyactiveresource/connection.py#L34)
unit_test.py:
from pyactiveresource.connection import UnauthorizedAccess, Response
class TestFoo(TestCase):
def test_exception_unauthorized_access(self):
foo = SomeThingFactory()
resp = Response(code=401,body='{"errors": "Login or wrong password"}')
with patch('module.MyTestClass.myfunction', side_effect=UnauthorizedAccess(response=resp)):
foo.function_that_call_myfuncion()
#assertions goes below here
...
But then I got this error from the Class Response:
#classmethod
def from_httpresponse(cls, response):
"""Create a Response object based on an httplib.HTTPResponse object.
Args:
response: An httplib.HTTPResponse object.
Returns:
A Response object.
"""
> return cls(response.code, response.read(),
dict(response.headers), response.msg, response)
E AttributeError: 'Response' object has no attribute 'read'
What am I missing? I just couldn't figure out how to set the 'read' attribute on the constructor, so that I can get the body value.
I'm using Python 3.8, Django 2.2
I managed to mock Shopify's ClientError exceptions by doing something along the lines of:
import urllib.error
from io import BytesIO
import pyactiveresource.testing.http_fake
pyactiveresource.testing.http_fake.initialize()
response = urllib.error.HTTPError('', 401, '', {}, BytesIO(b''))
pyactiveresource.testing.http_fake.TestHandler.set_response(response)
Which I learned about by digging into the Shopify/pyactiveresource tests.

Failing to send 404 HTTP status on Flask when client tries to get a nonexistent element

In a Python/Flask application, I have defined this endpoint that I expect to return 404 if a client tries to get an id that doesn't exist on my database.
For example:
#app.route('/plants/<int:plant_id>', methods=['GET'])
def get_plant(plant_id):
try:
plant = Plant.query.filter(Plant.id == plant_id).one_or_none()
if plant is None:
abort(404)
return jsonify({
'success': True,
'plant': plant.format()
})
except:
abort(422)
The problem is that when I try to execute it, it always seems to raise an exception and returns 422.
If I remove the try/except syntax, it works as expected and returns the 404. But I lose the capacity of handling exceptions... so it's not a solution for me.
Why am I doing wrong? How could I correctly trigger 404 without setting 404 as the except return?
Thanks!!
Ok, finally I was able to understand it and solve it. I post my findings here so maybe it could help someone in the future. :)
The answer is very basic, actually: every time I abort, I trigger an exception.
So, when I aborted, no matter the status code I used, I fell into my except statement, which was returning 422 by default.
What I did to solve it was to implement a custom RequestError, and every time I have a controlled error, I trigger my custom error, which output I can control separately.
This is the implementation of my custom error:
class RequestError(Exception):
def __init__(self, status):
self.status = status
def __str__(self):
return repr(self.status)
And I've changed my route implementation for something like this:
(note that I'm now handling first the custom error exception, and only then triggering a generic 422 error)
#app.route('/plants/<int:plant_id>', methods=['GET'])
def get_plant(plant_id):
try:
plant = Plant.query.filter(Plant.id == plant_id).one_or_none()
if plant is None:
raise RequestError(404)
return jsonify({
'success': True,
'plant': plant.format()
})
except RequestError as error:
abort(error.status)
except:
abort(422)
And that does it! \o/

Catch `Exception` globally in FastAPI

I am trying to catch unhandled exceptions at global level. So somewhere in main.py file I have the below:
#app.exception_handler(Exception)
async def exception_callback(request: Request, exc: Exception):
logger.error(exc.detail)
But the above method is never executed. However, if I write a custom exception and try to catch it (as shown below), it works just fine.
class MyException(Exception):
#some code
#app.exception_handler(MyException)
async def exception_callback(request: Request, exc: MyException):
logger.error(exc.detail)
I have gone through Catch exception type of Exception and process body request #575. But this bug talks about accessing request body. After seeing this bug, I feel it should be possible to catch Exception.
FastAPI version I am using is: fastapi>=0.52.0.
Thanks in advance :)
Update
There are multiple answers, I am thankful to all the readers and authors here.
I was revisiting this solution in my application. Now I see that I needed to set debug=False, default it's False, but I had it set to True in
server = FastAPI(
title=app_settings.PROJECT_NAME,
version=app_settings.VERSION,
)
It seems that I missed it when #iedmrc commented on answer given by #Kavindu Dodanduwa.
In case you want to capture all unhandled exceptions (internal server error), there's a very simple way of doing it. Documentation
from fastapi import FastAPI
from starlette.requests import Request
from starlette.responses import Response
app = FastAPI()
async def catch_exceptions_middleware(request: Request, call_next):
try:
return await call_next(request)
except Exception:
# you probably want some kind of logging here
return Response("Internal server error", status_code=500)
app.middleware('http')(catch_exceptions_middleware)
Make sure you place this middleware before everything else.
You can do something like this. It should return a json object with your custom error message also works in debugger mode.
from fastapi import FastAPI
from fastapi.responses import JSONResponse
app = FastAPI()
#app.exception_handler(Exception)
async def validation_exception_handler(request, err):
base_error_message = f"Failed to execute: {request.method}: {request.url}"
# Change here to LOGGER
return JSONResponse(status_code=400, content={"message": f"{base_error_message}. Detail: {err}"})
Adding a custom APIRoute can be also be used to handle global exceptions. The advantage of this approach is that if a http exception is raised from the custom route it will be handled by default Starlette's error handlers:
from typing import Callable
from fastapi import Request, Response, HTTPException, APIRouter, FastAPI
from fastapi.routing import APIRoute
from .logging import logger
class RouteErrorHandler(APIRoute):
"""Custom APIRoute that handles application errors and exceptions"""
def get_route_handler(self) -> Callable:
original_route_handler = super().get_route_handler()
async def custom_route_handler(request: Request) -> Response:
try:
return await original_route_handler(request)
except Exception as ex:
if isinstance(ex, HTTPException):
raise ex
logger.exception("uncaught error")
# wrap error into pretty 500 exception
raise HTTPException(status_code=500, detail=str(ex))
return custom_route_handler
router = APIRouter(route_class=RouteErrorHandler)
app = FastAPI()
app.include_router(router)
Worked for me with fastapi==0.68.1.
More on custom routes: https://fastapi.tiangolo.com/advanced/custom-request-and-route/
It is a known issue on the Fastapi and Starlette.
I am trying to capture the StarletteHTTPException globally by a following simple sample.
import uvicorn
from fastapi import FastAPI
from starlette.requests import Request
from starlette.exceptions import HTTPException as StarletteHTTPException
from starlette.responses import JSONResponse
app = FastAPI()
#app.exception_handler(StarletteHTTPException)
async def exception_callback(request: Request, exc: Exception):
print("test")
return JSONResponse({"detail": "test_error"}, status_code=500)
if __name__ == "__main__":
uvicorn.run("test:app", host="0.0.0.0", port=1111, reload=True)
It works. I open the browser and call the endpoint / and try to access http://127.0.0.1:1111/ , it will return the json {"detail":"test_error"} with HTTP code "500 Internal Server Error" .
However, when I only changed StarletteHTTPException to Exception in the #app.exception_handler,
import uvicorn
from fastapi import FastAPI
from starlette.requests import Request
from starlette.exceptions import HTTPException as StarletteHTTPException
from starlette.responses import JSONResponse
app = FastAPI()
#app.exception_handler(Exception)
async def exception_callback(request: Request, exc: Exception):
print("test")
return JSONResponse({"detail": "test_error"}, status_code=500)
if __name__ == "__main__":
uvicorn.run("test:app", host="0.0.0.0", port=1111, reload=True)
The method exception_callback could not capture the StarletteHTTPException when I accessed the http://127.0.0.1:1111/ . It reported 404 error.
The excepted behaviour should be: StarletteHTTPException error could be captured by the method exception_handler decorated by Exception because StarletteHTTPException is the child class of Exception.
However, it is a known issue reported in Fastapi and Starlette
https://github.com/tiangolo/fastapi/issues/2750
https://github.com/tiangolo/fastapi/issues/2683
https://github.com/encode/starlette/issues/1129
So we are not able to acheieve the goal currently.
First I invite to get familiar with exception base classes in python. You can read them in the document Built-in Exceptions
Secondly, read through fastApi default exception overriding behaviour Override the default exception handlers
What you must understand is that #app.exception_handler accepts any Exception or child classes derived from Exception. For example RequestValidationError is a subclass of python built in ValueError which itself a subclass of Exception.
So you must design your own exceptions or throw available exceptions with this background. I guess what went wrong is with your logger logger.error(exc.detail) by either not having a detail field or not having a proper logger configuration.
Sample code :
#app.get("/")
def read_root(response: Response):
raise ArithmeticError("Divide by zero")
#app.exception_handler(Exception)
async def validation_exception_handler(request, exc):
print(str(exc))
return PlainTextResponse("Something went wrong", status_code=400)
Output :
A stdout entry and a response with Something went wrong
I found a way to catch exceptions without the "Exception in ASGI application_" by using a middleware. Not sure if this has some other side effect but for me that works fine! #iedmrc
#app.middleware("http")
async def exception_handling(request: Request, call_next):
try:
return await call_next(request)
except Exception as exc:
log.error("Do some logging here")
return JSONResponse(status_code=500, content="some content")
I was searching for global handler for fast api for giving custome message for 429 status code i found and implemented, working fine for me
#app.exception_handler(429)
async def ratelimit_handler(request: Request, exc: Exception):
return JSONResponse({'message': "You have exceeded your request quota. Kindly try after some time.", 'status': 'failed'})

Not found view doesn't work on Pyramid using traversal

I'm using Pyramid (1.5.7) + traversal and following the documentation I've tried all possible ways to get the "Not found exception view" working.
from pyramid.view import notfound_view_config,forbidden_view_config, view_config
#notfound_view_config(renderer="error/not_found.jinja2")
def not_found_view(request):
request.response.status = 404
return {}
#forbidden_view_config(renderer="error/forbidden.jinja2")
def forbidden_view(request):
return {}
Using contexts:
from pyramid.view import view_config
from pyramid.httpexceptions import HTTPForbidden, HTTPUnauthorized
#view_config(context=HTTPNotFound, renderer="error/not_found.jinja2")
def not_found_view(request):
request.response.status = 404
return {}
#view_config(context=HTTPForbidden, renderer="error/forbidden.jinja2")
def forbidden_view(request):
return {}
I'm using the Scan mode, but I've tried also adding a custom function to the configuration:
def main(globals, **settings):
config = Configurator()
config.add_notfound_view(notfound)
Not luck either, all time getting the following unhandled exception:
raise HTTPNotFound(msg)
pyramid.httpexceptions.HTTPNotFound: /example-url
Ouch... My bad! I was using a tween which was preventing Pyramid to load the Exceptions:
def predispatch_factory(handler, registry):
# one-time configuration code goes here
def predispatch(request):
# code to be executed for each request before
# the actual application code goes here
response = handler(request)
# code to be executed for each request after
# the actual application code goes here
return response
return predispatch
Still I don't know why but after removing this tween all seems to work as expected.

Resources