I'm working on a FastAPI application where I need to test the controller behavior without actually triggering the internal service call. My goal is to implement something similar to an instance_double in RSpec
My controller code looks like this
# module_name.api.routers.v1.py
#router.post("/notes")
async def create(note: Note):
s3_persister = S3Persister()
kafka_persister = KafkaPersister()
service = Persist.create(
note=note,
persisters=[s3_persister, kafka_persister]
)
await service.invoke()
return JSONResponse(
status_code=status.HTTP_200_OK,
content=success_response(payload=jsonable_encoder(service.note)),
)
The corresponding unit test that I have managed to get working in a non Pythonic way is
#pytest.fixture(scope='session')
def client():
client = TestClient(app)
yield client # testing happens here
def test_create_success_response(client):
"""
When the required parameters are passed
it should return a success response
"""
# copies some sample request body data
data = payload.copy()
# creates a note object using Pydantic models
valid_note = Note(**valid_note_payload)
# creates an service object with no persisters
persist_svc = Persist(note=note, persisters=[], event_publisher_version='1.0')
# mock the method that I intend to avoid calling in the controller flow
persist_svc.invoke = AsyncMock()
# A objects that will reflect on the svc_object post processing
persist_svc.note = valid_note
# create a mock of on the factory method so it returns my modified object
Persist.create = Mock(return_value=persist_svc)
response = client.post('/api/v1/notes',
headers=default_headers,
data=json.dumps(data)
)
assert persist_svc.invoke.called
assert response.status_code == 200
While this works I am unable to replicate this using the recommended #patch decorator.
I notice that the patched instance returned by the decorator isn't called and Fastapi ends up calling the actual service methods.
I was forced to add a create class method to the Persist service purely because I couldn't work with the default constructor object to obtain the same result.
#patch('module_name.api.services.persist.Persist')
def test_create_success_response(client, mockPersist):
data = payload.copy()
# I expect `persistInstance` to behave like an instance variable
persistInstance = mockPersist.return_value
valid_note = Note(**valid_note_payload)
# I attempt to mock the instance method
persistInstance.invoke = AsyncMock(return_value=valid_note)
persistInstance.note.return_value = valid_note
response = client.post('/api/v1/notes',
headers=default_headers,
data=json.dumps(data)
)
assert persistInstance.invoke.called
assert response.status_code == 200
The idea here is that I don't create another controller entirely as recommended by the FastAPI docs but instead mock the service method call alone.
I would like help on how to get write the unitTest in the recommended Pythonic way.
Related
How it started
I'm testing a class, ClassToTest, that makes API calls using atlassian-python-api. The tests are going to ensure that ClassToTest performs correctly with the data it gets back from the API. Many of the atlassian-python-api API calls use instantiated classes which inherit from the same base class or group of top-level classes.
I'd like to write tests that will expose breaks in the API contract if the wrong data is returned or API calls fail, while also testing the class I wrote to ensure it does the correct things with the data returned from the API. In order to do this, I was hoping to use unittest.mock.patch("path.to.Comment", autospec=True) to copy the API spec into the MagicMock, but I don't believe it's working properly.
For the purposes of the question, ClassToTest is not that important; what I am aiming to solve is how to setup and configure the pytest fixtures in a way that I can use them to mimic the API endpoints that will return the data that ClassToTest will act upon. Ideally I'd like to reuse the fixtures without having patch conflicts. I've included relevant code from ClassToTest for illustrative purposes here:
class_to_test.py:
from atlassian.bitbucket import Cloud
from typing import NamedTuple
# these are hardcoded constants that work with the production API
from src.constants import (
PULL_REQUEST_ID,
REPOSITORY,
WORKSPACE,
)
CommentType = NamedTuple("CommentType", [("top_level", str), ("inline", str)])
class ClassToTest:
def _get_token(self):
"""this returns a token of type(str)"""
def __init__(self, workspace, repository, pull_request_id):
self.active_comments = None
self.environment = sys.argv[1]
self.comment_text = CommentType(
top_level=r"top_level_comment text", inline=r"inline_comment text"
)
self.cloud = Cloud(token=self._get_token(), cloud=True)
self.workspace = self.cloud.workspaces.get(workspace)
self.repository = self.cloud.repositories.get(workspace, repository)
self.pull_request = self.repository.pullrequests.get(id=pull_request_id)
def _get_active_comments(self):
"""Returns a list of active (non-deleted) comments"""
return [
c for c in self.pull_request.comments() if c.data["deleted"] is False
]
# a few more methods here
def main():
instance = ClassToTest(WORKSPACE, REPOSITORY, PULL_REQUEST_ID)
# result = instance.method() for each method I need to call.
# do things with each result
if __name__ == "__main__":
main()
The class has methods that retrieve comments from the API (_get_active_comments, above), act on the retrieved comments, retrieve pull requests, and so on. What I am trying to test is that the class methods act correctly on the data received from the API, so I need to accurately mock data returned from API calls.
How it's going
I started with a unittest.Testcase style test class and wanted the flexibility of pytest fixtures (and autospec), but removed Testcase entirely when I discovered that pytest fixtures don't really work with it. I'm currently using a pytest class and conftest.py as follows:
/test/test_class_to_test.py:
import pytest
from unittest.mock import patch
from src.class_to_test import ClassToTest
#pytest.mark.usefixtures("mocked_comment", "mocked_user")
class TestClassToTest:
# We mock Cloud here as ClassToTest calls it in __init__ to authenticate with the API
# _get_token retrieves an access token for the API; since we don't need it, we can mock it
#patch("src.test_class_to_test.Cloud", autospec=True)
#patch.object(ClassToTest, "_get_token").
def setup_method(self, method, mock_get_token, mock_cloud):
mock_get_token.return_value = "token"
self.checker = ClassToTest("WORKSPACE", "REPOSITORY", 1)
def teardown_method(self, method):
pass
def test_has_top_level_and_inline_comments(self, mocked_comment, mocked_pull_request):
mock_top_comment = mocked_comment(raw="some text to search for later")
assert isinstance(mock_top_comment.data, dict)
assert mock_top_comment.data["raw"] == "some text to search for later"
# the assert below this line is failing
assert mock_top_comment.user.account_id == 1234
conftest.py:
import pytest
from unittest.mock import patch, PropertyMock
from atlassian.bitbucket.cloud.common.comments import Comment
from atlassian.bitbucket.cloud.common.users import User
#pytest.fixture()
def mocked_user(request):
def _mocked_user(account_id=1234):
user_patcher = patch(
f"atlassian.bitbucket.cloud.common.users.User", spec_set=True, autospec=True
)
MockUser = user_patcher.start()
data = {"type": "user", "account_id": account_id}
url = "user_url"
user = MockUser(data=data, url=url)
# setup mocked properties
mock_id = PropertyMock(return_value=account_id)
type(user).id = mock_id
mockdata = PropertyMock(return_value=data)
type(user).data = mockdata
request.addfinalizer(user_patcher.stop)
return user
return _mocked_user
#pytest.fixture()
def mocked_comment(request, mocked_user):
def _mocked_comment(raw="", inline=None, deleted=False, user_id=1234):
comment_patcher = patch(
f"atlassian.bitbucket.cloud.common.comments.Comment", spec_set=True, autospec=True
)
MockComment = comment_patcher.start()
data = {
"type": "pullrequest_comment",
"user": mocked_user(user_id),
"raw": raw,
"deleted": deleted,
}
if inline:
data["inline"] = {"from": None, "to": 1, "path": "src/code_issues.py"}
data["raw"] = "this is an inline comment"
comment = MockComment(data)
# setup mocked properties
mockdata = PropertyMock(return_value=data)
type(comment).data = mockdata
# mockuser = PropertyMock(return_value=mocked_user(user_id))
# type(comment).user = mockuser
request.addfinalizer(comment_patcher.stop)
return comment
return _mocked_comment
The problem I am encountering is that the assert mock_top_comment.user.account_id == 1234 line fails when running the test, with the following error:
> assert mock_top_comment.user.account_id == 1234
E AssertionError: assert <MagicMock name='Comment().user.account_id' id='4399290192'> == 1234
E + where <MagicMock name='Comment().user.account_id' id='4399290192'> = <MagicMock name='Comment().user' id='4399634736'>.account_id
E + where <MagicMock name='Comment().user' id='4399634736'> = <NonCallableMagicMock name='Comment()' spec_set='Comment' id='4399234928'>.user
How do I get the mock User class to attach to the mock Comment class in the same way that the real API makes it work? Is there something about autospec that I'm missing, or should I be abandoning unittest.mock.patch entirely and using something else?
Extra credit (EDIT: in retrospect, this may be the most important part)
I'm using mocked_comment as a pytest fixture factory and want to reuse it multiple times in the same test (for example to create multiple mocked Comments returned in a list). So far, each time I've tried to do that, I've been met with the following error:
def test_has_top_level_and_inline_comments(self, mocked_comment, mocked_pull_request):
mock_top_comment = mocked_comment(raw="Some comment text")
> mock_inline_comment = mocked_comment(inline=True)
...
test/conftest.py:30: in _mocked_comment
MockComment = comment_patcher.start()
/opt/homebrew/Cellar/python#3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/unittest/mock.py:1585: in start
result = self.__enter__()
...
> raise InvalidSpecError(
f'Cannot autospec attr {self.attribute!r} from target '
f'{target_name!r} as it has already been mocked out. '
f'[target={self.target!r}, attr={autospec!r}]')
E unittest.mock.InvalidSpecError: Cannot autospec attr 'Comment' from target 'atlassian.bitbucket.cloud.common.comments' as it has already been mocked out. [target=<module 'atlassian.bitbucket.cloud.common.comments' from '/opt/homebrew/lib/python3.10/site-packages/atlassian/bitbucket/cloud/common/comments.py'>, attr=<MagicMock name='Comment' spec_set='Comment' id='4398964912'>]
I thought the whole point of a pytest fixture factory was to be reusable, but I believe that using an autospec mock complicates things quite a bit. I don't want to have to hand copy every detail from the API spec into the tests, as that will have to be changed if anything in the API changes. Is there a solution for this that involves automatically and dynamically creating the necessary classes in the mocked API with the correct return values for properties?
One thing I'm considering is separating the testing into two parts: API contract, and ClassToTest testing. In this way I can write the tests for ClassToTest without relying on the API and they will pass as long as I manipulate the received data correctly. Any changes to the API will get caught by the separate contract testing tests. Then I can use non-factory fixtures with static data for testing ClassToTest.
For now though, I'm out of ideas on how to proceed with this. What should I do here? Probably the most important thing to address is how to properly link the User instance with the Comment instance in the fixtures so that my method calls in test work the same way as they do in production. Bonus points if we can figure out how to dynamically patch multiple fixtures in a single test.
I've started looking at this answer, but given the number of interconnected classes and properties, I'm not sure it will work without writing out a ton of fixtures. After following the directions and applying them to the User mock inside the Comment mock, I started getting the error in the Extra Credit section above, where autospec couldn't be used as it has already been mocked out.
I'm working on my first ever REST API, so apologies in advance if I've missed something basic. I have a function that takes a JSON request from another server, processes it (makes a prediction based on the data), and returns another JSON with the results. I'd like to keep a log on the server's local disk of all requests to this endpoint along with their results, for evaluation purposes and for retraining the model. However, for the purposes of minimising the latency of returning the result to the user, I'd like to return the response data first, and then write it to the local disk. It's not obvious to me how to do this properly, as the FastAPI paradigm necessitates that the result of a POST method is the return value of the decorated function, so anything I want to do with the data has to be done before it is returned.
Below is a minimal working example of what I think is my closest attempt at getting it right so far, using a custom object with a log decorator - my idea was just to assign the result to the log object as a class attribute, then use another method to write it to disk, but I can't figure out how to make sure that that function gets called after get_data every time.
import json
import uvicorn
from fastapi import FastAPI, Request
from functools import wraps
from pydantic import BaseModel
class Blob(BaseModel):
id: int
x: float
def crunch_numbers(data: Blob) -> dict:
# does some stuff
return {'foo': 'bar'}
class PostResponseLogger:
def __init__(self) -> None:
self.post_result = None
def log(self, func, *args, **kwargs):
#wraps(func)
def func_to_log(*args, **kwargs):
post_result = func(*args, **kwargs)
self.post_result = post_result
# how can this be done outside of this function ???
self.write_data()
return post_result
return func_to_log
def write_data(self):
if self.post_result:
with open('output.json', 'w') as f:
json.dump(self.post_result, f)
def main():
app = FastAPI()
logger = PostResponseLogger()
#app.post('/get_data/')
#logger.log
def get_data(input_json: dict, request: Request):
result = crunch_numbers(input_json)
return result
uvicorn.run(app=app)
if __name__ == '__main__':
main()
Basically, my question boils down to: "is there a way, in the PostResponseLogger class, to automatically call self.write_data after every call to self.log?", but if I'm using the wrong approach altogether, any other suggestions are also welcome.
You could have a Background Task for that purpose. A background task "will run only once the response has been sent" (see Starlette documentation). "This is useful for operations that need to happen after a request, but that the client doesn't really have to be waiting for the operation to complete before receiving the response" (see FastAPI documentation).
You can define a task function to run in the background for writing the log data, as shown below:
def write_log_data():
logger.write_data()
Then, import BackgroundTasks and define a parameter in your endpoint with a type declaration of BackgroundTasks. Inside of your endpoint, pass your task function (i.e., write_log_data, as defined above) to the background_tasks object with the method .add_task():
from fastapi import BackgroundTasks
#app.post('/get_data/')
#logger.log
def get_data(input_json: dict, request: Request, background_tasks: BackgroundTasks):
result = crunch_numbers(input_json)
background_tasks.add_task(write_log_data)
return result
The same principle could be applied if a middleware was used to capture and log the response data, as described in this answer, or a custom APIRoute class, as demonstrated in this answer.
For future reference, if you (or anyone) ever need to use async/await syntax, and run into concurrency issues (such as the event loop getting blocked) while performing some heavy background computation, please have a look at this answer, which explains the difference between defining an endpoint or a background task function with async def and def (briefly, async def endpoints/background tasks will run in the event loop, whereas def functions will run in an external threadpool that is then awaited), as well as provides solutions when it comes to running blocking I/O-bound or CPU-bound operations in such functions.
I have a module which makes object of dynamoDB client. The module is external and does not belong to my code. I am trying to unit test enclosingClass
class enclosingClass:
def enclosing_function():
ddb_client = get_ddb_client(role)
ddb_client.query()
# Module in my code which returns dynamoDB client object
from x.y.common import get_aws_client
def get_ddb_client(role):
return get_aws_client('dynamodb', assume_role_arn=role)
In my code I am getting the aws client object and calling query on it. I want to mock both the things
return value of get_aws_client should be a mocked object
Call to query dynamoDB should happen on mocked object.
But when I run unit test, I am getting the actual dynamoDB object and calling query on the dynamoDB object returns 'expected bytes or bytearray, but got 'MagicMock'
I am not understanding what to do in this. very new to python and not sure how do I return a mocked dynamoDb client object
#mock.patch("x.y.common.get_aws_client")
#mock.patch("boto3.client")
def test_hello_world_task(get_aws_client_mock, mock_client):
get_aws_client_mock.return_value= mock_client
mock_client.query.return_value = None
enclosing_class.enclosing_function() # the method call to enclosing function
Your issue: dynamoDB client is being initiated within enclosing_function.
Solution:
Create the object (object being mocked) outside of function under test and pass it as a parameter [also known as dependency injection].
class enclosingClass:
def enclosing_function(get_ddb_client):
ddb_client = get_ddb_client # could avoid this line by directly taking the parameter as ddb_client
ddb_client.query()
#mock.patch("x.y.common.get_aws_client")
#mock.patch("boto3.client")
def test_hello_world_task(get_aws_client_mock, mock_client):
get_aws_client_mock.return_value= mock_client
mock_client.query.return_value = None
enclosing_class.enclosing_function(get_aws_client_mock) # the method call to enclosing function
Hope this answers. Thanks.
I have the class with .upload method. This method is wrapped using the decorator:
#retry(tries=3)
def upload(self, xml_file: BytesIO):
self.client.upload(self._compress(xml_file))
I need to test if it runs 3 times if some exception occurs.
My test looks like:
#mock.patch("api.exporter.jumbo_uploader.JumboZipUploader.upload")
def test_upload_xml_fail(self, mock_upload):
"""Check if decorator called the compress function 3 times"""
generator = BrandBankXMLGenerator()
file = generator.generate()
uploader = JumboZipUploader()
uploader.upload = retry(mock_upload)
mock_upload.side_effect = Exception("Any exception")
uploader.upload(file)
self.assertEqual(mock_upload.call_count, 3)
I have read that the default behavior of python decorators assumes that the function inside the test will be unwrapped and I need to wrap it manually.
I did that trick, but the code fails with AssertionError: 0 != 3.
So, what is the right way here to wrap the decorated method properly?
def fixed_given(self):
return #given(
test_df=data_frames(
columns=columns(
["float_col1"],
dtype=float,
),
rows=tuples(
floats(allow_nan=True, allow_infinity=True)),
)
)(self)
#pytest.fixture()
#fixed_given
def its_a_fixture(test_df):
obj = its_an_class(test_df)
return obj
#pytest.fixture()
#fixed_given
def test_1(test_df):
#use returned object from my fixture here
#pytest.fixture()
#fixed_given
def test_2(test_df):
#use returned object from my fixture here
Here, I am creating my test dataframe in a seperate function to use it commonly across all functions.
And then creating a pytest fixture to instantiate a class by passing the test dataframe generated by a fixed given function.
I am finding a way to get a return value from this fixture.
But the problem i am using a given decorator, its doesn't allow return values.
is there a way to return even after using given decorator?
It's not clear what you're trying to acheive here, but reusing inputs generated by Hypothsis gives up most of the power of the framework (including minimal examples, replaying failures, settings options, etc.).
Instead, you can define a global variable for your strategy - or write a function that returns a strategy with #st.composite - and use that in each of your tests, e.g.
MY_STRATEGY = data_frames(columns=[
column(name="float_col1", elements=floats(allow_nan=True, allow_infinity=True))
])
#given(MY_STRATEGY)
def test_foo(df): ...
#given(MY_STRATEGY)
def test_bar(df): ...
Specifically to answer the question you asked, you cannot get a return value from a function decorated with #given.
Instead of using fixtures to instantiate your class, try using the .map method of a strategy (in this case data_frames(...).map(its_an_class)), or the builds() strategy (i.e. builds(my_class, data_frames(...), ...)).