Python, mocking and wrapping methods without instantating objects - python-3.x

I want to mock a method of a class and use wraps, so that it is actually called, but I can inspect the arguments passed to it. I have seen at several places (here for example) that the usual way to do that is as follows (adapted to show my point):
from unittest import TestCase
from unittest.mock import patch
class Potato(object):
def foo(self, n):
return self.bar(n)
def bar(self, n):
return n + 2
class PotatoTest(TestCase):
spud = Potato()
#patch.object(Potato, 'foo', wraps=spud.foo)
def test_something(self, mock):
forty_two = self.spud.foo(n=40)
mock.assert_called_once_with(n=40)
self.assertEqual(forty_two, 42)
However, this instantiates the class Potato, in order to bind the mock to the instance method spud.foo.
What I need is to mock the method foo in all instances of Potato, and wrap them around the original methods. I.e, I need the following:
from unittest import TestCase
from unittest.mock import patch
class Potato(object):
def foo(self, n):
return self.bar(n)
def bar(self, n):
return n + 2
class PotatoTest(TestCase):
#patch.object(Potato, 'foo', wraps=Potato.foo)
def test_something(self, mock):
self.spud = Potato()
forty_two = self.spud.foo(n=40)
mock.assert_called_once_with(n=40)
self.assertEqual(forty_two, 42)
This of course doesn't work. I get the error:
TypeError: foo() missing 1 required positional argument: 'self'
It works however if wraps is not used, so the problem is not in the mock itself, but in the way it calls the wrapped function. For example, this works (but of course I had to "fake" the returned value, because now Potato.foo is never actually run):
from unittest import TestCase
from unittest.mock import patch
class Potato(object):
def foo(self, n):
return self.bar(n)
def bar(self, n):
return n + 2
class PotatoTest(TestCase):
#patch.object(Potato, 'foo', return_value=42)#, wraps=Potato.foo)
def test_something(self, mock):
self.spud = Potato()
forty_two = self.spud.foo(n=40)
mock.assert_called_once_with(n=40)
self.assertEqual(forty_two, 42)
This works, but it does not run the original function, which I need to run because the return value is used elsewhere (and I cannot fake it from the test).
Can it be done?
Note The actual reason behind my needs is that I'm testing a rest api with webtest. From the tests I perform some wsgi requests to some paths, and my framework instantiates some classes and uses their methods to fulfill the request. I want to capture the parameters sent to those methods to do some asserts about them in my tests.

In short, you can't do this using Mock instances alone.
patch.object creates Mock's for the specified instance (Potato), i.e. it replaces Potato.foo with a single Mock the moment it is called. Therefore, there is no way to pass instances to the Mock as the mock is created before any instances are. To my knowledge getting instance information to the Mock at runtime is also very difficult.
To illustrate:
from unittest.mock import MagicMock
class MyMock(MagicMock):
def __init__(self, *a, **kw):
super(MyMock, self).__init__(*a, **kw)
print('Created Mock instance a={}, kw={}'.format(a,kw))
with patch.object(Potato, 'foo', new_callable=MyMock, wrap=Potato.foo):
print('no instances created')
spud = Potato()
print('instance created')
The output is:
Created Mock instance a=(), kw={'name': 'foo', 'wrap': <function Potato.foo at 0x7f5d9bfddea0>}
no instances created
instance created
I would suggest monkey-patching your class in order to add the Mock to the correct location.
from unittest.mock import MagicMock
class PotatoTest(TestCase):
def test_something(self):
old_foo = Potato.foo
try:
mock = MagicMock(wraps=Potato.foo, return_value=42)
Potato.foo = lambda *a,**kw: mock(*a, **kw)
self.spud = Potato()
forty_two = self.spud.foo(n=40)
mock.assert_called_once_with(self.spud, n=40) # Now needs self instance
self.assertEqual(forty_two, 42)
finally:
Potato.foo = old_foo
Note that you using called_with is problematic as you are calling your functions with an instance.

Do you control creation of Potato instances, or at least have access to these instances after creating them? You should, else you'd not be able to check particular arg lists.
If so, you can wrap methods of individual instances using
spud = dig_out_a_potato()
with mock.patch.object(spud, "foo", wraps=spud.foo) as mock_spud:
# do your thing.
mock_spud.assert_called...

Your question looks identical to python mock - patching a method without obstructing implementation to me. https://stackoverflow.com/a/72446739/9230828 implements what you want (except that it uses a with statement instead of a decorator). wrap_object.py:
# Copyright (C) 2022, Benjamin Drung <bdrung#posteo.de>
#
# Permission to use, copy, modify, and/or distribute this software for any
# purpose with or without fee is hereby granted, provided that the above
# copyright notice and this permission notice appear in all copies.
#
# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
import contextlib
import typing
import unittest.mock
#contextlib.contextmanager
def wrap_object(
target: object, attribute: str
) -> typing.Generator[unittest.mock.MagicMock, None, None]:
"""Wrap the named member on an object with a mock object.
wrap_object() can be used as a context manager. Inside the
body of the with statement, the attribute of the target is
wrapped with a :class:`unittest.mock.MagicMock` object. When
the with statement exits the patch is undone.
The instance argument 'self' of the wrapped attribute is
intentionally not logged in the MagicMock call. Therefore
wrap_object() can be used to check all calls to the object,
but not differentiate between different instances.
"""
mock = unittest.mock.MagicMock()
real_attribute = getattr(target, attribute)
def mocked_attribute(self, *args, **kwargs):
mock.__call__(*args, **kwargs)
return real_attribute(self, *args, **kwargs)
with unittest.mock.patch.object(target, attribute, mocked_attribute):
yield mock
Then you can write following unit test:
from unittest import TestCase
from wrap_object import wrap_object
class Potato:
def foo(self, n):
return self.bar(n)
def bar(self, n):
return n + 2
class PotatoTest(TestCase):
def test_something(self):
with wrap_object(Potato, 'foo') as mock:
self.spud = Potato()
forty_two = self.spud.foo(n=40)
mock.assert_called_once_with(n=40)
self.assertEqual(forty_two, 42)

Related

Python: why do I need super().__init__() call in metaclasses?

I have got one question: why do I need to call super().--init--() in metaclasses? Because metaclass is factory of classes, I think we don`t need to call initialization for making objects of class Shop. Or with using super().--init-- we initializing the class? (Because my IDE says, that I should call it. But without super().--init-- nothing happens, my class working without mistakes).
Can you explane me, why?
Thanks in advance!
class Descriptor:
_counter = 0
def __init__(self):
self.attr_name = f'Descriptor attr#{Descriptor._counter}'
Descriptor._counter += 1
def __get__(self, instance, owner):
return self if instance is None else instance.__dict__[self.attr_name]
def __set__(self, instance, value):
if value > 0:
instance.__dict__[self.attr_name] = value
else:
msg = 'Value must be > 0!'
raise AttributeError(msg)
class Shop():
weight = Descriptor()
price = Descriptor()
def __init__(self, name, price, weight):
self.name = name
self.price = price
self.weight = weight
def __repr__(self):
return f'{self.name}: price - {self.price} weight - {self.weight}'
def buy(self):
return self.price * self.weight
class Meta(type):
def __init__(cls, name, bases, attr_dict):
super().__init__(name, bases, attr_dict) # <- this is that func. call
for key, value in attr_dict.items():
if isinstance(value, Descriptor): # Here I rename attributes name of descriptor`s object.
value.attr_name = key
#classmethod
def __prepare__(metacls, name, bases):
return OrderedDict()
You don't "need" to - and if your code use no other custom metaclasses, not calling the metaclass'__init__.super() will work just the same.
But if one needs to combine your metaclass with another, through inheritance, without the super() call, it won't work "out of the box": the super() call is the way to ensure all methods in the inheritance chain are called.
And if at first it looks like that a metaclass is extremely rare, and combining metaclasses would likely never take place: a few libraries or frameworks have their own metaclasses, including Python's "abc"s (abstract base classes), PyQT, ORM frameworks, and so on. If any metaclass under your control is well behaved with proper super() calls on the __new__, __init__ and __call__ methods, (if you override those), what you need to do to combine both superclasses and have a working metaclass can be done in a single line:
CompatibleMeta = type("CompatibleMeta", (meta, type(OtherClassBase)), {})
This way, for example, if you want to use the mechanisms in your metaclass in a class using the ABCMeta functionalities in Python, you just do it. The __init__ method in your Meta will call the other metaclass __init__. Otherwise it would not run, and some subtle unexpectd thing would not be initialized in your classes, and this could be a very hard to find bug.
On a side note: there is no need to declare __prepare__ in a metaclass if all it does is creating an OrderedDict on a Python newer than 3.6: Since that version, dicitionaries used as the "locals()" while executing class bodies are ordered by default. Also, if another metaclass you are combining with also have a __prepare__, there is no way to make that work automatically by using "super()" - you have to check the code and verify which of the two __prepare__s should be used, or create a new mapping type with features to attend both metaclasses.

Using singledispatch with custom class(CPython 3.8.2)

Let's say I want to set functions for each classes in module Named 'MacroMethods'. So I've set up singledispatch after seeing it in 'Fluent Python' like this:
#singledispatch
def addMethod(self, obj):
print(f'Wrong Object {str(obj)} supplied.')
return obj
...
#addMethod.register(MacroMethods.Wait)
def _(self, obj):
print('adding object wait')
obj.delay = self.waitSpin.value
obj.onFail = None
obj.onSuccess = None
return obj
Desired behavior is - when instance of class 'MacroMethods.Wait' is given as argument, singledispatch runs registered function with that class type.
Instead, it runs default function rather than registered one.
>>> Wrong Object <MacroMethods.Wait object at 0x0936D1A8> supplied.
However, type() clearly shows instance is class 'MacroMethods.Wait', and dict_keys property also contains it.
>>> dict_keys([<class 'object'>, ..., <class 'MacroMethods.Wait'>])
I suspect all custom classes I made count as 'object' type and don't run desired functions in result.
Any way to solve this problem? Entire codes are here.
Update
I've managed to mimic singledispatch's actions as following:
from functools import wraps
def state_deco(func_main):
"""
Decorator that mimics singledispatch for ease of interaction expansions.
"""
# assuming no args are needed for interaction functions.
func_main.dispatch_list = {} # collect decorated functions
#wraps(func_main)
def wrapper(target):
# dispatch target to destination interaction function.
nonlocal func_main
try:
# find and run callable for target
return func_main.dispatch_list[type(target)]()
except KeyError:
# If no matching case found, main decorated function will run instead.
func_main()
def register(target):
# A decorator that register decorated function to main decorated function.
def decorate(func_sub):
nonlocal func_main
func_main.dispatch_list[target] = func_sub
def register_wrapper(*args, **kwargs):
return func_sub(*args, **kwargs)
return register_wrapper
return decorate
wrapper.register = register
return wrapper
Used like:
#state_deco
def general():
return "A's reaction to undefined others."
#general.register(StateA)
def _():
return "A's reaction of another A"
#general.register(StateB)
def _():
return "A's reaction of B"
But still it's not singledispatch, so I find this might be inappropriate to post this as answer.
I wanted to do similar and had the same trouble. Looks like we have bumped into a python bug. Found a write-up that describes this situation.
Here is the link to the Python Bug Tracker.
Python 3.7 breaks on singledispatch_function.register(pseudo_type), which Python 3.6 accepted

python3 mock member variable get multiple times

I have a use case where I need to mock a member variable but I want it to return a different value every time it is accessed.
Example;
def run_test():
myClass = MyDumbClass()
for i in range(2):
print(myClass.response)
class MyDumbClass():
def __init__(self):
self.response = None
#pytest.mark.parametrize("responses", [[200,201]])
#patch("blah.MyDumbClass")
def test_stuff(mockMyDumbClass, responses)
run_test()
assert stuff
What I am hoping for here is in the run_test method the first iteration will print 200 then the next will print 201. Is this possible, been looking through unittest and pytest documentation but can't find anything about mocking a member variable in this fashion.
Just started learning pytest and unittest with python3 so forgive me if the style isn't the best.
If you wrap myDumbClass.response in a get function - say get_response() then you can use the side_effect parameter of the mock class.
side_effect sets the return_value of the mocked method to an iterator returning a different value each time you call the mocked method.
For example you can do
def run_test():
myClass = MyDumbClass()
for i in range(2):
print(myClass.get_response())
class MyDumbClass():
def __init__(self):
self.response = None
def get_response(self):
return self.response
#pytest.mark.parametrize("responses", [([200,201])])
def test_stuff( responses):
with mock.patch('blah.MyDumbClass.get_response', side_effect=responses):
run_test()
assert False
Result
----------------------------------- Captured stdout call ------------------------------------------------------------
200
201
Edit
No need to patch via context manager e.g with mock.patch. You can patch via decorator in pretty much the same way. For example this works fine
#patch('blah.MyDumbClass.get_response',side_effect=[200,100])
def test_stuff(mockMyDumbClass):
run_test()
assert False
----------------------------------- Captured stdout call ------------------------------------------------------------
200
201

Can I derive from classmethod in Python?

I have a special statemachine implemented in Python, which uses class methods as state representation.
class EntityBlock(Block):
def __init__(self, name):
self._name = name
#classmethod
def stateKeyword1(cls, parserState : ParserState):
pass
#classmethod
def stateWhitespace1(cls, parserState : ParserState):
token = parserState.Token
if isinstance(token, StringToken):
if (token <= "generate"):
parserState.NewToken = GenerateKeyword(token)
parserState.NewBlock = cls(....)
else:
raise TokenParserException("....", token)
raise TokenParserException("....", token)
#classmethod
def stateDelimiter(cls, parserState : ParserState):
pass
Visit GitHub for full source code off pyVHDLParser.
When I debug my parser FSM, I get the statenames printed as:
State: <bound method Package.stateParse of <class 'pyVHDLParser.DocumentModel.Sequential.Package.Package'>>
I would like to get better reports, so I would like to overwrite the default behavior of __repr__ of each bound method object.
Yes, I could write a metaclass or apply a second decorator, but I was questioning myself:
Is it possible to derive from classmethod and have only one decorator called e.g. state?
According to PyCharm's builtins.py (a collection of dummy code for Python's builtins), classmethod is a class-based decorator.
Yes, you can write your own class that derives from classmethod if you want. It's a bit complicated though. You'll need to implement the descriptor protocol (overriding classmethod's implementation of __get__) so that it returns an instance of another custom class that behaves like a bound method object. Unfortunately, you can't inherit from Python's builtin bound method type (I'm not sure why not).
Probably the best approach then is to wrap one of the normal method objects in an instance of a custom class. I'm not sure how much of the method API you need to replicate though, so that might get a bit complicated. (Do you need your states to be comparable to one another? Do they need to be hashable? Picklable?)
Anyway, here's a bare bones implementation that does the minimum amount necessary to get a working method (plus the new repr):
class MethodWrapper:
def __init__(self, name, method):
self.name = name if name is not None else repr(method)
self.method = method
def __call__(self, *args, **kwargs):
return self.method(*args, **kwargs)
def __repr__(self):
return self.name
class State(classmethod):
def __init__(self, func):
self.name = None
super().__init__(func)
def __set_name__(self, owner, name):
self.name = "{}.{}".format(owner.__name__, name)
def __get__(self, owner, instance):
method = super().__get__(owner, instance)
return MethodWrapper(self.name, method)
And a quick demo of it in action:
>>> class Foo:
#State
def foo(cls):
print(cls)
>>> Foo.foo
Foo.foo
>>> Foo.foo()
<class '__main__.Foo'>
>>> f = Foo()
>>> f.foo()
<class '__main__.Foo'>
Note that the __set_name__ method used by the State descriptor is only called by Python 3.6. Without that new feature, it would be much more difficult for the descriptor to learn its own name (you might need to make a decorator factory that takes the name as an argument).

Show docstrings on every function call

Let's say I have a code like this:
class NewTestCase(unittest.TestCase, CommonMethods):
def setUp(self):
self.shortDescription()
def test_01_sample test(self):
"""Testing something"""
self.create_account(self.arg['account'])
assert ...
...
class CommonMethods():
def create_account(self, account):
"""Creating account"""
...
if __name__ == '__main__':
unittest.main(verbosity=2, warnings='ignore')
I want to show the docstrings of all methods defined / created by me ('Testing something' and 'Creating account'), but the execution shows 'Testing something' only. Any tip?
Maybe there is an option for that in the unittest module, but I doubt it; otherwise, how would that module distinguish between your methods and functions and all sorts of library functions?
What you could do is to use another function to modify the existing functions to print their Docstring and/or other useful information whenever they are called. You could make this a decorator, or just call the function manually before running the tests.
This one should 'verbosify' all the methods of a given class (only slightly tested!), and you could make similar ones for individual functions or entire modules.
def verbosify(clazz):
for name in dir(clazz):
attr = getattr(clazz, name)
if not name.startswith("__") and callable(attr):
def attr_verbose(*args, **kwargs):
print("Calling", name, args, kwargs)
print(attr.__doc__)
return attr(*args, **kwargs)
setattr(clazz, name, attr_verbose)
Just call verbosify(CommonMethods) in your main block.

Resources