So I am quite new to mocking. I think I need to mock two functions.
Function under test
def get_text_from_pdf(next_pdfs_path):
# TODO test this function
"""
pulls all text from a PDF document and returns as a string.
Parameters:
next_pdfs_path (str): file path use r"" around path.
Returns:
text (str): string of text
"""
if os.path.isfile(next_pdfs_path): # check file is a real file/filepath
try:
text = ''
with fitz.open(next_pdfs_path) as doc: # using PyMuPDF
for page in doc:
text += page.getText()
return text
except (RuntimeError, IOError):
pass
pass
test code first try
from unittest import mock
#mock.patch("content_production.fitz.open", return_value='fake_file.csv', autospec=True)
def test_get_text_from_pdf(mock_fitz_open):
assert cp.get_text_from_pdf('fake_file.csv') == 'fake_file.csv'
error
E AssertionError: assert None == 'fake_file.csv'
E + where None = <function get_text_from_pdf at 0x00000245EDF8CAF0>('fake_file.csv')
E + where <function get_text_from_pdf at 0x00000245EDF8CAF0> = cp.get_text_from_pdf
Do I need to mock both fitz.open and os.path.isfile? How could that be done if yes?
EDIT
Following njriasan feedback I have tried this
#mock.patch("content_production.os.path.isfile", return_value=True, autospec=True)
#mock.patch("content_production.fitz.Page.getText")
#mock.patch("content_production.fitz.open")
def test_get_text_from_pdf(mock_fitz_open, mock_path_isfile, mock_gettext):
mock_fitz_open.return_value.__enter__.return_value = 'test'
assert cp.get_text_from_pdf('test') == 'test'
But now getting this error.
> text += page.getText()
E AttributeError: 'str' object has no attribute 'getText'
I think there are a couple issues with what you are doing. The first problem I see is that I think you are mocking the wrong function. By mocking fitz.open(next_pdfs_path) you are still expecting:
for page in doc:
text += page.getText()
to execute properly. I'd suggest that you wrap this entire with statement and text result updating in a helper function and then mock that. If the file path doesn't actually exist on your system then you will also need to mock os.path.isfile. I believe that can be done by adding a second decorator (I don't think there is any limit).
Let's say I want to set functions for each classes in module Named 'MacroMethods'. So I've set up singledispatch after seeing it in 'Fluent Python' like this:
#singledispatch
def addMethod(self, obj):
print(f'Wrong Object {str(obj)} supplied.')
return obj
...
#addMethod.register(MacroMethods.Wait)
def _(self, obj):
print('adding object wait')
obj.delay = self.waitSpin.value
obj.onFail = None
obj.onSuccess = None
return obj
Desired behavior is - when instance of class 'MacroMethods.Wait' is given as argument, singledispatch runs registered function with that class type.
Instead, it runs default function rather than registered one.
>>> Wrong Object <MacroMethods.Wait object at 0x0936D1A8> supplied.
However, type() clearly shows instance is class 'MacroMethods.Wait', and dict_keys property also contains it.
>>> dict_keys([<class 'object'>, ..., <class 'MacroMethods.Wait'>])
I suspect all custom classes I made count as 'object' type and don't run desired functions in result.
Any way to solve this problem? Entire codes are here.
Update
I've managed to mimic singledispatch's actions as following:
from functools import wraps
def state_deco(func_main):
"""
Decorator that mimics singledispatch for ease of interaction expansions.
"""
# assuming no args are needed for interaction functions.
func_main.dispatch_list = {} # collect decorated functions
#wraps(func_main)
def wrapper(target):
# dispatch target to destination interaction function.
nonlocal func_main
try:
# find and run callable for target
return func_main.dispatch_list[type(target)]()
except KeyError:
# If no matching case found, main decorated function will run instead.
func_main()
def register(target):
# A decorator that register decorated function to main decorated function.
def decorate(func_sub):
nonlocal func_main
func_main.dispatch_list[target] = func_sub
def register_wrapper(*args, **kwargs):
return func_sub(*args, **kwargs)
return register_wrapper
return decorate
wrapper.register = register
return wrapper
Used like:
#state_deco
def general():
return "A's reaction to undefined others."
#general.register(StateA)
def _():
return "A's reaction of another A"
#general.register(StateB)
def _():
return "A's reaction of B"
But still it's not singledispatch, so I find this might be inappropriate to post this as answer.
I wanted to do similar and had the same trouble. Looks like we have bumped into a python bug. Found a write-up that describes this situation.
Here is the link to the Python Bug Tracker.
Python 3.7 breaks on singledispatch_function.register(pseudo_type), which Python 3.6 accepted
I have a use case where I need to mock a member variable but I want it to return a different value every time it is accessed.
Example;
def run_test():
myClass = MyDumbClass()
for i in range(2):
print(myClass.response)
class MyDumbClass():
def __init__(self):
self.response = None
#pytest.mark.parametrize("responses", [[200,201]])
#patch("blah.MyDumbClass")
def test_stuff(mockMyDumbClass, responses)
run_test()
assert stuff
What I am hoping for here is in the run_test method the first iteration will print 200 then the next will print 201. Is this possible, been looking through unittest and pytest documentation but can't find anything about mocking a member variable in this fashion.
Just started learning pytest and unittest with python3 so forgive me if the style isn't the best.
If you wrap myDumbClass.response in a get function - say get_response() then you can use the side_effect parameter of the mock class.
side_effect sets the return_value of the mocked method to an iterator returning a different value each time you call the mocked method.
For example you can do
def run_test():
myClass = MyDumbClass()
for i in range(2):
print(myClass.get_response())
class MyDumbClass():
def __init__(self):
self.response = None
def get_response(self):
return self.response
#pytest.mark.parametrize("responses", [([200,201])])
def test_stuff( responses):
with mock.patch('blah.MyDumbClass.get_response', side_effect=responses):
run_test()
assert False
Result
----------------------------------- Captured stdout call ------------------------------------------------------------
200
201
Edit
No need to patch via context manager e.g with mock.patch. You can patch via decorator in pretty much the same way. For example this works fine
#patch('blah.MyDumbClass.get_response',side_effect=[200,100])
def test_stuff(mockMyDumbClass):
run_test()
assert False
----------------------------------- Captured stdout call ------------------------------------------------------------
200
201
I have a custom contextmanager I use (not a fixture) for setup and cleanup of a test:
#contextmanager
def db_content(*args, **kwargs):
instance = db_insert( ... )
yield instance
db_delete(instance)
def test_my_test():
with db_content( ... ) as instance:
# ...
assert result
The problem is that when the assertion fails, the db_delete() code - meaning the post yield statements, are not being executed.
I can see that if I use a fixture this does work.
#pytest.fixture
def db_instance():
instance = db_insert( ... )
yield instance
db_delete(instance)
def test_my_test(db_instance):
# ...
assert result
However, fixtures are very inflexible. I would like to pass different arguments to my context each test, while using fixtures would force me to define a different fixture for each case.
contextlib does not execute the post-yield statements if an exception was thrown. This is by design. To make it work you would have to write:
#contextmanager
def db_content(*args, **kwargs):
instance = db_insert( ... )
try:
yield instance
finally:
db_delete(instance)
In my opinion this is counter-intuitive as the try is not on the yield itself.
I took the implementation of contextmanager and made a safe version one that works as I expected, however its an entire code duplication, if anyone has a better workaround I'd love to see it.
I'm trying to wrap all methods of an existing Class (not of my creation) into a try/except suite. It could be any Class, but I'll use the pandas.DataFrame class here as a practical example.
So if the invoked method succeeds, we simply move on. But if it should generate an exception, it is appended to a list for later inspection/discovery (although the below example just issues a print statement for simplicity).
(Note that the kinds of data-related exceptions that can occur when a method on the instance is invoked, isn't yet known; and that's the reason for this exercise: discovery).
This post was quite helpful (particularly #martineau Python-3 answer), but I'm having trouble adapting it. Below, I expected the second call to the (wrapped) info() method to emit print output but, sadly, it doesn't.
#!/usr/bin/env python3
import functools, types, pandas
def method_wrapper(method):
#functools.wraps(method)
def wrapper(*args, **kwargs): #Note: args[0] points to 'self'.
try:
print('Calling: {}.{}()... '.format(args[0].__class__.__name__,
method.__name__))
return method(*args, **kwargs)
except Exception:
print('Exception: %r' % sys.exc_info()) # Something trivial.
#<Actual code would append that exception info to a list>.
return wrapper
class MetaClass(type):
def __new__(mcs, class_name, base_classes, classDict):
newClassDict = {}
for attributeName, attribute in classDict.items():
if type(attribute) == types.FunctionType: # Replace it with a
attribute = method_wrapper(attribute) # decorated version.
newClassDict[attributeName] = attribute
return type.__new__(mcs, class_name, base_classes, newClassDict)
class WrappedDataFrame2(MetaClass('WrappedDataFrame',
(pandas.DataFrame, object,), {}),
metaclass=type):
pass
print('Unwrapped pandas.DataFrame().info():')
pandas.DataFrame().info()
print('\n\nWrapped pandas.DataFrame().info():')
WrappedDataFrame2().info()
print()
This outputs:
Unwrapped pandas.DataFrame().info():
<class 'pandas.core.frame.DataFrame'>
Index: 0 entries
Empty DataFrame
Wrapped pandas.DataFrame().info(): <-- Missing print statement after this line.
<class '__main__.WrappedDataFrame2'>
Index: 0 entries
Empty WrappedDataFrame2
In summary,...
>>> unwrapped_object.someMethod(...)
# Should be mirrored by ...
>>> wrapping_object.someMethod(...)
# Including signature, docstring, etc. (i.e. all attributes); except that it
# executes inside a try/except suite (so I can catch exceptions generically).
long time no see. ;-) In fact it's been such a long time you may no longer care, but in case you (or others) do...
Here's something I think will do what you want. I've never answered your question before now because I don't have pandas installed on my system. However, today I decided to see if there was a workaround for not having it and created a trivial dummy module to mock it (only as far as I needed). Here's the only thing in it:
mockpandas.py:
""" Fake pandas module. """
class DataFrame:
def info(self):
print('pandas.DataFrame.info() called')
raise RuntimeError('Exception raised')
Below is code that seems to do what you need by implementing #Blckknght's suggestion of iterating through the MRO—but ignores the limitations noted in his answer that could arise from doing it that way). It ain't pretty, but as I said, it seems to work with at least the mocked pandas library I created.
import functools
import mockpandas as pandas # mock the library
import sys
import traceback
import types
def method_wrapper(method):
#functools.wraps(method)
def wrapper(*args, **kwargs): # Note: args[0] points to 'self'.
try:
print('Calling: {}.{}()... '.format(args[0].__class__.__name__,
method.__name__))
return method(*args, **kwargs)
except Exception:
print('An exception occurred in the wrapped method {}.{}()'.format(
args[0].__class__.__name__, method.__name__))
traceback.print_exc(file=sys.stdout)
# (Actual code would append that exception info to a list)
return wrapper
class MetaClass(type):
def __new__(meta, class_name, base_classes, classDict):
""" See if any of the base classes were created by with_metaclass() function. """
marker = None
for base in base_classes:
if hasattr(base, '_marker'):
marker = getattr(base, '_marker') # remember class name of temp base class
break # quit looking
if class_name == marker: # temporary base class being created by with_metaclass()?
return type.__new__(meta, class_name, base_classes, classDict)
# Temporarily create an unmodified version of class so it's MRO can be used below.
TempClass = type.__new__(meta, 'TempClass', base_classes, classDict)
newClassDict = {}
for cls in TempClass.mro():
for attributeName, attribute in cls.__dict__.items():
if isinstance(attribute, types.FunctionType):
# Convert it to a decorated version.
attribute = method_wrapper(attribute)
newClassDict[attributeName] = attribute
return type.__new__(meta, class_name, base_classes, newClassDict)
def with_metaclass(meta, classname, bases):
""" Create a class with the supplied bases and metaclass, that has been tagged with a
special '_marker' attribute.
"""
return type.__new__(meta, classname, bases, {'_marker': classname})
class WrappedDataFrame2(
with_metaclass(MetaClass, 'WrappedDataFrame', (pandas.DataFrame, object))):
pass
print('Unwrapped pandas.DataFrame().info():')
try:
pandas.DataFrame().info()
except RuntimeError:
print(' RuntimeError exception was raised as expected')
print('\n\nWrapped pandas.DataFrame().info():')
WrappedDataFrame2().info()
Output:
Unwrapped pandas.DataFrame().info():
pandas.DataFrame.info() called
RuntimeError exception was raised as expected
Wrapped pandas.DataFrame().info():
Calling: WrappedDataFrame2.info()...
pandas.DataFrame.info() called
An exception occurred in the wrapped method WrappedDataFrame2.info()
Traceback (most recent call last):
File "test.py", line 16, in wrapper
return method(*args, **kwargs)
File "mockpandas.py", line 9, in info
raise RuntimeError('Exception raised')
RuntimeError: Exception raised
As the above illustrates, the method_wrapper() decoratored version is being used by methods of the wrapped class.
Your metaclass only applies your decorator to the methods defined in classes that are instances of it. It doesn't decorate inherited methods, since they're not in the classDict.
I'm not sure there's a good way to make it work. You could try iterating through the MRO and wrapping all the inherited methods as well as your own, but I suspect you'd get into trouble if there were multiple levels of inheritance after you start using MetaClass (as each level will decorate the already decorated methods of the previous class).