First of all, I'm pretty new in multiprocessing and I'm here for learning of all of you. I have several files doing something similar to this:
SharedClass.py:
class simpleClass():
a = 0
b = ""
.....
MyProcess.py:
import multiprocessing
import SharedClass
class FirstProcess(multiprocessing.Process):
def __init__(self):
multiprocessing.Process.__init__(self)
def modifySharedClass():
# Here I want to modify the object shared with main.py defined in SharedClass.py
Main.py:
from MyProcess import FirstProcess
import sharedClass
if __name__ == '__main__':
pr = FirstProcess()
pr.start()
# Here I want to print the initial value of the shared class
pr.modifySharedClass()
# Here I want to print the modified value of the shared class
I want to define a shared class (in SharedClass.py), in a kind of shared memory that can be readed and writted for both files Main.py and MyProcess.py.
I have try to use the Manager of multiprocessing and multiprocessing.array but Im not having good results, the changes made in one file are not beeing reflected in the other file (maybe Im doing this in the wrong way).
Any ideas? Thank you.
so I am writing code where I generate certain data in a class and save it in a dictionary. I want to use that data in the second class . The first class is as fellows:
class DataAnalysis():
def __init__(self,matfile=None):
'''Constructor
'''
self.matfile= matfile
def get_alldata(self):
print('all dict data accessed')
print(bodedata_dict)
return bodedata_dict
if __name__ == '__main__':
obj1= DataAnalysis(matfile=matfile)
"do some work"
bodedata_dict.update(bode_data)
obj1.get_alldata()
I then access the dictionary in the second class as:
from A import DataAnalysis
class PlotComaparison(DataAnalysis):
if __name__ == '__main__':
obj= DataAnalysis(matfile=None)
obj1= PlotComaparison(obj)
dict_data= obj.get_alldata()
But when I run the script with the second class, it gives me the following error:
File "DataAnalysis.py", line 301, in get_alldata
print(bodedata_dict)
NameError: name 'bodedata_dict' is not defined
I am very new to the concept of classes in python, so please help me with how I can use data from one class into another.
The get_alldata() method in the DataAnalysis class you defined is returning a bodedata_dict which isn't defined anywhere. It's like printing the content of a variable without defining it first.
EDIT:
Looking further into it, bodedata_dictin the first example comes from outside the class. You would likely want to change the flow of your program so that when DataAnalysis has it's get data method called, it doesn't depend on an outside state.
I don't very understand the classes logic in python but cannot answer on web.
I have create a class to generate person info:
class person:
def fristnameGen(gender):
...
def emailGen(firstname,surname):
...
i create a bot to call it like this:
from person import *
class bots:
def __init__(self):
self.person = person()
def createDB(self):
print(self.person.name)
#do something...
finally i call it by a button with thread
from bots import *
import threading
class Panel:
def __init__(self):
self.top = tk.Tk()
self.bot = bots()
self.buildUI()
def foo(self):
self.bot.createDB(self.stringPhone.get())
def threadTheAction(func, *args):
t = threading.Thread(target=func, args=args)
t.setDaemon(True)
t.start()
def buildUI(self):
Button = tk.Button(self.top, text ="Start", command = lambda :self.threadTheAction(self.foo))
I get this error:
TypeError: 'Panel' object is not callable
However, i call it directly, its work
Button = tk.Button(self.top, text ="Start", command = lambda :self.foo())
How to fix the error?
...
2. Moreover, i tried create p1 = person() and p2= person() and print it. Found p1 and p2 is a same person, i prefer each new a class have a new one. How to generate "new person" using classes?
Thank you
You seem to have a lot of confusion about Object Oriented Programming in Python. Some of your methods have self parameters and some do not, seemingly at random. That's the source of your current error.
The threadTheAction method in your Panel class is getting the Panel instance passed in as its first argument, but that's named func in the method (since you omitted self). The real function you're passing as an argument gets caught in the variable argument *args. When the thread tries unsuccessfully to call it, you get an exception. Adding self before func would fix the immediate problem:
def threadTheAction(self, func, *args):
I suspect if your code got further along, you'd run into other errors with other methods without self in their parameter lists. For instance, none of the methods you've shown in person are likely to work correctly.
As for your second question, you haven't shown enough of person to know what's happening, but you're probably doing instance variables wrong somehow. With no self parameter in the methods, that's almost inevitable (since you assign to self.whatever to set a whatever attribute on the current instance). If you need help squaring that away, I suggest you ask a separate question (Stack Overflow is best when each question is self-contained) and provide the full code for your person class.
Lets say I the following class;
class CompositionClass(object):
def __init__(self):
self._redis = Redis()
self._binance_client = BinanceClient()
def do_processing(self, data):
self._redis.write(data)
self._binance_client.buy(data.amount_to_buy)
# logic to actually unittest
return process_val
I have other objects which call external API as composition in my ComplexClass. When I am unittesting the logic of do_processing, I do not want to call these expensive API calls. I have checked throughly in SO and Google about unittesting; all examples are simple not that really useful. In my case how can I use unittest.mock to mock these objects?
One way of mocking the Redis and BinanceClient classes is to use the patch decorator in your test class, such as:
from unittest import TestCase
from unittest.mock import patch
from package.module import CompositionClass
class TestCompositionClass(TestCase):
#patch('package.module.BinanceClient')
#patch('package.module.Redis')
def test_do_processing(self, mock_redis, mock_binance):
c = CompositionClass()
data = [...]
c.do_processing(data)
# Perform your assertions
# Check that mocks were called
mock_redis.return_value.write.assert_called_once_with(data)
mock_binance.return_value.buy.assert_called_once_with(data.amount_to_buy)
Note that the path specified to #patch is the path to module containing the CompositionClass and its imports for Redis and BinanceClient. The patching happens in that module, not the module containing the Redis and BinanceClient implementations themselves.
You need to set a value that must be returned by your API call, to this function:
from unittest.mock import MagicMock
class Tester(unittest.TestCase):
def setUp(self):
pass
def test_do_processing(self):
self.API_function = MagicMock(return_value='API_response')
# test logic
I want to mock a method of a class and use wraps, so that it is actually called, but I can inspect the arguments passed to it. I have seen at several places (here for example) that the usual way to do that is as follows (adapted to show my point):
from unittest import TestCase
from unittest.mock import patch
class Potato(object):
def foo(self, n):
return self.bar(n)
def bar(self, n):
return n + 2
class PotatoTest(TestCase):
spud = Potato()
#patch.object(Potato, 'foo', wraps=spud.foo)
def test_something(self, mock):
forty_two = self.spud.foo(n=40)
mock.assert_called_once_with(n=40)
self.assertEqual(forty_two, 42)
However, this instantiates the class Potato, in order to bind the mock to the instance method spud.foo.
What I need is to mock the method foo in all instances of Potato, and wrap them around the original methods. I.e, I need the following:
from unittest import TestCase
from unittest.mock import patch
class Potato(object):
def foo(self, n):
return self.bar(n)
def bar(self, n):
return n + 2
class PotatoTest(TestCase):
#patch.object(Potato, 'foo', wraps=Potato.foo)
def test_something(self, mock):
self.spud = Potato()
forty_two = self.spud.foo(n=40)
mock.assert_called_once_with(n=40)
self.assertEqual(forty_two, 42)
This of course doesn't work. I get the error:
TypeError: foo() missing 1 required positional argument: 'self'
It works however if wraps is not used, so the problem is not in the mock itself, but in the way it calls the wrapped function. For example, this works (but of course I had to "fake" the returned value, because now Potato.foo is never actually run):
from unittest import TestCase
from unittest.mock import patch
class Potato(object):
def foo(self, n):
return self.bar(n)
def bar(self, n):
return n + 2
class PotatoTest(TestCase):
#patch.object(Potato, 'foo', return_value=42)#, wraps=Potato.foo)
def test_something(self, mock):
self.spud = Potato()
forty_two = self.spud.foo(n=40)
mock.assert_called_once_with(n=40)
self.assertEqual(forty_two, 42)
This works, but it does not run the original function, which I need to run because the return value is used elsewhere (and I cannot fake it from the test).
Can it be done?
Note The actual reason behind my needs is that I'm testing a rest api with webtest. From the tests I perform some wsgi requests to some paths, and my framework instantiates some classes and uses their methods to fulfill the request. I want to capture the parameters sent to those methods to do some asserts about them in my tests.
In short, you can't do this using Mock instances alone.
patch.object creates Mock's for the specified instance (Potato), i.e. it replaces Potato.foo with a single Mock the moment it is called. Therefore, there is no way to pass instances to the Mock as the mock is created before any instances are. To my knowledge getting instance information to the Mock at runtime is also very difficult.
To illustrate:
from unittest.mock import MagicMock
class MyMock(MagicMock):
def __init__(self, *a, **kw):
super(MyMock, self).__init__(*a, **kw)
print('Created Mock instance a={}, kw={}'.format(a,kw))
with patch.object(Potato, 'foo', new_callable=MyMock, wrap=Potato.foo):
print('no instances created')
spud = Potato()
print('instance created')
The output is:
Created Mock instance a=(), kw={'name': 'foo', 'wrap': <function Potato.foo at 0x7f5d9bfddea0>}
no instances created
instance created
I would suggest monkey-patching your class in order to add the Mock to the correct location.
from unittest.mock import MagicMock
class PotatoTest(TestCase):
def test_something(self):
old_foo = Potato.foo
try:
mock = MagicMock(wraps=Potato.foo, return_value=42)
Potato.foo = lambda *a,**kw: mock(*a, **kw)
self.spud = Potato()
forty_two = self.spud.foo(n=40)
mock.assert_called_once_with(self.spud, n=40) # Now needs self instance
self.assertEqual(forty_two, 42)
finally:
Potato.foo = old_foo
Note that you using called_with is problematic as you are calling your functions with an instance.
Do you control creation of Potato instances, or at least have access to these instances after creating them? You should, else you'd not be able to check particular arg lists.
If so, you can wrap methods of individual instances using
spud = dig_out_a_potato()
with mock.patch.object(spud, "foo", wraps=spud.foo) as mock_spud:
# do your thing.
mock_spud.assert_called...
Your question looks identical to python mock - patching a method without obstructing implementation to me. https://stackoverflow.com/a/72446739/9230828 implements what you want (except that it uses a with statement instead of a decorator). wrap_object.py:
# Copyright (C) 2022, Benjamin Drung <bdrung#posteo.de>
#
# Permission to use, copy, modify, and/or distribute this software for any
# purpose with or without fee is hereby granted, provided that the above
# copyright notice and this permission notice appear in all copies.
#
# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
import contextlib
import typing
import unittest.mock
#contextlib.contextmanager
def wrap_object(
target: object, attribute: str
) -> typing.Generator[unittest.mock.MagicMock, None, None]:
"""Wrap the named member on an object with a mock object.
wrap_object() can be used as a context manager. Inside the
body of the with statement, the attribute of the target is
wrapped with a :class:`unittest.mock.MagicMock` object. When
the with statement exits the patch is undone.
The instance argument 'self' of the wrapped attribute is
intentionally not logged in the MagicMock call. Therefore
wrap_object() can be used to check all calls to the object,
but not differentiate between different instances.
"""
mock = unittest.mock.MagicMock()
real_attribute = getattr(target, attribute)
def mocked_attribute(self, *args, **kwargs):
mock.__call__(*args, **kwargs)
return real_attribute(self, *args, **kwargs)
with unittest.mock.patch.object(target, attribute, mocked_attribute):
yield mock
Then you can write following unit test:
from unittest import TestCase
from wrap_object import wrap_object
class Potato:
def foo(self, n):
return self.bar(n)
def bar(self, n):
return n + 2
class PotatoTest(TestCase):
def test_something(self):
with wrap_object(Potato, 'foo') as mock:
self.spud = Potato()
forty_two = self.spud.foo(n=40)
mock.assert_called_once_with(n=40)
self.assertEqual(forty_two, 42)