I hava a multiprocess program whcich using starmap. But when I run it, a TypeError occured. While normally when I run it in map.
def process_single_image(img_path, target_dir="", func=None, severity=1):
print(target_dir)
...
pool.starmap(
partial(
process_single_image,
target_dir=target_dir,
func=iaa.imgcorruptlike.apply_shot_noise,
severity=4,
),
img_paths,
)
I don't know why I got this error. I think it should run normally as pool.map(....).
Here is the traceback.
The arguments you pass to .starmap are unpacked to the target function. Therefore, the arguments must be an iterable of iterables (even if the argument length the target function accepts is one).
Therefore, if the argument length is one, then explicitly convert each element of img_path into a tuple before passing it to starmap:
img_path = [(arg,) for arg in img_path]
Related
Imagine I have a dict.
d = ['a': 1 , 'b':3]
I'm having a hard time to understand the difference between d.get and d.get().
I know that d.get() get the value from the key, like this:
print(d.get('a') )
output: 1
But when I write d.get, it shows this:
print(d.get)
output: <built-in method get of dict object at .........>
What is 'd.get' doing in my code?
I'm using python 3X
A method is literally just an attribute of an object that happens to be of type <class function>. The output you see is essentially what happens when you try to call print() on any function object, and is essentially a concise string representation that python creates for the function.
Actually calling a function is done with parentheses: d.get('a'), which means to execute the behavior the function refers to. It doesn't especially matter where the function is, though: I could do the following, and it would still work:
d = {'a': 1 , 'b':3}
freefunc = d.get
freefunc('a')
This is what the term "first class functions" refers to, when people compare python to something like Java. An entire function can be encapsulated in a variable and treated no differently than any other variable or attribute.
The short answer? There is no difference between the two methods. They are the same exact method.
The difference in your code is at when you write .get() you call the method, but when you write .get you just get a pointer (or location in the memory, to be exact) for that method, to call it later on if needed.
In the first scenario, you are calling print on the result of executing get('a'), which in this case is 1.
In your second scenario, you are calling print on the get function itself, instead of on an execution of it, which evaluates to its documentation, i.e. <built-in method get of dict object at... etc.
I have a piece of code that has used python exec() function. data is saved as an array in a file and when exec() is used, data type is specified as dictionary. I can't quit understand what's the output
style = dict()
# test.py includes one 10 x 10 array
with open('test.py')as output:
exec(output.read(), style)
Since you are passing empty dict() as globals argument for exec() output will not be defined when output.read() is executed. If you need to print the result of output.read() then you need to pass either globals() or locals() as second argument to exec. They return a dictionary containing objects available in global and local scope respectively. The new code may be:
style = dict()
with open('test.py') as output:
exec("print(output.read())", globals())
or
style = dict()
with open('test.py') as output:
exec("print(output.read())", locals())
Return value from exec statement is None so you need to use print to see the output of output.read()
I am measuring performance of different sorting methods using Python built-in library timeit. I would like to pass a function and an integer as arguments to the statement being tested in timeit(). I tried the following:
def sort_1(l):
...
return l_sorted
def test(f: Callable, l_len: int):
l = np.random.rand(low=-1000, high=1000, size=l_len)
f(l)
timeit.timeit(stmt=test(sort_1, l_len=10), number=1000)
... with a ValueError saying that stmt is neither a string nor callable. The error doesn't occur when I call it like this:
timeit.timeit(stmt=test, number=1000)
... but then I cannot pass any argument to test(). What is a general solution if someone wants to pass arguments to a function given as an argument? (let's say, when a method is already implemented and there is not way to pass arguments in a separate argument)
Cheers
Edit:
#jonrsharpe, thanks! The solution looks like this:
timeit.timeit(stmt='test(f=sort_1, l_len=10)', number=100, globals=globals())
I would like to use argparse to pass some values throughout my main function. When calling the python file, I would always like to include the flag for the argument, while either including or excluding its string argument. This is because some external code, where the python file is being called, becomes a lot simpler if this would be possible.
When adding the arguments by calling parser.add_argument, I've tried setting the default value to default=None and also setting it to default=''. I can't make this work on my own it seems..
def main():
parser = argparse.ArgumentParser()
parser.add_argument('-p', '--projects_to_build', default='')
args = parser.parse_args()
This call works fine:
py .\python_file.py -p proj_1,proj_2,proj_3
This call does not:
py .\python_file.py -p
python_file.py: error: argument -p/--projects_to_build: expected one argument
You need to pass a nargs value of '?' with const=''
parser.add_argument('-p', '--projects_to_build', nargs='?', const='')
You should also consider adding required=True so you don't have to pass default='' as well.
I'm testing code where one of two exceptions can be raised: MachineError or NotImplementedError. I would like to use pytest.raises to make sure that at least one of them is raised when I run my test code, but it only seems to accept one exception type as an argument.
This is the signature for pytest.raises:
raises(expected_exception, *args, **kwargs)
I tried using the or keyword inside a context manager:
with pytest.raises(MachineError) or pytest.raises(NotImplementedError):
verb = Verb("donner<IND><FUT><REL><SG><1>")
verb.conjugate()
but I assume this only checks whether the first pytest.raises is None and sets the second one as the context manager if it is.
Passing multiple exceptions as positional arguments doesn't work, because pytest.raises takes its second argument to be a callable. Every subsequent positional argument is passed as an argument to that callable.
From the documentation:
>>> raises(ZeroDivisionError, lambda: 1/0)
<ExceptionInfo ...>
>>> def f(x): return 1/x
...
>>> raises(ZeroDivisionError, f, 0)
<ExceptionInfo ...>
>>> raises(ZeroDivisionError, f, x=0)
<ExceptionInfo ...>
Passing the exceptions as a list doesn't work either:
Traceback (most recent call last):
File "<pyshell#4>", line 1, in <module>
with pytest.raises([MachineError, NotImplementedError]):
File "/usr/local/lib/python3.4/dist-packages/_pytest/python.py", line 1290, in raises
raise TypeError(msg % type(expected_exception))
TypeError: exceptions must be old-style classes or derived from BaseException, not <class 'list'>
Is there a workaround for this? It doesn't have to use a context manager.
Pass the exceptions as a tuple to raises:
with pytest.raises( (MachineError, NotImplementedError) ):
verb = ...
In the source for pytest, pytest.raises may:
catch expected_exception; or
pass expected_exception to a RaisesContext instance, which then uses issubclass to check whether the exception was one you wanted.
In Python 3, except statements can take a tuple of exceptions. The issubclass function can also take a tuple. Therefore, using a tuple should be acceptable in either situation.