I have a data augmentation script that has a class with a bunch of optional methods that are triggered by argparse arguments. I am curious how I can structure my code to process the argparse commands based on the order they are passed in from the terminal.
Goal: If I were to pass arguments as: python maths.py --add --multiply I would want it to add 10 first then multiply by 5 second.
If I were to pass arguments as: python maths.py --multiply --add I would want it to multiply 5 first then add 10.
For example:
class Maths:
def __init__(self):
self.counter = 0
def addition(self, num):
self.counter += num
return self
def multiply(self, num):
self.counter *= num
return self
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--add', required = False, action = 'store_true')
parser.add_argument('--multiply', required = False, action = 'store_true')
args = parser.parse_args()
maths = Maths()
maths.addition(10)
maths.multiply(5)
print(maths.counter)
if __name__ == "__main__":
main()
How can I accomplish ordering based on the order of how the arguments are passed in? Thank you!
This parser provides two ways of inputing lists of strings:
In [10]: parser = argparse.ArgumentParser()
...: parser.add_argument('--cmds', nargs='*', choices=['add','mult'])
...: parser.add_argument('--add', dest='actions', action='append_const', const='add')
...: parser.add_argument('--multiply', dest='actions', action = 'append_const', const='mult')
...: parser.print_help()
...:
...:
usage: ipython3 [-h] [--cmds [{add,mult} [{add,mult} ...]]] [--add]
[--multiply]
optional arguments:
-h, --help show this help message and exit
--cmds [{add,mult} [{add,mult} ...]]
--add
--multiply
As values of a '--cmds' argument:
In [11]: parser.parse_args('--cmds mult add'.split())
Out[11]: Namespace(actions=None, cmds=['mult', 'add'])
As separate flagged arguments:
In [12]: parser.parse_args('--mult --add'.split())
Out[12]: Namespace(actions=['mult', 'add'], cmds=None)
In both cases I create a list of strings. In the second the const values could be functions or methods.
const=maths.addition
Related
I have a class Data which I want to filter using the below api.
# Example: filter using where
inpt = {"a":np.array((1,2,3,4,2,5,6,2,3,3,2,1)),
"b":np.random.rand(12)}
data = (Data(inpt)
.where(col("a").equals(3)) # This is how where should be called.
)
data
where is a method from class Data
col("a").equals(3) is syntactic sugar for inpt["a"] == 3
I am able to achieve this using another class Expr which handles all the functionality within Data.where() using closures. Reason for this being that Expr doesn't have access to Data.
Questions: can someone provide me with an alternative approach not involving closures. My goal is to learn new approaches / directions.
Here is my code:
from __future__ import annotations
from typing import Dict, Any
import numpy as np
class Data:
def __init__(self, data: Dict):
self._data = data
def where(self, e: Expr) -> Data:
idx = e.collect(self)
for k,v in self._data.items():
self._data[k] = v[idx]
return self
def __repr__(self):
return str(self._data)
class Expr:
def __init__(self):
self.fs = []
def col(self, s: str) -> Self:
f = lambda x: x._data[s]
self.fs.append(f)
return self
def equals(self, el: Any) -> Self:
f = lambda x: x == el
self.fs.append(f)
return self
def collect(self, x: Data) -> Data:
args = x
for f in self.fs:
args = f(args)
return args
def col(s: str) -> Expr:
return Expr().col(s)
I don't really understand the point. Maybe if you give an example of what you're actually trying to do?
If you already know the right key, you can just check directly. If you want to find the right key, the pythonic way is to use a list comprehension.
In [2]: inpt = {
...: "a": (1,2,3,4,2,5,6,2,3,3,2,1),
...: "b": 3,
...: }
In [3]: inpt["a"] == 3
Out[3]: False
In [4]: inpt["b"] == 3
Out[4]: True
In [5]: [key for key, value in inpt.items() if value == 3][0]
Out[5]: 'b'
In [8]: from typing import Sequence
In [9]: [key for key, value in inpt.items() if isinstance(value, Sequence) and 3 in value][0]
Out[9]: 'a'
I want a function that takes in kwargs to recursively. How can I pass the kwargs on?
example of the code:
def recursion(a, b, **kwargs):
if a == 1:
print(a + b)
elif a == 2:
print(a + b + kwargs['name']
else:
a = a/2
recursion(what to put in here?)
def re(a, b, **kwargs):
print(a + b, kwargs['name'])
if a == 0:
return
else:
re(a-b,b,**kwargs)
re(5,1,name='Hello World!')
This will give you the following output
6 Hello World!
5 Hello World!
4 Hello World!
3 Hello World!
2 Hello World!
1 Hello World!
Having been faced with a similar problem recently, I've come up with the following:
from functools import wraps
from inspect import currentframe
def recursive_kwargs(fun):
''' Persist keyword arguments through recursive calls '''
#wraps(fun)
def wrapper(*args,**kwargs):
caller_name = currentframe().f_back.f_code.co_name
if caller_name is fun.__name__:
fun(*args,**fun.__kwargs__)
else: # Top of the recursive stack
fun.__kwargs__ = kwargs
fun(*args,**kwargs)
return wrapper
You can then just decorate your function and presto, it ought to work by itself without passing any kwargs along:
#recursive_kwargs
def fib(n, codename='spam', password='eggs'):
print(codename, password)
if n == 1:
return 1
return fib(n-1) # Don't pass any kwargs
Now fib(3) returns spam eggs (3 times), and fib(3,codename='Vikings') does indeed return Vikings eggs (3 times).
This is lovely and convenient, but SLOW, both due to the #wraps and the currentframe lookup. Here are some timeit results:
fib: [3.69, 3.24, 2.82, 2.57, 2.56] us # As defined above
fib2: [1.45, 1.14, 1.15, 1.08, 1.08] us # With just the #wraps (but doing nothing)
fib3: [0.58, 0.66, 0.68, 0.54, 0.48] us # Just passing the kwargs along manually
TLDR: #Harish's answer is probably the way to do this unless you don't care about efficiency. For example if you are dealing with a shallow (constant depth, even) recursion stack where the bottleneck is the body of the function itself, not the recursion (aka definitely not the example I've used here).
Also; don't use recursion without memoisation for calculating Fibonacci numbers (see e.g. here for more information)
First, see the relevant user code for running z3-solver. This works fine, and the solver returns some values after evaluating the expression:
# see source at https://github.com/Z3Prover/z3
# the relevant code is at: z3-master/src/api/python/z3/z3.py
# pip install z3-solver
from z3 import *
x = Int('x')
#print(type(x)) # <class 'z3.z3.ArithRef'>
y = Int('y')
s = Solver()
# print(type(s)) # <class 'z3.z3.Solver'>
s.add(x > 10) # this works
s.add(y == x + 2)
print(s)
print(s.check())
print(s.model())
Now, my short attempt at writing an evaluator like z3:
class Int():
def __init__(self,name):
self.name = name
self.value = None
def do_print(self):
print('Rand({})'.format(self.name))
def __str__(self):
return(str(self.value))
class Solver():
def add(self, *argv):
self.args = []
for arg in argv:
self.args.append(arg)
def do_print(self):
print('Constr(',end='')
for arg in self.args:
print(arg)
print(')')
x = Int('x')
s=Solver()
s.add(x > 10) # this does not work - but why is same kind of code in z3 working?
# err: TypeError: '<' not supported between instances of 'Int' and 'int'
x.do_print()
s.do_print()
I am getting an error on the s.add() line. I understand the error, that the object 'x' cannot be compared with an integer. This is happening because the expression in this line is getting evaluated before the s.add() is being called.
But my question is - how is z3-solver bypassing this error? How is it able to pass the args to add() without evaluating it first?
How do I fix the problem in my code, and pass the expression to s.add() without evaluating it first?
I want to add an argument of type string but restrict the length of the string to 1.
group.add_argument(
"--test_param",
dest="test_param",
type=str,
required=True,
default=None,
)
Is there any way I can do it here without having to raise an error by checking len(test_param) later?
You can try using a lambda function in type
Ex:
import argparse
group = argparse.ArgumentParser(description='Data.')
group.add_argument(
"--test_param",
dest="test_param",
type=lambda x: x if x.isalpha() and len(x) <=1 else False, #Sample Function
required=True,
default=None,
)
args = group.parse_args()
value = getattr(args, 'test_param')
if value:
print(value)
else:
print("error!!!")
Output:
python sample.py --test_param fddd #->>error!!
python sample.py --test_param f #->>f
You can do this with the type parameter of add_argument. The recommended way is to raise an exception during the type check:
import argparse
import sys
parser = argparse.ArgumentParser()
def len_gt_0(s):
if len(s) > 0:
return s
raise argparse.ArgumentTypeError("Value must have length greater than 0")
parser.add_argument("--foo", type=len_gt_0, required=True)
opts = parser.parse_args(sys.argv[1:])
print(f"{ opts.foo = }")
Output:
$ python3.8 /tmp/foo.py --foo=abc
opts.foo = 'abc'
$ python3.8 /tmp/foo.py --foo=
usage: foo.py [-h] --foo FOO
foo.py: error: argument --foo: Value must have length greater than 0
$ python3.8 /tmp/foo.py
usage: foo.py [-h] --foo FOO
foo.py: error: the following arguments are required: --foo
You can use the choices argument available while adding a argument for example in my code i just want to select from 1 or 2. I used the following snippet as follows:
parser.add_argument(
"report",
type=int,
metavar="[report #]",
help="1. for Annual Max/Min Temperature 2. for Hottest day of each year",
choices=[1, 2]
)
it will restrict the input from these choices only and will show the error result if you have given out of these choices like this:
usage: [report#] [data_dir]
[report #]
1 for Annual Max/Min Temperature
2 for Hottest day of each year
[data_dir]
Directory containing weather data files
main.py: error: argument [report #]: invalid choice: 3 (choose from 1, 2)
If you don't want to add the choices you can use lamda or a function for validating the data. You can get help from my example below:
parser.add_argument(
"data_dir",
type=check_directory_availability,
metavar="[data_directory]",
help="Directory containing weather data files",
)
def check_directory_availability(dir_name):
if not os.path.isdir(dir_name):
raise NotADirectoryError("Provided directory name doesn't exists")
return dir_name
Hope so it may help you.
all!
Normally, if I need to pass a dictionary to a function as parameter set, One simple way of doing that is through unpacking the dictionary and use them.
def some_function(a=0, b=0, **params):
print(a) # 1
print(b) # 2
d = {"a":1, "b":2}
some_function(**d)
I am working on a little project that will take very long time. So, I was trying to use Multiprocessing to help me get the job done faster. I simplified my code (below).
from multiprocessing import Pool
from itertools import repeat
import time
def some_function(n, param1=0, param2=0):
#some operation on n and **params
print(n)
print(param1)
input_k = [i for i in range(1,10,2)]
input_params = {'param1':1, 'param2':1}
if __name__ == '__main__':
with Pool(16) as p:
k = p.starmap(some_function, zip(input_k, repeat(input_params)))
############## output ##############
#1
#7
#9
#5
#3
#{'param1': 1, 'param2': 1}
#{'param1': 1, 'param2': 1}
#{'param1': 1, 'param2': 1}
#{'param1': 1, 'param2': 1}
#{'param1': 1, 'param2': 1}
I wanted to run that some_function on the corresponding input_k and input_params. But the output of the function is not as expected. It looks like the function take whole input_params dictionary and assigned it to param1 in the function?
I know I did not unpack the dictionary as I pass it to the function, but I don't know how to do that(zip() doesn't allow me to simply add "**" to the input_param). How do I do that?
Thanks!
I modified your code to:
from multiprocessing import Pool
from itertools import repeat
import time
def some_function(n, param1=0, param2=0):
#some operation on n and **params
print(n, param1, param2)
input_k = [i for i in range(1,10,2)]
input_params = {'param1':1, 'param2':1}
if __name__ == '__main__':
with Pool(16) as p:
k = p.starmap(some_function, [[x,*y.values()] for x,y in zip(input_k, repeat(input_params))])
And is now working fine.