I've got a script (foo.py) which I originally developed to be called from the command line with named arguments. I'd now like to refactor it so it may called both from the command line and via import as well. Calling the script as is after import throws an unexpected keyword argument error, as written (since the main function itself has no arguments). I was under the impression that the __name__=="__main__" chunk would've provided the same experience in both calls, but I've obviously misunderstood. How could I refactor this so it can take the named arguments interactively and upon import by another script?
foo.py
def parse_args():
parser = argparse.ArgumentParser()
parser.add_argument('--token', help='API token')
parser.add_argument('--namedArgA', help='Switch parameter', action='store_true')
return parser.parse_args()
def main():
args = parse_args()
token = args.token
# Other stuff with the args
print(argDict)
if __name__=="__main__":
main()
bar.py
import foo
baz = foo.main(token='myToken',namedArgA='True')
Related
I'd like to make a parser for a program like follows program --serve some/path /file/to/serve.html
Looking at the argparse documentation https://docs.python.org/3/library/argparse.html#type
I cannot for the life of me figure out how I could parse the first argument of --serve as a string and the second as argparse.FileType('r')
I'd like to do something like
parser.add_argument('--serve', nargs=2, type=(str, argparse.FileType('r')), action='append', help='...')
Is there a way to do this with argparse?
If you implement a custom type and instead of using --nargs=2 you use a delimiter to separate the two arguments, you could do something like this:
import os
import stat
import argparse
def mytype(v):
dirpath, filepath = v.split(':')
try:
res = os.stat(filepath)
except FileNotFoundError:
raise ValueError(f'{filepath} does not exist')
else:
if not stat.S_ISREG(res.st_mode):
raise ValueError(f'{filepath} is not a regular file')
return dirpath, filepath
p = argparse.ArgumentParser()
p.add_argument('--serve', type=mytype, action='append', help='...')
args = p.parse_args()
print(args)
If I run this in a directory that contains the file foo.txt, we see:
# with a file that does not exist
$ python argtest.py --serve somedir:bar.txt
usage: argtest.py [-h] [--serve SERVE]
argtest.py: error: argument --serve: invalid mytype value: 'somedir:bar.txt'
# with a file that *does* exist
$ python argtest.py --serve somedir:foo.txt
Namespace(serve=[('somedir', 'foo.txt')])
This isn't opening the file; it's testing that the second argument points to a regular file. You could instead of the file and return a file object instead of the path if you want.
I want to print the name of file along with line number, but it print the library name and its line number.
#!/bin/env python3
# library source code (lib.py)
from inspect import currentframe, getframeinfo
def get_file_name():
return __file__
def get_line():
frameinfo = getframeinfo(currentframe())
return f"[{frameinfo.filename}:{frameinfo.lineno}]"
def log(data):
print(f"{get_line()} {data}")
# program source code (program.py)
#!/bin/env python3
import lib
lib.log("hi")
Output:
[/home/shubhapp/python3/codathon/doubt/lib.py:9] hi
Expected output:
[/home/shubhapp/python3/codathon/doubt/program.py:7] hi
def get_file_name():
return __file__
__file__ is returned from the namespace of the module itself, not from the caller. That's why you didn't get main.py, but test.py.
def get_line():
frameinfo = getframeinfo(currentframe())
return f"[{frameinfo.filename}:{frameinfo.lineno}]"
You are not getting desired output because currentframe() gave you the frame in which get_line was being executed, not where it was invoked. To get to the invocation point use the following approach:
get get_line():
# fs means FrameSummary.
caller_fs = traceback.extract_stack()[0]
filename, lineno = caller_fs.filename, caller_fs.lineno
# Do whatever you want to do now.
If you want to see intermediate invocation points, you can use traceback.print_stack().
I have been given an existing script (let's call it existing.py) that in its MVCE form has the following structure.
import argparse
FLAGS = None
def func():
print(FLAGS.abc)
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument(
'--abc',
type=str,
default='',
help='abc.'
)
FLAGS, unparsed = parser.parse_known_args()
func()
As this is part of tool that gets constantly updated, I cannot change existing.py. Normally, existing.py is invoked with commandline arguments.
python -m existing.py --abc "Ok"
which prints the output 'Ok'.
I wish to call the functions (not the whole script) in existing.py using another script. How can I feed in the FLAGS object that is used in the functions of the script? I do not wish to use subprocess will just run the script in its entirety.
I know that argparse creates the FLAGS as a Namespace dictionary and I can construct it in calling.py (see code below) but I cannot then push it back into the function that is imported from existing.py into calling.py. The following is the calling.py that I've tried.
from existing import func
import argparse
args = argparse.Namespace()
args.abc = 'Ok'
FLAGS = args
func()
which throws an error
AttributeError: 'NoneType' object has no attribute 'abc'
This is different from other StackOverflow questions as this question explicitly forbids subprocess and the existing script cannot be changed.
Import existing and use
existing.FLAGS = args
Now functions defined in the existing namespace should see the desired FLAGS object.
I am trying to pass arguments from a pytest testcase to a module being tested. For example, using the main.py from Python boilerplate, I can run it from the command line as:
$ python3 main.py
usage: main.py [-h] [-f] [-n NAME] [-v] [--version] arg
main.py: error: the following arguments are required: arg
$ python3 main.py xx
hello world
Namespace(arg='xx', flag=False, name=None, verbose=0)
Now I am trying to do the same with pytest, with the following test_sample.py
(NOTE: the main.py requires command line arguments. But these arguments need to be hardcoded in a specific test, they should not be command line arguments to pytest. The pytest testcase only needs to send these values as command line arguments to main.main().)
import main
def test_case01():
main.main()
# I dont know how to pass 'xx' to main.py,
# so for now I just have one test with no arguments
and running the test as:
pytest -vs test_sample.py
This fails with error messages. I tried to look at other answers for a solution but could not use them. For example, 42778124 suggests to create a separate file run.py which is not a desirable thing to do. And 48359957 and 40880259 seem to deal more with command line arguments for pytest, instead of passing command line arguments to the main code.
I dont need the pytest to take command line arguments, the arguments can be hardcoded inside a specific test. But these arguments need to be passed as arguments to the main code. Can you give me a test_sample.py, that calls main.main() with some arguments?
If you can't modify the signature of the main method, you can use the monkeypatching technique to temporarily replace the arguments with the test data. Example: imagine writing tests for the following program:
import argparse
def main():
parser = argparse.ArgumentParser(description='Greeter')
parser.add_argument('name')
args = parser.parse_args()
return f'hello {args.name}'
if __name__ == '__main__':
print(main())
When running it from the command line:
$ python greeter.py world
hello world
To test the main function with some custom data, monkeypatch sys.argv:
import sys
import greeter
def test_greeter(monkeypatch):
with monkeypatch.context() as m:
m.setattr(sys, 'argv', ['greeter', 'spam'])
assert greeter.main() == 'hello spam'
When combined with the parametrizing technique, this allows to easily test different arguments without modifying the test function:
import sys
import pytest
import greeter
#pytest.mark.parametrize('name', ['spam', 'eggs', 'bacon'])
def test_greeter(monkeypatch, name):
with monkeypatch.context() as m:
m.setattr(sys, 'argv', ['greeter', name])
assert greeter.main() == 'hello ' + name
Now you get three tests, one for each of the arguments:
$ pytest -v test_greeter.py
...
test_greeter.py::test_greeter[spam] PASSED
test_greeter.py::test_greeter[eggs] PASSED
test_greeter.py::test_greeter[bacon] PASSED
A good practice might to have this kind of code, instead of reading arguments from main method.
# main.py
def main(arg1):
return arg1
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='My awesome script')
parser.add_argument('word', help='a word')
args = parser.parse_args()
main(args.word)
This way, your main method can easily be tested in pytest
import main
def test_case01():
main.main(your_hardcoded_arg)
I am not sure you can call a python script to test except by using os module, which might be not a good practice
I have a program (temp.py) in python3 that reads the temperature at sensors in a solar panel. I also have a program (stepper.py) that I use to control a stepper motor. The program stepper.py uses sys.argv to evaluate arguments and determine how far and how quickly the motor should turn. I now want to call stepper.py from temp.py to combine the functionality of the two programs. However, when I call stepper.py from temp.py the arguments are not passed in a way that sys.argv can use them. I have written a short script to illustrate my problem:
import sys
y=5
z=2
def called(a,b):
print(str(sys.argv))
print(len(sys.argv[1:]))
global p,q
p=a*b
q=a+b
called(y,z)
print(p,q)
This script returns the following:
['/home/pi/calling.py']
0
10 0
In other words, although the calculation of the arguments has been completed, sys.argv is saying that no arguments have been passed and merely shows argv[0] which is the program's name. I understand why this occurs but is there any way that I can get sys.argv to "see" the arguments?
I'm not aware of a way to manually fill argv, but that doesn't really seem like the best way of achieving your goal here to me. You're trying to call a python function in one module from another, so you should do that using Python's normal importing and function call mechanisms, not by repurposing argv, which is designed to handle command line arguments. Usig python's if __name__ == "__main__" construction, you can still pass command line arguments through to the function.
If you haven't already, this will require wrapping up the code you have in your modules into functions with well-defined arguments. Here's an example of the approach I'm describing:
# stepper.py
import sys
def turn_motor(rotations, speed=1):
"""Turns the stepper motor"""
if __name__ == "__main__":
# Code in this block will run when stepper.py is invoked from the command line
turn_motor(rotations=sys.argv[1], speed=sys.argv[2])
and in your other file
# temp.py
import sys
from stepper import turn_motor
def detect_temp():
"""Returns the current temperature"""
def turn_when_hot(threshold):
"""Turns stepper motor when temperature above given threshold"""
temperature = detect_temp()
if temperature > threshold:
# Calls the turn_motor function directly with whatever arguments
# you like, without having to mess with sys.argv
turn_motor(revolutions=3, speed=1.2)
if __name__ == "__main__":
turn_when_hot(sys.argv[1])
So you could call stepper.py from the command line with arguments for turns and speed, or you could call temp.py with an argument for a temperature threshold, and it would call stepper.turn_motor directly.
Here is the equivalent fixed code with notes below:
def called(a, b):
return a*b, a+b
if __name__ == '__main__':
import sys
y=5
z=2
print(str(sys.argv))
print(len(sys.argv[1:]))
p, q = called(y, z)
print(p, q)
As TroyHurts wrote, separate the core functionality to the functions, and the body to the if __name__ == '__main__': block.
Then you will be able to import stepper in your temp.py. If you need sys.argv in temp.py, import the sys in the temp.py.
The import sys in stepper.py can even be moved to the if block if it is not used in the function definitions.
The functions (in stepper.py) should get all inputs as function arguments. They should return everything through the return statement.
Never use global.
The called is not a nice identifier. Choose better one. ;)