Argparse Unit Test with IP Address Module - python-3.x

I'd like to write unit tests for the following function:
#!/usr/bin/env python3
"""IPv4 validation using `ipaddress module` and argparse."""
import argparse
from ipaddress import ip_address
def parse_cli_args():
"""
Command line parser for subnet of interest.
Args:
--ip 0.0.0.0
Returns:
String, e.g. 0.0.0.0
"""
parser = argparse.ArgumentParser(description="IPv4 address of interest.")
parser.add_argument("--ip", action="store", type=ip_address,\
required=True,\
help="IP address of interest, e.g. 0.0.0.0")
args = parser.parse_args()
return args
if __name__ == '__main__':
args = parse_cli_args()
print(args.ip)
which works as expected, e.g.:
python3 test.py --ip 192.168.1.1
192.168.1.1
python3 test.py --ip derp
usage: test.py [-h] --ip IP
test.py: error: argument --ip: invalid ip_address value: 'derp'
python3 test.py --ip
usage: test.py [-h] --ip IP
test.py: error: argument --ip: expected one argument
How can I mock these three conditions in unit tests?
I tried a few variations of this:
import unittest
from unittest.mock import patch
class ParseCLIArgs(unittest.TestCase):
"""Unit tests."""
#patch('builtins.input', return_value='192.168.1.1')
def test_parse_cli_args_01(self, input):
"""Valid return value."""
self.assertIsInstance(parse_cli_args(), ipaddress.IPv4Address)
if __name__ == '__main__':
unittest.main()
without success. What am I doing wrong, and how can I fix that?
EDIT I got a bit further with this:
class ParseCLIArgs(unittest.TestCase):
def setUp(self):
self.parser = parse_cli_args()
def test_parser_cli_args(self):
parsed = self.parser.parse_args(['--ip', '192.168.1.1'])
self.assertIs(parsed.ip, '192.168.1.1')
if __name__ == '__main__':
unittest.main()
Which fails with: TypeError: isinstance() arg 2 must be a type or tuple of types. I believe this is because the function actually transforms user input.

To test a parser you need to either modify sys.argv or provide your own substitute.
When called with
args = parser.parse_args(argv)
if argv is None (or not provided) it parses sys.argv[1:]. This is the list that the shell/interpreter gives it. Otherwise it will parse the equivalent list that you provide.
test_argparse.py uses both ways to test a parser - constructing a custom sys.argv and call parse_args with a custom argv.
Another thing to watch out for is error trapping. Most parsing errors display usage, error message and then exits. Capturing that exit and the stderr message takes some work. test_argparse makes a subclass of ArgumentParser with a custom error method.
In sum, doing unittest on code that depends on sys.argv and does a system exit, may require more work than it's worth. That said, I'm not an expert on unit testing and mocking tools; I've just studied the test_argparse file.

Hopefully this can serve as good sample for someone with a similar question. This test does most (but not all) of what I want it to do. I'm still working out a unit test to check for Type.
#!/usr/bin/env python3
"""IPv4 validation using `ipaddress module` and argparse."""
import argparse
from ipaddress import ip_address
import unittest
def parse_cli_args():
"""
Command line parser for subnet of interest.
Args:
--ip 0.0.0.0
Returns:
String, e.g. 0.0.0.0
"""
parser = argparse.ArgumentParser(description="IPv4 address of interest.")
parser.add_argument("--ip", action="store",\
required=True,\
help="IP address of interest, e.g. 0.0.0.0")
return parser
class ParseCLIArgs(unittest.TestCase):
def setUp(self):
self.parser = parse_cli_args()
def test_parser_cli_args(self):
parsed = self.parser.parse_args(['--ip', '192.168.1.1'])
self.assertEqual(parsed.ip, '192.168.1.1')
if __name__ == '__main__':
unittest.main()

Related

Is there a way to pass options values during mod_wsgi server start up in Django app

I am using mod_wsgi to run my Django app. As there are a plethora of options to define when the server run command is fired, I was trying to create some kind of python script to pass the options and their pre-set values.
For example:
Instead of using:
$python3 manage.py runmodwsgi --processes 3 --threads 1
I am trying to use a python script and use it as:
$python3 runme.py
where runme.py is the script file I am trying to use to start the server.
What I have done so far?
Created the script file:
import os
from django.core.management import execute_from_command_line
os.environ.setdefault('DJANGO_SETTINGS_MODULE', '<proj>.settings')
myargs = 'runmodwsgi'
list_of_args = ['', myargs]
execute_from_command_line(list_of_args)
As expected the server started with preset options, for example:
Now what I am trying to achieve is the pass values of certain options like:
--processes 3 --threads 1
and so on.
Is there a way I may pass the preset values (as I may be able to define in my script file runme.py), say something like adding to the list of arguments:
list_of_args = ['', myargs, addl_args]
I have been checking SO queries posted in addition to help available on python site, but could not get my head around to the problem.
I tried the following which is not very helpful though:
import os
import argparse # New import
from django.core.management import execute_from_command_line
# Addition of new code lines
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument("-p", "--processes", action="store_true")
group.add_argument("-t", "--threads", action="store_true")
parser.add_argument("pint", type=int, help="no. of processes")
parser.add_argument("tint", type=int, help="no. of threads")
args = parser.parse_args()
# End of new code
os.environ.setdefault('DJANGO_SETTINGS_MODULE', '<proj>.settings')
myargs = 'runmodwsgi'
# At this line if I add ", args" (like shown below:])
list_of_args = ['', myargs, args]
execute_from_command_line(list_of_args)
and run the command, it comes up with error:
TypeError: 'Namespace' object does not support indexing
If I simply run: python3 runme.py, I get the following error:
usage: runme.py [-h] [-p | -t] pint tint
runme.py: error: the following arguments are required: pint, tint
Whereas, using
python3 runme.py 3 1
starts the server but the options integers "3" and "1" does not have any effect (as intended for no. of processes and threads).
If I use:
python3 runme.py --processes 3 --threads 1
I get the following error:
usage: runme.py [-h] [-p | -t] pint tint
runme.py: error: argument -t/--threads: not allowed with argument -p/--processes
Tried with a single arg like:
python3 runme.py --processes 3 1
The server starts at this time but the options value/s are not affected.
How do I define the option values and pass these preset values of options to the run command?
It looks like you're just misunderstanding/misusing argparse.
import os
import sys
import argparse
os.environ.setdefault('DJANGO_SETTINGS_MODULE', '<proj>.settings')
parser = argparse.ArgumentParser()
parser.add_argument("-p", "--processes", type=int, default=1)
parser.add_argument("-t", "--threads", type=int, default=1)
args = parser.parse_args()
from django.core.management import execute_from_command_line
command = [sys.argv[0], 'runmodwsgi', '--processes', str(args.processes), '--threads', str(args.threads)]
execute_from_command_line(command)
might be closer to what you want; it will default processes and threads both to 1.

argparse with multiple flags and arguments

I want my code to be able to call different functions according to the flag and then use the argument passed after the flag as the input to the function.
Example of the expected output:
$ python3 test.py -a 2
4
$ python3 test.py -a human123
<Error message>
$ python3 test.py -h human123
Hello, human123
Here's my sample code:
class Test:
def __init__(self):
pass
def add(self, a):
return a+a
def hello(self, name):
return f"Hello, {name}"
parser = argparse.ArgumentParser()
parser.add_argument('-a', '--add', dest='command', action='store_consts', const='add', nargs=1, help='add a to itself')
parser.add_argument('-h', '--hello', dest='command', action='store_consts', const='hello', nargs=1, help='hello!')
args = parser.parse_args()
t = Test()
if args.command=='add':
print(t.add(args.add))
elif args.command=='sub':
print(t.hello(args.hello))
This sample code currently is doing what I want to achieve. I tried many things to fix the issue from removing the 'consts', changing the action to 'store', changing the value of nargs to '?', etc., however, it keeps giving me different kinds of errors like TypeError, etc.
Simplifying your arguments:
import argparse
class Test:
def __init__(self):
pass
def add(self, a):
return a+a
def hello(self, name):
return f"Hello, {name}"
parser = argparse.ArgumentParser()
parser.add_argument('-a', '--add', help='add a to itself')
parser.add_argument('-b', '--hello', help='hello!') # -h is already taken
args = parser.parse_args()
print(args)
t = Test()
if args.add: # or 'args.add is not None:'
print(t.add(args.add))
elif args.hello:
print(t.hello(args.hello))
test runs:
1936:~/mypy$ python3 stack62702524.py -a testing
Namespace(add='testing', hello=None)
testingtesting
1937:~/mypy$ python3 stack62702524.py -b other
Namespace(add=None, hello='other')
Hello, other
1937:~/mypy$ python3 stack62702524.py
Namespace(add=None, hello=None)
===
Your errors, which you did not show :{ When you get errors, don't just throw up your hands and randomly try alternatives. Read the docs and try to understand the error.
parser.add_argument('-c', action='store_consts', const='hello')
ValueError: unknown action "store_consts"
parser.add_argument('-c', action='store_const', const='hello', nargs=1)
TypeError: __init__() got an unexpected keyword argument 'nargs'
'store_consts' is an name error; with 'store_const' nargs is fixed at 0; you can't change that.
If I add 3 arguments - two store_const and one positional:
parser.add_argument('-c', dest='command', action='store_const', const='add')
parser.add_argument('-d', dest='command', action='store_const', const='hello')
parser.add_argument('foo')
Note the two new command and foo attributes, which you could use in your function call:
1945:~/mypy$ python3 stack62702524.py -c bar
Namespace(add=None, command='add', foo='bar', hello=None)
1945:~/mypy$ python3 stack62702524.py -d bar
Namespace(add=None, command='hello', foo='bar', hello=None)
Typically I use the dest parameter of add_argument to specify the variable name.
For example:
parser.add_argument("-a", "--add", dest="add", required=False, type=str help="some help")
Could be accessed by:
args = parser.parse_args()
print(args.add == "something")
I believe that you need one unique dest per argument.
Also, -h is reserved for help. You may wish to change this to -e or something.
parser.add_argument('-h', '--hello', ...)

Calling functions from a script using argparse without using subprocess

I have been given an existing script (let's call it existing.py) that in its MVCE form has the following structure.
import argparse
FLAGS = None
def func():
print(FLAGS.abc)
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument(
'--abc',
type=str,
default='',
help='abc.'
)
FLAGS, unparsed = parser.parse_known_args()
func()
As this is part of tool that gets constantly updated, I cannot change existing.py. Normally, existing.py is invoked with commandline arguments.
python -m existing.py --abc "Ok"
which prints the output 'Ok'.
I wish to call the functions (not the whole script) in existing.py using another script. How can I feed in the FLAGS object that is used in the functions of the script? I do not wish to use subprocess will just run the script in its entirety.
I know that argparse creates the FLAGS as a Namespace dictionary and I can construct it in calling.py (see code below) but I cannot then push it back into the function that is imported from existing.py into calling.py. The following is the calling.py that I've tried.
from existing import func
import argparse
args = argparse.Namespace()
args.abc = 'Ok'
FLAGS = args
func()
which throws an error
AttributeError: 'NoneType' object has no attribute 'abc'
This is different from other StackOverflow questions as this question explicitly forbids subprocess and the existing script cannot be changed.
Import existing and use
existing.FLAGS = args
Now functions defined in the existing namespace should see the desired FLAGS object.

how to pass command line argument from pytest to code

I am trying to pass arguments from a pytest testcase to a module being tested. For example, using the main.py from Python boilerplate, I can run it from the command line as:
$ python3 main.py
usage: main.py [-h] [-f] [-n NAME] [-v] [--version] arg
main.py: error: the following arguments are required: arg
$ python3 main.py xx
hello world
Namespace(arg='xx', flag=False, name=None, verbose=0)
Now I am trying to do the same with pytest, with the following test_sample.py
(NOTE: the main.py requires command line arguments. But these arguments need to be hardcoded in a specific test, they should not be command line arguments to pytest. The pytest testcase only needs to send these values as command line arguments to main.main().)
import main
def test_case01():
main.main()
# I dont know how to pass 'xx' to main.py,
# so for now I just have one test with no arguments
and running the test as:
pytest -vs test_sample.py
This fails with error messages. I tried to look at other answers for a solution but could not use them. For example, 42778124 suggests to create a separate file run.py which is not a desirable thing to do. And 48359957 and 40880259 seem to deal more with command line arguments for pytest, instead of passing command line arguments to the main code.
I dont need the pytest to take command line arguments, the arguments can be hardcoded inside a specific test. But these arguments need to be passed as arguments to the main code. Can you give me a test_sample.py, that calls main.main() with some arguments?
If you can't modify the signature of the main method, you can use the monkeypatching technique to temporarily replace the arguments with the test data. Example: imagine writing tests for the following program:
import argparse
def main():
parser = argparse.ArgumentParser(description='Greeter')
parser.add_argument('name')
args = parser.parse_args()
return f'hello {args.name}'
if __name__ == '__main__':
print(main())
When running it from the command line:
$ python greeter.py world
hello world
To test the main function with some custom data, monkeypatch sys.argv:
import sys
import greeter
def test_greeter(monkeypatch):
with monkeypatch.context() as m:
m.setattr(sys, 'argv', ['greeter', 'spam'])
assert greeter.main() == 'hello spam'
When combined with the parametrizing technique, this allows to easily test different arguments without modifying the test function:
import sys
import pytest
import greeter
#pytest.mark.parametrize('name', ['spam', 'eggs', 'bacon'])
def test_greeter(monkeypatch, name):
with monkeypatch.context() as m:
m.setattr(sys, 'argv', ['greeter', name])
assert greeter.main() == 'hello ' + name
Now you get three tests, one for each of the arguments:
$ pytest -v test_greeter.py
...
test_greeter.py::test_greeter[spam] PASSED
test_greeter.py::test_greeter[eggs] PASSED
test_greeter.py::test_greeter[bacon] PASSED
A good practice might to have this kind of code, instead of reading arguments from main method.
# main.py
def main(arg1):
return arg1
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='My awesome script')
parser.add_argument('word', help='a word')
args = parser.parse_args()
main(args.word)
This way, your main method can easily be tested in pytest
import main
def test_case01():
main.main(your_hardcoded_arg)
I am not sure you can call a python script to test except by using os module, which might be not a good practice

How to use argparse in the actual script

How does argparse work? I was told to 'hide' the password from the psycopg2 connection I am building so to be able to run the script automatically every week and being able to share it between departments. This is the beginning of the psycopg2 script where the password is asked:
#Connect to database
conn_string = "host='my_aws_postgresql_database.rds.amazonaws.com' dbname='my_database_name' user='my_username' password='my_password'"
# print the connection string we will use to connect
print ("\"Connecting to database\n ->%s\"" % (conn_string))
Now, how would I use argparse (and getpass) to hide my password? I found this script about this subject a couple of times (I would delete the print statement after getting it to work):
import argparse
import getpass
class Password(argparse.Action):
def __call__(self, parser, namespace, values, option_string):
if values is None:
values = getpass.getpass()
setattr(namespace, self.dest, values)
parser = argparse.ArgumentParser('Test password parser')
parser.add_argument('-p', action=Password, nargs='?', dest='password',
help='Enter your password')
args = parser.parse_args()
print (args.password)
I tried to add the argparse snippet above the #Connect to database code. And replaced the password part on line 2 with
conn_string =
"host='my_aws_postgresql_database.rds.amazonaws.com'
dbname='my_database_name'
user='my_username'
password='" + args + "'"
Then I tried to run the whole script with the command python3 my_script_file.py my_password -p I get asked for the password which I entered, but this rendered the following error
usage: Test password parser [-h] [-p [PASSWORD]]
Test password parser: error: unrecognized arguments: my_password
If I use python3 my_script_file.py my_password I get the same error, but I did not have to enter the password (again).
Am I close to the solution? Is this the standard way of doing this?
The problem was that I used python3 my_script_file.py my_password -p instead of the correct order python3 my_script_file.py -p my_password, see accepted answer below by #hpaulj and the comments to that answer.
This parser is gives the user 2 ways of entering the password, on the commandline, or with a separate getpass prompt:
import argparse
import getpass
class Password(argparse.Action):
def __call__(self, parser, namespace, values, option_string):
if values is None:
values = getpass.getpass()
setattr(namespace, self.dest, values)
parser = argparse.ArgumentParser('Test password parser')
parser.add_argument('-p', action=Password, nargs='?', dest='password',
help='Enter your password')
args = parser.parse_args()
print (args)
Sample runs:
0911:~/mypy$ python3 stack44571340.py
Namespace(password=None)
0912:~/mypy$ python3 stack44571340.py -p test
Namespace(password='test')
0912:~/mypy$ python3 stack44571340.py -p
Password:
Namespace(password='testing')
0912:~/mypy$ python3 stack44571340.py test
usage: Test password parser [-h] [-p [PASSWORD]]
Test password parser: error: unrecognized arguments: test
I tested without any arguments (got the default None)`; with '-p test' which uses the 'test' string; with just '-p', which asks; and without '-p', which produces the error.
I don't know why python3 my_script_file.py -p my_password produced an error; my best guess there's a typo in your parser definition (something wrong with nargs?).
It's not entirely clear how you merged this parser code into the larger script. Done right it shouldn't have changed the behavior of the parser.
The password argument would be used as:
password='" + args.password + "'"
The echo argument, is a positional one, which requires a string. In contrast the -p with nargs='?', is an optional flagged argument, which allows for the three way input I illustrated.
parser.add_argument("echo")
Thank you #CarlShiles, your answer didn't work with that long argparse/getpass snippet from above, but it made me realise that I could just echo the password in there. So I did a simple
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("echo")
args = parser.parse_args()
and then used your suggestion, password='" + args.echo + "'". And then ran the following command python3 my_script_file.py my_password. This worked just fine.

Resources