I am writing a script in python 3 which takes space separated arguments from command line. Depending on the first argument I call different methods.
When calling the script it looks like this:
python name.py [-k] [-p pr] number filename
The user will give or [-k] or [-p pr] (not both), number and filename.
for example:
1. python name.py -k 3 filename
if the user gives -k then I want to call a function which will have as arguments the number 3(n1) and the input_filename(filename).
def function1(n1, filename):
code
2. python name.py -p 2 3 filename
if the user gives -r then I want to call a function which will have as arguments the number 2(pr), the number 3(n1) and the input_filename(filename).
def function2(pr, n1, filename):
code
How can I separate the arguments from the command line and use them as arguments for the functions (using argparse)?
import argparse
def function1(n1, filename):
print('doing 1')
def function2(pr, n1, filename):
print('doing 2')
parser = argparse.ArgumentParser()
parser.add_argument('-k', action='store_true')
parser.add_argument('-p', '--pr')
parser.add_argument('number', type=int)
parser.add_argument('filename')
args = parser.parse_args()
print(args)
# could use mutually_exclusive_group, but with this simple alternative
# this is just as good. Tweak as needed.
if args.k:
function1(args.number, args.filename)
elif args.pr is not None:
function2(args.pr, args.number, args.filename)
add in your comment (corrected):
import sys
argv = sys.argv
print(argv)
if argv[1]=="-p":
function2(int(argv[2]), int(argv[3]), argv[4])
if argv[1]=="-k":
function1(int(argv[2]), argv[3])
test
1700:~/mypy$ python3 stack61218370.py -p 2 3 filename
Namespace(filename='filename', k=False, number=3, pr='2')
doing 2
['stack61218370.py', '-p', '2', '3', 'filename']
doing 2
1702:~/mypy$ python3 stack61218370.py -k 3 filename
Namespace(filename='filename', k=True, number=3, pr=None)
doing 1
['stack61218370.py', '-k', '3', 'filename']
doing 1
Related
This is my first time using Pytest, I have a program that is called with command line parameters, as in :
$ myprog -i value_a -o value_b
I am not sure how to use Pytest to test the output of this program. Given values of value_a and value_b, I expect a certain output that I want to test.
The Pytest examples that I see all refer to testing functions, for instance if there is a function such as:
import pytest
def add_nums(x,y):
return x + y
def test_add_nums():
ret = add_nums(2,2)
assert ret == 4
But I am not sure how to call my program using Pytest and not just test individual functions? Do I need to use os.system() and then call my program that way?
In my program I am using argparse module.
The solution is based on monkeypatch fixture. In below example myprog reads number from the file myprog_input.txt adds 2 to it and stores result in myprog_output.txt
Program under test
cat myprog.py
#!/usr/bin/python3.9
import argparse
import hashlib
def main():
parser = argparse.ArgumentParser(description='myprog')
parser.add_argument('-i')
parser.add_argument('-o')
args = parser.parse_args()
with open(args.i) as f:
input_data=int(f.read())
output_data=input_data+2
f.close()
with open(args.o,"w") as fo:
fo.write(str(output_data) + '\n')
fo.close()
with open(args.o) as fot:
bytes = fot.read().encode() # read entire file as bytes
fot.close()
readable_hash = hashlib.sha256(bytes).hexdigest();
return readable_hash
if __name__ == '__main__':
print(main())
Test
cat test_myprog.py
#!/usr/bin/python3.9
import sys
import myprog
def test_myprog(monkeypatch):
with monkeypatch.context() as m:
m.setattr(sys, 'argv', ['myprog', '-i', 'myprog_input.txt', '-o', 'myprog_output.txt'])
assert myprog.main() == 'f0b5c2c2211c8d67ed15e75e656c7862d086e9245420892a7de62cd9ec582a06'
Input file
cat myprog_input.txt
3
Running the program
myprog.py -i myprog_input.txt -o myprog_output.txt
f0b5c2c2211c8d67ed15e75e656c7862d086e9245420892a7de62cd9ec582a06
Testing the program
pytest test_myprog.py
============================================= test session starts =============================================
platform linux -- Python 3.9.5, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /home/<username>/py
plugins: hypothesis-6.23.1
collected 1 item
test_myprog.py . [100%]
============================================== 1 passed in 0.04s ==============================================
Using the following code:
import subprocess
while(1==1):
command_terminal= input('Enter the command you would like to pass: ')
command_terminal=command_terminal.split()
result=subprocess.run(command_terminal, stdout=subprocess.PIPE)
result=result.stdout.decode('utf-8')
print(result)
Basically I am trying to emulate terminal as a whole. But I am sort of failing.
Stuff like ps -A Doesn't give an output at all. Any help..?
Use subprocess.check_output. The encoding argument convert the output bytes to str.
import subprocess
while (1 == 1):
command_terminal = input('Enter the command you would like to pass: ')
command_terminal = command_terminal.split()
result = subprocess.check_output(command_terminal, encoding='utf-8')
print(result)
Note that CalledProcessError is raise if the exit code is not zero.
Alternatively, subprocess.run with capture_output=True may be used. The output is stored in the returned CompletedProcess.stdout and no exception whatever the exit code is.
import subprocess
while (1 == 1):
command_terminal = input('Enter the command you would like to pass: ')
command_terminal = command_terminal.split()
completed = subprocess.run(command_terminal, encoding='utf-8', capture_output=True)
result = completed.stdout
print(result)
I actually did what you did and hosted it over on github (https://github.com/Pythers/PyCMD/blob/master/main.py)
I believe your problem is that you need to set the stdout to the normal instead of subprocess.PIPE
import sys
import subprocess
while 1==1:
command_terminal= input('Enter the command you would like to pass: ')
command_terminal=command_terminal.split()
result=subprocess.run(command_terminal, stdout=sys.stdout)
Let's say I have a python script myscript.py, which takes multiple named arguments. The user calls the script as python myscript.py --arg1 value1 --arg2 value2. How to capture this whole command (i.e. python myscript.py --arg1 value1 --arg2 value2 ) and save it to a text file "command.selfie" using the same script that the user is calling?
sys.argv is a list with all the arguments.
This will help:
import sys
command = " ".join(sys.argv)
# do whatever you want to do with command: str
You can do something like this:
import argparse
import sys
def main(argv):
parser = argparse.ArgumentParser()
parser.add_argument("--arg1", default=False, help="Explain arg1")
parser.add_argument("--arg2", type=str, default="hello", help="Explain arg2")
# add all arguments you need
args = parser.parse_args(argv[1:])
params = {"arg1": args.arg1,
"arg2": args.arg2, }
print(params["arg1"])
print(params["arg2"])
if __name__ == "__main__":
sys.exit(main(sys.argv))
I try to use the concurrent.future multithreading in Python with subprocess.run to launch an external Python script. But I have some troubles with the shell=True part of the subprocess.run().
Here is an example of the external code, let's call it test.py:
#! /usr/bin/env python3
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('-x', '--x_nb', required=True, help='the x number')
parser.add_argument('-y', '--y_nb', required=True, help='the y number')
args = parser.parse_args()
print('result is {} when {} multiplied by {}'.format(int(args.x_nb) * int(args.y_nb),
args.x_nb,
args.y_nb))
In my main python script I have:
#! /usr/bin/env python3
import subprocess
import concurrent.futures
import threading
...
args_list = []
for i in range(10):
cmd = './test.py -x {} -y 2 '.format(i)
args_list.append(cmd)
# just as an example, this line works fine
subprocess.run(args_list[0], shell=True)
# this multithreading is not working
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
executor.map(subprocess.run, args_list)
The problem here is that I can't pass the shell=True option to the executor.map.
I have already tried without success:
args_list = []
for i in range(10):
cmd = './test.py -x {} -y 2 '.format(i)
args_list.append((cmd, eval('shell=True'))
or
args_list = []
for i in range(10):
cmd = './test.py -x {} -y 2 '.format(i)
args_list.append((cmd, 'shell=True'))
Anyone has an idea on how to solve this problem?
I don't think the map method can call a function with keyword args directly but there are 2 simple solutions to your issue.
Solution 1: Use a lambda to set the extra keyword argument you want
The lambda is basically a small function that calls your real function, passing the arguments through. This is a good solution if the keyword arguments are fixed.
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
executor.map(lambda args: subprocess.run(args, shell=True), args_list)
Solution 2: Use executor.submit to submit the functions to the executor
The submit method lets you specify args and keyword args to the target function.
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
for args in args_list:
executor.submit(subprocess.run, args, shell=True)
I do not find a way to create this kind of arguments structure with the argparse package:
$ python3 prog.py [A (a1 VAR | a2 | a3) | B | C c]
I would like A to be a root for a sub-commands family, in my case a blacklist (./prog blacklist [add PATH | rm | ...]), but I also need two other features : $ ./prog push FILE and finally $ ./prog --interactive.
I've already taken a serious look at the argparse documentation.
I've tried to use add_mutually_exclusive_group() to regroup a1, b1 and c1 but I'm stuck with this :
ValueError: mutually exclusive arguments must be optional
I did manage to have python3 prog.py (A a1 a2| B | C c1) using groups, but either I miss understood something, or argparse has nothing included for this case.
Any help would be appreciated. Thanks for your time !
EDIT : I've finally managed to do what I was looking for. However, I am not convinced this is very clean. It could be great to have some packages which does it better than I.
Thanks to this website, I've found a way to achieve my goal. I attach the code if ever it interests anyone.
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import argparse
from sys import argv
class CLI:
"""
Class which represent the CLI's behaviour.
Every function describes a subcommands tree and
how the CLI should respond.
"""
def __init__(self):
parser = argparse.ArgumentParser(
description='',
usage='''cli <command> [<args>]'
Commands list:
cli blacklist Interact with the blacklist.
cli interactive Run an interactive mode for queries.
cli read <path> Try to extract credentials.
'''
)
parser.add_argument('command', help='Subcommand to run')
# only parsing the subcommand
args = parser.parse_args(argv[1:2])
if not hasattr(self, args.command):
print('Unrecognized command')
parser.print_help()
exit(1)
# use dispatch pattern to invoke method with same name
getattr(self, args.command)()
def blacklist(self):
"""
Parser for blacklist methods.
"""
parser = argparse.ArgumentParser(
description='',
usage='''cli blacklist <subcommand> [<args>]
Subcommands list:
add <domain> Add <domain> from the blacklist.
export <path> Export blacklist as csv to <path>.
import <path> [-e] Import <path> as csv. Erase existing blacklist if -e.
remove <domain> Remove <domain> from the blacklist.
show Print the blacklist.
'''
)
parser.add_argument('subcommand')
args = parser.parse_args(argv[2:3])
if not hasattr(self, 'bl_%s' % (args.subcommand)):
print('Unrecognized command')
parser.print_usage()
exit(1)
getattr(self, 'bl_%s' % (args.subcommand))()
def bl_add(self):
print('add `%s` to the blacklist' % (argv[3:4][0]))
def bl_export(self):
print('export')
def bl_import(self):
print('import')
def bl_remove(self):
print('remove')
def bl_show(self):
print('show')
def read(self):
if len(argv) != 3:
print('Wrong number of arguments')
print('usage: cli read <path>')
exit(1)
print('process %s' % (argv[2:3][0]))
def interactive(self):
print('run interactive mode')
CLI()