I have the following code:
if __name__ == '__main__':
usage = 'mytool show/add/remove/discover cluter (-g $group -s $start -e $end)'
description = 'A tool to show/add/remove/ hosts, discover services and activate changes for a cluster'
parser = argparse.ArgumentParser(usage=usage, description=description,allow_abbrev=False)
parser.add_argument(dest='action', choices=('show', 'add', 'remove', 'discover'), type=str, help='Operations on a cluster')
parser.add_argument('-c', '--cluster', required=True, type=str, help='a cluster tag like "cluster:my-02"')
parser.add_argument('-g', '--group', type=str, help='Specifiy a group i.e. "platform/image"')
parser.add_argument('-s', '--start', type=int, help='Specifiy the first machine i.e. "-s 004"')
parser.add_argument('-e', '--end', required=('-s' or '--start') in sys.argv, type=int, help='Specifiy the last machine i.e. "-e 009"')
parser.add_argument('--simulate', action='store_true', help='Simulate the operations')
args = parser.parse_args()
action = args.action
cluster = args.cluster
group = args.group
start = args.start
end = args.end
Is there any way to to exclusively use one of the positional arguments without using the add_mutually_exclusive_group()?
The second thing is when I ever pass the -h option I always get the whole help message, is it possible to get the help message for each argument separately?
mytool add -h
mytool add cluster:my-02 -h
mytool add cluster:my-02 -g platform -h
etc...
Thanks
Related
I'm using Python 3.8 and pytest 6.0.1. How do I create a custom command line option for pytest? I thought it was as simple as adding this to conftest.py ...
def pytest_addoption(parser):
parser.addoption('--option1', action='store_const', const=True)
but when I pytest, I get an unrecognized option error
# pytest --option1=Y -c tests/my_test.py
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --option1=Y
What's the right way to add a custom option?
Edit: I tried the answer given. I included some other things in my tests/conftest.py file in case those are the reason the answer isn't working. File contains
def pytest_generate_tests(metafunc):
option1value = metafunc.config.getoption("--option1")
print(f'Option1 Value = {option1value}')
def pytest_configure(config):
use_docker = False
try:
use_docker = config.getoption("--docker-compose-remove-volumes")
except:
pass
plugin_name = 'wait_for_docker' if use_docker else 'wait_for_server'
if not config.pluginmanager.has_plugin(plugin_name):
config.pluginmanager.import_plugin("tests.plugins.{}".format(plugin_name))
But output when running is
$ pytest -s --option1 tests/shared/model/test_crud_functions.py
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --option1
inifile: /Users/davea/Documents/workspace/my_project/pytest.ini
rootdir: /Users/davea/Documents/workspace/my_project
As already mentioned in a comment action='store_const' makes your option a flag. The value you receive when you read option value provided it is specified on cli is the one specified by const i.e. True in your case.
Try this:
add below function to conftest.py
def pytest_generate_tests(metafunc):
option1value = metafunc.config.getoption("--option1")
print(f'Option1 Value = {option1value}')
pytest invoked with option pytest -s --option1
Output will have : Option1 Value = True
pytest invoked without option pytest -s
Output will have : Option1 Value = None
action=store might give you the desired behavior.
Solution:
# Change the action associated with your option to action='store'
def pytest_addoption(parser):
parser.addoption('--option1', action='store')
def pytest_configure(config):
x = config.getoption('option1')
print(x) # Any logic that uses option value
Output:
pytest -s --option1=Y -c=test.py
Y
============================================================================= test session starts ==============================================================================
platform darwin -- Python 3.8.5, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
You can find details on available action(s) and more here: https://docs.python.org/3/library/argparse.html#action
I think order matters in this case.
Try pytest -c tests/my_test.py --option1=Y
I am struggling with command-line parsing and argparse, how to handle global variables, subcommands and optional params to these subcommands
I'm writting a python3 wrapper around python-libvirt to manage my VMs. The wrapper will handle creation, removal, stop/start, snapshots, etc.
A partial list of the options follows, that shows the different ways to pass params to my script:
# Connection option for all commands:
# ---
# vmman.py [-c hypervisor] (defaults to qemu:///system)
# Generic VM commands:
# ---
# vmman.py show : list all vms, with their state
# vmman.py {up|down|reboot|rm} domain : boots, shuts down, reboots
or deletes the domain
# Snapshot management:
# ---
# vmman.py lssnap domain : list snapshots attached to domain
# vmman.py snaprev domain [snapsname] : reverts domain to latest
snapshot or to snapname
# Resource management:
# ---
# vmman.py domain resdel [disk name] [net iface]
And then some code used to test the first subcommand :
def setConnectionString(args):
print('Arg = %s' % args.cstring)
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers()
parserConnect = subparsers.add_parser('ConnectionURI')
parserConnect.set_defaults(func=setConnectionString)
parserConnect.add_argument('-c', '--connect', dest='host')
args = parser.parse_args()
args.func(args)
print("COMPLETED")
Now, the argparse() doc on docs.python.org is dense and a bit confusing of a python newbie as I am... I would have expected the output to be something like:
`Arg = oslo`
What I get is :
[10:21:40|jfgratton#bergen:kvmman.py]: ./argstest.py -c oslo
usage: argstest.py [-h] {ConnectionURI} ...
argstest.py: error: invalid choice: 'connectionURI' (choose from 'ConnectionURI')
I obviously miss something, and I'm only testing the one I thought would be the easiest of the lot (global param); haven't even figured yet on how to handle optional subparams and all.
Your error output lists 'connectionURI' with lowercase 'c' as invalid choice, while it also says "choose from 'ConnectionURI'" with capital letter 'C'.
Fix: Call your test with:
./argstest.py ConnectionURI oslo
Maybe you should start simple (without subparsers) and build from there:
import argparse
def setConnectionString(hostname):
print('Arg = {}'.format(hostname))
parser = argparse.ArgumentParser(description='python3 wrapper around python-libvirt to manage VMs')
parser.add_argument('hostname')
args = parser.parse_args()
setConnectionString(args.hostname)
print("COMPLETED")
When using azure-cli in python 3.5 and calling the commands from a script I have no control on the output in the console.
When a command is executed it prints the result to the console, but I'm struggling to just take the result and put it in a variable to analyze it.
from azure.cli.core import get_default_cli
class AzureCmd():
def __init__(self, username, password):
self.username = username
self.password = password
def login(self, tenant):
login_successfull = get_default_cli().invoke(['login',
'--tenant', tenant,
'--username', self.username,
'--password', self.password]) == 0
return login_successfull
def list_vm(self, tenant):
list_vm = get_default_cli().invoke(['vm', 'list', '--output', 'json'])
print(list_vm)
tenant = 'mytenant.onmicrosoft.com'
cmd = AzureCmd('login', 'mypassword')
cmd.login(tenant)
cmd.list_vm(tenant)
Here is my my script attempt.
What I want to achieve : not getting any output when cmd.login(tenant) is executed.
Instead of getting 0 (success) or 1 (failure) in my variables login_successfull and list_vm, I want to save the output of the get_default_cli().invoke() in it.
I ran into the same problem, and found a solution, I also found out many people offered the standard solution that normally works in most cases, but they didn't verify it works for this scenario, and it turns out az cli is an edge case.
I think the issue has something to do with az cli is based on python.
Win10CommandPrompt:\> where az
C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\az.cmd
If you look in that file you'll see something like this and discover that Azure CLI is just python:
python.exe -IBm azure.cli
So to do what you want to do, try this (it works for me):
import subprocess
out = subprocess.run(['python', '-IBm', 'azure.cli', '-h'], stdout=subprocess.PIPE).stdout.decode('utf-8')
print(out)
#this is equivalent to "az -h'
The above syntax won't work unless every single arg is a comma separated list of strings, I found a syntax I like alot more after reading how to do multiple args with python popen:
import subprocess
azcmd = "az ad sp create-for-rbac --name " + SPName + " --scopes /subscriptions/" + subscriptionid
out = subprocess.run(azcmd, shell=True, stdout=subprocess.PIPE).stdout.decode('utf-8')
print(out)
I faced the same problem while trying to save the log of Azure Container Instance. None of the above solutions worked exactly as they are. After debugging the azure cli python code
(File : \Python39\Lib\site-packages\azure\cli\command_modules\container\custom.py , function : container_logs() ), i see that the container logs are just printed to the console but not returned. If you want to save the logs to any variable, add the return line (Not exactly a great solution but works for now). Hoping MS Azure updates their azure cli in upcoming versions.
def container_logs(cmd, resource_group_name, name, container_name=None, follow=False):
"""Tail a container instance log. """
container_client = cf_container(cmd.cli_ctx)
container_group_client = cf_container_groups(cmd.cli_ctx)
container_group = container_group_client.get(resource_group_name, name)
# If container name is not present, use the first container.
if container_name is None:
container_name = container_group.containers[0].name
if not follow:
log = container_client.list_logs(resource_group_name, name, container_name)
print(log.content)
# Return the log
return(log.content)
else:
_start_streaming(
terminate_condition=_is_container_terminated,
terminate_condition_args=(container_group_client, resource_group_name, name, container_name),
shupdown_grace_period=5,
stream_target=_stream_logs,
stream_args=(container_client, resource_group_name, name, container_name, container_group.restart_policy))
With this modification and along with the above solutions given (Using the get_default_cli), we can store the log of the Azure container instance in a variable.
from azure.cli.core import get_default_cli
def az_cli(args_str):
args = args_str.split()
cli = get_default_cli()
res = cli.invoke(args)
if cli.result.result:
jsondata = cli.result.result
elif cli.result.error:
print(cli.result.error)
I think you can use the subprocess and call the az cli to get the output instead using get_default_cli.
import subprocess
import json
process = subprocess.Popen(['az','network', 'ddos-protection', 'list'], stdout=subprocess.PIPE)
out, err = process.communicate()
d = json.loads(out)
print(d)
Well, we can execute the Azure CLI commands in Python as shown below.
Here, the res variable usually stores a value of integer type and therefore we might not be able to access the json response. To store the response in a variable, we need to do cli.result.result.
from azure.cli.core import get_default_cli
def az_cli(args_str):
args = args_str.split()
cli = get_default_cli()
res = cli.invoke(args)```
if cli.result.result:
jsondata = cli.result.result
elif cli.result.error:
print(cli.result.error)
Firstly, I'd like to say I just begin to learn python, And I want to execute maven command inside my python script (see the partial code below)
os.system("mvn surefire:test")
But unfortunately, sometimes this command will time out, So I wanna to know how to set a timeout threshold to control this command.
That is to say, if the executing time is beyond X seconds, the program will skip the command.
What's more, can other useful solution deal with my problem? Thanks in advance!
use the subprocess module instead. By using a list and sticking with the default shell=False, we can just kill the process when the timeout hits.
p = subprocess.Popen(['mvn', 'surfire:test'])
try:
p.wait(my_timeout)
except subprocess.TimeoutExpired:
p.kill()
Also, you can use in terminal timeout:
Do like that:
import os
os.system('timeout 5s [Type Command Here]')
Also, you can use s, m, h, d for second, min, hours, day.
You can send different signal to command. If you want to learn more, see at:
https://linuxize.com/post/timeout-command-in-linux/
Simple answer
os.system not support timeout.
you can use Python 3's subprocess instead, which support timeout parameter
such as:
yourCommand = "mvn surefire:test"
timeoutSeconds = 5
subprocess.check_output(yourCommand, shell=True, timeout=timeoutSeconds)
Detailed Explanation
in further, I have encapsulate to a function getCommandOutput for you:
def getCommandOutput(consoleCommand, consoleOutputEncoding="utf-8", timeout=2):
"""get command output from terminal
Args:
consoleCommand (str): console/terminal command string
consoleOutputEncoding (str): console output encoding, default is utf-8
timeout (int): wait max timeout for run console command
Returns:
console output (str)
Raises:
"""
# print("getCommandOutput: consoleCommand=%s" % consoleCommand)
isRunCmdOk = False
consoleOutput = ""
try:
# consoleOutputByte = subprocess.check_output(consoleCommand)
consoleOutputByte = subprocess.check_output(consoleCommand, shell=True, timeout=timeout)
# commandPartList = consoleCommand.split(" ")
# print("commandPartList=%s" % commandPartList)
# consoleOutputByte = subprocess.check_output(commandPartList)
# print("type(consoleOutputByte)=%s" % type(consoleOutputByte)) # <class 'bytes'>
# print("consoleOutputByte=%s" % consoleOutputByte) # b'640x360\n'
consoleOutput = consoleOutputByte.decode(consoleOutputEncoding) # '640x360\n'
consoleOutput = consoleOutput.strip() # '640x360'
isRunCmdOk = True
except subprocess.CalledProcessError as callProcessErr:
cmdErrStr = str(callProcessErr)
print("Error %s for run command %s" % (cmdErrStr, consoleCommand))
# print("isRunCmdOk=%s, consoleOutput=%s" % (isRunCmdOk, consoleOutput))
return isRunCmdOk, consoleOutput
demo :
isRunOk, cmdOutputStr = getCommandOutput("mvn surefire:test", timeout=5)
I'm trying to use the groovy CliBuilder to parse command line options. I'm trying to use multiple long options without a short option.
I have the following processor:
def cli = new CliBuilder(usage: 'Generate.groovy [options]')
cli.with {
h longOpt: "help", "Usage information"
r longOpt: "root", args: 1, type: GString, "Root directory for code generation"
x args: 1, type: GString, "Type of processor (all, schema, beans, docs)"
_ longOpt: "dir-beans", args: 1, argName: "directory", type: GString, "Custom location for grails bean classes"
_ longOpt: "dir-orm", args: 1, argName: "directory", type: GString, "Custom location for grails domain classes"
}
options = cli.parse(args)
println "BEANS=${options.'dir-beans'}"
println "ORM=${options.'dir-orm'}"
if (options.h || options == null) {
cli.usage()
System.exit(0)
}
According to the groovy documentation I should be able to use multiple "_" values for an option when I want it to ignore the short option name and use a long option name only. According to the groovy documentation:
Another example showing long options (partial emulation of arg
processing for 'curl' command line):
def cli = new CliBuilder(usage:'curl [options] <url>')
cli._(longOpt:'basic', 'Use HTTP Basic Authentication')
cli.d(longOpt:'data', args:1, argName:'data', 'HTTP POST data')
cli.G(longOpt:'get', 'Send the -d data with a HTTP GET')
cli.q('If used as the first parameter disables .curlrc')
cli._(longOpt:'url', args:1, argName:'URL', 'Set URL to work with')
Which has the following usage message:
usage: curl [options] <url>
--basic Use HTTP Basic Authentication
-d,--data <data> HTTP POST data
-G,--get Send the -d data with a HTTP GET
-q If used as the first parameter disables .curlrc
--url <URL> Set URL to work with
This example shows a common convention. When mixing short and long
names, the short names are often one
character in size. One character
options with arguments don't require a
space between the option and the
argument, e.g. -Ddebug=true. The
example also shows the use of '_' when
no short option is applicable.
Also note that '_' was used multiple times. This is supported but
if any other shortOpt or any longOpt is repeated, then the behavior is undefined.
http://groovy.codehaus.org/gapi/groovy/util/CliBuilder.html
When I use the "_" it only accepts the last one in the list (last one encountered). Am I doing something wrong or is there a way around this issue?
Thanks.
not sure what you mean it only accepts the last one. but this should work...
def cli = new CliBuilder().with {
x 'something', args:1
_ 'something', args:1, longOpt:'dir-beans'
_ 'something', args:1, longOpt:'dir-orm'
parse "-x param --dir-beans beans --dir-orm orm".split(' ')
}
assert cli.x == 'param'
assert cli.'dir-beans' == 'beans'
assert cli.'dir-orm' == 'orm'
I learned that my original code works correctly. What is not working is the function that takes all of the options built in the with enclosure and prints a detailed usage. The function call built into CliBuilder that prints the usage is:
cli.usage()
The original code above prints the following usage line:
usage: Generate.groovy [options]
--dir-orm <directory> Custom location for grails domain classes
-h,--help Usage information
-r,--root Root directory for code generation
-x Type of processor (all, schema, beans, docs)
This usage line makes it look like I'm missing options. I made the mistake of not printing each individual item separate from this usage function call. That's what made this look like it only cared about the last _ item in the with enclosure. I added this code to prove that it was passing values:
println "BEANS=${options.'dir-beans'}"
println "ORM=${options.'dir-orm'}"
I also discovered that you must use = between a long option and it's value or it will not parse the command line options correctly (--long-option=some_value)