I would like to use argparse to pass some values throughout my main function. When calling the python file, I would always like to include the flag for the argument, while either including or excluding its string argument. This is because some external code, where the python file is being called, becomes a lot simpler if this would be possible.
When adding the arguments by calling parser.add_argument, I've tried setting the default value to default=None and also setting it to default=''. I can't make this work on my own it seems..
def main():
parser = argparse.ArgumentParser()
parser.add_argument('-p', '--projects_to_build', default='')
args = parser.parse_args()
This call works fine:
py .\python_file.py -p proj_1,proj_2,proj_3
This call does not:
py .\python_file.py -p
python_file.py: error: argument -p/--projects_to_build: expected one argument
You need to pass a nargs value of '?' with const=''
parser.add_argument('-p', '--projects_to_build', nargs='?', const='')
You should also consider adding required=True so you don't have to pass default='' as well.
Related
I have been given an existing script (let's call it existing.py) that in its MVCE form has the following structure.
import argparse
FLAGS = None
def func():
print(FLAGS.abc)
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument(
'--abc',
type=str,
default='',
help='abc.'
)
FLAGS, unparsed = parser.parse_known_args()
func()
As this is part of tool that gets constantly updated, I cannot change existing.py. Normally, existing.py is invoked with commandline arguments.
python -m existing.py --abc "Ok"
which prints the output 'Ok'.
I wish to call the functions (not the whole script) in existing.py using another script. How can I feed in the FLAGS object that is used in the functions of the script? I do not wish to use subprocess will just run the script in its entirety.
I know that argparse creates the FLAGS as a Namespace dictionary and I can construct it in calling.py (see code below) but I cannot then push it back into the function that is imported from existing.py into calling.py. The following is the calling.py that I've tried.
from existing import func
import argparse
args = argparse.Namespace()
args.abc = 'Ok'
FLAGS = args
func()
which throws an error
AttributeError: 'NoneType' object has no attribute 'abc'
This is different from other StackOverflow questions as this question explicitly forbids subprocess and the existing script cannot be changed.
Import existing and use
existing.FLAGS = args
Now functions defined in the existing namespace should see the desired FLAGS object.
That question will be strange in some point...
I'm trying to add the option strings in the parser after adding one argument.
For example
import argparse
p = argparse.ArgumentParser()
p.add_argument(dest = 'myvar')
p._actions[1].option_strings = ['-foo']
p.parse_args('-foo 1')
This example doesn't work, it says:
: error: the following arguments are required: -foo
Even though I'm supplying the argument...
Anyone know why this is occurring?
Is there some way to add the option strings after the add_argument method?
After looking at the source code of argparse, I found you also have to register the action by the option string it has:
import argparse
p = argparse.ArgumentParser()
p.add_argument(dest = 'myvar')
p._actions[1].option_strings = ['-foo']
# You also need to do this
p._option_string_actions['-foo'] = p._actions[1]
args = p.parse_args(['-foo', '100'])
print('myvar is', args.myvar) # myvar is 1
Note the change when calling parse_args.
I hope this help!!
Problem statement
I want the options supported in a python module to be overridable with an .yaml file, because in some cases there are too many options to be specified with non-default values.
I implemented the logic as follows.
parser = argparse.ArgumentParser()
# some parser.add statements that comes with default values
parser.add_argument("--config_path", default=None, type=str,
help="A yaml file for overriding parameters specification in this module.")
args = parser.parse_args()
# Override parameters
if args.config_path is not None:
with open(args.config_path, "r") as f:
yml_config = yaml.safe_load(f)
for k, v in yml_config.items():
if k in args.__dict__:
args.__dict__[k] = v
else:
sys.stderr.write("Ignored unknown parameter {} in yaml.\n".format(k))
The problem is, for some options I have specific functions/lambda expressions to convert the input strings, such as:
parser.add_argument("--tokens", type=lambda x: x.split(","))
In order to apply corresponding functions when parsing option specifications in YAML, adding so many if statements does not seem a good solution. Maintaining a dictionary that changes accordingly when new options are introduced in parser object seems redundant. Is there any solution to get the type for each argument in parser object?
If the elements that you add to the parser with add_argument start with -- then they
are actually optional and usually called options. You can find these walking over
the result of the _get_optonal_actions() method of the parser instance.
If you config.yaml looks like:
tokens: a,b,c
, then you can do:
import sys
import argparse
import ruamel.yaml
sys.argv[1:] = ['--config-path', 'config.yaml'] # simulate commandline
yaml = ruamel.yaml.YAML(typ='safe')
parser = argparse.ArgumentParser()
parser.add_argument("--config-path", default=None, type=str,
help="A yaml file for overriding parameters specification in this module.")
parser.add_argument("--tokens", type=lambda x: x.split(","))
args = parser.parse_args()
def find_option_type(key, parser):
for opt in parser._get_optional_actions():
if ('--' + key) in opt.option_strings:
return opt.type
raise ValueError
if args.config_path is not None:
with open(args.config_path, "r") as f:
yml_config = yaml.load(f)
for k, v in yml_config.items():
if k in args.__dict__:
typ = find_option_type(k, parser)
args.__dict__[k] = typ(v)
else:
sys.stderr.write("Ignored unknown parameter {} in yaml.\n".format(k))
print(args)
which gives:
Namespace(config_path='config.yaml', tokens=['a', 'b', 'c'])
Please note:
I am using the new API of ruamel.yaml. Loading this way is actually faster
than using from ruamel import yaml; yaml.safe_load() although your config files
are probably not big enough to notice.
I am using the file extension .yaml, as recommended in the official FAQ. You should do so as well, unless you cannot (e.g. if your
filesystem doesn't allow that).
I use a dash in the option string --config-path instead of an underscore, this is
somewhat more natural to type and automatically converted to an underscore to get valid
identifier name
You might want to consider a different approach where you parse sys.argv by hand for
--config-path, then set the defaults for all the options from the YAML config file and
then call .parse_args(). Doing things in that order allow you to override, on
the commandline, what has a value in the config file. If you do things your way, you always
have to edit a config file if it has all correct values except for one.
I have a tkinter GUI with an entry box I want to allow only numbers. Can someone explain to me what each command / line of code in validation does. I don't understand the vcmd variable and all the '%i' '%s' stuff. Thanks :)
UPDATE:
I have a different application to use this vcmd command with and dont understand it entirely. Here is my validation code:
def validate(self, action, index, value_if_allowed, prior_value, text, validation_type, trigger_type, widget_name):
if not int(action):
return True
elif text in '0123456789.-+':
try:
float(value_if_allowed)
return True
except ValueError:
return False
else:
return False
I dont get why in this code i need all of these:
action, index, value_if_allowed, prior_value, text, validation_type, trigger_type, widget_name
Why do i need all of these specific to my validation code for it to function correctly and what use are they?
The documentation you provided made sense but some of those '%s', '%i' stuff seemed unnecessary for my specific code yet it only works with all of them included like so:
vcmd = (self.master.register(self.validate), '%d', '%i', '%P', '%s', '%S', '%v', '%V', '%W')
I also want to know what self.master.register does please, i still cant figure that one out.
If you do not need any of the special arguments, you don't need to call register. For example, the following code will correctly call do_validation without any arguments:
import tkinter as tk
def do_validation():
value = entry.get()
if value == "" or value.isnumeric():
return True
return False
root = tk.Tk()
entry = tk.Entry(root, validate='key', validatecommand=do_validation)
entry.pack(fill="x", padx=20, pady=20)
root.mainloop()
However, in the above case the validation will always be one character behind. This is because validation happens before the character is inserted into the entry widget (ie: the first time it is called, entry.get() will return an empty string). The whole point of validation is to prevent illegal characters, so it makes sense to call the command before the character is inserted rather than after.
This is why the special arguments are useful -- they provide sufficient context in order to decide whether the character or characters are legal before they are inserted into the widget.
How register is useful
Tkinter is a wrapper around a Tcl interpreter, because tk (the core technology behind tkinter) is implemented in Tcl. Tcl is a programming language just like python.
In a Tcl program, you would use the validation features like this:
proc do_validation(new_value) {
return $new_value == "" || [string is integer $new_value]
}
entry .entry -validate key --validatecommand [list do_validation %P]
When Tcl detects a change, it performs substitution on the arguments, and then calls the defined procedure with the substituted arguments. For example, if you type "A", %P is converted to "A" and the function is called with "A" as the only argument.
In the case of Tkinter, there is no direct corollary for defining the function with arguments to be substituted at runtime.
Unfortunately, the validation feature wasn't implemented very well in the tkinter wrapper, so to properly use the validation we have to use a small workaround in order to get these special arguments passed to our function.
What register does is to create a new Tcl command that calls your python command. It also creates a new python command that is a reference to this new Tcl command. You can use this reference exactly like any other python command.
A simple example
In the simplest case, all you need is what the string would look like if the edit was to be allowed. You can then decide whether the edit is something that will result in valid input, or if it will result in invalid input. If the edit won't result in valid input, you can reject the edit before it actually happens.
The special argument that represents the value if the edit is allowed is %P 1. We can modify the function we are registering to accept this argument, and we can add this argument when we do the registering:
def do_validation(new_value):
return new_value == "" or new_value.isnumeric()
...
vcmd = (root.register(do_validation), '%P')
entry = tk.Entry(..., validatecommand=vcmd)
With that, when the underlying Tcl code detects a change it will call the new Tcl command which was created, passing in one argument corresponding to the special %P substitution.
1All of the mechanics of the validation are described thoroughly in the tcl documentation here: http://tcl.tk/man/tcl8.5/TkCmd/entry.htm#M7
I'm currently testing an argparse usage, but it's not working as expected. I have a couple of subparsers and optional arguments, being called the following way:
python3 myprogram.py positional argument --optional something
# Outcome
Namespace(optional='something')
The program works as expected if the optional is the last, but if it is in any other order, it is discarded.
python3 myprogram.py positional --optional argument
python3 myprogram.py --optional positional argument
# Outcome
Namespace(optional=None)
By looking at the argparse documentation I wasn't able to find a way to make the optional argument global.
I'm creating the the positional arguments for each positional in a for loop, which doesn't seem to be the best way. Because otherwise, it would add the optional arguments only to the last subparser.
import argparse
class Parsing(object):
def __init__(self):
parser = argparse.ArgumentParser(prog='python3 myprogram.py',
formatter_class=argparse.RawDescriptionHelpFormatter,
description='some description')
self.subparser = parser.add_subparsers(title='Positional', help='help description')
for sub in self.Generate(): # Method with a bunch of subparsers
self.Subparser(sub)
def Subparser(self, parsers):
for each in sorted(parsers):
positional = subparser.add_parser(each)
self.Optional(positional) # Method with some optional arguments for each of the second subparsers
self.Optional(parser) # Adding the optional arguments to the first subparser
def Optional(self, parser):
# ... Optional arguments
def Generate(self):
# ... Subparsers
I might be missing some code in the example above, tried to simplify the best I could and hope it to be perceptible.
Question: Is there a way to make the optional arguments across all subparsers?
Your description and code is hard to follow, but I've concluded that your problem lies with how defaults are handled when the main and subparsers share an argument dest.
I condensed your code a bit so I could make a test run:
import argparse
class Parsing(object):
def __init__(self):
self.parser = argparse.ArgumentParser(prog='prog',
description='some description')
self.subparser = self.parser.add_subparsers(dest='cmd', title='Cmds', help='help description')
self.make_subparsers(['cmd1','cmd2'])
def make_subparsers(self, parsers):
for each in parsers:
subp = self.subparser.add_parser(each)
self.optional(subp, default='sub')
self.optional(self.parser, default='main')
def optional(self, parser, default=None):
parser.add_argument('--foo', default=default)
args = Parsing().parser.parse_args()
print(args)
I get for 2 runs
1315:~/mypy$ python3.5 stack41431025.py cmd1 --foo 1
Namespace(cmd='cmd1', foo='1')
1316:~/mypy$ python3.5 stack41431025.py --foo 1 cmd1
Namespace(cmd='cmd1', foo='sub')
In the first, foo is set by the strings parsed by the cmd1 subparser.
In the second, foo gets the default value set by the subparser. The main parser parsed --foo, but its value was over written by the subparser.
There has been some discussion of this in bug/issues. http://bugs.python.org/issue9351 changed handling so that the subparser default has priority over main parser values. I think there are problems with that patch, but it's been in effect for a couple of years.
You retain more control if they are given different dest.
def make_subparsers(self, parsers):
for each in parsers:
subp = self.subparser.add_parser(each)
self.optional(subp, default='sub')
self.optional(self.parser, default='main', dest='main_foo')
def optional(self, parser, default=None, dest=None):
parser.add_argument('--foo', default=default, dest=dest)
1325:~/mypy$ python3.5 stack41431025.py --foo 1 cmd1
Namespace(cmd='cmd1', foo='sub', main_foo='1')
1325:~/mypy$ python3.5 stack41431025.py cmd1
Namespace(cmd='cmd1', foo='sub', main_foo='main')
1325:~/mypy$ python3.5 stack41431025.py --foo 1 cmd1 --foo 2
Namespace(cmd='cmd1', foo='2', main_foo='1')
====================
(earlier answer)
I'll try to sketch the possible combinations of arguments
parser = argparse.ArgumentParser()
parser.add_argument('mainpos', help='positional for main')
parser.add_argument('--mainopt', help='optional defined for main')
sp = parser.add_subparser(dest='cmd')
p1 = sp.add_parser('cmd1')
p1.add_argument('subpos', help='postional for sub')
p1.add_argument('--subopt', help='optional defined for sub')
A composite usage would look like:
python prog.py foo [--mainopt bar] cmd1 sfoo [--subopt baz]
The respective positionals have to be given in the right order. The subparser cmd is effectively a positional for the main.
The optional defined for main has to occur before the subparser name. The optional defined for the subparser has to occur after. They could have the same flag or dest, but they have to be defined separately. And if they have the same dest, there could be a conflict over values, especially defaults.
parser.parse_args() starts matching the input strings with its arguments. If it sees --mainopt is parses that optional argument. Otherwise it expects two postionals. The 2nd has to be one of the subparser names.
Once it gets a subparser name it passes the remaining strings to that subparser. The subparser handles the rest, and puts the values in the main namespace. And the first thing the subparser does is set its defaults. Whether that action overwrites values set by the main parser or not depends on just how the namespace is passed between the two.
================
Parsing is driven by the order of arguments in the command line. It tries to allow flagged arguments in any order. But once parsing is passed to the subparser, the main parser does not get another go at parsing. It just does a few clean up tasks.
But if I use parse_known_args, I can collect the strings that neither parser handled, and take another stab a parsing them.
parser1 = argparse.ArgumentParser()
parser1.add_argument('--foo')
sp = parser1.add_subparsers(dest='cmd')
sp1 = sp.add_parser('cmd1')
args, extra = parser1.parse_known_args()
parser2 = argparse.ArgumentParser()
parser2.add_argument('--foo')
if extra:
args = parser2.parse_args(extra)
print(args)
runs
1815:~/mypy$ python stack41431025.py --foo 1 cmd1
Namespace(cmd='cmd1', foo='1')
1815:~/mypy$ python stack41431025.py cmd1 --foo 2
Namespace(foo='2')
1815:~/mypy$ python stack41431025.py --foo 1 cmd1 --foo 3
Namespace(foo='3')
I haven't tested this idea in anything more complex, so there might be some interactions that I haven't thought of. But this is the closest I can come to a flagged argument that can occur anywhere, and not be subject to conflicting default problems.