I'm building a command-line interface using the argparse library to parse user input. At one stage, I'd like to take user input such as "'test', x=False" and use it in a function such as func('test', x=False).
I've tried using ast.literal_eval to do this but it encounters a syntax error at the equals sign. (I did ast.literal_eval("("+args+")") where args was above example)
Does anyone know of a safe way to parse the user input like that? Preferably without eval although worst-case scenario I could use eval as, well, it's a CLI tool.
Edit (to people that have said to use input manually(): I need the tool to parse input from when the command is run (it's a python module that I want to be able to be called like python3 -m hmcli.raw --args "'test', x=False" where the args can be flexible as the function used can differ.
Related
I'm writing a module with custom logging utilities to be imported in other scripts.
It's based on the standard-library logging module.
One of these utilities looks like this:
import argparse as ap
def parse_log_args() -> dict:
log_arg_parser = ap.ArgumentParser(description='Parses arguments that affect logging')
log_arg_parser.add_argument(
'--level',
dest='level',
help='Sets logging level',
choices=['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']
)
log_args, _ = log_arg_parser.parse_known_args()
return vars(log_args)
This function looks for arguments that have to do with logging (even though only --level is defined for the time being) and parses those independently of (and before) all others so that the logging can be configured early on and used in the rest of the script.
The goal here is to remain flexible and be able to quickly plug-in support for these arguments, both in scripts that expect no other arguments and in those that do.
From the point of view of simply parsing arguments this works: this function runs first, parses --level and then the script-specific parser comes and handles the rest.
The problem, however, is the help text. When I run a script that calls this function with --help it only displays the help text from this first parser and not from the script-specific one. So something like this:
Parses arguments that affect logging
optional arguments:
-h, --help show this help message and exit
--level {DEBUG,INFO,WARNING,ERROR,CRITICAL}
Sets logging level
Is there a way to combine the help-texts from all the ArgumentParser instances in a script?
Alternatively: Is there a different way to achieve this in a flexible/plug-in kind of way, that is, without modifying existing ArgumentParsers or having to add them to scripts that don't yet use them?
PS: A similar question has been asked before here: Argparse combine --help directives but the proposed ideas don't really solve the problem:
Define the first parser with add_help=False: This would hide the option from the user which I would prefer not to do.
Use subcommands somehow: doesn't seem to be applicable here.
I think this might fit the bill:
import argparse
part1 = argparse.ArgumentParser(add_help=False)
#... some parsing takes place ...
part2 = argparse.ArgumentParser(add_help=True, parents=[part1])
part1 parser must be fully initialized for parents to work.
More on the topic:
https://docs.python.org/3/library/argparse.html#parents
What is the recommended method of sanitizing user_input_parameter passed to the shell like
subprocess.Popen(['sudo', 'rm -rf', user_input_parameter])
The command should accept all parameters but malicious activies like breaking out of the command should be mitigated.
Python's implementation of subprocess protects against shell injection, documentation says so:
17.5.2. Security Considerations
Unlike some other popen functions, this implementation will never
implicitly call a system shell. This means that all characters,
including shell metacharacters, can safely be passed to child
processes. If the shell is invoked explicitly, via shell=True, it is
the application’s responsibility to ensure that all whitespace and
metacharacters are quoted appropriately to avoid shell injection
vulnerabilities.
When using shell=True, the shlex.quote() function can be used to
properly escape whitespace and shell metacharacters in strings that
are going to be used to construct shell commands.
This will however NOT protect against a user passing a malicious input - in your case for example deleting something that was not intended to be deleted. I would not pass user input to the command directly like that - you should verify if whatever you want to be deleted is being deleted and not something completely different. That is however part of application's logic already - regarding shell injection (breaking out of the command) - that should be fine with subprocess.
I made this little example:
#!/usr/bin/env python3
import subprocess
user_input_parameter = '/; id'
subprocess.Popen(['ls', user_input_parameter])
Which outputs this when executed:
$ python3 asdf.py
ls: /; id: No such file or directory
$
To demonstrate subprocess passes the input as an argument to the parameter.
All of this is true only if shell=False (default as of writing this answer) for subprocess methods, otherwise you basically enable shell (bash, etc.) execution and allow for injection to happen if inputs are not properly sanitized.
Btw, you need to pass each parameter separately, so you would need to run it like this (but please don't do that):
subprocess.Popen(['sudo', 'rm', '-rf', user_input_parameter])
I've got a python script where I run a cmd using subprocess.getoutput(), and store the resulting output in a list. Now, I need to be able to have the script change to a target dir and run the command there. It should be simple, but passing the cwd arg to getoutput() is not working.
Any ideas?
Example:
out = subprocess.getoutput(" ".join(cmd), cwd='/my/target/path').splitlines()
From the doc it looks like I can easily do this with subprocess.Popen, but then it's difficult to get the output into a list of strings. I've only been able to get the results into a list of binary strings.
subprocess.getoutput is a Legacy Shell Invocation Function. It doesn't take cwd argument and returns a tuple of (status, output). You've got several problems before you even get to the list of bytes.
When python runs a program, it doesn't know what encoding its output is going to have you need to supply that somehow. Assuming the encoding is `utf-8', the basic operation is
mylist = []
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, cwd='/my/target/path')
for line in mylist:
mylist.append(line.decode('utf-8'))
proc.wait()
In this implementation, anything written to stderr just goes to your programs stderr. Notice also that I kept the command as a list and didn't do shell=True. There are several helper functions that do some of the work for you, but that's pretty simple already.
I am writing detailed information for each function in a class, and hope when others use my python code, they could check the info for each function, by using something like help(function_name)
For example, I have created a python file called text_preprocess.py, in this file I have created a class which includes functions.
class Preprocess():
def str_process(self, row_string):
'''
This Standford CoreNLP package requires the text input as 1 single string.
The input annotators are in you command line input.
:param row_string: The string format input for Standford CoreNLP
:return: Json format output
'''
parsed_json = self.nlp.annotate(row_string, properties={
'annotators': self.standford_annotators,
'outputFormat': 'json'
})
return parsed_json
In this function, as you can see the info is within triple quotes.
But I don't know how could other users see this info without looking into my code.
I have checked many solutions online, people use help(function_name), but it seems that they wrote the code through terminal, and then type help(function_name), many of those examples do not have class either. Using --help through command line only gives them the parameter descriptions I added through argparse....
So, if I hope others could check the info of my function without looking into the code, where and how could they do that?
Either be in the same directory as your script, or make sure your script is in one of the directories listed in sys.path (usually, the first option is easier for simple things, but if you want to do the second, use a virtualenv rather than trying to install the module system-wide).
Run pydoc3 text_preprocess to get documentation for the whole module. This recursively includes all of the items below the module, such as classes, their members, and functions.
Run pydoc3 text_preprocess.Preprocess if you just want the class.
Run pydoc3 text_preprocess.Preprocess.str_process if you just want the method.
Use Sphinx if you want nicely-formatted HTML or other formats such as PDF.
You may also want to remove that empty line at the beginning of your docstring; some docstring-parsing code may misinterpret it.
Putting triple quoted strings just below the declaration of a class or method is called documentation. You can read this using Preprocess.str_process.__doc__.
Good Afternoon Everyone,
This is probably a no-brainer but, I'm currently having issues passing a variable to a program in my bash script.
Here's what I'm trying to do:
regions=ne,se,vt,ma,sw,nw and so on and so forth
After that variable has been defined in my bash script, I'd then like to pass that variable into GrADS, where my script will read each of the regions one after the other until it reaches the end.
The most reliable means of passing variables I've found is to generate a text file with the code (or just the string) you want to pass from within the code. Alternatively, you could call GrADS (?) from within whatever program is generating the variable, and pass "$regions" as an argument.