I'm writing a Python wrapper around another program. I want the user to be able to specify a few options for the wrapper and then pass the rest of the command-line through to the wrapped program. Something like this:
#click.command()
#click.option("--port", type=int)
#click.argument("args", nargs=-1)
def main(port, args):
call_the_wrapped_program(port=port, args=args)
But this dies with Error: no such option: -k because it treats any command-line switch as something it should parse rather than an argument that can be added to args.
Is this possible?
Assuming you invoke the program with something akin to a cli command, have you tried simply calling it like this?
cli --port 8080 -- -k arg1 arg2 -r etc
If I print args with that invocation, I get all the arguments out, albeit as a string, but I'm hoping that whatever third party you want to delegate to might be able to run its own parsing over that.
The Forwarding Unknown Options section in the documentation covers this use case. You should be able to write something like this:
#click.command(context_settings=dict(
ignore_unknown_options=True,
))
#click.option("--port", type=int)
#click.argument("args", nargs=-1, type=click.UNPROCESSED)
def main(port, args):
call_the_wrapped_program(port=port, args=args)
Related
I am busy writing a Python3 script which requires user input, the input is used as parameters in commands passed to the shell.
The script is only intended to be used by trusted internal users - however I'd rather have some contingencies in place to ensure the valid execution of commands.
Example 1:
import subprocess
user_input = '/tmp/file.txt'
subprocess.Popen(['cat', user_input])
This will output the contents of '/tmp/file.txt'
Example 2:
import subprocess
user_input = '/tmp/file.txt && rm -rf /'
subprocess.Popen(['cat', user_input])
Results in (as expected):
cat: /tmp/file.txt && rm -rf /: No such file or directory
Is this an acceptable method of sanitizing input? Is there anything else, per best practice, I should be doing in addition to this?
The approach you have chosen,
import subprocess
user_input = 'string'
subprocess.Popen(['command', user_input])
is quite good as command is static and user_input is passed as one single argument to command. As long as you don't do something really stupid like
subprocess.Popen(['bash', '-c', user_input])
you should be on the safe side.
For commands that require multiple arguments, I'd recommend that you request multiple inputs from the user, e.g. do this
user_input1='file1.txt'
user_input2='file2.txt'
subprocess.Popen(['cp', user_input1, user_input2])
instead of this
user_input="file1.txt file2.txt"
subprocess.Popen(['cp'] + user_input.split())
If you want to increase security further, you could:
explicitly set shell=False (to ensure you never run shell commands; this is already the current default, but defaults may change over time):
subprocess.Popen(['command', user_input], shell=False)
use absolute paths for command (to prevent injection of malicious executables via PATH):
subprocess.Popen(['/usr/bin/command', user_input])
explicitly instruct commands that support it to stop parsing options, e.g.
subprocess.Popen(['rm', '--', user_input1, user_input2])
do as much as you can natively, e.g. cat /tmp/file.txt could be accomplished with a few lines of Python code instead (which would also increase portability if that should be a factor)
I have a bash file with a bunch of sed commands like this :
sed -i 's/hello my name is Thibault/hello my name is Louis/g' "$1"
so for now i'm doing all of this "by hand", however, I have a python script with a tkinter GUI and several input fields for the user. I would like to find a trick so that if the user inputs "hello my name is Olivia" in the text field then the regex would look like this:
sed -i 's/hello my name is Thibault/hello my name is Olivia/g' "$1"
So I was thinking that i could store the python text input result in the variable to have the regex look like this:
sed -i 's/hello my name is Thibault/$my_variable/g' "$1"
but i don't know how or if this is even possible. Lastly I want to mention that i know i could just ask for the user input in the bash script but this is for my first internship and I have to go through the python GUI.
Edit: i'm on windows 10 if this is any important
Try it like this :
import os
original_text = 'hello my name is Thibault'
new_text = 'hello my name is Louis'
filename = 'test.txt'
os.system (f'sed -i "s/{original_text}/{new_text}/g" {filename}')
For passing data (in your case: some string) from your Python program to a subprocess running a bash script, you have first of all the same options like when calling one bash script from another one: Either design the called script to expect positional parameters (use it as $1 for example) and pass the string as parameter. For instance, if the string is stored in the Python variable parameter, it would look like:
import subprocess
subprocess.call ['bash', './script_to_be_called', parameter]
The other possibility is to design the bash script so that it expects the string to be stored in a variable of a certain name (use it as $PARSTRING for instance) and pass the data via the environment:
import os
os.environ['PARSTRING']=parameter
subprocess.call['bash', './script_to_be_called']
If the "script" executes only a single command, you could alternatively synthesize the command line in your Python program. Assume that you have a string bashcommand, which already holds the complete command which is supposed to be executed by bash, you could do a
import subprocess
subprocess.call ['bash', '-c', bashcommand]
While this should answer your question, I can't help but pointing out, that for executing a single external command, I would not create a shell process, but invoke this program directly as a child process. Also don't forget that spawning a child process takes time, and if you have many such invocations, it might make sense to redesign your approach, for instance by doing everything inside Python, or having only one child prcocess which gets as input the data for all the substitutions to be performed (typically via a file).
I'm attempting to run a linux command, curl, through groovy, and would like the output to be assign to a variable. I'd like to do this, so I can extract specific data from the curl output, and use in in m y groovy script.
Any ideas, how I can do this.
I've tried,
def after = "curl \"https://test.com\"".execute()
def after = "curl \"https://test.com\"".execute().text
but the variable is always empty. Any help would be appreciated.
linux command with spaces needs to be separated.
["/bin/sh", "-o", "your_command"].execute()
In your case it would be something like below:
["curl", "https://test.com"].execute()
OR
def after = ["curl", "https://test.com"].execute().text
Try and see if it works.
Currently, I have a command that looks something like the following:
my_command = Popen([activate_this_python_virtualenv_file, \
"-m", "my_command", "-l", \
directory_where_ini_file_for_my_command_is + "/" + my_ini_file_name], \
stderr=subprocess.STDOUT, stdout=subprocess.PIPE, shell=False,
universal_newlines=False, cwd=directory_where_my_module_is)
I have figured out how to access and process the output, deal with subprocess.PIPE, and make subprocess do a few other neat tricks.
However, it seems odd to me that the standard Python documentation for subprocess doesn't mention a way to just get the actual command line as subprocess.Popen puts it together from arguments to the Popen constructor.
For example, perhaps my_command.get_args() or something like that?
Is it just that getting the command line run in Popen should be easy enough?
I can just put the arguments together on my own, without accessing the command subprocess runs with Popen, but if there's a better way, I'd like to know it.
It was added in Python 3.3. According to docs:
The following attributes are also available:
Popen.args The args argument as it was passed to Popen – a sequence of
program arguments or else a single string.
New in version 3.3.
So sample code would be:
my_args_list = [] # yourlist
p = subprocess.Popen(my_args_list)
assert p.args == my_args_list
I have been experimenting with Haskell. I am trying to write a web crawler and I need to use external curl binary (due to some proxy settings, curl needs to have some special arguments which seem to be impossible/hard to set inside the haskell code, so i rather just pass it as a command line option. but that is another story...)
In the code at the bottom, if I change the marked line with curl instead of curl --help the output renders properly and gives:
"curl: try 'curl --help' or 'curl --manual' for more information
"
otherwise the string is empty - as the `curl --help' response is multiline.
I suspect that in haskell the buffer is cleared with every new line. (same goes for other simple shell commands like ls versus ls -l etc.)
How do I fix it?
The code:
import System.Process
import System.IO
main = do
let sp = (proc "curl --help"[]){std_out=CreatePipe} -- *** THIS LINE ***
(_,Just out_h,_,_)<- createProcess sp
out <-hGetContents out_h
print out
proc takes as a first argument the name of the executable, not a shell command. That, is when you use proc "foo bar" you are not referring to a foo executable, but to an executable named exactly foo bar, with the space in its file name.
This is a useful feature in practice, because sometimes you do have spaces in there (e.g. on Windows you might have c:\Program Files\Foo\Foo.exe). Using a shell command you would have to escape spaces in your command string. Worse, a few other characters need to be escaped as well, and it's cumbersome to check what exactly those are. proc sidesteps the issue by not using the shell at all but passing the string as it is to the OS.
For the executable arguments, proc takes a separate argument list. E.g.
proc "c:\\Program Files\\Foo\\Foo.exe" ["hello world!%$"]
Note that the arguments need no escaping as well.
If you want to pass arguments to curl you have to pass that it in the list:
sp = (proc "/usr/bin/curl" ["--help"]) {std_out=CreatePipe}
Then you will get the complete output in the entire string.