I have a python program, which calls the shell script through subprocess() module. I am looking for a way to pass a simple file, as an input to shell script. Does this happen through subproess and popen?
I have tried this code for an AWS lambda function
It would be nice/helpful if you could share some excerpt of your code in your question.
But assuming bits of it.
Here is a way to achieve this.
import shlex
from subprocess import PIPE, Popen
import logger
def run_script(script_path, script_args):
"""
This function will run a shell script.
:param script_path: String: the path of script that needs to be called
:param script_args: String: the arguments needed by the shell script
:return:
"""
logger.info("Running bash script {script} with parameters:{params}".format(script=script_path, params=script_args))
# Adding a whitespace in shlex.split because the path gets distorted if args are added without it
session = Popen(shlex.split(script_path + " " + script_args), stderr=PIPE, stdout=PIPE, shell=False)
stdout, stderr = session.communicate()
# Beware that stdout and stderr will be bytes so in order to get a proper python string decode the values.
logger.debug(stdout.decode('utf-8'))
if stderr:
logger.error(stderr)
raise Exception("Error " + stderr.decode('utf-8'))
return True
Now a couple of things to note here
Your bash script should be able to handle the args properly may it be $1 or named params like --file or -f
Just give all the params you want in the string array in shlex method.
Also note the comments mentioned in code above.
Related
I am trying to build a system where a list of the available wifi networks would be stored for some specific purpose. Now the problem is that executing a system command with os.system() in a variable 'res' only stores the return value of the command which is useless to me at this point.
I know of no approach that provide me the desired result.
import os
res = os.system('nmcli dev wifi')
The variable res must store all the desired result into it rather than the return value. Even if it stores result, it will do the work.
You can do this using the Popen method from the subprocess module
from subprocess import Popen, PIPE
#First argument is the program name.
arguments = ['ls', '-l', '-a']
#Run the program ls as subprocess.
process = Popen(arguments, stdout=PIPE, stderr=PIPE)
#Get the output or any errors. Be aware, they are going to be
#in bytes!!!
stdout, stderr = process.communicate()
#Print the output of the ls command.
print(bytes.decode(stdout))
I have the following function that executes as expected.
def get_files(paths):
for path in paths:
file_name = parse_path(path)
csv_command = "curl -b ./cookie {} > ./tmp/{}".format(path, file_name)
subprocess.run([csv_command], shell=True)
print("success")
My issue here is that I am also capturing standardout from the subprocess. How do I modify the function to ignore the standard out of the subprocess. I will be logging using a logger and need to make sure that logging will still occur to STDout
Pass a handle to the DEVNULL directory for the standard output. (which can be accessed using subprocess.DEVNULL)
This should suppress output from your function
subprocess.run([csv_command], shell=True, stdout=subprocess.DEVNULL)
You can see some more options by reading this page: https://docs.python.org/3/library/subprocess.html#subprocess.DEVNULL
Just close the file-descriptor:
process.stdout.close()
Where process is the process which you desire to not read from.
I need to run the following command via python.
/work/data/get_info name=Mike home
The error I am getting is No such file or directory: '/work/data/get_info name=Mike home'. Which isn't correct. the get_info program does exits.
It is working in a perl script I am trying to get the same functionality in python.
perl script
$ENV{work} = '/work/data';
my $myinfo = "$ENV{work}/bin/get_info";
$info = `$myinfo name=Mike home`;
Info dumps out information
my python script
import os, subprocess
os.environ['work'] = '/work/data'
run_info = "{}/bin/get_info name={} {}".format(os.environ['work'],'Mike','home')
p = subprocess.call([run_product_info], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
I get an error No such file or directory: '/work/data/get_info name=Mike
The Python subprocess.call is thinking that the entire string is the name of the program, as if you had double quoted it like "/work/data/get_info name=Mike home" since you passed it as an array.
Either pass it without the array for the shell (if you are sure all escaping/quoting is correct, and see warnings in the docs) or pass each as a separate array element.
subprocess.call(['/work/data/bin/get_info', 'name=Mike', 'home'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
subprocess.call('/work/data/bin/get_info name=Mike home', stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
https://docs.python.org/3.7/library/subprocess.html#frequently-used-arguments
args is required for all calls and should be a string, or a sequence of program arguments. Providing a sequence of arguments is generally preferred, as it allows the module to take care of any required escaping and quoting of arguments (e.g. to permit spaces in file names). If passing a single string, either shell must be True (see below) or else the string must simply name the program to be executed without specifying any arguments.
I run the python script using terminal command
python3 myScript.py
It's simply run my program but if i want to open python console after complete run of my script so that i can access my script's variables.
So, What should i do ? and How can i get my script's variables after run the code using terminal ?
Open a python terminal (type 'python' in cmd);
Paste this (replace 'myScript.py' with your script filename):
def run():
t = ""
with open('myScript.py') as f:
t = f.read()
return t
Type exec(run()). Now you will have access to the variables defined in myScript.py.
I needed to do this so I could explore the result of a request from the requests library, without having to paste the code to make the requests every time.
Make the program run the other program you want with the variables as arguments. For example:
#program1
var1=7
var2="hi"
import os
os.system("python %s %d %s" % (filename, var1, var2))
#program2
import sys
#do something such as:
print(sys.argv[1]) #for var1
print(sys.argv[2]) #for var2
Basically, you are running program2 with arguments that can be referenced later.
Hope this helps :)
Currently, I have a command that looks something like the following:
my_command = Popen([activate_this_python_virtualenv_file, \
"-m", "my_command", "-l", \
directory_where_ini_file_for_my_command_is + "/" + my_ini_file_name], \
stderr=subprocess.STDOUT, stdout=subprocess.PIPE, shell=False,
universal_newlines=False, cwd=directory_where_my_module_is)
I have figured out how to access and process the output, deal with subprocess.PIPE, and make subprocess do a few other neat tricks.
However, it seems odd to me that the standard Python documentation for subprocess doesn't mention a way to just get the actual command line as subprocess.Popen puts it together from arguments to the Popen constructor.
For example, perhaps my_command.get_args() or something like that?
Is it just that getting the command line run in Popen should be easy enough?
I can just put the arguments together on my own, without accessing the command subprocess runs with Popen, but if there's a better way, I'd like to know it.
It was added in Python 3.3. According to docs:
The following attributes are also available:
Popen.args The args argument as it was passed to Popen – a sequence of
program arguments or else a single string.
New in version 3.3.
So sample code would be:
my_args_list = [] # yourlist
p = subprocess.Popen(my_args_list)
assert p.args == my_args_list