I have the following function that executes as expected.
def get_files(paths):
for path in paths:
file_name = parse_path(path)
csv_command = "curl -b ./cookie {} > ./tmp/{}".format(path, file_name)
subprocess.run([csv_command], shell=True)
print("success")
My issue here is that I am also capturing standardout from the subprocess. How do I modify the function to ignore the standard out of the subprocess. I will be logging using a logger and need to make sure that logging will still occur to STDout
Pass a handle to the DEVNULL directory for the standard output. (which can be accessed using subprocess.DEVNULL)
This should suppress output from your function
subprocess.run([csv_command], shell=True, stdout=subprocess.DEVNULL)
You can see some more options by reading this page: https://docs.python.org/3/library/subprocess.html#subprocess.DEVNULL
Just close the file-descriptor:
process.stdout.close()
Where process is the process which you desire to not read from.
Related
I am trying to build a system where a list of the available wifi networks would be stored for some specific purpose. Now the problem is that executing a system command with os.system() in a variable 'res' only stores the return value of the command which is useless to me at this point.
I know of no approach that provide me the desired result.
import os
res = os.system('nmcli dev wifi')
The variable res must store all the desired result into it rather than the return value. Even if it stores result, it will do the work.
You can do this using the Popen method from the subprocess module
from subprocess import Popen, PIPE
#First argument is the program name.
arguments = ['ls', '-l', '-a']
#Run the program ls as subprocess.
process = Popen(arguments, stdout=PIPE, stderr=PIPE)
#Get the output or any errors. Be aware, they are going to be
#in bytes!!!
stdout, stderr = process.communicate()
#Print the output of the ls command.
print(bytes.decode(stdout))
I need to run the following command via python.
/work/data/get_info name=Mike home
The error I am getting is No such file or directory: '/work/data/get_info name=Mike home'. Which isn't correct. the get_info program does exits.
It is working in a perl script I am trying to get the same functionality in python.
perl script
$ENV{work} = '/work/data';
my $myinfo = "$ENV{work}/bin/get_info";
$info = `$myinfo name=Mike home`;
Info dumps out information
my python script
import os, subprocess
os.environ['work'] = '/work/data'
run_info = "{}/bin/get_info name={} {}".format(os.environ['work'],'Mike','home')
p = subprocess.call([run_product_info], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
I get an error No such file or directory: '/work/data/get_info name=Mike
The Python subprocess.call is thinking that the entire string is the name of the program, as if you had double quoted it like "/work/data/get_info name=Mike home" since you passed it as an array.
Either pass it without the array for the shell (if you are sure all escaping/quoting is correct, and see warnings in the docs) or pass each as a separate array element.
subprocess.call(['/work/data/bin/get_info', 'name=Mike', 'home'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
subprocess.call('/work/data/bin/get_info name=Mike home', stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
https://docs.python.org/3.7/library/subprocess.html#frequently-used-arguments
args is required for all calls and should be a string, or a sequence of program arguments. Providing a sequence of arguments is generally preferred, as it allows the module to take care of any required escaping and quoting of arguments (e.g. to permit spaces in file names). If passing a single string, either shell must be True (see below) or else the string must simply name the program to be executed without specifying any arguments.
I have a python program, which calls the shell script through subprocess() module. I am looking for a way to pass a simple file, as an input to shell script. Does this happen through subproess and popen?
I have tried this code for an AWS lambda function
It would be nice/helpful if you could share some excerpt of your code in your question.
But assuming bits of it.
Here is a way to achieve this.
import shlex
from subprocess import PIPE, Popen
import logger
def run_script(script_path, script_args):
"""
This function will run a shell script.
:param script_path: String: the path of script that needs to be called
:param script_args: String: the arguments needed by the shell script
:return:
"""
logger.info("Running bash script {script} with parameters:{params}".format(script=script_path, params=script_args))
# Adding a whitespace in shlex.split because the path gets distorted if args are added without it
session = Popen(shlex.split(script_path + " " + script_args), stderr=PIPE, stdout=PIPE, shell=False)
stdout, stderr = session.communicate()
# Beware that stdout and stderr will be bytes so in order to get a proper python string decode the values.
logger.debug(stdout.decode('utf-8'))
if stderr:
logger.error(stderr)
raise Exception("Error " + stderr.decode('utf-8'))
return True
Now a couple of things to note here
Your bash script should be able to handle the args properly may it be $1 or named params like --file or -f
Just give all the params you want in the string array in shlex method.
Also note the comments mentioned in code above.
I have this fun in my python script:
def start_pushdata_server(Logger):
Logger.write_event("Starting pushdata Server..", "INFO")
retcode, stdout, stderr = run_shell(create_shell_command("pushdata-server
start"))
we want to redirect the standard error from pushdata-server binary to /dev/null.
so we edit it like this:
def start_pushdata_server(Logger):
Logger.write_event("Starting pushdata Server..", "INFO")
retcode, stdout, stderr = run_shell(create_shell_command("pushdata-server
start 2>/dev/null"))
But adding the 2>/dev/null in the python code isn't valid.
So how we can in the python code to send all errors from "pushdata-server
start" to null?
This code added to a Python script running in Unix or Linux will redirect all stderr output to /dev/null
import os # if you have not already done this
fd = os.open('/dev/null',os.O_WRONLY)
os.dup2(fd,2)
If you want to do this for only part of your code:
import os # if you have not already done this
fd = os.open('/dev/null',os.O_WRONLY)
savefd = os.dup(2)
os.dup2(fd,2)
The part of your code to have stderr redirected goes here. Then to restore stderr back to where it was:
os.dup2(savefd,2)
If you want to do this for stdout, use 1 instead of 2 in the os.dup and os.dup2 calls (dup2 stays as dup2) and flush stdout before doing any group of os. calls. Use different names instead of fd and/or savefd if these are conflicts with your code.
Avoiding the complexities of the run_shell(create_shell_command(...)) part which isn't well-defined anyway, try
import subprocess
subprocess.run(['pushdata-server', 'start'], stderr=subprocess.DEVNULL)
This doesn't involve a shell at all; your command doesn't seem to require one.
Currently, I have a command that looks something like the following:
my_command = Popen([activate_this_python_virtualenv_file, \
"-m", "my_command", "-l", \
directory_where_ini_file_for_my_command_is + "/" + my_ini_file_name], \
stderr=subprocess.STDOUT, stdout=subprocess.PIPE, shell=False,
universal_newlines=False, cwd=directory_where_my_module_is)
I have figured out how to access and process the output, deal with subprocess.PIPE, and make subprocess do a few other neat tricks.
However, it seems odd to me that the standard Python documentation for subprocess doesn't mention a way to just get the actual command line as subprocess.Popen puts it together from arguments to the Popen constructor.
For example, perhaps my_command.get_args() or something like that?
Is it just that getting the command line run in Popen should be easy enough?
I can just put the arguments together on my own, without accessing the command subprocess runs with Popen, but if there's a better way, I'd like to know it.
It was added in Python 3.3. According to docs:
The following attributes are also available:
Popen.args The args argument as it was passed to Popen – a sequence of
program arguments or else a single string.
New in version 3.3.
So sample code would be:
my_args_list = [] # yourlist
p = subprocess.Popen(my_args_list)
assert p.args == my_args_list