I'm writing a small script that runs through a directory and attempts to obtain version numbers by running "filename --version". Now for the most part it works or fails in a manageable way. However I've come across one file "iptables-restore" in /sbin/ that fails when --version is passed to it. However the failure leaves the prompt in a state that requires a ctrl+z or ctrl+c to return to the prompt, and thus halts my script.
Here's the code:
try:
subOut = subprocess.check_output([fname, "--version"])
except subprocess.CalledProcessError as cpE:
fObj.write('{0:25}RETURN CODE:{1:15}\t\t{2:30}\n'.format(fnamecolon, cpE.returncode, cpE.output))
fnamecolon = ''
pass
except Exception as e:
pass
I just wondered if there's an elegant way to handle this - either via a return code or perhaps an Exception.
I'm fairly new to Python and am doing this for practice really - so any guidance is greatly appreciated.
Thanks all!
So this works better - probably still some things I'm not understanding though...
try:
# devnull=open(os.devnull, 'w')
try:
from subprocess import DEVNULL # Python 3
except ImportError:
DEVNULL = open(os.devnull, 'r+b', 0)
subOut = Popen([fname, "--version"], stdin=DEVNULL, stdout=PIPE, stderr=STDOUT, close_fds=True)
output = subOut.stdout.read()
except subprocess.CalledProcessError as cpE:
fObj.write('{0:25}RETURN CODE:{1:15}\t\t{2:30}\n'.format(fnamecolon, cpE.returncode, cpE.output))
fname = ''
pass
except Exception as e:
# fObj.write('{0:30}{1:30}{2:30}\n'.format(fname, e.returncode, e.output))
pass
I can reproduce it: iptables-restore --version commands waits for input without stdin=subprocess.DEVNULL suggested by #mata:
#!/usr/bin/env python3
import subprocess
cp = subprocess.run(['iptables-restore', '--version'],
stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
print(cp.returncode, cp.stdout, cp.stderr)
Related
I have a simple cmd_str containing a set of lines. Using exec, I can run those lines juts fine. However, running those lines in a separate process when shell=True is failing. Is this dues to missing quotes? what is happening under the hood?
import subprocess
cmd_str = """
import sys
for r in range(10):
print('rob')
"""
exec(cmd_str) # works ok
full_cmd = f'python3 -c "exec( "{cmd_str}" )"'
process = subprocess.Popen([full_cmd],
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
(output, error) = process.communicate()
exit_code = process.wait()
output_msg = output.decode("utf-8", 'ignore')
error_msg = error.decode("utf-8", 'ignore').strip()
Your approach is slightly inaccurate. I believe the problem you're having has to do with the subprocess usage. The first thing you must realise is that exec
is a way to send and execute python code to and from the interpreter directly. This is why it works inside python programs(and it is generally not a good approach). Subprocesses on the other hand handle command like they are being called directly from the terminal or shell. This means that you no longer need to include exec cause you are already interacting with the python interpreter when you call python -c.
To get this to run in a subprocess environment all you need to do is
full_cmd = f'python3 -c "{cmd_str}"'
process = subprocess.Popen(full_cmd,
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
Also, notice the absence of square brackets when calling subprocess.Popen, this is because that particular approach works slightly different and if you want to use the square brackets your command will have to be
full_cmd = ['python3', '-c', f'{cmd_str}']
process = subprocess.Popen(full_cmd,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
And with these few changes, everything should work OK
I'm not sure but I imagine that there may be issues similar to mine, but I have not found any that has been satisfactory.
When I open my Jupyter Lab and execute a cell as below (code01), it remains with the status of * (see figure01 below) meaning that it is still running the code, but the output of the out1.txt file is printed correctly.
I would like to know if it is normal for the cell to remain running in this circumstances described from code01.
code01:
import sys
file = open('out1.txt', 'a')
sys.stdout = file
print("house")
file.close()
figure01:
Because you redirect the stdout to a file and then close it you are breaking the IPython kernel underneath: there is no way for any stdout to be correctly processed by the kernel afterwards (cannot write to a closed file). You can reproduce it by executing your code in the IPython console rather than in a notebook. To fix this you could rebind the original stdout back:
import sys
file = open('out1.txt', 'a')
stdout = sys.stdout
sys.stdout = file
print("house")
# before close, not after!
sys.stdout = stdout
file.close()
But this is still not 100% safe; you should ideally use context managers instead:
from contextlib import redirect_stdout
with open('out1.txt', 'a') as f:
with redirect_stdout(f):
print('house')
But for this particular case why not to make use the file argument of the print() function?
with open('out1.txt', 'a') as f:
print('house', file=f)
Im trying to run a exe with some arguments within python using subprocesses Popen but the code is currently printing nothing as the output and a CMD window doesnt open when i run the file.
I know the command works as i manually paste it within CMD and it runs perfectly fine, what could i be doing wrong in my code?
import sys
import admin
import subprocess
from subprocess import Popen, PIPE
Path_2to3 = sys.executable[:-11] + "Scripts"
cmdCommand = '''cd "{}" && 2to3.exe -w "C:/Users/Desktop/bulk_converter/tests/test2.py"'''.format(Path_2to3)
process = subprocess.Popen(cmdCommand, stdout=subprocess.PIPE, shell=True)
output, error = process.communicate()
print(output, error)
retured values from print(output,error)
b'' None
Any help/suggestions would be appreciated.
Hi perhaps also specify stderr should be redirected to a pipe:
stderr=subprocess.PIPE
You may also be interested in passing a timeout parameter to communicate.
I'm attempting to create a python script to compile github surface kernel using their recommended steps.
https://github.com/dmhacker/arch-linux-surface
So far I'm stuck at a couple of sections.
Per the instructions for compiling the setup.sh must be run using sudo.
I've tried sending in the password before calling process using
preproc = subprocess.Popen(password, shell=True, stdout=subprocess.PIPE)
process = subprocess.Popen(["sudo", 'sh setup.sh'], shell=True, stdin=preproc.stdout, encoding='utf8')
I've tried sudo -S which doesn't seem to work at all. I've also tried lowercase -s.
I've tried changing subprocess.Popen to subprocess.call
password = getpass.getpass()
process = subprocess.Popen(["sudo", 'sh setup.sh'], shell=True,
stdin=subprocess.PIPE, encoding='utf8')
print(process.communicate(password + "\n"))
process.wait()
I expected the shell to be run at sudo level but it's not.
I'm not exactly sure what the difference is as I've since gone through many iterations, but finally got it to work and simplified. Hope this helps someone in the future.
import getpass
from subprocess import Popen, PIPE
password = getpass.getpass()
command = "./setup.sh"
process = Popen(['sudo', '-S', command], stdout=PIPE, encoding='utf8')
process.communicate(password)
I would like to easily test my python programs without constantly using the python shell since each time the program is modified you have to quit, re-enter the python shell and import the program again. I am using a 2012 Macbook pro with OSX. I have the following code:
import sys
def read_strings(filename):
with open(filename) as file:
return file.read().split('>')[1:0]
file1 = sys.argv[1]
filename = read_strings(file1)
Essentially I would like to read into and split a txt file containing:
id1>id2>id3>id4
I am entering this into my command line:
pal-nat184-102-127:python_stuff ceb$ python3 program.py string.txt
However when I try the sys.argv approach on the command line my program returns nothing. Is this a good approach to testing code, could anyone point me in the correct direction?
This is what I would like to happen:
pal-nat184-102-127:python_stuff ceb$ python3 program.py string.txt
['id1', 'id2', 'id3', 'id4']
Let's take this a piece at a time:
However when I try the sys.argv approach on the command line my
program returns nothing
The final result of your program is that it writes a string into the variable filename. It's a little strange to have a program "return" a value. Generally, you want a program to print it's something out or save something to a file. I'm guessing it would ease your debugging if you modified your program by adding,
print (filename)
at the end: you'd be able to see the result of your program.
could anyone point me in the correct direction?
One other debugging note: It can be useful to write your .py files so that they can be run both independently at the command line or in a python shell. How you've currently structured your code, this will work semi-poorly. (Starting a shell and then importing your file will cause an error because sys.argv[1] isn't defined.)
A solution to this is to change your the bottom section of your code as follows:
if __name__ == '__main__':
file1 = sys.argv[1]
filename = read_strings(file1)
The if guard at the top says, "If running as a standalone script, then run what's below me. If you imported me from some place else, then do not execute what's below me."
Feel free to follow up below if I misinterpreted your question.
You never do anything with the result of read_strings. Try:
print(read_strings(file1))