Given the following code
import subprocess
result = ( subprocess.run(
'scontrol --details show node',
shell=True, stdout=subprocess.PIPE, timeout=30) )
if result.returncode != 0:
print("Error: scontrol returned a non-zero exit code.")
exit(result.returncode)
Does this create a pipe, and, if so, does that pipe get closed automatically? I assume the subprocess will terminate automatically with that code, even if it hangs? One more question: To write good code, should I use with anywhere? Thanks.
Related
I have a simple cmd_str containing a set of lines. Using exec, I can run those lines juts fine. However, running those lines in a separate process when shell=True is failing. Is this dues to missing quotes? what is happening under the hood?
import subprocess
cmd_str = """
import sys
for r in range(10):
print('rob')
"""
exec(cmd_str) # works ok
full_cmd = f'python3 -c "exec( "{cmd_str}" )"'
process = subprocess.Popen([full_cmd],
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
(output, error) = process.communicate()
exit_code = process.wait()
output_msg = output.decode("utf-8", 'ignore')
error_msg = error.decode("utf-8", 'ignore').strip()
Your approach is slightly inaccurate. I believe the problem you're having has to do with the subprocess usage. The first thing you must realise is that exec
is a way to send and execute python code to and from the interpreter directly. This is why it works inside python programs(and it is generally not a good approach). Subprocesses on the other hand handle command like they are being called directly from the terminal or shell. This means that you no longer need to include exec cause you are already interacting with the python interpreter when you call python -c.
To get this to run in a subprocess environment all you need to do is
full_cmd = f'python3 -c "{cmd_str}"'
process = subprocess.Popen(full_cmd,
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
Also, notice the absence of square brackets when calling subprocess.Popen, this is because that particular approach works slightly different and if you want to use the square brackets your command will have to be
full_cmd = ['python3', '-c', f'{cmd_str}']
process = subprocess.Popen(full_cmd,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
And with these few changes, everything should work OK
I'm running a shell command in a Jupyter Notebook using subprocess or os.system() . The actual output is a dump of thousands of lines of code which takes at least a minute to stdout in terminal. In my notebook, I just want to know if the output is more than a couple of lines because if it was an error, the output would only be 1 or 2 lines. What's the best way to check if I'm receiving 20+ lines and then stop the process and move on to the next?
you could read line by line using subprocess.Popen and count the lines (redirecting & merging output and error streams, maybe merging is not needed, depends on the process)
If the number of lines exceeds 20, kill the process and break the loop.
If the loop ends before the number of lines reaches 20, print/handle an error
code:
p = subprocess.Popen(cmd,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
for lineno,line in enumerate(iter(p.stdout.readline, b'')):
if lineno == 20:
print("process okay")
p.kill()
break
else:
# too short, break wasn't reached
print("process failed return code: {}".format(p.wait()))
note that p.poll() is not None can help to figure out if the process has ended prematurely too
I am trying to write something that checks a specific service, puts that into a text file. Afterwords I am trying to determine if its stopped or running and do other things.
The file gets created and looks like this, I tried parsing this out individually or using .readlines() but no dice. Any helps/tips would be appreciated.
SERVICE_NAME: fax
TYPE : 10 WIN32_OWN_PROCESS
STATE : 1 STOPPED
WIN32_EXIT_CODE : 1077 (0x435)
SERVICE_EXIT_CODE : 0 (0x0)
CHECKPOINT : 0x0
WAIT_HINT : 0x0
but my code below returns empty or nothing
from subprocess import Popen
import datetime
today = datetime.datetime.now()
squery = ['sc', 'query', 'fax']
proc = Popen(['sc', 'query', 'fax'], stdout=open(str(today.date())+'_ServiceCheck.txt', 'w'))
if 'STOPPED' in open(str(today.date())+'_ServiceCheck.txt').read():
print("Uh Oh")
#Do Something about it
As written, there's a good chance the parent process will open the file, check for STOP and close long before the subprocess even starts running. You can use subprocess.call to force the parent process to block until the child finishes executing, which might enable the idea of waiting for your Selenium script's process to finish execution.
Consider this:
# some_script.py
from time import sleep
print("subprocess running!")
for i in range(5):
print("subprocess says %s" % i)
sleep(1)
print("subprocess stopping!")
# main.py
import subprocess
while True:
print("parent process starting child...")
proc = subprocess.call(["python", "some_script.py"])
print("parent process noticed child stopped running")
Output excerpt from running python main.py:
parent process starting child...
subprocess running!
subprocess says 0
subprocess says 1
subprocess says 2
subprocess says 3
subprocess says 4
subprocess stopping!
parent process noticed child stopped running
parent process starting child...
subprocess running!
subprocess says 0
subprocess says 1
subprocess says 2
subprocess says 3
subprocess says 4
subprocess stopping!
parent process noticed child stopped running
...
This seems much better. The parent blocks completely until the child stops execution, then immediately restarts the child.
Otherwise, to do what you're doing, it sounds like you'll need to poll the file periodically like:
import datetime
from subprocess import Popen
from time import sleep
delay = 10
while True:
today = datetime.datetime.now()
fname = '%s_ServiceCheck.txt' % today.date()
file_content = open(fname).read()
if 'STOPPED' in file_content:
print('Uh oh')
proc = Popen(['sc', 'query', 'fax'], stdout=open(fname, 'w'))
sleep(delay)
But be careful--what if the Selenium process stops at 11:59:59? Polling this text file is pretty brittle, so this script is probably nowhere near robust enough to handle all cases. If you can redirect your Selenium script's output directly to the parent process, that would make it a lot more reliable. The parent process can also write the log to disk on behalf of the script if needed.
Either way, a lot of it depends on details about your environment and what you're trying to accomplish.
This is a simple code that logs into a linux box, and execute a grep to a resource on the box. I just need to be able to view the output of the command I execute, but nothing is returned. Code doesn't report any error but the desired output is not written.
Below is the code:
import socket
import re
import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('linux_box', port=22, username='abc', password='xyz')
stdin, stdout, stderr = ssh.exec_command('grep xyz *.set')
output2 = stdout.readlines()
type(output2)
This is the output I get:
C:\Python34\python.exe C:/Python34/Paramiko.py
Process finished with exit code 0
You never actually print anything to standard output.
Changing last line to print(output2) should print value correctly.
Your code was likely based on interactive Python shell experiments, where return value of last executed command is printed to standard output implicitly. In non-interactive mode such behavior does not occur. That's why you need to use print function explicitly.
I'm writing a small script that runs through a directory and attempts to obtain version numbers by running "filename --version". Now for the most part it works or fails in a manageable way. However I've come across one file "iptables-restore" in /sbin/ that fails when --version is passed to it. However the failure leaves the prompt in a state that requires a ctrl+z or ctrl+c to return to the prompt, and thus halts my script.
Here's the code:
try:
subOut = subprocess.check_output([fname, "--version"])
except subprocess.CalledProcessError as cpE:
fObj.write('{0:25}RETURN CODE:{1:15}\t\t{2:30}\n'.format(fnamecolon, cpE.returncode, cpE.output))
fnamecolon = ''
pass
except Exception as e:
pass
I just wondered if there's an elegant way to handle this - either via a return code or perhaps an Exception.
I'm fairly new to Python and am doing this for practice really - so any guidance is greatly appreciated.
Thanks all!
So this works better - probably still some things I'm not understanding though...
try:
# devnull=open(os.devnull, 'w')
try:
from subprocess import DEVNULL # Python 3
except ImportError:
DEVNULL = open(os.devnull, 'r+b', 0)
subOut = Popen([fname, "--version"], stdin=DEVNULL, stdout=PIPE, stderr=STDOUT, close_fds=True)
output = subOut.stdout.read()
except subprocess.CalledProcessError as cpE:
fObj.write('{0:25}RETURN CODE:{1:15}\t\t{2:30}\n'.format(fnamecolon, cpE.returncode, cpE.output))
fname = ''
pass
except Exception as e:
# fObj.write('{0:30}{1:30}{2:30}\n'.format(fname, e.returncode, e.output))
pass
I can reproduce it: iptables-restore --version commands waits for input without stdin=subprocess.DEVNULL suggested by #mata:
#!/usr/bin/env python3
import subprocess
cp = subprocess.run(['iptables-restore', '--version'],
stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
print(cp.returncode, cp.stdout, cp.stderr)