I am reading some output of remote server using SSH bash(command). I need to do it constantly, so I spawn subprocess. It works fine, unless server server become unavailable for brief period.
How do I restart SSH command (whole subprocess) if it fails?
I have following piece of code:
(...)
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True, universal_newlines=True)
while True:
line = process.stdout.readline()
if lines == "" and process.poll() is not None:
break
(...)
I would think that process.poll() is not None should do the trick
but it seems to hang on
1000 21431 0.0 0.0 0 0 pts/0 Z+ Oct22 0:00 [ssh] <defunct>
And does not break out of while True:
I had silly typo error in my code preventing if lines == "" and process.poll() is not None: from executing properly.
Another thing to look at is ssh_config, It is wise to set values to disconnect #60 seconds/1 attempt.
And avoid process.communicate() as it is blocking the whole thread.
Related
I have two files in Heroku like "app.py" and "notify.py".
when I run the "app.py".
In some case, I'd like to run "notify.py", so I use this function "subprocess.Popen" to call "notify.py" .and I hope that process "notify.py" can run in loop, even after the main process ends.
How can I do that?
By the way, I get the error like this
/bin/sh: 1: ./notify.py: Permission denied
I use some methods like below
child = subprocess.Popen(['./notify.py'], shell=True, cwd='/app', stdin=None, stdout=subprocess.PIPE)
child = subprocess.call(['./notify.py'], shell=True, cwd='/app', stdin=None, stdout=subprocess.PIPE)
but all doesn't work
and I try another like this
i make a function notify() in app.py
and change the code like below
child = subprocess.Popen([notify(text)], shell=True)
it work.
But it will run "notify(text)" then print "test" and return.
Please tell me why and how can I solve this problem.
if event.message.text[0:6] == 'push**':
t = event.message.text
text = t.split('**')
child = subprocess.Popen(['./notify.py'], shell=True, cwd='/app', stdin=None, stdout=subprocess.PIPE)
print('test')
return 0
/bin/sh: 1: ./notify.py: Permission denied
I am calling some java binary in unix environment wrapped inside python script
When I call script from bash, output comes clean and also being stored in desired variable , However when i run the same script from Cron, Output stored(in a Variable) is incomplete
my code:
command = '/opt/HP/BSM/PMDB/bin/abcAdminUtil -abort -streamId ETL_' \
'SystemManagement_PA#Fact_SCOPE_OVPAGlobal'
proc = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
(output, err) = proc.communicate() # Storing Output in output variable
Value of output variable when running from shell:
Abort cmd output:PID:8717
Executing abort function
hibernateConfigurationFile = /OBRHA/HPE-OBR/PMDB/lib/hibernate-core-4.3.8.Final.jar
Starting to Abort Stream ETL_SystemManagement_PA#Fact_SCOPE_OVPAGlobal
Aborting StreamETL_SystemManagement_PA#Fact_SCOPE_OVPAGlobal
Value of output variable when running from cron:
PID:830
It seems output after creating new process is not being stored inside variable , i don't know why ?
Kintul.
You question seems to be very similar to this one: Capture stdout stderr of python subprocess, when it runs from cron or rc.local
See if that helps you.
This happened because Java utility was trowing exception which is not being cached by subprocess.Popen
However exception is catched by subprocess.check_output
Updated Code :
try:
output = subprocess.check_output(command, shell=True, stderr=subprocess.STDOUT, stdin=subprocess.PIPE)
except subprocess.CalledProcessError as exc:
print("Status : FAIL", exc.returncode, exc.output)
else:
print("Output of Resume cmd: \n{}\n".format(output))
file.write("Output of Resume cmd: \n{}\n".format(output) + "\n")
Output of code:
('Status : FAIL', -11, 'PID:37319\n')
('Status : FAIL', -11, 'PID:37320\n')
Hence , command is throwing exception is being cached by subprocess.check_output but not by subprocess.Popen
Extract form official page of subprocess.check_output
If the return code was non-zero it raises a CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute and any output in the output attribute.
Here's the Python code to run an arbitrary command returning its stdout data, or raise an exception on non-zero exit codes:
proc = subprocess.Popen(
cmd,
stderr=subprocess.STDOUT, # Merge stdout and stderr
stdout=subprocess.PIPE,
shell=True)
The subprocess module does not support execution-time and if it exceeds specific threshold => timeout(ability to kill a process running for more than X number of seconds)
What is the simplest way to implement get_execution_time and timeout in Python2.6 program meant to run on Linux?
Good question. Here is the complete code for this:
import time, subprocess # Importing modules.
timeoutInSeconds = 1 # Our timeout value.
cmd = "sleep 5" # Your desired command.
proc = subprocess.Popen(cmd,shell=True) # Starting main process.
timeStarted = time.time() # Save start time.
cmdTimer = "sleep "+str(timeoutInSeconds) # Waiting for timeout...
cmdKill = "kill "+str(proc.pid)+" 2>/dev/null" # And killing process.
cmdTimeout = cmdTimer+" && "+cmdKill # Combine commands above.
procTimeout = subprocess.Popen(cmdTimeout,shell=True) # Start timeout process.
proc.communicate() # Process is finished.
timeDelta = time.time() - timeStarted # Get execution time.
print("Finished process in "+str(timeDelta)+" seconds.") # Output result.
I have a python program which should execute a command line (command line is a psexec command to call a batch file on the remote server)
I used popen to call the command line. The batch on the remote server produces a return code of 0.
Now I have to wait for this return code and on the basis of the return code I should continue my program execution.
I tried to use .wait() or .check_output() but for some reason did not work for me.
cmd = """psexec -u CORPORATE\user1 -p force \\\sgeinteg27 -s cmd /c "C:\\Planview\\Interfaces\\ProjectPlace_Sree\\PP_Run.bat" """
p = subprocess.Popen(cmd, bufsize=2048, shell=True,
stdin=subprocess.PIPE, stdout=subprocess.PIPE)
p.wait()
print(p.returncode)
##The below block should wait until the above command runs completely.
##And depending on the return code being ZERO i should continue the rest of
##the execution.
if p.returncode ==0:
result = tr.test_readXid.readQuery(cert,planning_code)
print("This is printed depending if the return code is zero")
Here is the EOF the batch file execution and the return code
Can anybody help me with this ?
When controlling a process using a PTY master/slave pair, I would like to indicate to the process in question that stdin has closed and I have no more content to send, but I would still like to receive output from the process.
The catch is that I only have one file descriptor (the PTY "master") which handles both input from the child process and output to the child process. So closing the descriptor would close both.
Example in python:
import subprocess, pty, os
master,slave = pty.openpty()
proc = subprocess.Popen(["/bin/cat"], stdin=slave, stdout=slave)
os.close(slave) # now belongs to child process
os.write(master,"foo")
magic_close_fn(master) # <--- THIS is what I want
while True:
out = os.read(master,4096)
if out:
print out
else:
break
proc.wait()
You need to get separate read and write file descriptors. The simple way to do that is with a pipe and a PTY. So now your code would look like this:
import subprocess, pty, os
master, slave = pty.openpty()
child_stdin, parent_stdin = os.pipe()
proc = subprocess.Popen(["/bin/cat"], stdin=child_stdin, stdout=slave)
os.close(child_stdin) # now belongs to child process
os.close(slave)
os.write(parent_stdin,"foo") #Write to the write end (our end) of the child's stdin
#Here's the "magic" close function
os.close(parent_stdin)
while True:
out = os.read(master,4096)
if out:
print out
else:
break
proc.wait()
I had to do this today, ended up here and was sad to see no answer. I achieved this using a pair of ptys rather than a single pty.
stdin_master, stdin_slave = os.openpty()
stdout_master, stdout_slave = os.openpty()
def child_setup():
os.close(stdin_master) # only the parent needs this
os.close(stdout_master) # only the parent needs this
with subprocess.Popen(cmd,
start_new_session=True,
stderr=subprocess.PIPE,
stdin=stdin_slave,
stdout=stdout_slave,
preexec_fn=child_setup) as proc:
os.close(stdin_slave) # only the child needs this
os.close(stdout_slave) # only the child needs this
stdin_pty = io.FileIO(stdin_master, "w")
stdout_pty = io.FileIO(stdout_master, "r")
stdin_pty.write(b"here is your input\r")
stdin_pty.close() # no more input (EOF)
output = b""
while True:
try:
output += stdout_pty.read(1)
except OSError:
# EOF
break
stdout_pty.close()
I think that what you want is to send the CTRL-D (EOT - End Of Transmission) caracter, isn't you? This will close the input in some applications, but others will quit.
perl -e 'print qq,\cD,'
or purely shell:
echo -e '\x04' | nc localhost 8080
Both are just examples. BTW the CTRL-D caracter is \x04 in hexa.