I am new to python and programming and I am trying to understand this code. I have spent the past few hours reading documentation and watching videos on subprocessing but I am still confused (I added snidbits of information of what I found online to comment the code as best I could).
Here are some questions I have pertaining to the code below:
when is subprocess used?
when should I use Popen verses the more convenient handles with subprocess?
what does PIPE do?
what does close_fds do?
basically I need this line of code explained
my_process=Popen(['player',my_video_File_path], stdin=PIPE, close_fds=True)
full code here:
#run UNIX commands we need to create a subprocess
from subprocess import Popen, PIPE
import os
import time
import RPi.GPIO as GPIO
my_video_file_path='/home/pi/green1.mp4'
#stdin listens for information
# PIPE connnects the stdin with stdout
#pipe, (like a pipe sending info through a tunnel from one place to another )
#STDIN (channel 0):
#Where your command draws the input from. If you don’t specify anything special this will be your keyboard input.
#STDOUT (channel 1):
#Where your command’s output is sent to. If you don’t specify anything special the output is displayed in your shell.
#to send data to the process’s stdin, you need to create the Popen object with stdin=PIPE.
#Popen interface can be used directly with subprocess
# with pipe The return value is an open file object connected to the pipe, which can be read or written depending on whether mode is 'r' (default) or 'w'.
#If we pass everything as a string, then our command is passed to the shell;
#"On Unix, if args is a string, the string is interpreted as the name or path of the program to execute. "
my_process=Popen(['player',my_video_File_path], stdin=PIPE, close_fds=True)
GPIO.setmode(GPIO.BCM)
GPIO.setup(17,GPIO.IN,pull_up_down=GPIO.PUD_UP)
GPIO.setup(22,GPIO.IN,pull_up_down=GPIO.PUD_UP)
while True:
button_state=GPIO.input(17)
button_state1=GPIO.input(22)
if button_state==False:
print("quite video")
my_process.stdin.write("q")
time.sleep(.09)
if button_state1==False:
print("full video")
my_process.stdin.write("fs")
time.sleep(5)
Regarding the difference between subprocess and Popen, here is a line from the python 3.5 docs:
For more advanced use cases, the underlying Popen interface can be used directly. (compared to using subprocess)
So, this means that subprocess is a wrapper on Popen.
PIPE is used so that your python script communicate with the subprocess via standard IO operations (you can think of print in python as a type of stdout).
So, when you do my_process.stdin.write("fs") you are sending this text (or piping it) to the standard input of your subprocess. Then your subprocess reads this text and does whatever processing it needs to do.
To further understand subprocessing, try to read standard input into a python program to see how it works. You can follow this How do you read from stdin in Python? question to work on this exercise.
Also, it might be worthwhile to learn about piping in the more general linux style. Try to read through this article.
Related
My case is a little bit specific. I'm trying to run a Python program using Python for testing purposes. The case is as follows:
# file1.py
print("Hello world")
# file1.test.py
import io
import sys
import os
import unittest
EXPECTED_OUTPUT = "Hello world"
class TestHello(unittest.TestCase):
def test_hello(self):
sio = io.StringIO()
sys.stdout = sio
os.system("python3 path/to/file1.py")
sys.stdout = sys.__stdout__
print("captured value:", sio.getvalue())
self.assertEqual(sio.getvalue(), EXPECTED_STDOUT)
if __name__ == "__main__":
unittest.main()
But nothing ends up in the sio variable. This way and similar ways are introduced online but they don't seem to work for me. My Python version is 3.8.10 but it doesn't really matter if this works better in some other version, I can switch to that.
Note: I know that if I was using an importable object this might be easier, but right now I need to know how to catch the output of another file.
Thanks!
stdout redirection does not work like this - this will change the stdout variable inside your Python process. But by using os.system, you are running another process, that will re-use the same terminal pseudo-files your parent process is using.
If you want to log a subprocess, the way to do it is to use the subprocess modules calls, which allow you to redirect the subprocess output. https://docs.python.org/3/library/subprocess.html
Also, the subprocess won't be able to use a StringIO object from the parent process (it is not an O.S. level object, just an in-process Python object with a write method). The docs above include instructions about using the special object subprocess.PIPE which allows for in-memory communication, or, you can just pass an ordinary filesystem file, which you can read afterwards.
I'm developing a Python code that can run two applications and exchange information between them during their run time.
The basic scheme is something like:
start a subprocess with the 1st application
start a subprocess with the 2nd application
1st application performs some calculation, writes a file A, and waits for input
2nd application reads file A, performs some calculation, writes a file B, and waits for input
1st application reads file B, performs some calculation, writes a file C, and waits for input
...and so on until some condition is met
I know how to start one Python subprocess, and now I'm learning how to pass/receive information during run time.
I'm testing my Python code using a super-simple application that just reads a file, makes a plot, closes the plot, and returns 0.
I was able to pass an input to a subprocess using subprocess.communicate() and I could tell that the subprocess used that information (plot opens and closes), but here the problems started.
I can only send an input string once. After the first subprocess.communicate() in my code below, the subprocess hangs there. I suspect I might have to use subprocess.stdin.write() instead, since I read subprocess.communicate() will wait for the end of the file and I wish to send multiple times different inputs during the application run instead. But I also read that the use of stdin.write() and stdout.read() is discouraged. I tried this second alteranative (see #alternative in the code below), but in this case the application doesn't seem to receive the inputs, i.e. it doesn't do anything and the code ends.
Debugging is complicated because I haven't found a neat way to output what the subprocess is receiving as input and giving as output. (I tried to implement the solutions described here, but I must have done something wrong: Python: How to read stdout of subprocess in a nonblocking way, A non-blocking read on a subprocess.PIPE in Python)
Here is my working example. Any help is appreciated!
import os
import subprocess
from subprocess import PIPE
# Set application name
app_folder = 'my_folder_path'
full_name_app = os.path.join(app_folder, 'test_subprocess.exe')
# Start process
out_app = subprocess.Popen([full_name_app], stdin=PIPE, stdout=PIPE)
# Pass argument to process
N = 5
for n in range(N):
str_to_communicate = f'{{\'test_{n+1}.mat\', {{\'t\', \'y\'}}}}' # funny looking string - but this how it needs to be passed
bytes_to_communicate = str_to_communicate.encode()
output_communication = out_app.communicate(bytes_to_communicate)
# output_communication = out_app.stdin.write(bytes_to_communicate) # alternative
print(f'Communication command #{n+1} sent')
# Terminate process
out_app.terminate()
I have a command-line program which when triggered provides a list of options and prompts the user to select an option to continue.
Now I have launched the command line program using subprocess call, now the command line program prompts for the value, how do I pass the value from user to the command line program via subprocess call?
subprocess.call just waits for the process to finish and gives return code, no way to interact with it. if you instead use subprocess.Popen that gives you the ability to communicate with the subprocess while it is running via stdin and stdout
import subprocess, sys
program = subprocess.Popen("python3",
# give us a pipes to coommunicate
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
data = input("input to subprocess")
[out,err] = program.communicate((data+"\n").encode())
print(out.decode())
print(err.decode(), file=sys.stderr)
Doing a bit of input then some output then more input can get messy though since there reading from stdout is blocking so determining when the output has stopped for more input is tricky.
I am currently working on a project that contains many system tray utilities on mac osx, one of these utilities is a 'fake friend soundboard' where I am to enable the user to click on the module then select as example "Discord Ping x1" and the program playback a soundfile of the ping.
Im using RUMPS which enables me to add menus to the system tray however when trying to use libraries to play sounds directly through upon the #rumps.clicked, nothing seems to happen
Instead I've programmed it to run another script that can easily play the sound inside another folder containing all the sounds.
When running the script and clicking a sound, it plays the sound exactly as intended,
but then the program stops responding...
Is this something I'm doing wrong with my code or is there another way i could play the sound using this library?
Heres the code:
import rumps
import subprocess
class sound(rumps.App):
def __init__(self):
super(sound, self).__init__("🔊")
self.menu = ["Fake friend soundboard",
None,
"Discord Sounds:",
"Ping x1",
"Incoming Call",
"Enter Call",
"Leave Call",
"Mute",
"Deafen",
None,
"Skype Sounds:",
"Incoming Call",
None,
"Random:",
"okbuddyretard",
None]
#rumps.clicked("Ping x1")
def about(sender):
subprocess.run("python3 soundboard/discord_pingx1.py", shell=True)
if __name__ == "__main__":
sound().run()
In the soundboard folder there is the sounds folder containing the discord folder containing **
discord_ping.wav
The soundboard folder also hold the discordd_pingx1.py script which is below.
import pyglet
def sound():
sound = pyglet.resource.media('sounds/discord/discord_ping.wav', streaming=False)
sound.play()
pyglet.app.run()
sound()
after running my main.py (opens all the modules)
and clicking the sound tab and then clicking the button set to make the sound,
it plays, but then stops responding.
Any help at all would be much appreciated.
I highly recommend avoiding the insert of subprocess or sys-calls inside of a Python script that executes Objective-C code. This leads to Aborts and Traps. Instead, refer to Apple's APIs and utilize AppleScript (osascript).
Here is a better implementation that calls a script to call your script from osascript:
exec.py
import platform
import subprocess, sys, os
applescript = '''\
do shell script "bash /path/to/file/myShellScript.sh"\
'''
# parse and stdout
args = [item for x in [("-e",l.strip()) for l in applescript.split('\n') if l.strip() != ''] for item in x]
proc = subprocess.Popen(["osascript"] + args ,stdout=subprocess.PIPE )
progname = proc.stdout.read().strip()
sys.stdout.write(str(progname))
The parser takes the string with the osascript in it and parses it for a stdout write. Stdout writes are very safe, considering they are handling data from the same thread.
/path/to/file/myShellScript.sh (add the shebang on line 1 as well)
#!/bin/bash
python3 soundboard/discord_pingx1.py
This 100% handles your problem without causing subprocess errors. If you get hung with subprocess, your computer will continue running Python3. If the Application doesn't stop freezing, exit the app by cmd+Space and typing Activity Monitor. Then, find Python3 by cmd+f and typing python3. Click it and press Quit in the top left corner (the X or stop-light symbol).
I am trying to write a shell that needs to run socket connections on a seperate thread. On my testings, when print() is used while cmd.Cmd.cmdloop() waiting for input, the print is displaying wrong.
from core.shell import Shell
import time
import threading
def test(shell):
time.sleep(2)
shell.write('Doing test')
if __name__ == '__main__':
shell = Shell(None, None)
testThrd = threading.Thread(target=test, args=(shell,))
testThrd.start()
shell.cmdloop()
When the above command runs, here is what happens:
python test.py
Welcome to Test shell. Type help or ? to list commands.
>>asd
*** Unknown syntax: asd
>>[17:59:25] Doing test
As you can see, printing from another threads add output after prompt >> not in a new line. How can I do it so that it appears in a new line and prompt appears?
What you can do, is redirect stdout from your core.shell.Shell to a file like object such as StringIO. You would also redirect the output from your thread into a different file like object.
Now, you can have some third thread read both of these objects and print them out in whatever fashion you want.
You said core.shell.Shell inherits from cmd.Cmd, which allows redirection as a parameter to the constructor:
import io
import time
import threading
from core.shell import Shell
def test(output_obj):
time.sleep(2)
print('Doing test', file=output_obj)
cmd_output = io.StringIO()
thr_output = io.StringIO()
shell = Shell(stdout=cmd_output)
testThrd = threading.Thread(target=test, args=(thr_output,))
testThrd.start()
# in some other process/thread
cmd_line = cmd_output.readline()
thr_line = thr_output.readline()
That's quite difficult. Both your threads are sharing the same stdout. So the output from each of those threads are concurrently sent to your stdout buffer where they are printed in some arbitrary order.
What you need to do is coordinate the output from both threads, and that's a tough nut to crack. Even bash doesn't do that!
That said, maybe you can try using a lock to make sure your threads access stdout in a controlled manner. Check out: http://effbot.org/zone/thread-synchronization.htm