I have a requirement of executing a process that has to run with different arguments for every run and I am using the python subprocess to launch the application in a server. In order for the subprocess cmd to work I need to first make a connection to the application and then change the input parameters every time:
cmd = 'path of an Executable of an application' + 'arguments'
out = subprocess.Popen(cmd, bufsize=1, stdin=subprocess.PIPE, stdout=subprocess.PIPE, shell=True)
Given this now I don't want to make a connection to the internal application every time. So is there an option to make the subprocess thread persistent and send the new application command to the existing thread?
Any help is appreciated!!!
Thanks in Advance!!!
Related
I have a nodejs program running as background process with nssm (I build an executable and then have it be managed by nssm).
My program needs to open a desktop app when the user requests it, the program is runned fine but the problem is that it is runned as a background process as well !
I want the program to be opened on the desktop because the user has to interact with it.
Here is the code running the program ( I tried running the exec directly as well, to no avail)
const batchfile = `
start "PROGRAM" "${this.EXEC_PROGRAM_PATH + '\\' + this.EXEC_NAME}" ${this.ARGS.join(' ')}
`
fs.writeFileSync('launch.bat', batchfile);
// execute the batch file
const program = child.execFile('launch.bat');
If i run the nodejs program directly (not through nssm) it works, but I need to program to run in the background as it needs to be ready at startup.
Thank you for your help
A subprocess is being created that requires use of the internal cmd.exe command start, so a shell is required.
>>> p = subprocess.Popen(['start', 'MCC', '/wait'], shell=True)
The 'MCC' takes place of the title parameter of the start command, and tells the Windows Console Host to load the settings for the console from the registry at HKCU\Console\MCC to open the customized console. Once opened, using the Popen interface, the console will remain after a call to terminate:
>>> p.poll()
>>> p.terminate()
>>> p.poll()
1
My initial diagnosis is that the call to terminate is terminating the subprocess correctly, but its "invisible", and the console that is showing is another process created by start that is not directly accessible from the variable p. I have been successful in terminating the program and closing the console but by running a secondary subprocess:
>>> pp = subprocess.Popen(['taskkill', '/f', '/pid', f'{p.pid}', '/t'])
While it works and is documented, ideally I'd like to avoid invoking another process that could yield zombie issues; thereby leaving me wondering how to terminate a subprocess p and any children it may have created.
For reference I have reviewed the following:
How to terminate a python subprocess launched with shell=True: Linux resolution, gave me the taskkill idea
Environment details:
Windows 10
CPython 3.6.0
Is it possible to run dask from a python script?
In interactive session I can just write
from dask.distributed import Client
client = Client()
as described in all tutorials. If I write these lines however in a script.py file and execute it python script.py, it immediately crashes.
I found another option I found, is to use MPI:
# script.py
from dask_mpi import initialize
initialize()
from dask.distributed import Client
client = Client() # Connect this local process to remote workers
And then run the script with mpirun -n 4 python script.py. This doesn't crash, however if you print the client
print(client)
# <Client: scheduler='tcp://137.250.37.84:35145' processes=0 cores=0>
you see that no cores are used, accordingly scripts run forever without doing anything.
How do I set my scripts up correctly?
If you want to create processes from within a Python script you need to protect that code in an if __name__ == "__main__": block
from dask.distributed import Client
if __name__ == "__main__":
client = Client()
If you want to use dask-mpi then you need to run it with mpirun or mpiexec with a suitable number of processes.
What I am trying to do
I have a front end system that is generating output. I am accessing this data(JSON) with a post request using bottle. My post receives the json without issue. I need to execute a backend python program(blender automation) and pass this JSON data to that program.
What I have tried to do
Subprocess - Using subprocess call the program and pass the input. In appearance seems to execute but when i check System Monitor the program is not starting but my server continues to run as it should. This subprocess command runs perfectly fine when executed independently from the server.
blender, script, and json are all string objects with absolute file paths
sub = subprocess.Popen([blender + " -b -P " + script + " -- " + json], stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)
C Style os.fork() - Same as above which i found reading pydoc that subprocess operates using these methods
Double Fork - From a posting on here tried forking from the server and then calling subprocess from that fork and terminating the parent of the subprocess to create an orphan. My subprocess command still does not execute and never shows up in System Monitor.
What I need
I need a solution that will run from the bottle server in its own process. It will handle multiple requests so the subprocess cannot block in the server. The process being called is fully automated and just requires sending the JSON data in the execution command. The result of the subprocess program will be string path to a file created on the server.
The above subprocess works perfectly fine when called from my test driver program. I just need to connect the execution to the webservice so my front end can trigger its execution.
My bottle post method - prints json when called without issue.
#post('/getData')
def getData():
json_text = request.json
print(json_text)
I am not sure where to go from here. From what i have read thus far, subprocess should work. Any help or suggestions would be very much appreciated. If additional information is needed please let me know. I will edit with more details. Thank you.
Relevant Information:
OS: Ubuntu 16.04 LTS,
Python 3.x
*EDIT
This isn't an elegant solution but my subprocess call works now.
cmd = blender
cmd += " -b -P "
cmd += script
cmd += " -- "
cmd += str(json)
sub = subprocess.Popen([cmd], shell=True)
It seems by setting shell=True and removing the stdout, stderr=PIPE allowed me to see output where i was throwing an unhandled exception because my json data was a list and not a string.
When using python for executing your scripts a process created by Popen.subprocess will unintentionally inherited and keeps open a file descriptor.
You need to close that so that process can run independently. (close_fds=True)
subprocess.Popen(['python', "-u", Constant.WEBAPPS_FOLDER + 'convert_file.py', src, username], shell=False, bufsize=-1, close_fds=True)
Alsso, u dont have to use shell for creating another process. It might have unintended consequences.
I had the exact same problem where bottle was not returning/hangs. It works now.
I have a Python 3 script which, among other things, launches Chrome with certain command-line parameters. The relevant portion of the code looks like this:
import multiprocessing as mp
from subprocess import call
import time
import logging
def launch_tab(datadir, url):
# Constructs the command line and launches Chrome via subprocess.call()
def open_browser(urls):
'''Arranges for Chrome to be launched in a subprocess with the specified
URLs, using the already-configured profile directory.
urls: a list of URLs to be loaded one at a time.'''
first_run = True
for tab in urls:
logging.debug('open_browser: {}'.format(tab))
proc = mp.Process(name=tab, target=launch_tab, args=(config.chromedir, tab))
proc.start()
if first_run:
first_run = False
time.sleep(10)
else:
time.sleep(0.5)
What Happens
When I run the script with Chrome already running as launched by the script:
Chrome launches as expected, sees that it is already running, follows the instructions provided on the command line, then terminates the process started on the command line.
Since the process is now terminated, my script also terminates. This is the behavior I want.
When I run the script while Chrome is not running:
Chrome sees that it is not already running, and so doesn't terminate the process started by the command line.
Because the process hasn't terminated, my script doesn't exit until Chrome does, despite its having nothing to do. I have to remember to place the script in the background.
What I Want
I want the subprocess which launches Chrome to be completely independent of the main process, such that I get my command prompt back immediately after Chrome is launched. After launching Chrome, my script has completed its job. I thought of using the daemonize module, but it apparently doesn't work with Python 3.
I'm not wedded to the multiprocessing module. Any reasonable approach which produces the desired end result is acceptable. Switching to Python 2 so I can try daemonize would be too difficult.
The script will only ever run on Linux, so portability isn't an issue.