What I am trying to do
I have a front end system that is generating output. I am accessing this data(JSON) with a post request using bottle. My post receives the json without issue. I need to execute a backend python program(blender automation) and pass this JSON data to that program.
What I have tried to do
Subprocess - Using subprocess call the program and pass the input. In appearance seems to execute but when i check System Monitor the program is not starting but my server continues to run as it should. This subprocess command runs perfectly fine when executed independently from the server.
blender, script, and json are all string objects with absolute file paths
sub = subprocess.Popen([blender + " -b -P " + script + " -- " + json], stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)
C Style os.fork() - Same as above which i found reading pydoc that subprocess operates using these methods
Double Fork - From a posting on here tried forking from the server and then calling subprocess from that fork and terminating the parent of the subprocess to create an orphan. My subprocess command still does not execute and never shows up in System Monitor.
What I need
I need a solution that will run from the bottle server in its own process. It will handle multiple requests so the subprocess cannot block in the server. The process being called is fully automated and just requires sending the JSON data in the execution command. The result of the subprocess program will be string path to a file created on the server.
The above subprocess works perfectly fine when called from my test driver program. I just need to connect the execution to the webservice so my front end can trigger its execution.
My bottle post method - prints json when called without issue.
#post('/getData')
def getData():
json_text = request.json
print(json_text)
I am not sure where to go from here. From what i have read thus far, subprocess should work. Any help or suggestions would be very much appreciated. If additional information is needed please let me know. I will edit with more details. Thank you.
Relevant Information:
OS: Ubuntu 16.04 LTS,
Python 3.x
*EDIT
This isn't an elegant solution but my subprocess call works now.
cmd = blender
cmd += " -b -P "
cmd += script
cmd += " -- "
cmd += str(json)
sub = subprocess.Popen([cmd], shell=True)
It seems by setting shell=True and removing the stdout, stderr=PIPE allowed me to see output where i was throwing an unhandled exception because my json data was a list and not a string.
When using python for executing your scripts a process created by Popen.subprocess will unintentionally inherited and keeps open a file descriptor.
You need to close that so that process can run independently. (close_fds=True)
subprocess.Popen(['python', "-u", Constant.WEBAPPS_FOLDER + 'convert_file.py', src, username], shell=False, bufsize=-1, close_fds=True)
Alsso, u dont have to use shell for creating another process. It might have unintended consequences.
I had the exact same problem where bottle was not returning/hangs. It works now.
Related
A subprocess is being created that requires use of the internal cmd.exe command start, so a shell is required.
>>> p = subprocess.Popen(['start', 'MCC', '/wait'], shell=True)
The 'MCC' takes place of the title parameter of the start command, and tells the Windows Console Host to load the settings for the console from the registry at HKCU\Console\MCC to open the customized console. Once opened, using the Popen interface, the console will remain after a call to terminate:
>>> p.poll()
>>> p.terminate()
>>> p.poll()
1
My initial diagnosis is that the call to terminate is terminating the subprocess correctly, but its "invisible", and the console that is showing is another process created by start that is not directly accessible from the variable p. I have been successful in terminating the program and closing the console but by running a secondary subprocess:
>>> pp = subprocess.Popen(['taskkill', '/f', '/pid', f'{p.pid}', '/t'])
While it works and is documented, ideally I'd like to avoid invoking another process that could yield zombie issues; thereby leaving me wondering how to terminate a subprocess p and any children it may have created.
For reference I have reviewed the following:
How to terminate a python subprocess launched with shell=True: Linux resolution, gave me the taskkill idea
Environment details:
Windows 10
CPython 3.6.0
I have a requirement of executing a process that has to run with different arguments for every run and I am using the python subprocess to launch the application in a server. In order for the subprocess cmd to work I need to first make a connection to the application and then change the input parameters every time:
cmd = 'path of an Executable of an application' + 'arguments'
out = subprocess.Popen(cmd, bufsize=1, stdin=subprocess.PIPE, stdout=subprocess.PIPE, shell=True)
Given this now I don't want to make a connection to the internal application every time. So is there an option to make the subprocess thread persistent and send the new application command to the existing thread?
Any help is appreciated!!!
Thanks in Advance!!!
So, I have a python script, that I can run in cmd using python [path to script]
I have it set in scheduler, but it doesn't run through scheduler and finds an error. Cmd closes out before being able to read the error. I created a batch file to launch to script, and it shows an error that a package doesn't exist [lxml]. But, the package exists as the script will run when manually executed
Any thoughts?
Script scrapes data from a website, creates a dataframe, posts dataframe to a google sheet, then pulls the full google sheet that it posts to, turns that into a dataframe with all of the data, then creates a plotly graph, turns that plotly into an html file, then sends the html file to a SFTP server
Figured it out...
import sys
import platform
import imp
print("Python EXE : " + sys.executable)
print("Architecture : " + platform.architecture()[0])
#print("Path to arcpy : " + imp.find_module("arcpy")[1])
#raw_input("\n\nPress ENTER to quit")
Run this to get the proper path to your Python.exe, and place this in Program/Script.
Then you need to verify that EVERY directory in your script starts at C:/ and works its way through the full path.
I am building a python tool to update an application. To do so i have to stop the apache service, do some update related stuff and then start it again, after the update ist finished.
Im currently using python 3.7.2 on Windows 10.
I have tried to somehow build a working process using these questions as a reference:
Run process as admin with subprocess.run in python
Windows can't find the file on subprocess.call()
Python subprocess call returns "command not found", Terminal executes correctly
def stopApache():
processName = config["apacheProcess"]
#stopstr = f'stop-service "{processName}"'
# the line abpove should be used once im finished, but for testing
# purposes im going with the one below
stopstr = 'stop-service "Apache2.4(x64)"'
print(stopstr)
try:
subprocess.run(stopstr, shell=True)
#subprocess.run(stopstr)
# the commented line here is for explanatory purposes, and also to
# show where i started.
except:
print('subprocess failed', sys.exc_info())
stopExecution()
From what i have gathered so far, the shell=TRUE option, is a must, since python does not check PATH.
Given the nature of what im trying to do, i expected the service to get stoppped. In reality the console error looks like this :
stopstr = get-service "Apache2.4(x64)"
Der Befehl "get-service" ist entweder falsch geschrieben oder
konnte nicht gefunden werden.
Which roughly translates to : the command "stop-service" is either incorrectly spelled or could not be found.
If i run the same command directly in powershell i get a similar error. If i open the shell as admin, and run i again, everything works fine.
But, if i use the very same admin shell to run the python code, i am back to square one.
What am i missing here, aparently there is some issue with permissions but i cannot wrap my head arround it
The command for stopping a service in MS Windows is net stop [servicename]
So change stopstr to 'net stop "Apache2.4(x64)"'
You will need to run your script as admin.
shell=True runs commands against cmd, not powershell.
Powershell is different to the regular command line. So to get stop-service to work you'd have to pass it to an instance of powershell.
stopstr = 'stop-service \\"Apache2.4(x64)\\"'
subprocess.run(['powershell', stopstr], shell=True)
As you noted in a comment, it's neccessary to escape the " around the service name, as first python will unescape them, then powershell will use them.
I have a Python 3 script which, among other things, launches Chrome with certain command-line parameters. The relevant portion of the code looks like this:
import multiprocessing as mp
from subprocess import call
import time
import logging
def launch_tab(datadir, url):
# Constructs the command line and launches Chrome via subprocess.call()
def open_browser(urls):
'''Arranges for Chrome to be launched in a subprocess with the specified
URLs, using the already-configured profile directory.
urls: a list of URLs to be loaded one at a time.'''
first_run = True
for tab in urls:
logging.debug('open_browser: {}'.format(tab))
proc = mp.Process(name=tab, target=launch_tab, args=(config.chromedir, tab))
proc.start()
if first_run:
first_run = False
time.sleep(10)
else:
time.sleep(0.5)
What Happens
When I run the script with Chrome already running as launched by the script:
Chrome launches as expected, sees that it is already running, follows the instructions provided on the command line, then terminates the process started on the command line.
Since the process is now terminated, my script also terminates. This is the behavior I want.
When I run the script while Chrome is not running:
Chrome sees that it is not already running, and so doesn't terminate the process started by the command line.
Because the process hasn't terminated, my script doesn't exit until Chrome does, despite its having nothing to do. I have to remember to place the script in the background.
What I Want
I want the subprocess which launches Chrome to be completely independent of the main process, such that I get my command prompt back immediately after Chrome is launched. After launching Chrome, my script has completed its job. I thought of using the daemonize module, but it apparently doesn't work with Python 3.
I'm not wedded to the multiprocessing module. Any reasonable approach which produces the desired end result is acceptable. Switching to Python 2 so I can try daemonize would be too difficult.
The script will only ever run on Linux, so portability isn't an issue.