Call to Pythons mutliprocessing.Process.join() fails - python-3.x

Question

When I run the code below it occasionally fails
time_start = time.time()
job = multiprocessing.Process(target=load_cpu, args=(deadline, ))
job.start() # This is line 37 in the source code linked below
# timeout=None in the call to join() solves the problem
job.join(deadline)
elapsed = time.time()-time_start
if elapsed < deadline and job.is_alive():
# I am getting here from time to time
logger.error(f"#{job_counter}: job.join() returned while process {job.pid} is still alive elapsed={elapsed} deadline={deadline}")
The Python 3.7 container (Docker) demonstrating the problem is here https://github.com/larytet-py/multiprocess
If I run the code for a few minutes on a 4 cores Ubuntu 18.04 host I get
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "main.py", line 37, in spawn_job
job.start()
File "/usr/lib/python3.7/multiprocessing/process.py", line 111, in start
_cleanup()
File "/usr/lib/python3.7/multiprocessing/process.py", line 56, in _cleanup
if p._popen.poll() is not None:
AttributeError: 'NoneType' object has no attribute 'poll'
What am I doing wrong?
My workaround is to replace the call to job.join() with a polling and check is_alive(). Unfortunately this approach impacts latency. Are there better alternatives?
def join_process(job, timeout):
time_start = time.time()
# Typical processing time is 100ms I want to reduce latency impact
# 10ms looks ok.
# TODO I can end up in a tight loop here.
polling_time = min(0.1*timeout, 0.010)
while time.time()-time_start < timeout and job.is_alive():
time.sleep(polling_time)
continue
Update. I tried multiprocessing.Event() instead of Process.join() The code fails with the same exception
Update2. I have reproduced the problem in the code not calling Process.join() at all. It require more time and more load, but eventually Process.start() crashes.
Update3. https://bugs.python.org/issue40860 is accepted? I am still looking for a workaround.

Answer 1

Synchronizing the call to Process.start() helps. This is a fair workaround. There no other answers. I am accepting my own answer.
diff --git a/main.py b/main.py
index d09dc53..49d68f0 100644
--- a/main.py
+++ b/main.py
## -26,17 +26,24 ## def load_cpu(deadline):
while time.time() - start < 0.2*deadline:
math.pow(random.randint(0, 1), random.randint(0, 1))
+def join_process(job, timeout):
+ time_start = time.time()
+ while time.time()-time_start < timeout and job.is_alive():
+ time.sleep(0.1 * timeout)
+ continue
+
job_counter = 0
+lock = threading.Lock()
def spawn_job(deadline):
'''
Creat a new Process, call join(), process errors
'''
global job_counter
time_start = time.time()
- job = multiprocessing.Process(target=load_cpu, args=(deadline, ))
- job.start()
- # timeout=None in the call to join() solves the problem
- job.join(deadline)
+ with lock:
+ job = multiprocessing.Process(target=load_cpu, args=(deadline, ))
+ job.start()
+ join_process(job, deadline)
My final version uses os.fork(). I drop multiprocessing altogether. The multiprocessing is not thread safe (I am kidding not) https://gist.github.com/larytet/3ca9f9a32b1dc089a24cb7011455141f