I'm pretty new to asyncio, correct me if my terminology is wrong.
Suppose I have a CPU-bound task that runs a for loop.
I have another task that pings every 1sec.
If we initiate both, the CPU-bound task will block the pinging task.
How do split the CPU-bound task so as it doesn't block the pinging task.
Currently I only have a dumb way of doing this, which is to divide the CPU-bound task into 1000 pieces and sleep 1sec after finishing every 1 piece.
Any better solutions to this?
Thanks
import time
import asyncio
async def ping():
print(f'ping - {time.time()}')
while True:
await asyncio.sleep(1)
print(f'ping - {time.time()}')
async def process():
result = 0.0
print(f'process - {time.time()}')
for i in range(100000000):
result += i * i
print(f'process - {time.time()}')
return result
async def run():
task1 = asyncio.create_task(ping())
task2 = asyncio.create_task(process())
await asyncio.gather(task1, task2)
asyncio.run(run())
The output of the above code follows
ping - 1659977867.4815755
process - 1659977867.4816022
process - 1659977874.5827787
ping - 1659977874.5828712
ping - 1659977875.5845447
ping - 1659977876.5862029
ping - 1659977877.5878963
ping - 1659977878.5894375
Related
I am running FastAPI app with gunicorn with the following config:
bind = 0.0.0.0:8080
worker_class = "uvicorn.workers.UvicornWorker"
workers = 3
loglevel = ServerConfig.LOG_LEVEL.lower()
max_requests = 1500
max_requests_jitter = 300
timeout = 120
Inside this app, I am doing some task (not very long running) every 0.5 seconds (through a Job Scheduler) and doing some processing on the data.
In that Job scheduler, I am calling "perform" method (See code below):
class BaseQueueConsumer:
def __init__(self, threads: int):
self._threads = threads
self._executor = ThreadPoolExecutor(max_workers=1)
def perform(self, param1, param2, param3) -> None:
futures = []
for _ in range(self._threads):
futures.append(
self._executor.submit(
BaseQueueConsumer.consume, param1, param2, param3
)
)
for future in futures:
future.done()
#staticmethod
def consume(param1, param2, param3) -> None:
# Doing some work here
The problem is, whenever this app is under a high load, I am getting the following error:
cannot schedule new futures after shutdown
My guess is that the gunicorn process restarts every 1500 requests (max_requests) and the tasks that are already submitted are causing this issue.
What I am not able to understand is that whatever thread gunicorn process starts due to threadpoolexecutor should also end when the process is terminated but that is not the case.
Can someone help me explain this behaviour and a possible solution for gracefully ending the gunicorn process without these threadpoolexecutor tasks causing errors?
I am using python 3.8 and gunicorn 0.15.0
The example I am running is mentioned in this PyMOTW3 link. I am reproducing the code here:
from concurrent import futures
import os
def task(n):
return (n, os.getpid())
ex = futures.ProcessPoolExecutor(max_workers=2)
results = ex.map(task, range(5, 0, -1))
for n, pid in results:
print('ran task {} in process {}'.format(n, pid))
As per source, I am supposed to get following output:
ran task 5 in process 40854
ran task 4 in process 40854
ran task 3 in process 40854
ran task 2 in process 40854
ran task 1 in process 40854
Instead, I'm getting a long message with following concluding line -
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
I am using Windows machine and running Python 9. All other examples are otherwise running fine. What is going wrong here?
I've finally been able to resolve the issue. The issue seems to be Windows specific. Following a related Stack Overflow post, I used if __name__=="__main__" idiom. The modified code is:
from concurrent import futures
import os
def task(n):
return (n, os.getpid())
def main():
ex = futures.ProcessPoolExecutor(max_workers=2)
results = ex.map(task, range(5, 0, -1))
for n, pid in results:
print('ran task {} in process {}'.format(n, pid))
if __name__ == '__main__':
main()
It worked, although I'm still not sure why this worked.
I am writing a job scheduler where I schedule M jobs across N co-routines (N < M). As soon as one job finishes, I add a new job so that it can start immediately and run in parallel with the other jobs. Additionally, I would like to ensure that no single job takes more than a certain fixed amount of time. Any jobs that take too long should be cancelled. I have something pretty close, like this:
def update_run_set(waiting, running, max_concurrency):
number_to_add = min(len(waiting), max_concurrency - len(running))
for i in range(0, number_to_add):
next_one = waiting.pop()
running.add(next_one)
async def _run_test_invocations_asynchronously(jobs:List[MyJob], max_concurrency:int, timeout_seconds:int):
running = set() # These tasks are actively being run
waiting = set() # These tasks have not yet started
waiting = {_run_job_coroutine(job) for job in jobs}
update_run_set(waiting, running, max_concurrency)
while len(running) > 0:
done, running = await asyncio.wait(running, timeout=timeout_seconds,
return_when=asyncio.FIRST_COMPLETED)
if not done:
timeout_count = len(running)
[r.cancel() for r in running] # Start cancelling the timed out jobs
done, running = await asyncio.wait(running) # Wait for cancellation to finish
assert(len(done) == timeout_count)
assert(len(running) == 0)
else:
for d in done:
job_return_code = await d
if len(waiting) > 0:
update_run_set(waiting, running, max_concurrency)
assert(len(running) > 0)
The problem here is that say my timeout is 5 seconds, and I'm scheduling 3 jobs across 4 cores. Job A takes 2 seconds, Job B takes 6 seconds and job C takes 7 seconds.
We have something like this:
t=0 t=1 t=2 t=3 t=4 t=5 t=6 t=7
-------|-------|-------|-------|-------|-------|-------|-------|
AAAAAAAAAAAAAAA
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
However, at t=2 the asyncio.await() call returns because A completed. It then loops back up to the top and runs again. At this point B has already been running for 2 seconds, but since it starts the countdown over, and only has 4 seconds remaining until it completes, B will appear to be successful. So after 4 seconds we return again, B is successful, then we start the loop over and now C completes.
How do I make it so that B and C both fail? I somehow need the time to be preserved across calls to asyncio.wait().
One idea that I had is to do my own bookkeeping of how much time each job is allowed to continue running, and pass the minimum of these into asyncio.wait(). Then when something times out, I can cancel only those jobs whose time remaining was equal to the value I passed in for timeout_seconds.
This requires a lot of manual bookkeeping on my part though, and I can't help but wonder about floating point problems which cause me to decide that it's not time to cancel a job even though it really is). So I can't help but think that there's something easier. Would appreciate any ideas.
You can wrap each job into a coroutine that checks its timeout, e.g. using asyncio.wait_for. Limiting the number of parallel invocations could be done in the same coroutine using an asyncio.Semaphore. With those two combined, you only need one call to wait() or even just gather(). For example (untested):
# Run the job, limiting concurrency and time. This code could likely
# be part of _run_job_coroutine, omitted from the question.
async def _run_job_with_limits(job, sem, timeout):
async with sem:
try:
await asyncio.wait_for(_run_job_coroutine(job), timeout)
except asyncio.TimeoutError:
# timed out and canceled, decide what you want to return
pass
async def _run_test_invocations_async(jobs, max_concurrency, timeout):
sem = asyncio.Semaphore(max_concurrency)
return await asyncio.gather(
*(_run_job_with_limits(job, sem, timeout) for job in jobs)
)
I'm using Monix Task for async control.
scenario
tasks are executed in parallel
if failure occurs over X times
stop all tasks that are not yet in complete status (as quick as better)
my solution
I come up the ideas that race between 1. result and 2. error counter, and cancel the loser.
Via Task.race if the error-counter get to threshold first, then the tasks would be canceled by Task.race.
experiment
on Ammonite REPL
{
import $ivy.`io.monix::monix:3.1.0`
import monix.eval.Task
import monix.execution.atomic.Atomic
import scala.concurrent.duration._
import monix.execution.Scheduler
//import monix.execution.Scheduler.Implicits.global
implicit val s = Scheduler.fixedPool("race", 2) // pool size
val taskSize = 100
val errCounter = Atomic(0)
val threshold = 3
val tasks = (1 to taskSize).map(_ => Task.sleep(100.millis).map(_ => errCounter.increment()))
val guard = Task(f"stop because too many error: ${errCounter.get()}")
.restartUntil(_ => errCounter.get() >= threshold)
val race = Task
.race(guard, Task.gather(tasks))
.runToFuture
.onComplete { case x => println(x); println(f"completed task: ${errCounter.get()}") }
}
issue
The outcome is depends on thread pool size !?
For pool size 1
the outcome is almost always a task success i.e. no stop.
Success(Right(.........))
completed task: 100 // all task success !
For pool size 2
it is very un-deterministic between success and failure and the cancelling is not accurate.
for example:
Success(Left(stop because too many error: 1))
completed task: 98
the canceling is as late as 98 tasks has completed.
the error count is weird small to threshold.
The default global scheduler get this same outcome behavior.
For pool size 200
it is more deterministic and the stopping is earlier thus more accurate in sense that less task was completed.
Success(Left(stop because too many error: 2))
completed task: 8
the larger of the pool size the better.
If I change Task.gather to Task.sequence execution, all issues disappeared!
What is the cause for this dependency on pool size ?
How to improve it or is there better alternative for stopping tasks once too many error occurs ?
What you're seeing is likely an effect of the monix scheduler and how it aims for fairness. It's a fairly complex topic but the documentation and scaladocs are excellent (see: https://monix.io/docs/3x/execution/scheduler.html#execution-model)
When you have only one thread (or few) it takes a while until the "guard" Task gets another turn to check. With Task.gather you start 100 tasks at once, so the scheduler is very busy and the "guard" cannot check again until the other tasks are already done.
If you have one thread per task the scheduler cannot guarantee fairness and therefore the "guard" unfairly checks much more frequently and can finish sooner.
If you use Task.sequence those 100 tasks are executed sequentially, which is why the "guard" task gets much more opportunities to finish as soon as needed. If you want to keep your code the way it is, you could use Task.gatherN(parallelism = 4) which will limit the parallelism and therefore allow your "guard" to check more often (a middleground between Task.sequence and Task.gather).
It seems a bit like Go code to me (using Task.race like Go's select) and you're also using side-effects unconstrained which further complicates understanding what's going on. I've tried to rewrite your program in a way that's more idiomatic and for complicated concurrency I usually reach for streams like Observable:
import cats.effect.concurrent.Ref
import monix.eval.Task
import monix.execution.Scheduler
import monix.reactive.Observable
import scala.concurrent.duration._
object ErrorThresholdDemo extends App {
//import monix.execution.Scheduler.Implicits.global
implicit val s: Scheduler = Scheduler.fixedPool("race", 2) // pool size
val taskSize = 100
val threshold = 30
val program = for {
errCounter <- Ref[Task].of(0)
tasks = (1 to taskSize).map(n => Task.sleep(100.millis).flatMap(_ => errCounter.update(_ + (n % 2))))
tasksFinishedCount <- Observable
.fromIterable(tasks)
.mapParallelUnordered(parallelism = 4) { task =>
task
}
.takeUntilEval(errCounter.get.restartUntil(_ >= threshold))
.map(_ => 1)
.sumL
errorCount <- errCounter.get
_ <- Task(println(f"completed tasks: $tasksFinishedCount, errors: $errorCount"))
} yield ()
program.runSyncUnsafe()
}
As you can see I no longer use global mutable side-effects but instead Ref which interally also uses Atomic but provides a functional api which we can use with Task.
For demonstration purposes I also changed the threshold to 30 and only every other task will "error". So the expected output is always around completed tasks: 60, errors: 30 no matter the thread-pool size.
I'm still using polling with errCounter.get.restartUntil(_ >= threshold) which might burn a bit too much CPU for my taste but it's close to your original idea and works well.
Usually I don't create a list of tasks up front but instead throw the inputs into the Observable and create the tasks inside of .mapParallelUnordered. This code keeps your list which is why there is no real mapping involved (it already contains tasks).
You can choose your desired parallelism much like with Task.gatherN which is pretty nice imo.
Let me know if anything is still unclear :)
Tell me, where I am wrong? I create a TornadoScheduler in the context of a Tornado server. Assign a listener and a execute function. But I get not exactly what I need.
from tornado.concurrent import return_future, run_on_executor
class Users:
#return_future
#run_in_executor
def save(self, callback=None):
some code
callback(some data)
scheduler.add_listener(_scheduler_listener, apscheduler.events.EVENT_ALL)
......
async def processing(event: JobEvent):
data = await Users.get_all_users() <-- comunicate with DB
if isinstance(data.result(), Exception):
raise data.result()
.....
done = await users.save() <-- comunicate with DB
.....
def _scheduler_listener(event: JobEvent):
asyncio.ensure_future(processing(event))
The scheduler starts the task and all 3 events with the code 512 - added, 32768 - submitted and 4096 - executed come to the listener, but it asyncio.ensure_future() performed only 2 times, when 512 and 32768 codes.
I use Tornado 5, TornadoScheduler. When I create a job, the scheduler sends code 512 to the listener, then code 32768 and then code 4096. But, when code 4096 arrives, asyncio.ensure_future is not executed. When I press Crtl+C the console is displayed:
RuntimeWarning: coroutine 'processing' was never awaited
2019-01-30 19:04:24,473 - asyncio - ERROR - Task was destroyed but it is pending! task: <Task pending coro=<processing() running at ....
What is the reason?
To run async functions, you must use AsyncIOScheduler and declare your callable as a (native) coroutine function.