Async socket.send() exception - python-3.x

Question

Hello I have the following for my async loop
async def start_process(restore_items, args, loop):
with GlacierRestorer(args.temp_dir, args.error_log_bucket, loop) as restorer:
restorer.initiate_restore_all(restore_items)
tasks = []
semaphore = asyncio.BoundedSemaphore(4)
for item in restore_items:
tasks.append(asyncio.ensure_future(restorer.transfer(item, semaphore)))
await asyncio.gather(*tasks)
def main():
args = get_args()
restore_items = get_restore_items(args)
for item in restore_items:
print(item.source, ':', item.destination)
try:
loop = asyncio.get_event_loop()
loop.run_until_complete(start_process(restore_items, args, loop))
except KeyboardInterrupt:
pass
My job and files get larger I see that I keep getting an
socket.send() exception
After reading the documentation it seems to be coming from loop.run_until_complete
The exception doesn't come the program to crash, but eventually bogs it down so much it gets stuck printing the exception.
How do I modify the current code to fix this?

Answer 1

run_until_complete only propagates the exception raised inside start_process. This means that if an exception happens at any point during start_process, and start_process doesn't catch it, run_until_complete(start_process()) will re-raise the same exception.
In your case the exception likely originally gets raised somewhere in restorer.transfer(). The call to gather returns the results of the coroutines, which includes raising an exception, if one occurred.
The exception doesn't come the program to crash, but eventually bogs it down so much it gets stuck printing the exception. How do I modify the current code to fix this?
Ideally you would fix the cause of the exception - perhaps you are sending in too many requests at once, or you are using the GlacierRestorer API incorrectly. But some exceptions cannot be avoided, e.g. ones caused by a failing network. To ignore such exceptions, you can wrap the call to restorer.transfer in a separate coroutine:
async def safe_transfer(restorer, item, semaphore):
try:
return await restorer.transfer(item, semaphore)
except socket.error as e:
print(e) # here you can choose not to print exceptions you
# don't care about if doing so bogs down the program
In start_process you would call this coroutine instead of restorer_transfer:
coros = []
for item in restore_items:
coros.append(safe_transfer(restorer, item, semaphore))
await asyncio.gather(*coros)
Note that you don't need to call asyncio.ensure_future() to pass a coroutine to asyncio.gather; it will be called automatically.