Async version of built-in print (stdout) - python-3.x

Question

I have a problem understanding some of the limitations using print inside an async function. Basically this is my code:
#!/usr/bin/env python
import sys
import asyncio
import aiohttp
async amain(loop):
session = aiohttp.ClientSession(loop=loop)
try:
# using session to fetch a large json file which is stored
# in obj
print(obj) # for debugging purposes
finally:
await session.close()
def main():
loop = asyncio.get_event_loop()
res = 1
try:
res = loop.run_until_complete(amain(loop, args))
except KeyboardInterrupt:
# silence traceback when pressing ctrl+c
pass
loop.close()
return res
if __name__ == '__main__':
sys.exit(main())
If I execute this, then the json object is printed on stdout and the suddenly dies with this error
$ dwd-get-sensor-file ; echo $?
Traceback (most recent call last):
File "/home/yanez/anaconda/py3/envs/mondas/bin/dwd-get-sensor-file", line 11, in <module>
load_entry_point('mondassatellite', 'console_scripts', 'dwd-get-sensor-file')()
File "/home/yanez/projects/mondassatellite/mondassatellite/mondassatellite/bin/dwd_get_sensor_file.py", line 75, in main
res = loop.run_until_complete(amain(loop, args))
File "/home/yanez/anaconda/py3/envs/mondas/lib/python3.7/asyncio/base_events.py", line 579, in run_until_complete
return future.result()
File "/home/yanez/projects/mondassatellite/mondassatellite/mondassatellite/bin/dwd_get_sensor_file.py", line 57, in amain
print(obj)
BlockingIOError: [Errno 11] write could not complete without blocking
1
The funny thing is that when I execute my code redirecting stdout to a file like this
$ dwd-get-sensor-file > output.txt ; echo $?
0
the exception doesn't happen and the whole output is correctly redirected to output.txt.
For testing purposes I converted the json object to a string and instead of doing print(obj) I do sys.stdout.write(obj_as_str) then I get this
exception:
BlockingIOError: [Errno 11] write could not complete without blocking
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
I've searched for this BlockingIOError exception but all threads I find have something to do with network sockets or CI builds. But I found one
interesting github comment:
The make: write error is almost certainly EAGAIN from stdout. Pretty much every command line tool expects stdout to be in blocking mode, and does not properly retry when in nonblocking mode.
So when I executed this
python -c 'import os,sys,fcntl; flags = fcntl.fcntl(sys.stdout, fcntl.F_GETFL); print(flags&os.O_NONBLOCK);'
I get 2048, which means blocking (or is this the other way round? I'm confused). After executing this
python -c 'import os,sys,fcntl; flags = fcntl.fcntl(sys.stdout, fcntl.F_GETFL); fcntl.fcntl(sys.stdout, fcntl.F_SETFL, flags&~os.O_NONBLOCK);'
I don't get the BlockingIOError exceptions anymore, but I don't like this solution though.
So, my question is: how should we deal when writing to stdout inside an async function? If I know that I'm dealing with stdout, should I
set stdout to non-blocking and revert it back when my program exits? Is there a specific strategy for this?

Answer 1

Give aiofiles a try, using stdout FD as the file object.
aiofiles helps with this by introducing asynchronous versions of files that support delegating operations to a separate thread pool.
In terms of actually using aiofiles with an FD directly, you could probably extend the aiofiles.os module, using wrap(os.write).