This question already has an answer here:
Calling a coroutine from asyncio.Protocol.data_received
(1 answer)
Closed 6 years ago.
I'm playing with asyncio UDP server example and would like to have a sleep from within the datagram_received method.
import asyncio
class EchoServerProtocol:
def connection_made(self, transport):
self.transport = transport
def datagram_received(self, data, addr):
message = data.decode()
print('Received %r from %s' % (message, addr))
# Sleep
await asyncio.sleep(1)
print('Send %r to %s' % (message, addr))
self.transport.sendto(data, addr)
loop = asyncio.get_event_loop()
print("Starting UDP server")
# One protocol instance will be created to serve all client requests
listen = loop.create_datagram_endpoint(EchoServerProtocol,
local_addr=('127.0.0.1', 9999))
transport, protocol = loop.run_until_complete(listen)
try:
loop.run_forever()
except KeyboardInterrupt:
pass
transport.close()
loop.close()
This fails with a SyntaxError on the sleep line (Python 3.5.1). Using time.sleep is obviously not working as it prevents any other datagram to be received. Any hints on how to solve this?
The goal is to replace this sleep with a real non-blocking I/O call.
It seems that the await has to live in an async def (coroutine). To do so, you must fire a call via asyncio.ensure_future.
def datagram_received(self, data, addr):
asyncio.ensure_future(self.reply(data, addr))
async def reply(self, data, addr):
await asyncio.sleep(1)
self.transport.sendto(data, addr)
Related
I can't cancel my aiohttp websocket server from within the application. I want to stop the server and shutdown when I get a "cancel" string
from the client. Yes, I get it, and I finish my co-routine (websocket_handler), but there are three co-routines from the aiohttp library which still continue working.
Of course, I can invoke asyncio.get_event_loop().stop() at the end of my co-routine, but is there a graceful way for stopping aiohttp server?
From my code one can see that I've tried to use Application().on_shutdown.append(), but it failed.
What is the right way?
#!/usr/bin/env python
# -- coding: utf-8 --
import os
import asyncio
import signal
import weakref
import aiohttp.web
from aiohttp import ClientConnectionError, WSCloseCode
# This restores the default Ctrl+C signal handler, which just kills the process
#https://stackoverflow.com/questions/27480967/why-does-the-asyncios-event-loop-suppress-the-keyboardinterrupt-on-windows
import signal
signal.signal(signal.SIGINT, signal.SIG_DFL)
HOST = os.getenv('HOST', 'localhost')
PORT = int(os.getenv('PORT', 8881))
async def testhandle(request):
#Сопрограмма одрабатывающая http-запрос по адресу "http://127.0.0.1:8881/test"
print("server: into testhandle()")
return aiohttp.web.Response(text='Test handle')
async def websocket_handler(request):
#Сопрограмма одрабатывающая ws-запрос по адресу "http://127.0.0.1:8881"
print('Websocket connection starting')
ws = aiohttp.web.WebSocketResponse()
await ws.prepare(request)
request.app['websockets'].add(ws)
print('Websocket connection ready')
try:
async for msg in ws:
if msg.type == aiohttp.WSMsgType.TEXT:
if msg.data == 'close':
print(msg.data)
break
else:
print(msg.data)
await ws.send_str("You said: {}".format(msg.data))
elif msg.type == aiohttp.WSMsgType.ERROR:
print('ws connection closed with exception %s' %
ws.exception())
except (asyncio.CancelledError, ClientConnectionError):
pass # Тут оказываемся когда, клиент отвалился.
# В будущем можно тут освобождать ресурсы.
finally:
print('Websocket connection closed')
request.app['websockets'].discard(ws)
#pending = asyncio.Task.all_tasks()
#asyncio.get_event_loop().stop()
return ws
async def on_shutdown(app):
for ws in set(app['websockets']):
await ws.close(code=WSCloseCode.GOING_AWAY, message='Server shutdown')
def main():
loop = asyncio.get_event_loop()
app = aiohttp.web.Application()
app['websockets'] = weakref.WeakSet()
app.on_shutdown.append(on_shutdown)
app.add_routes([aiohttp.web.get('/', websocket_handler)]) #, aiohttp.web.get('/test', testhandle)
try:
aiohttp.web.run_app(app, host=HOST, port=PORT, handle_signals=True)
print("after run_app")
except Exception as exc:
print ("in exception")
finally:
loop.close()
if __name__ == '__main__':
main()
I believe the correct answer is simply:
raise aiohttp.web.GracefulExit()
After catching the exception, it calls all the handlers appended to the on_shutdown and on_cleanup signals and dies.
One can see in the source code the aiohttp.web.run_app() waits for two exceptions: GracefulExit and KeyboardInterrupt. While the latter is rather uninteresting, following the trace of GracefulExit can lead you to this place in web_runner.py, that registers SIGINT and SIGTERM signal handlers to a function with raise GracefulExit().
Indeed, I also managed to gracefully shut it down by raising the signal.SIGINT or signal.SIGTERM from itself, e.g.
import signal
signal.raise_signal(signal.SIGINT)
This was tested to work on Fedora Linux 34, Python 3.9.7, aiohttp 3.7.4.
https://docs.aiohttp.org/en/v3.0.1/web_reference.html#aiohttp.web.Application.shutdown
app.shutdown()
app.cleanup()
After shutdown you should also do cleanup()
I want to send data through websockets as soon as a client is connected.
The Data is at an other place then the Websocket Handler. How can i get the data to the client ?
The server should hold the loop and the Handler. In the connector i connect to a tcp socket to get the data out of some hardware. I expect to have not more then 6 Websockets open once a time. The Data comes as a stream out of the TCP socket.
server.py
import os
from tornado import web, websocket
import asyncio
import connector
class StaticFileHandler(web.RequestHandler):
def set_default_headers(self):
self.set_header("Access-Control-Allow-Origin", "*")
def get(self):
self.render('index.html')
class WSHandler(websocket.WebSocketHandler):
def open(self):
print('new connection')
self.write_message("connected")
def on_message(self, message):
print('message received %s' % message)
self.write_message("pong")
def on_close(self):
print('connection closed')
public_root = 'web_src'
handlers = [
(r'/', StaticFileHandler),
(r'/ws', WSHandler),
]
settings = dict(
template_path = os.path.join(os.path.dirname(__file__), public_root),
static_path = os.path.join(os.path.dirname(__file__), public_root),
debug = True
)
app = web.Application(handlers, **settings)
sensorIP = "xxx.xxx.xxx.xxx"
if __name__ == "__main__":
app.listen(8888)
asyncio.ensure_future(connector.main_task(sensorIP))
asyncio.get_event_loop().run_forever()
connector.py
import yaml
import asyncio
class RAMReceiver:
def __init__(self, reader):
self.reader = reader
self.remote_data = None
self.initParams = None
async def work(self):
i = 0
while True:
data = await self.reader.readuntil(b"\0")
self.remote_data = yaml.load(data[:-1].decode("utf-8",
"backslashreplace"))
# here i want to emit some data
# send self.remote_data to websockets
if i == 0:
i += 1
self.initParams = self.remote_data
# here i want to emit some data after open event is
# triggered
# send self.initParams as soon as a client has connected
async def main_task(host):
tasks = []
(ram_reader,) = await asyncio.gather(asyncio.open_connection(host,
51000))
receiver = RAMReceiver(ram_reader[0])
tasks.append(receiver.work())
while True:
await asyncio.gather(*tasks)
You can use Tornado's add_callback function to call a method on your websocket handler to send the messages.
Here's an example:
1. Create an additional method on your websocket handler which will receive a message from connector.py and will send to connected clients:
# server.py
class WSHandler(websocket.WebSocketHandler):
# make it a classmethod so that
# it can be accessed directly
# from class without `self`
#classmethod
async def send_data(cls, data):
# write your code for sending data to client
2. Pass the currently running IOLoop and WSHandler.send_data to your connector.py:
# server.py
from tornado import ioloop
...
if __name__ == "__main__":
...
io_loop = ioloop.IOLoop.current() # current IOLoop
callback = WSHandler.send_data
# pass io_loop and callback to main_task
asyncio.ensure_future(connector.main_task(sensorIP, io_loop, callback))
...
3. Then modify main_task function in connector.py to receive io_loop and callback. Then pass io_loop and callback to RAMReceiver.
4. Finally, use io_loop.add_callback to call WSHandler.send_data:
class RAMReceiver:
def __init__(self, reader, io_loop, callback):
...
self.io_loop = io_loop
self.callback = callback
async def work(self):
...
data = "Some data"
self.io_loop.add_callback(self.callback, data)
...
I'm trying to write an asyncio-based server. The problem is, that it stops to respond after the first request.
My code is built upon this template for echo-server and this method to pass parameters to coroutines.
class MsgHandler:
def __init__(self, mem):
# here (mem:dict) I store received metrics
self.mem = mem
async def handle(self, reader, writer):
#this coroutine handles requests
data = await reader.read(1024)
print('request:', data.decode('utf-8'))
# read_msg returns an answer based on the request received
# My server closes connection on every second request
# For the first one, everything works as intended,
# so I don't thik the problem is in read_msg()
response = read_msg(data.decode('utf-8'), self.mem)
print('response:', response)
writer.write(response.encode('utf-8'))
await writer.drain()
writer.close()
def run_server(host, port):
mem = {}
msg_handler = MsgHandler(mem)
loop = asyncio.get_event_loop()
coro = asyncio.start_server(msg_handler.handle, host, port, loop=loop)
server = loop.run_until_complete(coro)
try:
loop.run_forever()
except KeyboardInterrupt:
pass
server.close()
loop.run_until_complete(server.wait_closed())
loop.close()
On the client-side I either get an empty response or ConnectionResetError (104, 'Connection reset by peer').
You are closing the writer with writer.close() in the handler, which closes the socket.
From the 3.9 docs on StreamWriter:
Also, if you don't close the stream writer, then you would still have store it somewhere in order to keep receiving messages over that same connection.
The HandlerWebsockets does work fine and is just replying with what has been send at the moment throught messageToSockets(msg). However both tries to send messages to the websocket from the coroutine of the web application are not working. Looks like everything is blocked by these attempts...
class webApplication(tornado.web.Application):
def __init__(self):
handlers = [
(r'/', HandlerIndexPage),
(r'/websocket', HandlerWebSocket, dict(msg='start')),
]
settings = {
'template_path': 'templates'
}
tornado.web.Application.__init__(self, handlers, **settings)
#gen.coroutine
def generateMessageToSockets(self):
while True:
msg = str(randint(0, 100))
print ('new messageToCon: ', msg)
yield [con.write_message(msg) for con in HandlerWebSocket.connections]
yield gen.sleep(1.0)
if __name__ == '__main__':
ws_app = webApplication()
server = tornado.httpserver.HTTPServer(ws_app)
port = 9090
print('Listening on port:' + str(port))
server.listen(port)
IOLoop.current().spawn_callback(webApplication.generateMessageToSockets)
IOLoop.current().set_blocking_log_threshold(0.5)
IOLoop.instance().start()
Here the WebSockets Handler
class HandlerWebSocket(tornado.websocket.WebSocketHandler):
connections = set()
def initialize(self, msg):
print('HWS:' + msg)
def messageToSockets(self, msg):
print ('return message: ', msg)
[con.write_message(msg) for con in self.connections]
def open(self):
self.connections.add(self)
print ('new connection was opened')
pass
def on_message(self, message):
print ('from WebSocket: ', message)
self.messageToSockets(message)
def on_close(self):
self.connections.remove(self)
print ('connection closed')
pass
I am a bit lost in the examples, questions here, documentation etc. So any hint how to properly start a continous calling websocket routine is greatly appreciated
generateMessageToSockets will loop endlessly, generating messages as fast as it can without waiting for those messages to be sent. Since it starts first and never yields, the HTTPServer will never actually be able to accept a connection.
If you really want to send messages as fast as you can, the minimal solution without blocking would be
yield [con.write_message(msg) for con in HandlerWebSocket.connections]
yield gen.moment
But it would probably be better to use gen.sleep to send messages at regular intervals, instead of "as fast as possible".
unfortunately all the gen.routines tries didn't work for me. Moved back to threads
def generateMessageToSockets():
while True:
msg = str(randint(0, 100))
print ('new messageToCon: ', msg)
[con.write_message(msg) for con in HandlerWebSocket.connections]
sleep(1.0)
class WebApplication(tornado.web.Application):
def __init__(self):
handlers = [
(r'/', HandlerIndexPage),
(r'/websocket', HandlerWebSocket, dict(msg='start')),
]
settings = {
'template_path': 'templates'
}
tornado.web.Application.__init__(self, handlers, **settings)
if __name__ == '__main__':
tGenarate = threading.Thread(target=generateMessageToSockets)
tGenarate.start()
ws_app = WebApplication()
server = tornado.httpserver.HTTPServer(ws_app)
port = 9090
print('Listening on port:' + str(port))
server.listen(port)
ioloop.IOLoop.instance().start()
which works
I wrote a script for a socket server that simply listens for incoming connections and processes the incoming data. The chosen architecture is the asyncio.start_server for the socket management and the asyncio.Queues for passing the data between the producer and consumer coroutines. The problem is that the consume(q1) function is executed only once (at the first script startup). Then it is not more executed. Is the line run_until_complete(asyncio.gather()) wrong?
import asyncio
import functools
async def handle_readnwrite(reader, writer, q1): #Producer coroutine
data = await reader.read(1024)
message = data.decode()
await writer.drain()
await q1.put(message[3:20])
await q1.put(None)
writer.close() #Close the client socket
async def consume(q1): #Consumer coroutine
while True:
# wait for an item from the producer
item = await q1.get()
if item is None:
logging.debug('None items') # the producer emits None to indicate that it is done
break
do_something(item)
loop = asyncio.get_event_loop()
q1 = asyncio.Queue(loop=loop)
producer_coro = asyncio.start_server(functools.partial(handle_readnwrite, q1=q1), '0.0.0.0', 3000, loop=loop)
consumer_coro = consume(q1)
loop.run_until_complete(asyncio.gather(consumer_coro,producer_coro))
try:
loop.run_forever()
except KeyboardInterrupt:
pass
loop.close()
handle_readnwrite always enqueues the None terminator, which causes consume to break (and therefore finish the coroutine). If consume should continue running and process other messages, the None terminator must not be sent after each message.