process socket data in near realtime python - python-3.x

Question

I have upto 30 nodes each capable of sending data upto 1000 messages/second. Each messages can have 256-512 bytes of data.
Each node use unique tcp port for communication. Each data received is pre-processed, inserted into database and post-processed.
Below are the approaches I have tried with observations :-
Case-1. using asyncio with processing data as soon as received.
async def process_packets(reader, writer, db):
while True:
data = reader.read(4096)
data = pre_process(data)
save_in_db(data)
post_process(data)
writer.close()
Observation:- For single packet, processing usually takes 10-20msec. But as packet frequency increases, tcp buffering starts to happen i.e single call to reader.read(), gets multiple packets.
This increases the processing of the current node as well as other nodes.
Case-2. using asyncio with data pushed in a queue and worker thread consuming this queue.
async def process_packets(reader, writer, q):
while True:
data = reader.read(4096)
q.put(data)
writer.close()
def worker_thread(q, db):
while True:
data = q.get()
data = pre_process(data)
save_in_db(data)
post_process(data)
Observation:- As no processing is done while receiving packets, all nodes are able put data in the queue as fast as possible. The issue comes in the worker thread where q.get() becomes very slow as time progress.
Case-3. creating socket server thread for each node
def server_thread(port, db):
s = socket.socket()
s.bind()
s.listen(1)
while True:
(conn, addr) = s.accept()
while True:
try:
data = conn.recv(4096)
except Exception:
conn.close()
break
data = pre_process(data)
save_in_db(data)
post_process(data)
Observation:- The advantage is this case is that each node has dedicated thread for receiving and processing data so other threads are not affected. But here I am facing multiple packets returned by socket.recv(). This increases processing time.
I need a way to process data from these node as fast as possible with application running 24x7 with no downtime.
OS = Ubuntu20.04-lts
System = intel i3 8th gen, 8GB ram, 4core

Answer 1

But here I am facing multiple packets returned by socket.recv()
TCP is a byte stream, i.e. there are no packets at this level. You likely mean application level messages. Your code MUST be able to deal with multiple or partial application messages itself since TCP does not provide a message syntax by itself. While you seem to get only full messages when reading fast enough there is no guarantee for this, eventually your application might stall for a short time (due to scheduling) and messages will accumulate.
Dealing with multiple messages returned from a socket.recv() can even be an advantage. Reading multiple messages at once means that a single system call returns more application data, which increases the efficiency of the application (less system calls needed for the same amount of work). So it is better to read as much as possible within a single recv instead of hoping to get only a single message.
As for the other design: the last approach with a thread per node scales best, since in this case the work (and this the load) is spread over multiple CPU cores. The other approaches only use a single CPU core. But none of the approaches actually guarantees that your specific system is able to process that much data. They only differ in how good they make use of the resources offered by the underlying system.