Python36 and sockets - python-3.x

Question

So I'm connected to an IRC chat using socket.connec
I log in by passing my variables via socket.send
The log in is successful and then I sit in a while true loop using
Socket.recv(1024)
If I just continually print the response everything looks fine, but let's say I want to add to the end of the string... I noticed that socket.recv doesn't always get the full message (only grabs up to 1024 as expected) and the remainder of the message is in the next iteration of the loop.
This makes its it impossible to process the feedback line by line.
Is there a better way to constantly read the data without it getting trunked? Is it possible to figure out the size of the response before receiving it so the buffer can be set dynamically?

Answer 1

TCP is a stream-based protocol. Buffer the bytes received and only extract complete messages from the stream.
For complete lines, look for newline characters in the buffer.
Example server:
import socket
class Client:
def __init__(self,socket):
self.socket = socket
self.buffer = b''
def getline(self):
# if there is no complete line in buffer,
# add to buffer until there is one.
while b'\n' not in self.buffer:
data = self.socket.recv(1024)
if not data:
# socket was closed
return ''
self.buffer += data
# break the buffer on the first newline.
# note: partition(n) return "left of n","n","right of n"
line,newline,self.buffer = self.buffer.partition(b'\n')
return line + newline
srv = socket.socket()
srv.bind(('',5000))
srv.listen(1)
conn,where = srv.accept()
client = Client(conn)
print(f'Client connected on {where}')
while True:
line = client.getline()
if not line:
break
print(line)
Example client:
s=socket()
s.connect(('127.0.0.1',5000))
s.sendall(b'line one\nline two\nline three\nincomplete')
s.close()
Output on server:
Client connected on ('127.0.0.1', 2667)
b'line one\n'
b'line two\n'
b'line three\n'