I have written a python read script which has the following code`
def read_serial():
while True:
# prepare for response
out = ''
while ser.inWaiting() > 0:
out += ser.read(1).decode('unicode_escape')
if out != '':
response_header = out.encode('unicode_escape').strip()
print( "Response recieved " ,response_header)
print((response_header[0],response_header[1],response_header[2])) //line 2
So the bytes array that I'm sending is:
data[] = {0x9c,0x10,0x01,0x05,0x07,0x08};
and on python side..in line 2 I'm receiving this:
Response recieved b'\\\x9c\\\x10\\\x01\\\x05\\\x07\\\x08'
(92, 120, 57)
Why 92,120,57 what does this mean because for 9C decimal would be 156?
Any suggestions? Or which encoding technique you suggest for bytes?
Just receive the bytes. Something like this since I can't run your code:
def read_serial():
while True:
# prepare for response
out = b'' # bytes type
while ser.inWaiting() > 0:
out += ser.read(1)
if out != b'':
print("Response received:",out.hex())
Related
I'm trying to link my IP camera to my a AWS service and there's 2 ways I can do this, either with my built in computer camera (runs fine) and a IP camera. The code I'm using is from https://github.com/aws-samples/amazon-rekognition-video-analyzer which is writing in Python 2.7(but I'm doing it in python 3), I already converted the code to python 3 (using python 2to3).but when I run the code I keep getting this error of only concatenate strings not bytes:
I'm new to python so for what I have research is that 2to3 will do must of the work, but I'm pretty sure this part of converting bytes to strings is not in there and I'm not sure how to handle this conversion/parsing.
Traceback (most recent call last):
File "video_cap_ipcam.py", line 140, in <module>
main()
File "video_cap_ipcam.py", line 104, in main
bytes += stream.read(16384*2)
TypeError: can only concatenate str (not "bytes") to str
video_cap_ipcam.py file:
def main():
ip_cam_url = ''
capture_rate = default_capture_rate
argv_len = len(sys.argv)
if argv_len > 1:
ip_cam_url = sys.argv[1]
if argv_len > 2 and sys.argv[2].isdigit():
capture_rate = int(sys.argv[2])
else:
print("usage: video_cap_ipcam.py <ip-cam-url> [capture-rate]")
return
print(("Capturing from '{}' at a rate of 1 every {} frames...".format(ip_cam_url, capture_rate)))
stream = urllib.request.urlopen(ip_cam_url)
bytes = ''
pool = Pool(processes=3)
frame_count = 0
while True:
# Capture frame-by-frame
frame_jpg = ''
bytes += stream.read(16384*2)
b = bytes.rfind('\xff\xd9')
a = bytes.rfind('\xff\xd8', 0, b-1)
if a != -1 and b != -1:
#print 'Found JPEG markers. Start {}, End {}'.format(a,b)
frame_jpg_bytes = bytes[a:b+2]
bytes = bytes[b+2:]
if frame_count % capture_rate == 0:
img_cv2_mat = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
rotated_img = cv2.transpose(cv2.flip(img_cv2_mat, 0))
retval, new_frame_jpg_bytes = cv2.imencode(".jpg", rotated_img)
#Send to Kinesis
result = pool.apply_async(send_jpg, (bytearray(new_frame_jpg_bytes), frame_count, True, False, False,))
frame_count += 1
if __name__ == '__main__':
main()
When you initially set the variable bytes to '', the variable becomes a string, which in Python 3 is considered a sequence of characters rather than a sequence of bytes. (A character can be expressed using multiple bytes.)
If you want bytes to be a sequence of bytes, initialize it as b'' instead. Then you can concatenate further bytes to it.
I am trying to use a modified MFRC522 module on UART, but I am stuck finding the correct formatting for serial read/write in python 3.
There is a library for python 2 which works for me:
The readme on that link also describes the modification to the module.
I'm having trouble getting the .self.ser.write and .self.ser.read in functions writeRegister and readRegister to work in python 3. These get string inputs, which I now understand is fine for python 2, but must be converted to bytes for python 3.
def writeRegister(self, addr, val, size=None):
if size is None:
count = 0
while True:
self.ser.flushInput()
self.ser.write(chr(addr&0x7F))
self.ser.write(chr(val))
tmp = ord(self.ser.read(1))
if(tmp == addr):
return True
count+=1
if(count > 10):
print ("Error de escritura en: "+ hex(addr))
return False
else:
self.ser.flushInput()
for txBytes in range (0, size):
self.ser.write(chr(addr&0x7F))
tmp = ord(self.ser.read(1))
if(tmp == addr):
self.ser.write(chr(val[txBytes]))
else:
print ("Error de escritura en bloque")
return False
return True
def readRegister(self, addr):
self.ser.flushInput()
self.ser.write(chr(addr|0x80))
val = self.ser.read(1)
return ord(val)
I suspected it's a matter of correctly applying .encode('utf-8') or similar, but I can't find a working solution. If I try
chr(0x80).encode('utf-8')
I get
UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 0: ordinal not in range(128)
Maybe I am going down the wrong path with that.
I'll try bytes:
bytes(chr(0x80), 'utf-8')
Gives 2 bytes in this case (>128 i guess):
b'\xc2\x80'
Maybe this is getting closer, but I get stuck on how to read back and decode. I don't know how or if to modify the ord parts. So I can't seem to get the response from the mfrc522.
Does anyone have any suggestions?
Anybody successfully used this mfrc522 UART on python 3?
There should be no need to use char in either Python 2.x or 3.x, you can just write bytes to the port:
self.ser.write(bytes[addr&0x7F])
For the reading function just drop the ord() from the return sentence, it's not needed for Python 3.x.
See here for more details.
That did it, thanks!
Here are the corrected functions:
def writeRegister(self, addr, val, size=None):
if size is None:
count = 0
while True:
self.ser.flushInput()
self.ser.write(bytes([addr&0x7F]))
self.ser.write(bytes([val]))
tmp = int.from_bytes(self.ser.read(1), "big")
if(tmp == addr):
return True
count+=1
if(count > 10):
print ("Write error at: "+ hex(addr))
return False
else:
self.ser.flushInput()
if(tmp == addr):
self.ser.write(bytes([val]))
else:
print ("Block write error")
return False
return True
def readRegister(self, addr):
self.ser.flushInput()
self.ser.write(bytes([addr|0x80]))
val = self.ser.read(1)
return int.from_bytes(val, "big")
When I start the program to share files to client it received it but when i request for another file download it failed with this error.
Now i keep getting this error from the client
socket1.send(bytes('0', 'UTF-8'))
BrokenPipeError: [Errno 32] Broken pipe
line 46 client.py
I tried breaking out of the server's filedownload loop but still not working.
server
#! /usr/bin/env python3
import socket
import sys
import os
import hashlib
import time
HOST = 127.0.0.1
PORT = 5000
c = 0 #used to count cycles
bufsize = 4096
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
print('Server Created')
except OSError as e:
print('Failed to create socket. Error Code : ' + str(msg[0]) + ' Message ' + msg[1])
sys.exit()
try:
s.bind((HOST, PORT))
except OSError as e:
print('Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1])
sys.exit()
print('Socket bind complete')
s.listen(1)
print('Server now listening')
while 1:
conn, addr = s.accept()
print('Connected with ' + addr[0] + ':' + str(addr[1]))
reqCommand = conn.recv(1024).decode("utf-8", errors='ignore')
print('Client> %s' % reqCommand)
string = reqCommand.split(' ', 1)
if reqCommand == 'quit':
break
elif reqCommand == 'lls':
toSend = ""
path = os.getcwd()
dirs = os.listdir(path)
for f in dirs:
toSend = toSend + f + ' '
conn.send(toSend.encode('utf-8'))
# print path
else:
string = reqCommand.split(' ', 1) # in case of 'put' and 'get' method
if len(string) > 1:
reqFile = string[1]
if string[0] == 'FileDownload':
with open(reqFile, 'rb') as file_to_send1:
# get the entire filesize, which sets the read sector to EOF
file_size = len(file_to_send1.read())
# reset the read file sector to the start of the file
file_to_send1.seek(0)
# take filesize and write it to a temp file
with open('temp01',mode='w', encoding='UTF-8') as file_to_send2:
file_to_send2.write(str(file_size))
# pass the file size over to client in a small info chunk
with open('temp01', 'rb') as file_to_send3:
conn.send(file_to_send3.read(1024))
#send the total file size off the client
while (c*bufsize) < file_size:
send_data = file_to_send1.read(bufsize)
conn.send(send_data)
c += 1
# get bool (0 is bad | 1 is good) from client
chunk_write_flag = int(conn.recv(1024))
while chunk_write_flag != 1: #while whole data was not sent..retry until successful
conn.send(send_data)
#get status from client after a retry
chunk_write_flag = int(conn.recv(1024))
# used on the last chunk of the file xfer
# if the file.read() is less than buffer size do last tasks
if (file_size - (c*bufsize)) < bufsize:
send_data = file_to_send1.read(bufsize)
conn.send(send_data)
file_to_send1.close()
break
#for data in file_to_send:
#conn.sendall(data)
print('Send Successful')
conn.close()
s.close()
client
#! /usr/bin/env python3
import socket
import sys
import os
import hashlib
import time
HOST = 127.0.0.1
PORT = 5000
c = 0
bufsize = 4096
def get(commandName):
socket1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket1.connect((HOST, PORT))
socket1.send(commandName.encode("utf-8"))
string = commandName.split(' ', 1)
inputFile = string[1]
c = 0
# before starting to write new file, get file size
file_size = int(socket1.recv(1024)) # from file_to_send3
# print (file_size)
# set byte buffer size
bufsize = 4096
# start writing at the beginning and use following variable to track
write_sectors = 0
# this only opens the file, the while loop controls when to close
with open(inputFile, 'wb+') as file_to_write:
# while written bytes to out is less than file_size
while write_sectors < file_size:
# write the BUFSIZE while the write_sector is less than file_size
file_to_write.write(socket1.recv(bufsize))
c += 1
with open(inputFile, 'rb') as verify:
write_check = (len(verify.read()) / c)
verify.seek(0) # read cycle moves seek location, reset it
while write_check != bufsize:
# where the original write started, to send back to server
if c > 1: file_to_write.seek((c-1) * bufsize)
if c == 1: file_to_write.seek(0)
# send to server that the write was not successful
socket1.send(bytes('0', 'UTF-8'))
file_to_write.write(socket1.recv(bufsize))
write_check = int(len(verify.read()) /c )
# if last packet, smaller than bufsize
socket1.send(bytes('1', 'UTF-8')) #send SUCCESS back to server
if (file_size - (write_check * c)) < bufsize:
#file_to_write.write(socket1.recv(bufsize))
verify.close()
#file_to_write.close()
file_size = 0
write_sectors += bufsize # successful write, move 'while' check
# add the written sectors by the bufsize.
# example if filesize in bytes is 4096 you need to track how much
# was written to figure out where the EOF is
file_to_write.write(socket1.recv(bufsize)) # write the last chunk missed by while loop
#data = socket1.recv(4096).decode("utf-8", errors="ignore")
#if not data: break
#break
# print data
#file_to_write.write(bytes(data.encode()))
#file_to_write.close()
print('Download Successful')
socket1.close()
return
def serverList(commandName):
socket1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket1.connect((HOST, PORT))
socket1.send(commandName.encode('utf-8'))
fileStr = socket1.recv(1024)
fileList = fileStr.decode('utf-8').split(' ')
for f in fileList:
print(f)
socket1.close()
return
msg = input('Enter your name: ')
while 1:
print("\n")
print('"FileDownload [filename]" to download the file from the server ')
print('"lls" to list all files in the server')
sys.stdout.write('%s> ' % msg)
inputCommand = sys.stdin.readline().strip()
if inputCommand == 'lls':
serverList('lls')
else:
string = inputCommand.split(' ', 1)
if string[0] == 'FileDownload':
Please can anyone help me,i don't know to fix it. I'll appreciate any help.
Thanks
Server.py
class BufferedReceiver():
def __init__(self, sock):
self.sock = sock
self.buffer = ""
self.bufferPos = 0
def _fetch(self):
while self.bufferPos >= len(self.buffer):
self.buffer = self.sock.recv(1024)
# print(self.buffer)
self.bufferPos = 0
def take(self, amount):
result = bytearray()
while(len(result) < amount):
# Fetch new data if necessary
self._fetch()
result.append(self.buffer[self.bufferPos])
self.bufferPos += 1
return bytes(result)
def take_until(self, ch):
result = bytearray()
while True:
# Fetch new data if necessary
self._fetch()
nextByte = self.buffer[self.bufferPos]
self.bufferPos += 1
result.append(nextByte)
if bytes([nextByte]) == ch:
break
return bytes(result)
Then, I simplified your server send routine after the else::
string = reqCommand.split(' ', 1) # in case of 'put' and 'get' method
if len(string) > 1:
reqFile = string[1]
if string[0] == 'FileDownload':
with open(reqFile, 'rb') as file_to_send1:
# get the entire filesize, which sets the read sector to EOF
file_size = len(file_to_send1.read())
# reset the read file sector to the start of the file
file_to_send1.seek(0)
# pass the file size over to client in a small info chunk
print('Filesize:', file_size)
conn.send((str(file_size)+'\n').encode())
#send the total file size off the client
c = 0
while (c*bufsize) < file_size:
send_data = file_to_send1.read(bufsize)
conn.send(send_data)
c += 1
print('Send Successful')
Client.py
def get(commandName):
socket1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket1.connect((HOST, PORT))
socket1.send(commandName.encode("utf-8"))
receiver = BufferedReceiver(socket1)
string = commandName.split(' ', 1)
inputFile = string[1]
# before starting to write new file, get file size
file_size = int(receiver.take_until(b'\n').decode().strip()) # from file_to_send3
print ('Filesize:', file_size)
# set byte buffer size
bufsize = 4096
# this only opens the file, the while loop controls when to close
with open(inputFile, 'wb+') as file_to_write:
# while written bytes to out is less than file_size
c = 0
while True:
# Compute how much bytes we have left to receive
bytes_left = file_size - bufsize * c
# If we are almost done, do the final iteration
if bytes_left <= bufsize:
file_to_write.write(receiver.take(bytes_left))
break
# Otherwise, just continue receiving
file_to_write.write(receiver.take(bufsize))
c += 1
#TODO open file again, verify.
# Generate MD5 on server while sending. Then, send MD5 to client.
# Open finished file in client again and compare MD5s
print('Download Successful')
socket1.close()
return
I have a binary string (bytes), and a potentially infinite byte stream. I need to concatenate them into a single byte stream in Python 3.6.
The concatenation must act like a standard byte stream, returning bytes from the string first, and then from the byte stream:
string = b'something'
stream = open(filename, 'rb')
concatenated = ?concat?(string, stream) # <=== need this
x = concantenated.read(5) # b"somet"
y = concatenated.read(2) # b"hi"
z = concatentated.read(26) # b"ngFIRST_24_BYTES_OF_STREAM"
…
I created a class called BufferedStringReader which functions as a byte steam, reading the string first and then some infinite byte stream next. I have assumed that the byte stream you're talking about is a file, like a block device or character device. Those are the only infinite byte streams I'm aware of though.
A quick briefing of how I implemented it: My class was not inherited from IOBase, BufferedReader or any other python IO object because many of the members and methods had the same names. Since this class must act like both a string stream and a file stream, these objects are stored as members, my class does not inherit from them.
I decided against using the tell() method in BufferReader to implement position because there is a weird quirk when using (some BufferReader object).tell(): Instead of returning a positive number, it returns a value between the negative of some block size and 0. So when the file is opened, the position is zero, but when you start reading from it, the position is at -4096 + the actual position, which wraps around after reading 4096 bytes. And the block size may not even be 4096 on your system, so it's very wacky. (Let me know if it's different on yours.)
Also, since your question is about infinite byte streams, my class does not support writing. For the case of block devices which I had taken as an example, it's not supported anyway.
The final thing is that unlimited reads aren't supported because it's used with infinite streams anyway, such calls would run out of memory.
from io import BufferedReader, StringIO, UnsupportedOperation
# Because this is a read-only stream, write methods raise an exception.
class BufferedStringReader:
def __init__(self, string, name, mode):
if mode != 'rb' and mode != 'br':
raise ValueError("invalid mode: " + mode)
self.reader = open(name, mode)
self.stringio = StringIO(string)
self.string = string
self.stringlen = len(string)
self.readerpos = 0
# self.reader.tell() doesn't seem to work properly
self.pos = self.stringio.tell() + self.readerpos
self.closed = (self.reader.closed and self.stringio.closed)
# This is my own method, not a method in the io module
def __update_state__(self):
self.pos = self.stringio.tell() + self.readerpos
self.closed = (self.reader.closed and self.stringio.closed)
def __repr__(self):
return "{BufferedStringReader reader=" + str(self.reader) + " stringio=" + str(self.stringio) + "}"
def getvalue(self):
# Reads the entire file. Since this is an infinite byte stream this is not supported.
raise UnsupportedOperation("BufferedStringReader does not support getvalue()")
def close(self):
self.reader.close()
self.stringio.close()
self.closed = True
def flush(self):
raise UnsupportedOperation("BufferedStringReader does not support flush()")
# This function leaves the file stream in an undefined state
def detach(self):
self.reader.detach()
#self.stringio.detach() Not supported
def isatty(self):
return False
def peek(self, size=0):
# We cannot guarrentee one single read will be done in this class but
# we can guarrentee that only one read will be done to the BufferedReader.
# Note that the number of bytes returned may be less or more than requested.
# Also in BufferedReader the size can apparently be negative and it's ignored
# in any case (on my linux system it returns 4K bytes even for positive size).
# Long story short; Python io's peek function ignores size.
# Nevertheless, we *try* to return exactly size bytes.
peek_size = (size if size > 0 else 1)
string_pos = self.stringio.tell()
string_read = self.string[string_pos:string_pos+peek_size]
if len(string_read) < peek_size:
reader_read = self.reader.peek(peek_size)
else:
reader_read = b''
return (bytes(string_read, 'utf-8') + reader_read)[0:peek_size]
def read(self, size=-1):
# size=-1 will cause your machine to run out of memory since you're dealing with
# and infinitely sized byte stream.
if size <= -1:
raise UnsupportedOperation("BufferedStringReader does not support read(size=-1)")
string_read = ''
reader_read = b''
if self.pos <= self.stringlen and self.pos+size <= self.stringlen:
string_read = self.stringio.read(size);
elif self.pos <= self.stringlen and self.pos+size > self.stringlen:
# No exceptions below, will return a smaller string if necessary
string_read = self.stringio.read(size);
reader_read = self.reader.read(size-len(string_read));
elif self.pos > self.stringlen:
reader_read = self.reader.read(size);
else:
# Impossible condition, detatch for security
raise RuntimeError
self.readerpos += len(reader_read)
self.__update_state__()
return bytes(string_read, 'utf-8') + reader_read;
def readable(self):
return True
def read1(self, size=-1):
# This is impossible to implement due to the arhitecture of this class.
# This method is supposed to call read() exactly once but since there are
# two IO streams, it is not possible.
raise UnsupportedOperation("BufferedStringReader does not support read1()")
def readinto(self, bytarray):
temp_read = self.read(len(bytarray))
temp_array = bytearray(temp_read)
if len(temp_array) <= 0:
return 0
bytarray[0:len(temp_array)] = temp_array
return len(temp_array)
def readinto1(self, bytarray):
raise UnsupportedOperation("BufferedStringReader does not support readinto1()")
def readlines(self, size=-1):
# Unlike read(), we stop when we encounter \n. No exceptions.
# Note that it's still possible we run out of memory if a large enough
# size is used and we don't find a newline.
# A trailing newline is kept in the string.
read_size = 0;
if size >= 0:
# If size is given try to find up to size
temp_read = self.peek(size)
newline = temp_read.find(b'\n')
if newline == -1:
read_size = len(temp_read)
else:
read_size = newline+1
else:
# Keep increasing size by 4K until newline is found
newline = -1
while newline == -1:
read_size += 4096
temp_read = self.peek(read_size);
newline = temp_read.find(b'\n')
read_size = newline+1
return self.read(read_size);
def seek(self, pos, whence=0):
if whence==2: # Seek from end of infinite file doesn't make sense
UnsupportedOperation("BufferedStringReader does not support seek(whence=2)")
elif whence==1:
self.pos = max(self.pos+pos, 0)
elif whence==0:
if pos < 0:
raise OSError("[Errno 22] Invalid argument")
self.pos = pos
else:
raise ValueError("whence value " + str(whence) + " unsupported")
self.stringio.seek(min(self.pos, self.stringlen), 0)
self.reader.seek(max(self.pos - self.stringlen, 0), 0)
self.readerpos = max(self.pos - self.stringlen, 0)
self.__update_state__()
return self.pos
def seekable(self):
return True
def tell(self):
return self.pos
def truncate(self, pos=None):
raise UnsupportedOperation("BufferedStringReader does not support truncate()")
def write(self, s):
raise UnsupportedOperation("BufferedStringReader does not support write()")
def writelines(self, s):
raise UnsupportedOperation("BufferedStringReader does not support writelines()")
def writable(self):
return False
# To use:
reader = BufferedStringReader("something", "/dev/urandom", "rb")
print(reader)
assert reader.readable() == True
assert reader.seekable() == True
assert reader.writable() == False
assert reader.isatty() == False
assert reader.tell() == 0
assert reader.read(4) == b"some"
assert reader.tell() == 4
reader.seek(pos=-2, whence=1)
assert reader.read(2) == b"me"
reader.seek(pos=0, whence=0)
assert reader.tell() == 0
print(reader.read(12)) # "something<First 3 characters from /dev/urandom>"
assert reader.tell() == 12
print(reader.read(12)) # "<Next 12 characters from /dev/urandom>"
assert reader.tell() == 24
zero_array = bytearray(0)
assert reader.readinto(zero_array) == 0
assert zero_array == b''
zero_array = bytearray(10)
assert reader.readinto(zero_array) == 10
print(zero_array)
reader.seek(pos=0, whence=0)
assert reader.peek(4) == b"some"
assert reader.read(4) == b"some"
assert reader.tell() == 4
print(reader.peek(8)) # "thing<First 3 characters from /dev/urandom>"
print(reader.read(8)) # Note: This is a block device, so output may not be the same as above.
assert reader.tell() == 12
print(reader.peek(8)) # "FNext20 characters from /dev/urandom>"
assert reader.tell() == 12
reader.close()
reader = BufferedStringReader("something\nspam\neggs\n", "/dev/urandom", "rb")
assert reader.readlines() == b"something\n"
assert reader.readlines(3) == b"spa"
assert reader.readlines() == b"m\n"
assert reader.readlines(5) == b"eggs\n"
reader.detach()
reader = BufferedStringReader("something\nspam\neggs\n", "/dev/urandom", "rb")
try:
reader.flush()
except UnsupportedOperation as e:
print(e)
try:
reader.getvalue()
except UnsupportedOperation as e:
print(e)
try:
reader.read(size=-1)
except UnsupportedOperation as e:
print(e)
try:
reader.read1(10)
except UnsupportedOperation as e:
print(e)
try:
reader.readinto1(10)
except UnsupportedOperation as e:
print(e)
try:
reader.seek(pos=100, whence=2)
except UnsupportedOperation as e:
print(e)
try:
reader.truncate(10)
except UnsupportedOperation as e:
print(e)
try:
reader.write('eggs')
except UnsupportedOperation as e:
print(e)
try:
reader.writelines('eggs')
except UnsupportedOperation as e:
print(e)
try:
reader = BufferedStringReader("something\nspam\neggs\n", "/dev/urandom", "wb")
except ValueError as e:
print(e)
Here are some examples of using BufferedStringReader:
>>> reader = BufferedStringReader("something", "/dev/urandom", "rb")
>>> bytes_object = reader.read(20)
>>> # Because this is /dev/urandom the following output might be different than this.
>>> print(bytes_object)
b'somethingc\xab\xf6\xab\xea\xd1q C(\x05'
I have a simple streamreader that is listening to a TCP port for a websocket implementation. The stream occasionally reads (every 30-300 second) blank data and throws an error.
loop = asyncio.get_event_loop()
coro = asyncio.start_server(handle_echo, 'not.real.ip.address', 8888, loop=loop)
server = loop.run_until_complete(coro)
server.close()
loop.run_until_complete(server.wait_closed())
loop.close()
#asyncio.coroutine
def handle_echo(reader, writer):
#Starts the connection with upgrade handshake
while True:
#read frame data
first_byte = yield from reader.readexactly(1)
opcode = first_byte & 0xf
second_byte = yield from reader.readexactly(1)
second_byte = ord(second_byte)
mask = second_byte >> 7
len1 = second_byte & 0x7f
if len1 == 126:
len2 = yield from reader.readexactly(2)
length = int.from_bytes(len2, byteorder='big')
elif len1 == 127:
len2 = yield from reader.readexactly(8)
length = int.from_bytes(len2, byteorder='big')
else:
length = len1
if mask:
masking_key = yield from reader.readexactly(4)
if length:
payload = yield from reader.readexactly(length)
payload = bytearray(payload)
if mask:
for i,b in enumerate(payload):
payload[i] = payload[i] ^ masking_key[i%4]
if opcode & 0x1:
payload = payload.decode("UTF-8")
Every once in a while, the following error is thrown:
Future/Task exception was never retrieved
future: Task(<handle_echo>)<exception=IncompleteReadError('0 bytes read on a total of 1 expected bytes',)>
Traceback (most recent call last):
File "/usr/lib/python3.4/asyncio/tasks.py", line 300, in _step
result = coro.send(value)
File "server.py", line 76, in handle_echo
first_byte = yield from reader.readexactly(1)
File "/usr/lib/python3.4/asyncio/streams.py", line 464, in readexactly
raise IncompleteReadError(partial, len(partial) + n)
asyncio.streams.IncompleteReadError: 0 bytes read on a total of 1 expected bytes
I am having a hard time tracking down what is causing this. I have used tcptrack to watch the port, and nothing is coming through. Am I improperly reading from the port, is there some sort of clean up that I need to do, or does the reader occasionally misread? I have tried other read functions (read and readline), and they throw similar errors occasionally.
readexactly() is a picky method. From the documentation:
Read exactly n bytes. Raise an IncompleteReadError if the end of the stream is reached before n can be read, the IncompleteReadError.partial attribute of the exception contains the partial read bytes.
Since the server seems to be sending empty bytes strings sometimes, it is probably a good idea for you to put the readexactly() call on a try / except block:
try:
first_byte = yield from reader.readexactly(1)
except asyncio.IncompleteReadError:
pass
else:
# Rest of your code
You could even check if you received more than 0 (zero) bytes and buffer them if your use case requires it:
except asyncio.IncompleteReadError as e:
if len(e.partial) > 0:
# Handle the bytes read
else:
pass