I'm trying to link my IP camera to my a AWS service and there's 2 ways I can do this, either with my built in computer camera (runs fine) and a IP camera. The code I'm using is from https://github.com/aws-samples/amazon-rekognition-video-analyzer which is writing in Python 2.7(but I'm doing it in python 3), I already converted the code to python 3 (using python 2to3).but when I run the code I keep getting this error of only concatenate strings not bytes:
I'm new to python so for what I have research is that 2to3 will do must of the work, but I'm pretty sure this part of converting bytes to strings is not in there and I'm not sure how to handle this conversion/parsing.
Traceback (most recent call last):
File "video_cap_ipcam.py", line 140, in <module>
main()
File "video_cap_ipcam.py", line 104, in main
bytes += stream.read(16384*2)
TypeError: can only concatenate str (not "bytes") to str
video_cap_ipcam.py file:
def main():
ip_cam_url = ''
capture_rate = default_capture_rate
argv_len = len(sys.argv)
if argv_len > 1:
ip_cam_url = sys.argv[1]
if argv_len > 2 and sys.argv[2].isdigit():
capture_rate = int(sys.argv[2])
else:
print("usage: video_cap_ipcam.py <ip-cam-url> [capture-rate]")
return
print(("Capturing from '{}' at a rate of 1 every {} frames...".format(ip_cam_url, capture_rate)))
stream = urllib.request.urlopen(ip_cam_url)
bytes = ''
pool = Pool(processes=3)
frame_count = 0
while True:
# Capture frame-by-frame
frame_jpg = ''
bytes += stream.read(16384*2)
b = bytes.rfind('\xff\xd9')
a = bytes.rfind('\xff\xd8', 0, b-1)
if a != -1 and b != -1:
#print 'Found JPEG markers. Start {}, End {}'.format(a,b)
frame_jpg_bytes = bytes[a:b+2]
bytes = bytes[b+2:]
if frame_count % capture_rate == 0:
img_cv2_mat = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
rotated_img = cv2.transpose(cv2.flip(img_cv2_mat, 0))
retval, new_frame_jpg_bytes = cv2.imencode(".jpg", rotated_img)
#Send to Kinesis
result = pool.apply_async(send_jpg, (bytearray(new_frame_jpg_bytes), frame_count, True, False, False,))
frame_count += 1
if __name__ == '__main__':
main()
When you initially set the variable bytes to '', the variable becomes a string, which in Python 3 is considered a sequence of characters rather than a sequence of bytes. (A character can be expressed using multiple bytes.)
If you want bytes to be a sequence of bytes, initialize it as b'' instead. Then you can concatenate further bytes to it.
Related
I have written a python read script which has the following code`
def read_serial():
while True:
# prepare for response
out = ''
while ser.inWaiting() > 0:
out += ser.read(1).decode('unicode_escape')
if out != '':
response_header = out.encode('unicode_escape').strip()
print( "Response recieved " ,response_header)
print((response_header[0],response_header[1],response_header[2])) //line 2
So the bytes array that I'm sending is:
data[] = {0x9c,0x10,0x01,0x05,0x07,0x08};
and on python side..in line 2 I'm receiving this:
Response recieved b'\\\x9c\\\x10\\\x01\\\x05\\\x07\\\x08'
(92, 120, 57)
Why 92,120,57 what does this mean because for 9C decimal would be 156?
Any suggestions? Or which encoding technique you suggest for bytes?
Just receive the bytes. Something like this since I can't run your code:
def read_serial():
while True:
# prepare for response
out = b'' # bytes type
while ser.inWaiting() > 0:
out += ser.read(1)
if out != b'':
print("Response received:",out.hex())
I am trying to use a modified MFRC522 module on UART, but I am stuck finding the correct formatting for serial read/write in python 3.
There is a library for python 2 which works for me:
The readme on that link also describes the modification to the module.
I'm having trouble getting the .self.ser.write and .self.ser.read in functions writeRegister and readRegister to work in python 3. These get string inputs, which I now understand is fine for python 2, but must be converted to bytes for python 3.
def writeRegister(self, addr, val, size=None):
if size is None:
count = 0
while True:
self.ser.flushInput()
self.ser.write(chr(addr&0x7F))
self.ser.write(chr(val))
tmp = ord(self.ser.read(1))
if(tmp == addr):
return True
count+=1
if(count > 10):
print ("Error de escritura en: "+ hex(addr))
return False
else:
self.ser.flushInput()
for txBytes in range (0, size):
self.ser.write(chr(addr&0x7F))
tmp = ord(self.ser.read(1))
if(tmp == addr):
self.ser.write(chr(val[txBytes]))
else:
print ("Error de escritura en bloque")
return False
return True
def readRegister(self, addr):
self.ser.flushInput()
self.ser.write(chr(addr|0x80))
val = self.ser.read(1)
return ord(val)
I suspected it's a matter of correctly applying .encode('utf-8') or similar, but I can't find a working solution. If I try
chr(0x80).encode('utf-8')
I get
UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 0: ordinal not in range(128)
Maybe I am going down the wrong path with that.
I'll try bytes:
bytes(chr(0x80), 'utf-8')
Gives 2 bytes in this case (>128 i guess):
b'\xc2\x80'
Maybe this is getting closer, but I get stuck on how to read back and decode. I don't know how or if to modify the ord parts. So I can't seem to get the response from the mfrc522.
Does anyone have any suggestions?
Anybody successfully used this mfrc522 UART on python 3?
There should be no need to use char in either Python 2.x or 3.x, you can just write bytes to the port:
self.ser.write(bytes[addr&0x7F])
For the reading function just drop the ord() from the return sentence, it's not needed for Python 3.x.
See here for more details.
That did it, thanks!
Here are the corrected functions:
def writeRegister(self, addr, val, size=None):
if size is None:
count = 0
while True:
self.ser.flushInput()
self.ser.write(bytes([addr&0x7F]))
self.ser.write(bytes([val]))
tmp = int.from_bytes(self.ser.read(1), "big")
if(tmp == addr):
return True
count+=1
if(count > 10):
print ("Write error at: "+ hex(addr))
return False
else:
self.ser.flushInput()
if(tmp == addr):
self.ser.write(bytes([val]))
else:
print ("Block write error")
return False
return True
def readRegister(self, addr):
self.ser.flushInput()
self.ser.write(bytes([addr|0x80]))
val = self.ser.read(1)
return int.from_bytes(val, "big")
I have a binary string (bytes), and a potentially infinite byte stream. I need to concatenate them into a single byte stream in Python 3.6.
The concatenation must act like a standard byte stream, returning bytes from the string first, and then from the byte stream:
string = b'something'
stream = open(filename, 'rb')
concatenated = ?concat?(string, stream) # <=== need this
x = concantenated.read(5) # b"somet"
y = concatenated.read(2) # b"hi"
z = concatentated.read(26) # b"ngFIRST_24_BYTES_OF_STREAM"
…
I created a class called BufferedStringReader which functions as a byte steam, reading the string first and then some infinite byte stream next. I have assumed that the byte stream you're talking about is a file, like a block device or character device. Those are the only infinite byte streams I'm aware of though.
A quick briefing of how I implemented it: My class was not inherited from IOBase, BufferedReader or any other python IO object because many of the members and methods had the same names. Since this class must act like both a string stream and a file stream, these objects are stored as members, my class does not inherit from them.
I decided against using the tell() method in BufferReader to implement position because there is a weird quirk when using (some BufferReader object).tell(): Instead of returning a positive number, it returns a value between the negative of some block size and 0. So when the file is opened, the position is zero, but when you start reading from it, the position is at -4096 + the actual position, which wraps around after reading 4096 bytes. And the block size may not even be 4096 on your system, so it's very wacky. (Let me know if it's different on yours.)
Also, since your question is about infinite byte streams, my class does not support writing. For the case of block devices which I had taken as an example, it's not supported anyway.
The final thing is that unlimited reads aren't supported because it's used with infinite streams anyway, such calls would run out of memory.
from io import BufferedReader, StringIO, UnsupportedOperation
# Because this is a read-only stream, write methods raise an exception.
class BufferedStringReader:
def __init__(self, string, name, mode):
if mode != 'rb' and mode != 'br':
raise ValueError("invalid mode: " + mode)
self.reader = open(name, mode)
self.stringio = StringIO(string)
self.string = string
self.stringlen = len(string)
self.readerpos = 0
# self.reader.tell() doesn't seem to work properly
self.pos = self.stringio.tell() + self.readerpos
self.closed = (self.reader.closed and self.stringio.closed)
# This is my own method, not a method in the io module
def __update_state__(self):
self.pos = self.stringio.tell() + self.readerpos
self.closed = (self.reader.closed and self.stringio.closed)
def __repr__(self):
return "{BufferedStringReader reader=" + str(self.reader) + " stringio=" + str(self.stringio) + "}"
def getvalue(self):
# Reads the entire file. Since this is an infinite byte stream this is not supported.
raise UnsupportedOperation("BufferedStringReader does not support getvalue()")
def close(self):
self.reader.close()
self.stringio.close()
self.closed = True
def flush(self):
raise UnsupportedOperation("BufferedStringReader does not support flush()")
# This function leaves the file stream in an undefined state
def detach(self):
self.reader.detach()
#self.stringio.detach() Not supported
def isatty(self):
return False
def peek(self, size=0):
# We cannot guarrentee one single read will be done in this class but
# we can guarrentee that only one read will be done to the BufferedReader.
# Note that the number of bytes returned may be less or more than requested.
# Also in BufferedReader the size can apparently be negative and it's ignored
# in any case (on my linux system it returns 4K bytes even for positive size).
# Long story short; Python io's peek function ignores size.
# Nevertheless, we *try* to return exactly size bytes.
peek_size = (size if size > 0 else 1)
string_pos = self.stringio.tell()
string_read = self.string[string_pos:string_pos+peek_size]
if len(string_read) < peek_size:
reader_read = self.reader.peek(peek_size)
else:
reader_read = b''
return (bytes(string_read, 'utf-8') + reader_read)[0:peek_size]
def read(self, size=-1):
# size=-1 will cause your machine to run out of memory since you're dealing with
# and infinitely sized byte stream.
if size <= -1:
raise UnsupportedOperation("BufferedStringReader does not support read(size=-1)")
string_read = ''
reader_read = b''
if self.pos <= self.stringlen and self.pos+size <= self.stringlen:
string_read = self.stringio.read(size);
elif self.pos <= self.stringlen and self.pos+size > self.stringlen:
# No exceptions below, will return a smaller string if necessary
string_read = self.stringio.read(size);
reader_read = self.reader.read(size-len(string_read));
elif self.pos > self.stringlen:
reader_read = self.reader.read(size);
else:
# Impossible condition, detatch for security
raise RuntimeError
self.readerpos += len(reader_read)
self.__update_state__()
return bytes(string_read, 'utf-8') + reader_read;
def readable(self):
return True
def read1(self, size=-1):
# This is impossible to implement due to the arhitecture of this class.
# This method is supposed to call read() exactly once but since there are
# two IO streams, it is not possible.
raise UnsupportedOperation("BufferedStringReader does not support read1()")
def readinto(self, bytarray):
temp_read = self.read(len(bytarray))
temp_array = bytearray(temp_read)
if len(temp_array) <= 0:
return 0
bytarray[0:len(temp_array)] = temp_array
return len(temp_array)
def readinto1(self, bytarray):
raise UnsupportedOperation("BufferedStringReader does not support readinto1()")
def readlines(self, size=-1):
# Unlike read(), we stop when we encounter \n. No exceptions.
# Note that it's still possible we run out of memory if a large enough
# size is used and we don't find a newline.
# A trailing newline is kept in the string.
read_size = 0;
if size >= 0:
# If size is given try to find up to size
temp_read = self.peek(size)
newline = temp_read.find(b'\n')
if newline == -1:
read_size = len(temp_read)
else:
read_size = newline+1
else:
# Keep increasing size by 4K until newline is found
newline = -1
while newline == -1:
read_size += 4096
temp_read = self.peek(read_size);
newline = temp_read.find(b'\n')
read_size = newline+1
return self.read(read_size);
def seek(self, pos, whence=0):
if whence==2: # Seek from end of infinite file doesn't make sense
UnsupportedOperation("BufferedStringReader does not support seek(whence=2)")
elif whence==1:
self.pos = max(self.pos+pos, 0)
elif whence==0:
if pos < 0:
raise OSError("[Errno 22] Invalid argument")
self.pos = pos
else:
raise ValueError("whence value " + str(whence) + " unsupported")
self.stringio.seek(min(self.pos, self.stringlen), 0)
self.reader.seek(max(self.pos - self.stringlen, 0), 0)
self.readerpos = max(self.pos - self.stringlen, 0)
self.__update_state__()
return self.pos
def seekable(self):
return True
def tell(self):
return self.pos
def truncate(self, pos=None):
raise UnsupportedOperation("BufferedStringReader does not support truncate()")
def write(self, s):
raise UnsupportedOperation("BufferedStringReader does not support write()")
def writelines(self, s):
raise UnsupportedOperation("BufferedStringReader does not support writelines()")
def writable(self):
return False
# To use:
reader = BufferedStringReader("something", "/dev/urandom", "rb")
print(reader)
assert reader.readable() == True
assert reader.seekable() == True
assert reader.writable() == False
assert reader.isatty() == False
assert reader.tell() == 0
assert reader.read(4) == b"some"
assert reader.tell() == 4
reader.seek(pos=-2, whence=1)
assert reader.read(2) == b"me"
reader.seek(pos=0, whence=0)
assert reader.tell() == 0
print(reader.read(12)) # "something<First 3 characters from /dev/urandom>"
assert reader.tell() == 12
print(reader.read(12)) # "<Next 12 characters from /dev/urandom>"
assert reader.tell() == 24
zero_array = bytearray(0)
assert reader.readinto(zero_array) == 0
assert zero_array == b''
zero_array = bytearray(10)
assert reader.readinto(zero_array) == 10
print(zero_array)
reader.seek(pos=0, whence=0)
assert reader.peek(4) == b"some"
assert reader.read(4) == b"some"
assert reader.tell() == 4
print(reader.peek(8)) # "thing<First 3 characters from /dev/urandom>"
print(reader.read(8)) # Note: This is a block device, so output may not be the same as above.
assert reader.tell() == 12
print(reader.peek(8)) # "FNext20 characters from /dev/urandom>"
assert reader.tell() == 12
reader.close()
reader = BufferedStringReader("something\nspam\neggs\n", "/dev/urandom", "rb")
assert reader.readlines() == b"something\n"
assert reader.readlines(3) == b"spa"
assert reader.readlines() == b"m\n"
assert reader.readlines(5) == b"eggs\n"
reader.detach()
reader = BufferedStringReader("something\nspam\neggs\n", "/dev/urandom", "rb")
try:
reader.flush()
except UnsupportedOperation as e:
print(e)
try:
reader.getvalue()
except UnsupportedOperation as e:
print(e)
try:
reader.read(size=-1)
except UnsupportedOperation as e:
print(e)
try:
reader.read1(10)
except UnsupportedOperation as e:
print(e)
try:
reader.readinto1(10)
except UnsupportedOperation as e:
print(e)
try:
reader.seek(pos=100, whence=2)
except UnsupportedOperation as e:
print(e)
try:
reader.truncate(10)
except UnsupportedOperation as e:
print(e)
try:
reader.write('eggs')
except UnsupportedOperation as e:
print(e)
try:
reader.writelines('eggs')
except UnsupportedOperation as e:
print(e)
try:
reader = BufferedStringReader("something\nspam\neggs\n", "/dev/urandom", "wb")
except ValueError as e:
print(e)
Here are some examples of using BufferedStringReader:
>>> reader = BufferedStringReader("something", "/dev/urandom", "rb")
>>> bytes_object = reader.read(20)
>>> # Because this is /dev/urandom the following output might be different than this.
>>> print(bytes_object)
b'somethingc\xab\xf6\xab\xea\xd1q C(\x05'
# Python program to count and
# print all palindrome numbers in a list.
def palindromeNumbers(list_a):
c = 0
# loop till list is not empty
for i in list_a:
# Find reverse of current number
t = i
rev = 0
while t > 0:
rev = rev * 10 + t % 10
t = t / 10
# compare rev with the current number
if rev == i:
print (i),
c = c + 1
print
print ("Total palindrome nos. are" + str(c))
print
def main():
list_a = [10, 121, 133, 155, 141, 252]
palindromeNumbers(list_a)
list_b = [ 111, 220, 784, 565, 498, 787, 363]
palindromeNumbers(list_b)
if __name__ == "__main__":
main() # main function call
This code, obtained from
https://www.geeksforgeeks.org/all-palindrome-numbers-in-a-list/, has been written in Python 2.
When I run this program in Python 3.6 it returns the value as 0 for both lists.
Can someone tell me how to change it to be compatible with Python 3?
One of the important changes between Python2 and Python3 is integer division, that in Python2 returns a truncated, integer result while in Python3 returns a floating point number. To have a real integer division you have to use a double slash, "//".
In summary, change the line t = t/10 to t = t//10.
Ok, so I have changed the code a bit using a different method to check if the number is the same reversed... I tried not to change too much of your code...
The function reverse() just returns the string given to it reversed...
If I'm not mistaken the function makes a list out of the str by split function then [::-1] gives the list reversed...
def reverse(str):
return str[::-1]
def palindromeNumbers(list_a):
c = 0
# loop till list is not empty
for i in list_a:
# Find reverse of current number
t = i
t = int(reverse((str(t))))
if t == i:
c += 1
print ("Total palindrome nos. are " + str(c))
def main():
list_a = [10, 121, 133, 155, 141, 252]
palindromeNumbers(list_a)
list_b = [ 111, 220, 784, 565, 498, 787, 363]
palindromeNumbers(list_b)
if __name__ == "__main__":
main() # main function call
Try it out! I hope you find this helpful...
I first converted the number to an iterable string str(i). Then I shortened the for loop by only comparing the first half of the number t[a] to the second half of the number t[~a], and then using all() to check that all of the comparisons were true. (using Python 3.6.8)
for i in list_a:
# convert interger to iterable string
t = str(i)
# compare first half of the number to second half of the number
if all([t[a]==t[~a] for a in range(len(t)//2)]):
print (i),
c = c + 1
I have a simple streamreader that is listening to a TCP port for a websocket implementation. The stream occasionally reads (every 30-300 second) blank data and throws an error.
loop = asyncio.get_event_loop()
coro = asyncio.start_server(handle_echo, 'not.real.ip.address', 8888, loop=loop)
server = loop.run_until_complete(coro)
server.close()
loop.run_until_complete(server.wait_closed())
loop.close()
#asyncio.coroutine
def handle_echo(reader, writer):
#Starts the connection with upgrade handshake
while True:
#read frame data
first_byte = yield from reader.readexactly(1)
opcode = first_byte & 0xf
second_byte = yield from reader.readexactly(1)
second_byte = ord(second_byte)
mask = second_byte >> 7
len1 = second_byte & 0x7f
if len1 == 126:
len2 = yield from reader.readexactly(2)
length = int.from_bytes(len2, byteorder='big')
elif len1 == 127:
len2 = yield from reader.readexactly(8)
length = int.from_bytes(len2, byteorder='big')
else:
length = len1
if mask:
masking_key = yield from reader.readexactly(4)
if length:
payload = yield from reader.readexactly(length)
payload = bytearray(payload)
if mask:
for i,b in enumerate(payload):
payload[i] = payload[i] ^ masking_key[i%4]
if opcode & 0x1:
payload = payload.decode("UTF-8")
Every once in a while, the following error is thrown:
Future/Task exception was never retrieved
future: Task(<handle_echo>)<exception=IncompleteReadError('0 bytes read on a total of 1 expected bytes',)>
Traceback (most recent call last):
File "/usr/lib/python3.4/asyncio/tasks.py", line 300, in _step
result = coro.send(value)
File "server.py", line 76, in handle_echo
first_byte = yield from reader.readexactly(1)
File "/usr/lib/python3.4/asyncio/streams.py", line 464, in readexactly
raise IncompleteReadError(partial, len(partial) + n)
asyncio.streams.IncompleteReadError: 0 bytes read on a total of 1 expected bytes
I am having a hard time tracking down what is causing this. I have used tcptrack to watch the port, and nothing is coming through. Am I improperly reading from the port, is there some sort of clean up that I need to do, or does the reader occasionally misread? I have tried other read functions (read and readline), and they throw similar errors occasionally.
readexactly() is a picky method. From the documentation:
Read exactly n bytes. Raise an IncompleteReadError if the end of the stream is reached before n can be read, the IncompleteReadError.partial attribute of the exception contains the partial read bytes.
Since the server seems to be sending empty bytes strings sometimes, it is probably a good idea for you to put the readexactly() call on a try / except block:
try:
first_byte = yield from reader.readexactly(1)
except asyncio.IncompleteReadError:
pass
else:
# Rest of your code
You could even check if you received more than 0 (zero) bytes and buffer them if your use case requires it:
except asyncio.IncompleteReadError as e:
if len(e.partial) > 0:
# Handle the bytes read
else:
pass