Python http.client.Incomplete Read(0 bytes read) error - python-3.x

Question

I have seen this error on the forum and read through the responses yet I still don't understand what it is or how to address it. I'm scraping data from the internet from 16k links, my script scrapes similar information from each link and writes it to a .csv some of the date gets written before this error.
Traceback (most recent call last):
File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 541, in _get_chunk_left
chunk_left = self._read_next_chunk_size()
File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 508, in _read_next_chunk_size
return int(line, 16)
ValueError: invalid literal for int() with base 16: b''
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 558, in _readall_chunked
chunk_left = self._get_chunk_left()
File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 543, in _get_chunk_left
raise IncompleteRead(b'')
http.client.IncompleteRead: IncompleteRead(0 bytes read)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "MoviesToDb.py", line 91, in <module>
html = r.read()
File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 455, in read
return self._readall_chunked()
File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 565, in _readall_chunked
raise IncompleteRead(b''.join(value))
http.client.IncompleteRead: IncompleteRead(17891 bytes read)
I would like to know:1) What does this error mean? 2) How do I prevent it?

Answer 1

try to import :
from http.client import IncompleteRead
and add this in your script :
except IncompleteRead:
# Oh well, reconnect and keep trucking
continue

Answer 2

requests.exceptions.ChunkedEncodingError: (‘Connection broken: IncompleteRead(0 bytes read)’, IncompleteRead(0 bytes read)).
It is because the server of http protocal is 1.0 version，while python use 1.1 version. The solution is to assign the protocal version of client, like this
Python3 Version please add:
> import http.client
> http.client.HTTPConnection._http_vsn = 10
> http.client.HTTPConnection._http_vsn_str = 'HTTP/1.0'
Python2 Version please add:
> import http.client
> http.client.HTTPConnection._http_vsn = 10
> http.client.HTTPConnection._http_vsn_str = 'HTTP/1.0'
See the reference How to deal with "http.client.IncompleteRead: IncompleteRead(0 bytes read)" problem