Python requests module exception SSLError - python-3.x

I am getting a requests module exception error when I try to access the http://www.acastipharma.com/ website. I am not having problems with any other website so I believe this is a website specific issue. Here is some example code
import requests
initialURL = 'http://www.acastipharma.com/'
r = requests.get(initialURL)
When I run this code I get an error message that terminates with
requests.exceptions.SSLError: HTTPSConnectionPool(host='www.acastipharma.com', port=443): Max retries exceeded with url: /investors/ (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:645)'),))
An internet search indicates that the problem might be with acastipharma's SSL certificate. I tried installing pyopenssl to make sure I had the latest version of the the module thats checks SSL certificates but that did not solve the problem. I also tried running the requests.get statement with the verify=False option but that was also unsuccessful.
r = requests.get(initialURL, verify=False)
If anybody has any ideas on how to resolve this issue I would appreciate the assistance. I also tried using the older urllib.request package but ran into the same error.
This is an update to my original question: The error message I posted was from trying to run the requests command on one of the acastipharma's website subpages, here is the complete error message I get when I run the code exactly as shown in this question:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/urllib3/connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn
conn.connect()
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/urllib3/connection.py", line 326, in connect
ssl_context=context)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/urllib3/util/ssl_.py", line 329, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 376, in wrap_socket
_context=self)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 747, in __init__
self.do_handshake()
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 983, in do_handshake
self._sslobj.do_handshake()
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 628, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:645)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/adapters.py", line 440, in send
timeout=timeout
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/urllib3/connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/urllib3/util/retry.py", line 388, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.acastipharma.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:645)'),))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/sessions.py", line 640, in send
history = [resp for resp in gen] if allow_redirects else []
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/sessions.py", line 640, in <listcomp>
history = [resp for resp in gen] if allow_redirects else []
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/sessions.py", line 218, in resolve_redirects
**adapter_kwargs
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/requests/adapters.py", line 506, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='www.acastipharma.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:645)'),))
I am using Python 3.5.1. I am on a Mac using High Sierra version 10.13.2. I am using requests 2.18.4. Since posting the question I believe the problem lies with my IDE PyCharm's environment. If I use my Python 3.5 environment I have the problem as shown in this question, if I switch the project interpreter to a Python 3.6 Anaconda environment requests will work but unfortunately mysql won't import. Thanks

The OpenSSL version is OpenSSL 0.9.8zh 14 Jan 2016
This (very old and long unsupported) version of OpenSSL does not support the newer ciphers required by this specific web server. This server only supports ECDHE key exchange, which is not supported yet by OpenSSL 0.9.8. This means that the client only offers ciphers to the server which the server will not accept and due to no common ciphers the server will close the connection with an SSL handshake alert.

We can ignore this SSL Error using the following:
import warnings
from urllib3.exceptions import InsecureRequestWarning
warnings.simplefilter('ignore',InsecureRequestWarning)

Related

im getting a SSLV3_ALERT_HANDSHAKE_FAILURESSL

I am encountering some errors when using the requests library for making requests to a particular website. At times, it works fine, but other times it returns an error. Can anyone assist me in identifying the issue and providing a solution? I am aware of the verify=False option, but I cannot use it due to security concerns.
requests.exceptions.SSLError: HTTPSConnectionPool(host='*', port=443): Max retries exceeded with url: /token (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:997)')))
Traceback (most recent call last):
File /app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py, line 703, in urlopen
{code_line}
File /app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py, line 386, in _make_request
{code_line}
File /app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py, line 1042, in _validate_conn
{code_line}
File /app/.venv/lib/python3.10/site-packages/urllib3/connection.py, line 414, in connect
{code_line}
File /app/.venv/lib/python3.10/site-packages/urllib3/util/ssl_.py, line 449, in ssl_wrap_socket
{code_line}
File /app/.venv/lib/python3.10/site-packages/urllib3/util/ssl_.py, line 493, in _ssl_wrap_socket_impl
{code_line}
File /usr/local/lib/python3.10/ssl.py, line 513, in wrap_socket
{code_line}
File /usr/local/lib/python3.10/ssl.py, line 1071, in _create
{code_line}
File /usr/local/lib/python3.10/ssl.py, line 1342, in do_handshake
{code_line}
ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:997)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File /app/.venv/lib/python3.10/site-packages/requests/adapters.py, line 489, in send
{code_line}
File /app/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py, line 787, in urlopen
{code_line}
File /app/.venv/lib/python3.10/site-packages/urllib3/util/retry.py, line 592, in increment
{code_line}
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='*', port=443): Max retries exceeded with url: /token (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:997)')))
During handling of the above exception, another exception occurred:
I have also tried to update the request library and OpenSSL

openssl unable to get local issuer certificate- but why do I need it?

I admit that I'm not a sysadmin by training, so I don't "get" some things that come up. This is one of those times.
I was setting up a server for a project. Stupidly, because the last time I'd been shoved into a sysadmin role I was working with Centos7, I opted for that OS out of familiarity. When installing Python 3.10+ (I specifically need 3.10+ for some packages I intend to use), openssl was an issue; after a certain version, 1.0.2 (the last shipped version with Centos7) is not an option, and I had to spot-setup 1.1.1k for this Python3 installation to work.
So I have this Python3.10 installation working:
python3 -c "import ssl; print(ssl.OPENSSL_VERSION)"
OpenSSL 1.1.1k 25 Mar 2021
Now, I need to run queries against a remote postgres database's API using their Python3 sdk. I get openssl problems when I do. Minimum to reproduce:
from algosdk.v2client import indexer
indexer_url="https://mainnet-idx.algonode.cloud"
indexer_client = indexer.IndexerClient(indexer_token='', indexer_address=indexer_url)
indexer_client.search_assets(asset_id=0)
This fails:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/urllib/request.py", line 1348, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/usr/local/lib/python3.10/http/client.py", line 1282, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/lib/python3.10/http/client.py", line 1328, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.10/http/client.py", line 1277, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.10/http/client.py", line 1037, in _send_output
self.send(msg)
File "/usr/local/lib/python3.10/http/client.py", line 975, in send
self.connect()
File "/usr/local/lib/python3.10/http/client.py", line 1454, in connect
self.sock = self._context.wrap_socket(self.sock,
File "/usr/local/lib/python3.10/ssl.py", line 513, in wrap_socket
return self.sslsocket_class._create(
File "/usr/local/lib/python3.10/ssl.py", line 1071, in _create
self.do_handshake()
File "/usr/local/lib/python3.10/ssl.py", line 1342, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.10/site-packages/algosdk/v2client/indexer.py", line 801, in search_assets
return self.indexer_request("GET", req, query, **kwargs)
File "/usr/local/lib/python3.10/site-packages/algosdk/v2client/indexer.py", line 74, in indexer_request
resp = urlopen(req)
File "/usr/local/lib/python3.10/urllib/request.py", line 216, in urlopen
return opener.open(url, data, timeout)
File "/usr/local/lib/python3.10/urllib/request.py", line 519, in open
response = self._open(req, data)
File "/usr/local/lib/python3.10/urllib/request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/usr/local/lib/python3.10/urllib/request.py", line 496, in _call_chain
result = func(*args)
File "/usr/local/lib/python3.10/urllib/request.py", line 1391, in https_open
return self.do_open(http.client.HTTPSConnection, req,
File "/usr/local/lib/python3.10/urllib/request.py", line 1351, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)>
Okay. My initial impression was that this could be a firewall issue, so I put my firewall down. No change. Put the firewall back up. (_ssl.c:997) seems to mean "Unable to get local issuer certificate", and trying to find a resolution brings me to sites like this one.
This is where I start to lose my patience, because I just don't understand the issue. I've only handled ssl certificates in the context of setting up a website; I put up a site, the site needs a signature specifying certain things about it (which earns it an s at the end of http), so I obtain a certificate it and tie it to the site. That's the extent of my understanding. I'm not talking about a website here. Yes, I'll be hosting a website on this server, but I'm literally just talking about a Python3 script. The error above seems to indicate that the issue is that I don't have an ssl certificate. Why does that matter? This code runs on my local machine, and I never had to obtain an ssl certificate.
tl;dr: help a noob out, please. I don't understand the problem, and that also means that I don't know what to look up to fix it.

Is there an Eclipse Pydev setting or something in my Python get request preventing me from connecting with a host?

I am trying to read data from an ESA server in a PyDev Eclipse 3.8.1 Environment using Python 3.5.3. An example of a product link is given here:
"https://scihub.copernicus.eu/dhus/odata/v1/Products('c7208694-dedb-4f47-96c0-c8fb03512ff5')
You will need authentication credentials to access the website.
In my Eclipse environment I have manually added in proxy settings by going to Windows-> General -> Network Connections. Authentication is not required for the proxy.
To get the content of the webpage, I use a Python get request to send my query as below:
url = r"https://scihub.copernicus.eu/dhus/odata/v1/Products('c5dc59f0-b041-4f76-a685-49be63491270')"
r = requests.get(url, verify=False, proxies=proxies, auth=auth)
I need the verify=False to disable the certificate check and proxies is a dictionary storing the proxy addresses with the relevant port. Server credentials are given with auth which is a tuple of a username and password.
When I send the request to the server, I get the following error:
Traceback (most recent call last):
File "/lib/python3.5/site-packages/urllib3/connection.py", line 159, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/lib/python3.5/site-packages/urllib3/util/connection.py", line 80, in create_connection
raise err
File "/lib/python3.5/site-packages/urllib3/util/connection.py", line 70, in create_connection
sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/lib/python3.5/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/lib/python3.5/site-packages/urllib3/connectionpool.py", line 343, in _make_request
self._validate_conn(conn)
File "/lib/python3.5/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
conn.connect()
File "/lib/python3.5/site-packages/urllib3/connection.py", line 301, in connect
conn = self._new_conn()
File "/lib/python3.5/site-packages/urllib3/connection.py", line 168, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7fecf00acbe0>: Failed to establish a new connection: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/lib/python3.5/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/lib/python3.5/site-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/lib/python3.5/site-packages/urllib3/util/retry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='scihub.copernicus.eu', port=443): Max retries exceeded with url: /dhus/odata/v1/Products('c5dc59f0-b041-4f76-a685-49be63491270') (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fecf00acbe0>: Failed to establish a new connection: [Errno 110] Connection timed out',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "download_lib2/download_lib/scihub/download.py", line 325, in <module>
download_url(dl_link, auth, temp_dir)
File "download_lib2/download_lib/scihub/download.py", line 153, in download_url
online = check_url_online(url_to_download, auth)
File "download_lib2/download_lib/scihub/download.py", line 51, in check_url_online
r = requests.get(url, verify=False, proxies=proxies, auth=auth)
File "/lib/python3.5/site-packages/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/lib/python3.5/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/lib/python3.5/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/lib/python3.5/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/lib/python3.5/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='scihub.copernicus.eu', port=443): Max retries exceeded with url: /dhus/odata/v1/Products('c5dc59f0-b041-4f76-a685-49be63491270') (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fecf00acbe0>: Failed to establish a new connection: [Errno 110] Connection timed out',))
I have tried the same request outside of Eclipse, in IDLE, and I am able to successfully connect and read the content. Is there something wrong with my current Eclipse/PyDev environment that will not allow me to send a get request?
Why is Python not able to connect? I am able to send get requests to this and other websites using other projects in Eclipse (e.g. https://podaac-opendap.jpl.nasa.gov/opendap/allData/oscar/preview/L4/oscar_third_deg/) with no problem whatsoever.
Edit: Added in versions
The socks timeout indicates that your problem is actually the connection to your proxy as opposed to the ESA server. Additionally, I believe the proxy settings in Eclipse are only used for plugins/software updates and not by any code you write.
As an initial step, check your proxy dictionary keys are correct and verify your proxy URLS if you can.
You mention you have other Eclipse projects that can access the internet; it's worth checking what proxy settings they implement and seeing if there are any discrepancies with your current setup.

Accessing https pages with python3

I do not understand how to use urllib3 or requests to connect to an https web site. This is driving me nuts. I have installed certifi and I see the default .pem file it provides. I have tried to set the requests.verify option to requests to every .pem and .crt file on the machine my script runs on [I am not an admin on this device]. I get nothing but errors.
I switched to using urllib3 and am now getting:
H:\Projects\MyScraper\venv\Scripts\python.exe H:/Projects/MyScraper/MyScraper.py
Traceback (most recent call last):
File "H:\Projects\MyScraper\venv\lib\site-packages\urllib3\connectionpool.py", line 600, in urlopen
chunked=chunked)
File "H:\Projects\MyScraper\venv\lib\site-packages\urllib3\connectionpool.py", line 343, in _make_request
self._validate_conn(conn)
File "H:\Projects\MyScraper\venv\lib\site-packages\urllib3\connectionpool.py", line 839, in _validate_conn
conn.connect()
File "H:\Projects\MyScraper\venv\lib\site-packages\urllib3\connection.py", line 344, in connect
ssl_context=context)
File "H:\Projects\MyScraper\venv\lib\site-packages\urllib3\util\ssl_.py", line 342, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "C:\Program Files (x86)\Python36-32\lib\ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "C:\Program Files (x86)\Python36-32\lib\ssl.py", line 814, in __init__
self.do_handshake()
File "C:\Program Files (x86)\Python36-32\lib\ssl.py", line 1068, in do_handshake
self._sslobj.do_handshake()
File "C:\Program Files (x86)\Python36-32\lib\ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "H:/Projects/MyScraper/MyScraper.py", line 15, in <module>
raw_html = HTTP.request('GET', 'https://portal.xsede.org/course-calendar/')
File "H:\Projects\MyScraper\venv\lib\site-packages\urllib3\request.py", line 68, in request
**urlopen_kw)
File "H:\Projects\MyScraper\venv\lib\site-packages\urllib3\request.py", line 89, in request_encode_url
return self.urlopen(method, url, **extra_kw)
File "H:\Projects\MyScraper\venv\lib\site-packages\urllib3\poolmanager.py", line 323, in urlopen
response = conn.urlopen(method, u.request_uri, **kw)
File "H:\Projects\MyScraper\venv\lib\site-packages\urllib3\connectionpool.py", line 667, in urlopen
**response_kw)
File "H:\Projects\MyScraper\venv\lib\site-packages\urllib3\connectionpool.py", line 667, in urlopen
**response_kw)
File "H:\Projects\MyScraper\venv\lib\site-packages\urllib3\connectionpool.py", line 667, in urlopen
**response_kw)
[Previous line repeated 6 more times]
File "H:\Projects\MyScraper\venv\lib\site-packages\urllib3\connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "H:\Projects\MyScraper\venv\lib\site-packages\urllib3\util\retry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='portal.xsede.org', port=443): Max retries exceeded with url: /course-calendar/ (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)'),))
Process finished with exit code 1
My code looks like:
#!/home/me/virtualenv/python3.6/3.6/bin/python
import certifi
import urllib3
from bs4 import BeautifulSoup
HTTP = urllib3.PoolManager(
cert_reqs='CERT_REQUIRED',
ca_certs=certifi.where(),
retries=10
)
raw_html = HTTP.request('GET', 'https://portal.xsede.org/course-calendar/')
html = BeautifulSoup(raw_html, 'html.parser')
It blows up on the raw_html = HTTP.request(... line. Ideas?
Edit
Huh, this has something to do with my target host. If I go to google.com then several of my pem/crt files work.
The problem is, you are using wrong certificate to make request.
you can run this command to verify which certificate is used when any request is made, and then use that certificate in your request,
openssl s_client -showcerts -connect google.com:443
Please also make sure that you are passing verify the path to CA_BUNDLE file or directory with certificates of trusted CAs.
This list of trusted CAs can also be specified through the REQUESTS_CA_BUNDLE environment variable.
If this doesn't work out for you can explicitly merge the environment settings into your session,
When you are using the prepared request flow, keep in mind that it
does not take into account the environment. This can cause problems if
you are using environment variables to change the behaviour of
requests. For example: Self-signed SSL certificates specified in
REQUESTS_CA_BUNDLE will not be taken into account. As a result an SSL:
CERTIFICATE_VERIFY_FAILED is thrown. You can get around this behaviour
by explicity merging the environment settings into your session:
from requests import Request, Session
s = Session()
req = Request('GET', url)
prepped = s.prepare_request(req)
# Merge environment settings into session
settings = s.merge_environment_settings(prepped.url, None, None, None, None)
resp = s.send(prepped, **settings)
print(resp.status_code)

SSLError using Requests and Python 3.6

I'm getting an SSLError while submitting a request against https://myaccount.xcelenergy.com (and am not nearly enough of an SSL expert to understand why). Can you help me see what the issue is (and how to address it)?
Environment
macOS 10.13.2
LibreSSL 2.2.7 (installed via Homebrew)
Python 3.6.4 (installed via Homebrew)
Requests 2.18.4 (installed via pipenv)
Certifi 2017.11.5 (installed via pipenv)
Code
import requests
requests.get('https://myaccount.xcelenergy.com')
Stacktrace
Traceback (most recent call last):
File "/Users/abach/.local/share/virtualenvs/pyxcel-PG9DCiyE/lib/python3.6/site-packages/urllib3/connectionpool.py", line 595, in urlopen
self._prepare_proxy(conn)
File "/Users/abach/.local/share/virtualenvs/pyxcel-PG9DCiyE/lib/python3.6/site-packages/urllib3/connectionpool.py", line 816, in _prepare_proxy
conn.connect()
File "/Users/abach/.local/share/virtualenvs/pyxcel-PG9DCiyE/lib/python3.6/site-packages/urllib3/connection.py", line 326, in connect
ssl_context=context)
File "/Users/abach/.local/share/virtualenvs/pyxcel-PG9DCiyE/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 329, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/local/Cellar/python3/3.6.4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/usr/local/Cellar/python3/3.6.4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 814, in __init__
self.do_handshake()
File "/usr/local/Cellar/python3/3.6.4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1068, in do_handshake
self._sslobj.do_handshake()
File "/usr/local/Cellar/python3/3.6.4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/abach/.local/share/virtualenvs/pyxcel-PG9DCiyE/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
timeout=timeout
File "/Users/abach/.local/share/virtualenvs/pyxcel-PG9DCiyE/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "/Users/abach/.local/share/virtualenvs/pyxcel-PG9DCiyE/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='myaccount.xcelenergy.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)'),))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/abach/.local/share/virtualenvs/pyxcel-PG9DCiyE/lib/python3.6/site-packages/requests/api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "/Users/abach/.local/share/virtualenvs/pyxcel-PG9DCiyE/lib/python3.6/site-packages/requests/api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "/Users/abach/.local/share/virtualenvs/pyxcel-PG9DCiyE/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "/Users/abach/.local/share/virtualenvs/pyxcel-PG9DCiyE/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "/Users/abach/.local/share/virtualenvs/pyxcel-PG9DCiyE/lib/python3.6/site-packages/requests/adapters.py", line 506, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='myaccount.xcelenergy.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)'),))
What I've Tried (to no avail)
# Specifically calling out local cert chain:
requests.get('https://myaccount.xcelenergy.com', verify='/usr/local/etc/openssl/cert.pem')
# Using Certifi's old chain:
requests.get('https://myaccount.xcelenergy.com', verify=certifi.old_where())
There is currently a known issue with OSX's Python3.6 and OpenSSL that might be related to what you're seeing. I was able to fix it by installing certifi, then making a symlink in OpenSSL's directory. All of that can be done for you by running the following from bash:
Applications/Python 3.6/Install Certificates.command
See this related question: ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:747) on OS X

Resources