I'm making a secure SSL connection to a server using python and M2Crypto. See code below.
from M2Crypto import SSL, m2,x509
from M2Crypto.m2xmlrpclib import Server, SSL_Tranport
ctx = SSL.Context()
m2.ssl_ctx_use_pkey_privkey(ctx.ctx,myKey.pkey)
m2.ssl_ctx_use_x509(ctx.ctx,myCert.x509)
server = Server(serverUrl, SSL_Transport(ctx))
server.ping()
The above works fine. If I try to change the default socket timeout by adding the following two lines at the beginning of the code, I get a protocol error.
import socket
socket.setdefaulttimeout(40)
This is the error I receive:
File "/usr/local/lib/python2.4/xmlrpclib.py", line 1096, in call
return self.__send(self.__name, args)
File "/usr/local/lib/python2.4/xmlrpclib.py", line 1383, in __request
verbose=self.__verbose
File "/usr/local/lib/python2.4/site-packages/M2Crypto/m2xmlrpclib.py", line 68, in request
headers
xmlrpclib.ProtocolError:
Why is the default socket timeout causing problems?
There is a patch that can fix this. It is for Linux only. See Bug 2341 --> https://bugzilla.osafoundation.org/show_bug.cgi?id=2341
I have not tried the patch. I will use a different work around. I set the socket timeout to None then run my M2Crypto code then set the socket timeout back to the value I need for the rest of my code.
origTimeout = socket.getdefaulttimeout()
socket.setdefaulttimeout(None)
# run M2Crypto code
socket.setdefaulttimeout(origTimeout)
Related
This question just recently posted has some useful answers, but it's not the same as mine. I'm running urllib3 1.26.4 and Python 3.7 from an ArcGIS Pro Notebook. I also have Fiddler 4 open because I want to track web traffic while troubleshooting a script. I only get the following error when I have Fiddler open. If I close Fiddler I get <Response [200]>. Is it not possible to use the requests module with Fiddler open? I'm new to Fiddler.
Truncated script:
import requests
#url
idph_data = 'https://idph.illinois.gov/DPHPublicInformation/api/covidVaccine/getVaccineAdministrationCurrent'
#headers
headers = {'user-agent': 'Mozilla/5.0'}
response = requests.get(idph_data, headers=headers, verify=True)
Error:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
In [35]:
Line 4: response = requests.get(idph_data,verify=True)
File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\api.py, in get:
Line 76: return request('get', url, params=params, **kwargs)
File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\api.py, in request:
Line 61: return session.request(method=method, url=url, **kwargs)
File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\sessions.py, in request:
Line 542: resp = self.send(prep, **send_kwargs)
File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\sessions.py, in send:
Line 655: r = adapter.send(request, **kwargs)
File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\adapters.py, in send:
Line 449: timeout=timeout
File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\connectionpool.py, in urlopen:
Line 696: self._prepare_proxy(conn)
File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\connectionpool.py, in _prepare_proxy:
Line 964: conn.connect()
File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\connection.py, in connect:
Line 359: conn = self._connect_tls_proxy(hostname, conn)
File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\connection.py, in _connect_tls_proxy:
Line 506: ssl_context=ssl_context,
File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\util\ssl_.py, in ssl_wrap_socket:
Line 432: ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls)
File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\urllib3\util\ssl_.py, in _ssl_wrap_socket_impl:
Line 474: return ssl_context.wrap_socket(sock)
File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\ssl.py, in wrap_socket:
Line 423: session=session
File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\ssl.py, in _create:
Line 827: raise ValueError("check_hostname requires server_hostname")
ValueError: check_hostname requires server_hostname
---------------------------------------------------------------------------
I am running into this issue as well with the environment provided by the current version of ArcGIS Pro. Per a lower-rated answer in the question you linked, I ran pip install urllib3==1.25.11 in the desired environment (in my case a clone of the default), and the issue appears to be resolved.
This is apparently due to a new feature in the urllib3 version provided by ArcGIS Pro. The above command downgrades to a relatively recent, but working, version. This will not be resolved in newer versions of urllib3, but instead, there is currently a pull request pending to fix the underlying issue in Python itself.
By the way, while it's possible to configure pip to be able to run through the fiddler proxy, it's not too easy, so it is best to turn off Fiddler while running any pip commands.
The pertinent bug report is found here. The issue appears to be that there is a very old bug in how Windows system proxy settings are being parsed by CPython / built-in urllib, causing the proxy entry for use with https URLs to always receive a HTTPS prefix (instead of HTTP). Newer version of urllib3 actually support using proxies over HTTPS, which was not previously the case. So before, urllib3 would ignore the prefix, but now, it attempts to use HTTPS to communicate with a HTTP url.
I've updated to requests v. 2.7.0, the latest, and I'm no longer receiving the error. If it was a version-specific issue related to v. 2.25.1, which was what I was using, I'm not sure. I haven't came across any evidence of that.
In a Windows command prompt in the same directory as my Python executable:
python -m pip install requests==2.7.0
Now if I run my original script with Fiddler capturing, I get a HTTP status of 200 and my script no longer gives me the error.
I use bazel to publish docker images to gitlab regitry. Last week, the bazel commands started failing. I was able to narrow down the issue to httplib2.
The code sample below can be used to reproduce the issue.
import httplib
import httplib2
conn = httplib.HTTPSConnection("registry.gitlab.com")
conn.request("GET", "/")
r1 = conn.getresponse()
print r1.status, r1.reason
httplib2.Http().request('https://registry.gitlab.com')
The output for the above is:
200 OK
Traceback (most recent call last):
File "deleteMe.py", line 9, in <module>
httplib2.Http().request('https://registry.gitlab.com')
File "/Users/joint/Library/Python/2.7/lib/python/site-packages/httplib2/__init__.py", line 2135, in request
cachekey,
File "/Users/joint/Library/Python/2.7/lib/python/site-packages/httplib2/__init__.py", line 1796, in _request
conn, request_uri, method, body, headers
File "/Users/joint/Library/Python/2.7/lib/python/site-packages/httplib2/__init__.py", line 1701, in _conn_request
conn.connect()
File "/Users/joint/Library/Python/2.7/lib/python/site-packages/httplib2/__init__.py", line 1411, in connect
raise SSLHandshakeError(e)
httplib2.SSLHandshakeError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)
Error shown in Wireshark is 'Description: Unknown CA (48)'
I have tried verifying the gitlab certs via openssl and I don't see any issue with them.
I have tried specifying the gitlab cert in httplib2 definition but I get the same error.
h = httplib2.Http(ca_certs='./registrygitlabcom.crt')
h.request('https://registry.gitlab.com')
Any pointers on what I should be doing or trying out... thanks!
I think I have figured out the answer. Posting it here for anyone else who might run into this.
The root certificates used by httplib2 are coming from the cacerts.txt file.
(https://github.com/httplib2/httplib2/blob/master/python2/httplib2/cacerts.txt)
registry.gitlab.com probably switched the root CA last week and that has triggered the problem.
The program should allow to run several https get requests with one aiohttp.ClientSession as the documentation suggests. It is intended to run a telegram bot.
I was not able to catch the exception with try ... except. Therefore the program hangs when exiting. During extended sessions the error is printed in the command windows (but not in the error log).
SSL error in data received
protocol: <asyncio.sslproto.SSLProtocol object at 0x0000016A581E4400>
transport: <_SelectorSocketTransport fd=644 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
File "C:\Users\annet\Anaconda3\lib\asyncio\sslproto.py", line 526, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File "C:\Users\annet\Anaconda3\lib\asyncio\sslproto.py", line 207, in feed_ssldata
self._sslobj.unwrap()
File "C:\Users\annet\Anaconda3\lib\ssl.py", line 767, in unwrap
return self._sslobj.shutdown()
ssl.SSLError: [SSL: KRB5_S_INIT] application data after close notify (_ssl.c:2592)
^C
As the error information is very unspecific I could not really isolate the source and have a short code to reproduce the error.
A sample code is on github under https://github.com/fhag/telegram2.git
In order to run the code you will need an API token from telegram of your own bot.
This error showed up the first time when I upgraded to python 3.7.1.
Python is running on Windows 10.
I want to create a sniffer script for monitoring windows and linux. For linux, it is easy, but for windows it is not. I met an error as follows:
Traceback (most recent call last):
File "test_scapy.py", line 45, in <module>
main()
File "test_scapy.py", line 38, in main
sniff(filter="tcp port 80", prn=packet_callback, count=10)
File "C:\Python27\lib\site-packages\scapy\sendrecv.py", line 575, in sniff
sel = select([s],[],[],remain)
select.error: (10038, '')
My code is
from scapy.all import *
def packet_callback(packet):
print packet.show()
sniff(filter="tcp port 80", prn=packet_callback, count=10)
I have read the post Scapy sniff() "an operation was performed on something that is not a socket"
and read the select document https://docs.python.org/2/library/select.html
"File objects on Windows are not acceptable, but sockets are. On Windows, the underlying select() function is provided by the WinSock library, and does not handle file descriptors that don’t originate from WinSock."
I think that means I have to re-define the socket for sniffing. So I upgrade the code
# create a raw socket and bind it to the public interface
s = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_IP)
s.bind((HOST, 0))
# Include IP headers
s.setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1)
# receive all packages
s.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON)
sniff(filter="tcp port 80", prn=packet_callback, count=10, opened_socket=s)
s.ioctl(socket.SIO_RCVALL, socket.RCVALL_OFF)
That is ok now and the error disappear. But I have to parse the TCP RAW data manually.
Now the question is: is there any easy way for me to sniffer for
windows platform like using linux scapy layer wrapper, like call package[TCP].dst
Any suggestion is appreciated.
Which version of Scapy are you using? You should probably update to the latest (2.3.3) version or event better, to the current development version from GitHub.
I am trying to implement websockets on a openshift.com server (which should support them).
openshift.com provides me a WSGI, so I embed my cherrypy to it, so that my wsgi.pyscript define an application object.
Also, cherrypy has a websocket tool, as defined by ws4py.
This is a minimal cherrypy application that works under WSGI in OpenShift, and that should use websockets too!
import cherrypy
from ws4py.server.cherrypyserver import WebSocketPlugin, WebSocketTool
from ws4py.websocket import EchoWebSocket
import atexit
import logging
# see http://tools.cherrypy.org/wiki/ModWSGI
cherrypy.config.update({'environment': 'embedded'})
if cherrypy.__version__.startswith('3.0') and cherrypy.engine.state == 0:
cherrypy.engine.start(blocking=False)
atexit.register(cherrypy.engine.stop)
class Root(object):
def index(self): return 'I work!'
def ws(self): print('THIS IS NEVER PRINTED :(')
index.exposed=True
ws.exposed=True
# registering the websocket
conf={'/ws':{'tools.websocket.on': True,'tools.websocket.handler_cls': EchoWebSocket}}
WebSocketPlugin(cherrypy.engine).subscribe()
cherrypy.tools.websocket = WebSocketTool()
#show stacktraces in console (for some reason this is not default in cherrypy+WSGI)
logger = logging.getLogger()
logger.setLevel(logging.INFO)
stream = logging.StreamHandler()
stream.setLevel(logging.INFO)
logger.addHandler(stream)
application = cherrypy.Application(Root(), script_name='', config=conf)
Everything work wonderfully, except when I create a websocket ( connecting to ws://myserver:8000/ws ), this is the stacktrace I get:
cherrypy/_cplogging.py, 214, HTTP Traceback (most recent call last):
File "cherrypy/_cprequest.py", line 661, in respond
self.hooks.run('before_request_body')
File "cherrypy/_cprequest.py", line 114, in run
raise exc
File "cherrypy/_cprequest.py", line 104, in run
hook()
File "cherrypy/_cprequest.py", line 63, in __call__
return self.callback(**self.kwargs)
File "ws4py/server/cherrypyserver.py", line 200, in upgrade
ws_conn = get_connection(request.rfile.rfile)
AttributeError: 'mod_wsgi.Input' object has no attribute 'rfile'
(I manually deleted the absolute path from the filenames)
PS: I use python3.3, cherrypy==3.5.0, ws4py==0.3.4.
It is not clear to me:
if this is a lack of compatibility between cherrypy and ws4py when in an WSGI environment.
if it is a problem of ws4py when in a WSGI environment
if it is because Openshift websockets have a different port than the http one
PPS: this is a complete OpenShift project, that you can run and try this yourself: https://github.com/spocchio/wsgi-cherrypy-ws4py
I don't think it is possible at all. WSGI is a synchronous protocol (1, 2), WebSocket protocol is asynchronous. Wiki states that for a Python application interface OpenShift uses WSGI (3). Alas.
However I've recently played with ws4py in pub/sub scenario and it works really well on top of CherryPy standard HTTP-server deployment. So it shouldn't be a problem on a generic virtual server with no application interface constraints.