I am trying to access a SOAP server using zeep. My server uses SSL with a custom certificate, and connection to that server works, with my cert, or ignoring it:
python -mzeep "https://<server-ip>/servicemanager/1?wsdl" --no-verify
I get a long list of Prefixes, Global elements, Global types, Bindings and Service. The latter one says:
Service: ServiceManager
Port: servicemanager_1 (Soap11Binding: {http://soap.client.<snipped>.at}servicemanager_1Binding)
Operations:
getServices() -> return: ns0:service[]
So, from what I can say by now, I can create a client object and call it's service named getServices().
from zeep import CachingClient as Client
from zeep.wsse.signature import Signature
from zeep.transports import Transport
from requests import Session, Request
session = Session()
session.verify = False
transport = Transport(session=session)
c = Client('https://<server-ip>/servicemanager/1?wsdl', transport=transport)
c.service.getServices()
But that leads to an error in urllib3 (~/.virtualenvs/soap/lib/python3.5/site-packages/urllib3/util/connection.py):
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
[...]
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='localhost',
port=443): Max retries exceeded with url: /servicemanager/1 (Caused by
NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object
at 0x7f4e2a6f7d30>: Failed to establish a new connection: [Errno 111]
Connection refused',))
It does not matter if I ignore the SSL verification, or provide a CA_BUNDLE. both are accepted, the client is created, but I can't call the getServices() method.
What did I forget here? I don't think this is a zeep problem, as the underlying urllib3 throws the exception. But I tried for hours and searched the internet for a solution, without success.
Apart of the XML I get from the endpoint is:
<service name="ServiceManager">
<port name="servicemanager_1" binding="tns:servicemanager_1Binding">
<soap:address location="http://localhost/servicemanager/1"/>
</port>
</service>
And I don't know why it returns a "localhost" there - is zeep using that for its call? Then I would understand why permanent errors occur.
Any hints?
To change the endpoint address I use it this way:
client.service._binding_options['address'] = 'https://mynewaddress.com/service.wsdl'
As always, after days of searching, in the moment I ask at Stackoverflow, the answer came up through other channels.
If anyone has the same problems, here is the solution. My server provides me with the WSDL file, like said above:
<service name="ServiceManager">
<port name="servicemanager_1" binding="tns:servicemanager_1Binding">
<soap:address location="http://localhost/servicemanager/1"/>
</port>
</service>
And there it stands: localhost. Zeep (IMHO correctly) uses that service endpoint to communicate then with the server.
What I did for testing: I SSH-tunnelled the ports 80/443 to localhost, so zeep thought it talked to localhost.
And Shazaam, it worked.
So my server was the culprit - too bad I can't change that, as I have no control over it.
But now a workaround is possible.
Related
I am trying to read and send emails (O365) using OAuth2. This code works fine on my local machine, but
when I am trying to deploy this in a server where there is a proxy to access the internet, I am getting the below error.
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='login.microsoftonline.com', port=443): Max retries exceeded with url: /2d11ad74-bbf6-403f-32c7-6ff0e039a923/oauth2/v2.0/token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000002507067C348>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond'))
Below is my code I tried:
import settings
from exchangelib import Credentials, Account, DELEGATE, BASIC, \
Configuration, NTLM, Message, Mailbox, FileAttachment, HTMLBody, FaultTolerance, IMPERSONATION
from exchangelib.protocol import BaseProtocol, NoVerifyHTTPAdapter
from exchangelib import OAuth2LegacyCredentials
credentials = OAuth2LegacyCredentials(
client_id= settings.clientId, client_secret= settings.clientSecret,
tenant_id=settings.tenentId,
username=settings.eUserName, password=settings.ePassword
)
config = Configuration(server = 'smtp.office365.com', credentials=credentials)
account = Account(settings.eUserName, config=config, access_type=DELEGATE)
m = Message(
account=account,
subject='Any Subject OAth2 Authentication',
body=HTMLBody('OAth2 Authentication'),
to_recipients=map(lambda x: Mailbox(email_address=x), ['myEmail#gmail.com'])
)
for item in account.inbox.all()[:10]:
print (item.sender.email_address)
m.send()
print('Email is being sent...')
You probably need to add setup proxy configuration. Here's how to do that in exchangelib:
https://ecederstrand.github.io/exchangelib/#proxies-and-custom-tls-validation
Hello fellow developers ! I'm stuck in a corner case and I'm starting to be out of hairs to pull... Here is the plot :
load-balancer.example.com:443 (TCP passthrough)
/\
/ \
/ \
/ \
s1.example.com:443 s2.example.com:443
(SSL/SNI) (SSL/SNI)
The goal is to stress-test the upstreams s1 and s2 directly using aiohttp with certificate-validation enable. Since the load-balancer does not belong to me I don't want to do the stress-test over it.
the code is not supposed to run on other platforms than GNU Linux with at least Python-v3.7 (but I can use any recent version if needed)
all servers serve a valid certificate for load-balancer.example.com
openssl validates the certificate from the upstreams when using openssl s_connect s1.example.com:443 -servername load-balancer.example.com
cURL needs curl 'https://load-balancer.example.com/' --resolve s1.example.com:443:load-balancer.example.com and also validates successfully
I am able to launch a huge batch of async ClientSession.get requests on both upstreams in parallel but for each request I need to somehow tell asyncio or aiohttp to use load-balancer.example.com as server_hostname, otherwise the SSL handshake fails.
Is there an easy way to setup the ClientSession to use a specific server_hostname when setting up the SSL socket ?
Does someone have already done something like that ?
EDIT : here is the most simple snippet with just a single request :
import aiohttp
import asyncio
async def main_async(host, port, uri, params=[], headers={}, sni_hostname=None):
if sni_hostname is not None:
print('Setting SNI server_name field ')
#
# THIS IS WHERE I DON'T KNOW HOW TO TELL aiohttp
# TO SET THE server_name FIELD TO sni_hostname
# IN THE SSL SOCKET BEFORE PERFORMING THE SSL HANDSHAKE
#
try:
async with aiohttp.ClientSession(raise_for_status=True) as session:
async with session.get(f'https://{host}:{port}/{uri}', params=params, headers=headers) as r:
body = await r.read()
print(body)
except Exception as e:
print(f'Exception while requesting ({e}) ')
if __name__ == "__main__":
asyncio.run(main_async(host='s1.example.com', port=443,
uri='/api/some/endpoint',
params={'apikey': '0123456789'},
headers={'Host': 'load-balancer.example.com'},
sni_hostname='load-balancer.example.com'))
When running it with real hosts, it throws
Cannot connect to host s1.example.com:443 ssl:True
[SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] '
certificate verify failed: certificate has expired (_ssl.c:1131)')])
Note that the error certificate has expired indicates that the certificate proposed to the client is the default certificate since the SNI hostname is s1.example.com which is unknow by the webserver running there.
When running it against the load-balancer it works just fine, the SSL handshake happens with the upstreams which serve the certificate and everything is valid.
Also note that
sni_callback does not help since it is called after the handshake has started and the certificate was received (and at this point server_hostname is a read-only property anyway)
it does not seem to be possible to set server_hostname when creating an SSLContext allthough SSLContext.wrap_socket does support server_hostname but I was not able to make that work
I hope someone knows how to fill the comment block in that snippet ;-]
I'm using the "requests" module for getting JSON from my web service, using the next code:
import requests
import SSL
# With or without this line of code below, the output is the same
ssl.match_hostname = lambda cert, hostname: True
response = requests.get("MY_URL", cert=("client.pem", "client-key.pem"), verify="CAcert.cer")
When the SSL step seems to fail with the following message:
HTTPSConnectionPool(host='x.x.x.x', port=443): Max retries exceeded with url: {WEBSERVICE_URL_PATTERN} (Caused by SSLError(CertificateError("hostname 'x.x.x.x' doesn't match 'x.x.x.x'")))
I'm using Python 3.10.5 with the latest version of the "requests" module.
Does anyone know what could cause this kind of error and how to fix it?
I assume you've redacted actual names which are in fact different, because if you really did have a host named x.x.x.x using a cert with the same name it would match (unless it wasn't really the same because the CA, or a potentially-bogus 'subject'/'subscriber', used lookalike characters).
From the documentation of match_hostname
Changed in version 3.7: The function is no longer used to TLS connections. Hostname matching is now performed by OpenSSL. ...
Deprecated since version 3.7.
At the python.ssl level, or http.client or urllib.requests, you can still turn off only hostname checking with check_hostname=False in the SSLContext. However AFAIK requests doesn't give you access to the SSL level except for setting the cert(s) as you do or the sledgehammer option of turning off all verification with verify=False.
If at all possible, you should try to use a hostname and a host cert that do match. Note changing either the name you request or the cert can accomplish this.
The problem was solved, using a Subject Alternative Name (SAN) for the server, with a value of its own IP address.
I've found out that we use Simple-CA, and the request of getting a signed certificate from it was with a Common Name (CN), when we don't have a domain name.
After changing the signing action to SAN instead of CN, the problem was solved.
Thanks for the helpers!
I am attempting to make a web scraper in Python 3. I keep getting a WinError 10060 stating that the connection failed because the connected party did not properly respond or the connected host failed to respond. Using both the urllib and also trying with requests libraries both create the 10060 error. When using requests the error states they the max retries exceeded with the URL.
import urllib.request
with urllib.request.urlopen('http://python.org') as response:
html = response.read()
People have mentioned that it is likely a proxy or firewall issue as I am attempting to do this on my work network.
Turns out this was a proxy authentication error. Simply adding the proxy with your credentials in the get command solved this.
proxies = {'http': 'http://user:pass#url:8080', 'https': 'http://user:pass#url:8080'}
page = requests.get(webpage, proxies)
I've been trying to get the kaazing stock ticker excel demo working, and I'm running into some problems with the websocket connection. I have my gateway running and the stock feed service seems to be working:
[Stock Feed] Stock Ticker demo connected to tcp://localhost:61616
When I attempt to run the javascript JMS messaging demo, I get the following:
CONNECT: ws://localhost:8001/jms
EXCEPTION: ConnectionFailedException: WebSocket connection failed
It typically takes about 30 seconds to fail. In that time, my current sessions on the kaazing dashboard do spike to 1, so I know the connection is at least being attempted.
The same thing is occurring if I use the simple websocket client chrome extension. In my error logs, I'm seeing:
2016-03-11 11:06:18,723 [New I/O worker #6] INFO [ws://localhost:8001/jms x-kaazing-handshake]
[tcp://[0:0:0:0:0:0:0:1]:52340 http/1.1] - [localhost:8001] "GET /jms?.kl=Y HTTP/1.1 " "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0"
2016-03-11 11:06:19,860 [EagerCP_0-1] WARN Unable to establish JMS Connection due to the following exception: Could not connect to broker URL: tcp://localhost:61616. Reason: java.net.ConnectException: Connection refused: connect
The error is addressed on the kaazing site here: http://kaazing.com/doc/jms/4.0/integration-jms/p_jms_integrate_tshoot.html#problem4 but it doesn't really provide a solution. I would try to implement the "workaround" suggested but I can't find the configuration file. None of the gateway configurations contain the given block of XML.
As a last ditch effort, I attempted to modify the activemq service configuration to include websockets by adding the ws to the transportConnector:
<transportConnectors>
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
<transportConnector name="ssl" uri="ssl://0.0.0.0:61617"/>
<transportConnector name="stomp" uri="stomp://0.0.0.0:61613"/>
<transportConnector name="websocket" uri="ws://0.0.0.0:61614"/>
</transportConnectors>
Still no luck. What am I missing? It seems like this should work right out of the box...
The issue was a permission level on the activemq.bat load. The connection never actually establish itself because of admin privileges on our networks. I was able to fix it by tweaking our admin rights.
Of course, I never looked at the activemq batch logs which gave me the hint right away.
http://developer.kaazing.com/documentation/jms/4.0/about/setup-guide.html#localhost_install