Hello fellow developers ! I'm stuck in a corner case and I'm starting to be out of hairs to pull... Here is the plot :
load-balancer.example.com:443 (TCP passthrough)
/\
/ \
/ \
/ \
s1.example.com:443 s2.example.com:443
(SSL/SNI) (SSL/SNI)
The goal is to stress-test the upstreams s1 and s2 directly using aiohttp with certificate-validation enable. Since the load-balancer does not belong to me I don't want to do the stress-test over it.
the code is not supposed to run on other platforms than GNU Linux with at least Python-v3.7 (but I can use any recent version if needed)
all servers serve a valid certificate for load-balancer.example.com
openssl validates the certificate from the upstreams when using openssl s_connect s1.example.com:443 -servername load-balancer.example.com
cURL needs curl 'https://load-balancer.example.com/' --resolve s1.example.com:443:load-balancer.example.com and also validates successfully
I am able to launch a huge batch of async ClientSession.get requests on both upstreams in parallel but for each request I need to somehow tell asyncio or aiohttp to use load-balancer.example.com as server_hostname, otherwise the SSL handshake fails.
Is there an easy way to setup the ClientSession to use a specific server_hostname when setting up the SSL socket ?
Does someone have already done something like that ?
EDIT : here is the most simple snippet with just a single request :
import aiohttp
import asyncio
async def main_async(host, port, uri, params=[], headers={}, sni_hostname=None):
if sni_hostname is not None:
print('Setting SNI server_name field ')
#
# THIS IS WHERE I DON'T KNOW HOW TO TELL aiohttp
# TO SET THE server_name FIELD TO sni_hostname
# IN THE SSL SOCKET BEFORE PERFORMING THE SSL HANDSHAKE
#
try:
async with aiohttp.ClientSession(raise_for_status=True) as session:
async with session.get(f'https://{host}:{port}/{uri}', params=params, headers=headers) as r:
body = await r.read()
print(body)
except Exception as e:
print(f'Exception while requesting ({e}) ')
if __name__ == "__main__":
asyncio.run(main_async(host='s1.example.com', port=443,
uri='/api/some/endpoint',
params={'apikey': '0123456789'},
headers={'Host': 'load-balancer.example.com'},
sni_hostname='load-balancer.example.com'))
When running it with real hosts, it throws
Cannot connect to host s1.example.com:443 ssl:True
[SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] '
certificate verify failed: certificate has expired (_ssl.c:1131)')])
Note that the error certificate has expired indicates that the certificate proposed to the client is the default certificate since the SNI hostname is s1.example.com which is unknow by the webserver running there.
When running it against the load-balancer it works just fine, the SSL handshake happens with the upstreams which serve the certificate and everything is valid.
Also note that
sni_callback does not help since it is called after the handshake has started and the certificate was received (and at this point server_hostname is a read-only property anyway)
it does not seem to be possible to set server_hostname when creating an SSLContext allthough SSLContext.wrap_socket does support server_hostname but I was not able to make that work
I hope someone knows how to fill the comment block in that snippet ;-]
I wrote a java code. In the code, I used com.microsoft.sqlserver.jdbc.SQLServerDataSource to establish a JDBC connection with my Azure sql database . I found that no matter whether I used " ds.setEncrypt(true);" or not, the JDBC connection was encrypted by TLS ( I use wireshark to catch the TCP packaege , all the package is TLS whether I used " ds.setEncrypt(true);" or not ).
Why ? I checked many official documents, but I couldn't find the answer . It's too difficult...
Azure sql database TLS is always enable ? Are there relevant official documents to prove it ?
The question is : I use ds.setEncrypt(true) or not ,even i set this to "false" , the TCP packages are encrypted by TLS . Why ?
Below is my code to establish the JDBC connection .
public static Connection getConnectionObject() {
SQLServerDataSource ds = new SQLServerDataSource();
ds.setServerName("azuresqldbserver0821.database.windows.net");
ds.setDatabaseName("azuresqldb0821");
ds.setPortNumber(1433);
ds.setUser("root0817");
ds.setPassword("<YourStrong#Passw0rd>");
ds.setEncrypt(false);// I use this method or not ,even i set this to "false" , the TCP packages are encrypted by TLS
ds.setTrustServerCertificate(true);
Connection conn;
try {
conn = ds.getConnection();
} catch (Exception e) {
e.printStackTrace();
return null;
}
return conn;
}
}
When a client first attempts a connection to SQL Azure, it sends an initial connection request. Consider this a "pre-pre-connection" request. At this point the client does not know if TLS/SSL/Encryption is required and waits an answer from SQL Azure to determine if TLS/SSL is indeed required throughout the session (not just the login sequence, the entire connection session). A bit is set on the response indicating so. Then the client library disconnects and reconnects armed with this information.
When you set "Encrypt connection" setting on the connetion string you avoid the "pre-pre-connection", you are preventing any proxy from turning off the encryption bit on the client side of the proxy, this way attacks like man-in-the-middle attack are avoided.
When secure connections are needed, please enable "Encrypt connection" setting.
In-transit encryption to Azure SQL is always enabled.
Transport Layer Security (TLS) was previously known as Secure Sockets Layer (SSL).
I would like to create a TLS connection to a server. Then, I want to send some encrypted data to the server. I know the hostname and port and I have the certificate. Surprisingly, I also received the private key of the server. However, I think it is not normal that I received the private key.
The first question is that, do I really need the private key to make a TLS connection?
By the way, I am using this python script
import socket
import ssl
server_addr = '**.**.**.**'
server_port = ****
server_cert = 'server.crt'
server_key = 'server.key' # I use the private key
context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
context.verify_mode = ssl.CERT_REQUIRED
context.load_cert_chain(certfile=server_cert, keyfile=server_key)
bindsocket = socket.socket()
bindsocket.connect((server_addr, server_port))
I am using the private key in the above script. It works without any error. However, when I try to bind() instead of connect(), i.e.,
bindsocket.bind((server_addr, server_port))
I get the following error:
OSError: [Errno 99] Cannot assign requested address
I have read many related questions about the above error, however, I still do not understand why this happens. Since, I have the hostname, port, certificate, and the key, I expect to create a TLS connection successfully.
The second question is that how can I establish a TLS connection? Is my script correct?
I highly appreciate any comment to improve the script.
So, first of all, you should absolutely not have the private key! As the name says, it is private and not necessary to establish a connection.
You could have the public key, but even that is not necessary as long as you use standard SSL and you trust the CA that signed the servers certificate.
Are you sure, it is the private key? Does the file begin with -----BEGIN PRIVATE KEY-----? Check with openssl rsa -noout -text -in server.key.
Refer to the wikipedia article and this post for more on asymmetric cryptography.
Further along the way:
With socket.bind() you bind a socket to a port on your local machine. This is not possible, as your machine does not have the address (you provide a server address).
From your code, it looks like you are trying to open the socket as a server. You will need the private key for that, but then you will be accepting connections and not connect to other machines yourself. I have a feeling, that you are mixing up two things here.
Refer to the python documentation of socket.bind() and to this question as this seems to be closely related.
Also check out the python documentation on ssl. I took the example, that does what you are asking for from said documentation:
import socket, ssl, pprint
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# require a certificate from the server
ssl_sock = ssl.wrap_socket(s,
ca_certs="/etc/ca_certs_file",
cert_reqs=ssl.CERT_REQUIRED)
ssl_sock.connect(('www.verisign.com', 443))
pprint.pprint(ssl_sock.getpeercert())
# note that closing the SSLSocket will also close the underlying socket
ssl_sock.close()
Also have a look on the example on how to open a SSL socket in server mode.
Further thoughts:
Do you really need to do all that TLS stuff yourself? If the server, for example, uses HTTPS (SSL encrypted HTTP), you can just use the http.client library.
Feel free to ask, if you need me to clarify something. I'll update my answer accordingly.
EDIT:
As you indicated, you want to open a port in server mode, I made an example for you (it heavily leans on the python documentation example):
import socket, ssl
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
context.load_cert_chain(certfile="cert.pem", keyfile="key.pem")
bindsocket = socket.socket()
bindsocket.bind(('127.0.0.1', 10023))
bindsocket.listen(5)
def deal_with_client(connstream):
data = connstream.recv(1024)
# empty data means the client is finished with us
while data:
print(data)
data = connstream.recv(1024)
while True:
newsocket, fromaddr = bindsocket.accept()
connstream = context.wrap_socket(newsocket, server_side=True)
try:
deal_with_client(connstream)
finally:
connstream.shutdown(socket.SHUT_RDWR)
connstream.close()
Running it:
% python3 ssltest.py
b'hello server!\n'
b'this is data\n'
The client side:
% openssl s_client -connect 127.0.0.1:10023
CONNECTED(00000005)
depth=0 C = SE, ST = Some-State, O = Internet Widgits Pty Ltd
verify error:num=18:self signed certificate
verify return:1
depth=0 C = SE, ST = Some-State, O = Internet Widgits Pty Ltd
verify return:1
---
Certificate chain
0 s:C = SE, ST = Some-State, O = Internet Widgits Pty Ltd
i:C = SE, ST = Some-State, O = Internet Widgits Pty Ltd
---
Server certificate
-----BEGIN CERTIFICATE-----
... certificate ...
-----END CERTIFICATE-----
subject=C = SE, ST = Some-State, O = Internet Widgits Pty Ltd
issuer=C = SE, ST = Some-State, O = Internet Widgits Pty Ltd
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 2272 bytes and written 404 bytes
Verification error: self signed certificate
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
... session stuff ...
Extended master secret: yes
---
hello server!
this is data
^C
If you use some usual protocol like HTTP, you should use a library though. Here is a example with flask:
>>> from flask import Flask
>>> app = Flask(__name__)
>>>
>>> #app.route("/")
... def hello():
... return "Hello World!"
...
>>> if __name__ == "__main__":
... app.run(ssl_context=('cert.pem', 'key.pem'))
...
* Serving Flask app "__main__" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
* Running on https://127.0.0.1:5000/ (Press CTRL+C to quit)
127.0.0.1 - - [06/Aug/2020 11:45:50] "GET / HTTP/1.1" 200 -
User who has authorized TLS certificate only able to connect to Open-sip server from application (Android and iOS).
What we need to change in config file for only TLS connection to Open-sip server.
You can configure the TLS certificate information in opensips.cfg file
tls_certificate="/usr/local/etc/opensips/tls/glob/glob-cert.pem"
tls_private_key="/usr/local/etc/opensips/tls/glob/glob-privkey.pem"
tls_ca_list="/usr/local/etc/opensips/tls/glob/glob-calist.pem"
## turn on the strictest and strongest authentication possible
tls_verify_client = 1
tls_require_client_certificate = 1
tls_method = TLSv1
tls_verify_client = 1 will ensure the client with authorized certificate configured in tls_ca_list file
Can you try uncommenting the line of startTLS from config file and make it true as a value?
It should work!
Also make sure that your Android and iOS clients are configured to accept TLS connections(though most of the time it's default behaviour).
I use Tyrus webSocket implementation to connect to the server from my JavaFX application. When I try to establish connection over SSL I get this error: javax.net.ssl.SSLException: SSL handshake error has occurred - more data needed for validating the certificate
I tried to use a dummy certificate and host verification as described in Disable Certificate Validation in Java SSL Connections but to no avail.
There is also not much information on Tyrus documentation.
I simply don't know what to do!
P.S. For what it's worth I managed to get around this issue by using Grizzly client
//final WebSocketContainer container = ContainerProvider.getWebSocketContainer();
final ClientManager client = ClientManager.createClient();
URI uri = URI.create(this.uri + "?" + System.currentTimeMillis());
session = client.connectToServer(this, uri);
It sounds like you need to install a certificate chain. I believe you can import the signing certificate using keytool -import. Have you setup the certificate store?