RabbitMQ' pika handshaking fails when SSL is set - python-3.x

Question

I am setting up the SSL layer on RabbitMQ on both server and clients. But the clients are failing when creating the connection to the server. At this point I am running the RabbitMQ server on a docker locally and the client locally using a conda environment.
Once the RabbitMQ server is up I see that the secure connection is accepting incoming connections:
test-rabbitmq-1 | 2023-01-20 08:22:01.692731+00:00 [info] <0.726.0> started TCP listener on [::]:5672
test-rabbitmq-1 | 2023-01-20 08:22:01.694836+00:00 [info] <0.746.0> started TLS (SSL) listener on [::]:7575
But the client refuses to connect with:
(rabbitmq-test) ➜ RabbitMQ-TSL ✗ python3 test.py
Enter PEM pass phrase: ********
INFO:pika.adapters.utils.connection_workflow:Pika version 1.3.1 connecting to ('127.0.0.1', 7575)
INFO:pika.adapters.utils.io_services_utils:Socket connected: <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 43142), raddr=('127.0.0.1', 7575)>
ERROR:pika.adapters.utils.io_services_utils:SSL do_handshake failed: error=SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:997)'); <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 43142), raddr=('127.0.0.1', 7575)>
Traceback (most recent call last):
File "/.../.local/lib/python3.10/site-packages/pika/adapters/utils/io_services_utils.py", line 636, in _do_ssl_handshake
self._sock.do_handshake()
File "/usr/lib/python3.10/ssl.py", line 1342, in do_handshake
self._sslobj.do_handshake()
ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:997)
ERROR:pika.adapters.utils.connection_workflow:Attempt to create the streaming transport failed: SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:997)'); 'localhost'/(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 7575)); ssl=True
ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorTransportSetupError: SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:997)')
ERROR:pika.adapters.utils.connection_workflow:AMQP connection workflow failed: AMQPConnectionWorkflowFailed: 1 exceptions in all; last exception - AMQPConnectorTransportSetupError: SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:997)'); first exception - None.
ERROR:pika.adapters.utils.connection_workflow:AMQPConnectionWorkflow - reporting failure: AMQPConnectionWorkflowFailed: 1 exceptions in all; last exception - AMQPConnectorTransportSetupError: SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:997)'); first exception - None
ERROR:pika.adapters.blocking_connection:Connection workflow failed: AMQPConnectionWorkflowFailed: 1 exceptions in all; last exception - AMQPConnectorTransportSetupError: SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:997)'); first exception - None
ERROR:pika.adapters.blocking_connection:Error in _create_connection().
Traceback (most recent call last):
File "/.../.local/lib/python3.10/site-packages/pika/adapters/blocking_connection.py", line 451, in _create_connection
raise self._reap_last_connection_workflow_error(error)
File "/.../.local/lib/python3.10/site-packages/pika/adapters/utils/io_services_utils.py", line 636, in _do_ssl_handshake
self._sock.do_handshake()
File "/usr/lib/python3.10/ssl.py", line 1342, in do_handshake
self._sslobj.do_handshake()
ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:997)
Traceback (most recent call last):
File "/.../test.py", line 16, in <module>
with pika.BlockingConnection(conn_params) as conn:
File "/.../.local/lib/python3.10/site-packages/pika/adapters/blocking_connection.py", line 360, in __init__
self._impl = self._create_connection(parameters, _impl_class)
File "/.../.local/lib/python3.10/site-packages/pika/adapters/blocking_connection.py", line 451, in _create_connection
raise self._reap_last_connection_workflow_error(error)
File "/.../.local/lib/python3.10/site-packages/pika/adapters/utils/io_services_utils.py", line 636, in _do_ssl_handshake
self._sock.do_handshake()
File "/usr/lib/python3.10/ssl.py", line 1342, in do_handshake
self._sslobj.do_handshake()
ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:997)
Any idea of what I am not setting properly on pika or at the RabbitMQ server?
On the server side I set the SSL layer at rabbitmq.conf as:
# Enable AMQPS
listeners.ssl.default = 7575
ssl_options.cacertfile = /etc/rabbitmq/cer/ca_certificate.pem
ssl_options.certfile = /etc/rabbitmq/cer/server_certificate.pem
ssl_options.keyfile = /etc/rabbitmq/cer/server_key.pem
ssl_options.verify = verify_peer
ssl_options.fail_if_no_peer_cert = true
# Enable HTTPS
management.listener.port = 15671
management.listener.ssl = true
management.listener.ssl_opts.cacertfile = /etc/rabbitmq/cer/ca_certificate.pem
management.listener.ssl_opts.certfile = /etc/rabbitmq/cer/server_certificate.pem
management.listener.ssl_opts.keyfile = /etc/rabbitmq/cer/server_key.pem
The docker compose file contains:
version: '3.8'
services:
rabbitmq:
image: rabbitmq:3-management
hostname: rabbitmq-server
volumes:
- ./rabbitmq-config/rabbitmq-cert:/etc/rabbitmq/cer
- ./rabbitmq-config/rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf
- ./rabbitmq/data:/var/lib/rabbitmq/mnesia/rabbit#my-rabbit
- ./rabbitmq/logs:/var/log/rabbitmq/log
ports:
- 5672:5672
- 7575:7575
- 15672:15672
Then, and to simply, I am using the demo client from RabbitMQ's guide as:
import logging
import pika
import ssl
from pika.credentials import ExternalCredentials
logging.basicConfig(level=logging.INFO)
context = ssl.create_default_context(
cafile = '/rabbitmq-config/rabbitmq-cert/ca_certificate.pem'
)
context.load_cert_chain(
'/rabbitmq-config/rabbitmq-cert/client_certificate.pem',
'/rabbitmq-config/rabbitmq-cert/client_key.pem'
)
ssl_options = pika.SSLOptions(context, "localhost")
conn_params = pika.ConnectionParameters(
port = 7575,
ssl_options = ssl_options,
credentials = ExternalCredentials()
)
with pika.BlockingConnection(conn_params) as conn:
ch = conn.channel()
ch.queue_declare("foobar")
ch.basic_publish("", "foobar", "Hello, world!")
print(ch.basic_get("foobar"))

Answer 1

Your Python code is set up to do X509 certificate authentication (you're not using username/password but are using ExternalCredentials). However, you have not configured RabbitMQ to accept X509 certificates for authentication (docs).
The Pika docs need to be updated, so I opened this issue - https://github.com/pika/pika/issues/1413
You would probably get a hint of this if you look at the RabbitMQ log file at the time your Python client tries to connect.
In order to enable X509 certificate authentication, do the following:
rabbitmq-plugins enable rabbitmq_auth_mechanism_ssl (docs)
Edit your rabbitmq.conf file and add the following section:
auth_mechanisms.1 = PLAIN
auth_mechanisms.1 = AMQPLAIN
auth_mechanisms.1 = EXTERNAL
Restart RabbitMQ
Add a password-less user that exactly matches the CN= value in your client certificate: rabbitmqctl add_user 'foobar, O=baz'. You can also get this value by attempting to connect after making the above changes. The failed auth attempt will be logged.
If you continue to have problems, I suggest asking on the mailing list as this is not a good forum for detailed analysis.
NOTE: the RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.