How to connect to database via asyncssh (Tunnel) and psycop2? - python-3.x

Question

python 3.11 / asyncssh 2.13 / psycopg2-binary 2.9.5
After using the sshtunnel library i wanted to switch to the asyncssh library because its more maintained with newer Python Versions and brings async benefits.
After reading the asyncssh Doc i wrote a little test script and could connect via SSH from my local computer to my Server. I execute some ls terminal statement on my server and could print the outcome with my python script. After that small success i wanted to connect to my postgreSQL Database via the asyncssh Tunnel, so i can push my local panda data to my server database.
In the past it worked well with the sshtunnel library. Unfortunately i fail with asyncssh library to establish a connection to my database on my server.
Problem
My main Problem is to wrap psycopg2 into the tunnel like in sshtunnel with remote_bind_address. I tried with forward_local_port or forward_remote_port and it seems the connection is established but how to funnel psycopg2 into it? Instead of connecting to my 'Server-Database Port 5432' i should connect to the tunnel port (see example 2)?
Example: How sshtunnel worked.
(this is an from geo.rocks but i used that structure also).
import psycopg2
from sshtunnel import SSHTunnelForwarder
try:
with SSHTunnelForwarder(
('some.linktodb.com', 22), # port 22 as standard SSH port
ssh_username="username",
ssh_pkey="your/private/key", # your private key file
ssh_private_key_password="****",
remote_bind_address=('localhost', 5432)) as server: # mirroring to local port 5432
server.start()
params = { # database params
'database': 'test',
'user': 'dome',
'password': '*****',
'host': 'localhost',
'port': server.local_bind_port
}
conn = psycopg2.connect(**params)
curs = conn.cursor() # if this works, you are connected
print("DB connected")
except:
print("Connection failed")
Example: My current asyncssh approach.
async def run_client() -> None:
async with asyncssh.connect(
host=os.environ.get('SSH_HOST'),
known_hosts=None,
username=os.environ.get('SSH_USER'),
passphrase=os.environ.get('SSH_PW'),
client_keys=os.environ.get('SSH_PRIVAT_KEY')) as tunnel:
listener = await tunnel.forward_local_port(
listen_host='',
listen_port=8084,
dest_host='127.0.0.1',
dest_port=5432)
conn = psycopg2.connect(
host=os.environ.get('DB_HOST'),
port=os.environ.get('DB_PORT'), # <- `local_bind_port` like in sshtunnel?
database=os.environ.get('DB_NAME'),
user=os.environ.get('DB_USER'),
password=os.environ.get('DB_PW'),
)
OUTPUT:
...
[ 2023-01-23,12:17:17.+0100 ] [INFO] logging.py - log - [conn=0] Creating local TCP forwarder from port 8084 to 127.0.0.1, port 5432
[ 2023-01-23,12:17:17.+0100 ] [INFO] logging.py - log - [conn=0] Closing connection
[ 2023-01-23,12:17:17.+0100 ] [INFO] logging.py - log - [conn=0] Sending disconnect: Disconnected by application (11)
[ 2023-01-23,12:17:17.+0100 ] [INFO] logging.py - log - [conn=0] Connection closed
Traceback (most recent call last):
File "/home/user/atlas/user_atlas/mono/tutorials/python/db_ssh_activity.py", line 135, in <module>
loop.run_until_complete(coroutine)
File "/usr/lib64/python3.11/asyncio/base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/home/user/atlas/user_atlas/mono/tutorials/python/db_ssh_activity.py", line 125, in main
await run_client()
File "/home/user/atlas/user_atlas/mono/tutorials/python/db_ssh_activity.py.py", line 82, in run_client
conn = psycopg2.connect(
^^^^^^^^^^^^^^^^^
File "/home/user/atlas/mono/mono_env/lib64/python3.11/site-packages/psycopg2/__init__.py", line 122, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.OperationalError: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: No such file or directory
Is the server running locally and accepting connections on that socket?
EXAMPLE 2:
Instead of connecting to my 'Server-Database Port 5432' i should connect to the tunnel port
async def run_client() -> None:
async with asyncssh.connect(
host=os.environ.get('SSH_HOST'),
known_hosts=None,
username=os.environ.get('SSH_USER'),
passphrase=os.environ.get('SSH_PW'),
client_keys=os.environ.get('SSH_PRIVAT_KEY')) as tunnel:
tunnel.forward_remote_port
listener = await tunnel.forward_local_port(
listen_host='localhost',
listen_port=1,
dest_host='127.0.0.1',
dest_port=5432)
conn = psycopg2.connect(
host='localhost',
port=listener.get_port(),
database=os.environ.get('DB_NAME'),
user=os.environ.get('DB_USER'),
password=os.environ.get('DB_PW'),
)
listener.wait_closed()
OUTPUT:
[ 2023-01-23,12:40:46.+0100 ] [INFO] logging.py - log - [conn=0] Auth for user hendrix succeeded
[ 2023-01-23,12:40:46.+0100 ] [INFO] logging.py - log - [conn=0] Creating local TCP forwarder from localhost, port 1 to 127.0.0.1, port 5432
[ 2023-01-23,12:40:46.+0100 ] [DEBUG] logging.py - log - [conn=0] Failed to create local TCP listener: [Errno 13] error while attempting to bind on address ('::1', 1, 0, 0): Permission denied
[ 2023-01-23,12:40:46.+0100 ] [INFO] logging.py - log - [conn=0] Closing connection
[ 2023-01-23,12:40:46.+0100 ] [INFO] logging.py - log - [conn=0] Sending disconnect: Disconnected by application (11)
[ 2023-01-23,12:40:46.+0100 ] [INFO] logging.py - log - [conn=0] Connection closed
Traceback (most recent call last):
File "/home/user/atlas/mono/monokapi_jupyter/tutorials/python/db_ssh_activity.py", line 136, in <module>
loop.run_until_complete(coroutine)
File "/usr/lib64/python3.11/asyncio/base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/home/user/atlas/user_atlas/mono/tutorials/python/db_ssh_activity.py", line 126, in main
await run_client()
File "/home/user/atlas/user_atlas/mono/tutorials/python/db_ssh_activity.py.py", line 76, in run_client
listener = await tunnel.forward_local_port(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/atlas/user_atlas/mono_env/lib64/python3.11/site-packages/asyncssh/connection.py", line 2944, in forward_local_port
listener = await create_tcp_forward_listener(self, self._loop,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/atlas/user_atlas/mono_env/lib64/python3.11/site-packages/asyncssh/listener.py", line 341, in create_tcp_forward_listener
return await create_tcp_local_listener(conn, loop, protocol_factory,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/atlas/user_atlas/mono_env/lib64/python3.11/site-packages/asyncssh/listener.py", line 314, in create_tcp_local_listener
raise OSError(exc.errno, 'error while attempting ' # type: ignore
PermissionError: [Errno 13] error while attempting to bind on address ('::1', 1, 0, 0): Permission denied