Connecting to Azure PostgreSQL server from python psycopg2 client - python-3.x

I have trouble connecting to the Azure postgres database from python. I am following the guide here - https://learn.microsoft.com/cs-cz/azure/postgresql/connect-python
I have basically the same code for setting up the connection.
But the psycopg2 and SQLalchemy throw me the same error:
OperationalError: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
I am able to connect to the instance by other client tools like dbeaver but from python it does not work.
When I investigate in Postgres logs I can see that the server actually authorized the connection but the next line says
could not receive data from client: An existing connection was forcibly closed by the remote host.
Python is 3.7
psycopg's version is 2.8.5
Azure Postgres region is in West Europe
Does someone has any suggestion on what should I try to make it work?
Thank you!
EDIT:
The issue resolved itself. I tried the same setup a few days later and it started working. Might have been something wrong with the Azure West Europe.

I had this issue too. I think I read somewhere (I forget where) that Azure has an issue with the # you have to for the username (user#serverName).
I created variables and an f-string and then it worked OK.
import sqlalchemy
username = 'user#server_name'
password = 'PassWord!'
host = 'server_name.postgres.database.azure.com'
database = 'your_database'
conn_str = f'postgresql+psycopg2://{username}:{password}#{host}/{database}'
After that:
engine = sqlalchemy.create_engine(conn_str, pool_pre_ping=True)
conn = engine.connect()
Test it with a simple SQL statement.
sql = 'SELECT * FROM public.some_table;'
results = conn.engine.execute(sql)
This was a connection in UK South. Before that it did complain about the format of the username having to use #, although the username was correct, as tested from the command line with PSQL and another SQL client.

Related

How to create the connection string for a database generated with pgAdmin4

I developed my app with SQLAlchemy testing it on an SQLite database.
Now I would like to testing more using PostgreSQL but in a localhost once again.
For this reason, I tried to create a local Postgre database with pdAdmin4 (my laptop has Windows 7 so I installed the v4 version of the database manager).
Now I am trying to generate the connection string for this Postgre database but I cannot understand how to do it.
This is what I created in pgAdmin4:
while the details of the server are these:
The connection string I am trying to use is this:
selected_engine = create_engine('postgresql+psycopg2://postgres:123#localhost/test_server/test_database')
The password 123 is the password I set at the first access to pgdAmin4.
However, I got this error:
OperationalError: (psycopg2.OperationalError) connection to server at "localhost" (::1), port 5432 failed: FATAL: database "test_server/test_database" does not exist
I am sure I am wrong and I cannot figure out what is the correct URL for my database.

SQLanywhere & SqllanyDB Specified database not found', -83

I'm trying to connect to a SAP SQLANYWHERE server using syqlanydb library and even sqlalchemy however both scenarios I get
sqlanydb.OperationalError: (b'Specified database not found', -83)
I can use the same connection details to connect using clients such as DBeaver but no matter what can't connect to this server, I've also tried ODBC drivers but server wasn't returning anything.
I have sqlanywhere 17 installed matching server and port 2638 is open. Have in mind this is a remote server.
My connection string:
sybase_connection_string = "sqlalchemy_sqlany://{user}:{pwd}#{host}:{port}/{db}".format(user=user, pwd=pwd, host=host, port=port, db=database)
engine = create_engine(sybase_connection_string)
engine.connect(sybase_connection_string)
Has anyone experienced this?
Thanks

Azure SQL serverless is not waking up on connection attempt

I'm testing Azure SQL Serverless and from SSMS it seems to work fine, but from my ASP.NET Core application it never wakes up.
Using SSMS I can open a connection to a sleeping Serverless SQL database and after a delay the connection will go through.
Using my ASP.NET Core application I tried the same. From the login page I tried to login, which opens a connection to the database. After 10 or 11 seconds (I looked up the default timeout and its supposed to be 15 seconds but in this case it always seems to be about 10.5 seconds +/-0.5s). According to the docs, the first connection attempt may fail but subsequent ones should succeed, but I can send multiple queries to the database and it always fails with the following error:
Microsoft.Data.SqlClient.SqlException (0x80131904): Database 'myDb' on server
'MyDbSvr.database.windows.net' is not currently available. Please retry the connection later. If the
problem persists, contact customer support, and provide them the session tracing ID of
'{XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX}'.
If I wake the database up using SSMS then the login web page can connect to the database and succeeds.
I have added Connect Timeout=120; to the connection string.
The connection does happen during an HTTP request that is marked async on the Controller, thought I don't know if that makes any difference.
Am I doing something wrong or is there something additional I need to do to get the DB to wake?
[updte]
as an extra test wrote the following test
void Main()
{
SqlConnection con = new SqlConnection("Server=mydbsvr.database.windows.net;Database=mydb;User Id=abc;Password=xyz;Connect Timeout=120;");
Console.WriteLine(con.ConnectionTimeout);
con.Open();
var cmd = con.CreateCommand();
cmd.CommandText = "select getdate();";
Console.WriteLine(cmd.ExecuteScalar());
}
and got the same error.
I figured it out and its the dumbest thing.
This Azure SQL Server instance was migrated from another subscription and the group that migrated it gave it a new name, but they did something that allowed the use of the old name also. I'm researching to figure out how that was done. I will update this answer when I find out what that was.
As it turns out, using the old name with an Serverless Database won't wake up the db. Don't know why. But if you change to use the new/real server name it works. you do have to add a retry to the connection as it may fail the first few times.
[Update]
The new server allows logins using the old name by using a Azure SQL Database Alias https://learn.microsoft.com/en-us/azure/sql-database/dns-alias-overview

Connecting to oracle db from python

We are connecting to oracle from python using cx_oracle package.
But the user_id, password and SID details are hardcoded in that.
My question is, is there any way to create a Datasource kind of thing? Or how we will deploy such python script sin production?
The database is in a Linux box and python is installed in another Linux box(Weblogic server is also installed in this Linux box).
import cx_Oracle
con = cx_Oracle.connect('pythonhol/welcome#127.0.0.1/orcl')
print con.version
Expectation is :
Can we deploy python in a production instance?
If yes how can we connect to the database by hiding the DB credentials?
Use some kind of 'external authentication', for example a wallet. See the cx_Oracle documentation https://cx-oracle.readthedocs.io/en/latest/user_guide/connection_handling.html#connecting-using-external-authentication
In summary:
create a wallet with mkstore which contains the username/password credentials.
copy the wallet to the machines that are running Python
make sure no bad people can access the wallet
configure Oracle Net files to point to the wallet
your scripts would connect like
connection = cx_Oracle.connect(dsn="mynetalias", encoding="UTF-8")
or
pool = cx_Oracle.SessionPool(externalauth=True, homogeneous=False, dsn="mynetalias",
encoding="UTF-8")
pool.acquire()

mariadb connection string to r

I am trying to use a VM machine with R on Azure. I want to connect it to a mariaDB on Azure.
I use the RmySQL package, and I use the following connection string:
require(RMySQL)
con <- dbConnect(RMySQL::MySQL(),
dbname="energidb",
host="energidb.mariadb.database.azure.com",
port=3306,
user="USER",
password="MY_PWD")
However, No luck I have in trying this, as R says. "Error in .local(drv, ...) :
Failed to connect to database: Error: Unknown database 'energidb'"
On azure, I promise and guarentee with my life, the name of the mariaDB is "energidb"
What am I doing wrong?
If you just create a Maria DB resource on Azure portal, then you should only have a database server.
You can connect to the Maria DB without specifying a database name. And then list all the existing databases as following:
con <- dbConnect(RMySQL::MySQL(),
host="your host address, jackmariadb.mariadb.database.azure.com",
port=3306,
user="user name, jack#jackmariadb",
password="password, d*******5")
rs = dbSendQuery(con, "show databases")
fetch(rs, n=-1)
In my case, it will just show the 3 system databases:
It turns out that on Azure, I created a mariaDB server (without any database). The database reference i made was to the server, which ofcourse is meaningless.
The solution is first to create a database (which is done by pointing at the server). From there I can point on the database.

Resources