Need overview explanation of dbase connectivity using python - python-3.x

I have been working with python and postgreql for over a year. I can connect and work with postgres databases by blindly using various libraries. But whenever I change platform (most recently from macOS laptop to remote ubuntu server) I go through a day or so of trying to get libraries working eg. I was using 'pyodbc' in some modules but when I migrated the code to the server I had to switch to 'pg8000' because the modules as they were kept throwing errors.
Can someone explain or point me to a link explaining how python connects to dB's? For example, why do I need a MS ODBC driver for 'pyodbc' to connect to an Azure SQL or postgresql but 'pg8000' seems to need nothing at all to connect to a postgresql? When I move to an Ubuntu environment and install ODBC drivers they show up on root under /etc, and /opt (for MS ODBC) but also in my Conda environment (/anaconda3/envs/) and I don't know which is the correct choice for 'ODBC.ini'?
Like I say, I can get things working but really have no understanding as to why they are working and that means I waste time experimenting every time I deal with a change in environment. I've not yet found an explanation online that covers more than a very specific circumstance eg. 'here's how to install our driver ...' Any help would be appreciated.
Final Update:
Following the responses esp. #Thompson the diagram below seems to be the final interpretation and I have a better idea of where to look for answers. For the record pyodbc, SQLAlchemy and pg8000 have been my tools of choice with no problems except as described in the question.

pyodbc is not actually a driver and doesn't contain one, its a 'module for ODBC databases', so it's more of an interface from python to ODBC driver to some database. That's why to use it you have to have an actual separate driver to connect to. Azure SQL being owned by Microsoft would reasonably require Microsoft's ODBC driver, while Postgres will require a Postgres ODBC driver, etc...
The ODBC driver manager is platform-specific, while the ODBC driver is database-specific. That would explain why if you are you are changing platforms or databases, you need to change drivers.
As Adrian noted, you don't need ODBC drivers for postgres, it is more common to use postgres/python drivers (eg: https://wiki.postgresql.org/wiki/Python)
psycopg2 is an actual PostgresSQL driver. It serves as client from Python to postgres, no intermediary required. That's why you don't need to install anything else when you use it. I haven't used pg8000, but based on this list it's a driver too, so you wan't need anything else.
EDITED TO ADD:
Think of a database as some 'black box' you need to activate, and its drivers as electrical sockets. ODBC driver is a specific type of socket (ODBC is a standard developed by Microsoft). If you are using ODBC plug from python (like pyodbc) to a database, you need to make sure the database has an ODBC socket installed/activated.
But your database can have other sockets too, like python-compatible DBAPI that's available on postgres. In that case you use a different direct DBAPI connector, like psycopg2.

Drivers are specific to a database. ODBC is a two stage process. There is the ODBC driver manager and then there are the database specific drivers that allow you to talk to a database. You don't need ODBC to connect to a Postgresql server. If you are going through Python you just need one of the Postgres drivers. You have already found pg8000. My preference is psycopg2.

Related

What is the importance of Oracle client in Python script?

I'm working with python 3.6 on ubuntu 16.04 and trying to connect with Oracle database which is installed in another machine. So i have installed "cx_Oracle" python module by following this link.
Now, my doubt is what is the necessity to install oracle client in my machine when i need to access the database from another machine in my python script?
The Oracle Client libraries provide all the necessary network connectivity (e.g. things like network encryption), connection management (e.g. connection pooling), high availability features, cross platform and Oracle-version support, data caching, etc etc etc. They are used by many different user and applications. They have significant engineering in them, not to mention testing.
cx_Oracle makes calls to the Oracle Client libraries (which are freely available in Oracle Instant Client), so you need those libraries.

ODBC driver connection string error

This is a general question that I can't seem to find any specific answer to. I am using a node module: https://www.npmjs.com/package/odbc
The module says to install a ODBC driver for the specified database and also have a properly configured odbc.ini and odbcinst.ini.
I am also using docker to run my node script.
My question is on my docker container, do I install each driver for the database? And then on the database server I have to configure the odbc.ini? Or on my node server I set this up? Or does it already come pre configured on some databases?
I keep getting the error: "Data source name not found, and no default driver specified"
Which I am guessing since I probably don't have the driver setup in my node container and the odbc.ini properly setup on my database server.
I am currently testing on a mysql database but want to be able to connect to any ODBC database.
Here is a link to setup mysql on ubuntu:
http://www.kaffeetalk.de/how-to-setup-and-configure-mysql-with-unixodbc-under-ubuntu-14-04/
odbc.ini contains DSN (Data Source Name) definitions, which are based on drivers registered in the odbcinst.ini, both found along with the drivers themselves on the ODBC data consuming host/environment (here, your docker container). These configuration files are typically (and optimally) respectively targeted by $ODBCINI and $ODBCINSTINI environment variables, but sometimes found at default locations like /etc/odbc.ini or ~/.odbc.ini.
You might look at some documentation, like the ODBC spec on MSDN, or developer guidance on iODBC.org. Do be sure to use ODBC's DBMS-agnostic SQL syntax for your queries, and not the DBMS-specific dialect for MySQL (or any other DBMS)!

Access to database without installing any client/lib

I am currently on a project where I have 2 VM (virtual machine), a Windows and Linux one.
I also have an Oracle database where I have a simple table called "Material".
On the 2 VM, I want to connect to my Oracle database without any client or libraries. The thing is I want to create a script which would run on the VM and can connect to my database and insert some datas to my table "Material" but I can't install anything on my VM (like the mysqlclient for exemple).
So is it possible to connect to a database without installing anything on my VM? Or perhaps can I access to an online client to send my SQL to my Oracle Database?
I know it's quite difficult to understand my problem so if you have any question, feel free to ask.

Error when running Lotus Domino agent

I wrote an Lotus Domino agent to transfer Domlog.nsf data to DB2 database. When I run agent I get:
Error: Cannot find Connector 'DB2COPY2'
DB2COPY2 is name of my DB2 client or instance, don't know how you call it.
And next error is (probably due to first error)
NO RESUME
In the LotusScript I got line Dim conn As New LCConnection("DB2COPY2")
Any ideas on first error? Thank you.
You want to access a data source without installing a driver for it. Its like wanting to drive a car without a key... This does not work. To access odbc data sources, you need tominstall an odbc driver (from the vendor of your database system). To accesss db2 databases, you need a db2- client installed on the server (delivered by ibm if you have the relevant licenses).
In addition you could install decs (for free) or lei (quite cost intensive) to make it easier to configure the access to the data sources, but it will not make the drivers obsolete...

MySql with Mac os

I want to use MySql database as centralised database for storing user informations(credentials) for an iPhone application.MySql is not installed in my system.Can anyone tell me how to use MySql in mac os.
The MySQL manual has a section on installing MySQL on OS X
Obviously, to access it from an iPhone application you will then have to configure it to be exposed over the network (not a good idea except possibly for LAN use only) or put something in between the database and the world (a Web Service is the usual approach).
If you mean that you want to install it on iOS, then forget it. There isn't an iOS version. Look to SQLite instead. AFAIK it is the usual approach for SQL on that platform.

Resources