I am trying to connect to an Oracle Data instance (ORAD) with a Python script.
Here is the basic script:
import cx_Oracle
conn = cx_Oracle.connect("username/password#//server:1560/orad")
c = conn.cursor()
c.execute('select distinct * from table1')
for row in c:
print(row)
conn.close()
I currently have the instance's port, SID, and hostname, too, if that helps.
Running this script yields a: cx_Oracle.DatabaseError: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor error,
while using the other connections (that is commented out) yields an error SyntaxError: invalid syntax
I am unsure of what I am doing wrong. I did check my TNSNAMES.ORA file which contains a few ifile links to DBA secured (I don't have access to see or edit) other files.
I have viewed this post and this post, but I don't have the IP, just the host name.
Any assistance would be appreciated.
The following answer and script worked:
def get_data(database, username, password, sql_statement):
import cx_Oracle
dsn_tns = cx_Oracle.makedsn('<server>', '<port>', '<sid>')
connection = cx_Oracle.connect(username, password, dsn_tns)
c = connection.cursor()
c.execute(sql_statement)
# Print the returning dataset
for row in c:
print(row)
# Close the connection
connection.close()
Related
I've to transfer data from one postgreSQL DB (old) into another postgresSQL DB (new).
Old is encoded in win1252. New is encoded in utf-8.
I've already tried different methods ex. pandas.to_sql, sqlalchemy, psycopg2 and so on but failing all the time due to encoding "issues". I've done some researches and the most valid thing looks like an issue on the driver side. As far as I know psycopg2 uses the unicode driver but with my source database version (PostgreSQL 9.4.20 on x86_64) I've to use ANSI to bypass these encoding issues.
I've tested with an ETL tool if it's possible to export the affected table without encoding issues. It was possible without issues. Due to this test I'm pretty sure it's no real encoding issue instead of an driver handling issue.
When I just used a sample to test if loading the data in general works, I already noticed pandas is to slow. I've to load 1.2 mio reccords. But this runs for ever. Therefore the postgreSQL copy method is may prefered method. From my perspective psycopg2 is useing the standard connection string (https://halvar.at/python/odbc_dsn_connection_strings/). But I've to use the ANSI driver.
I tried to pass an SQLAlchemy to thy psycopg2 connector. But this does not work.
stage_engine_string = ("{PostgreSQL ANSI}+psycopg2://" + str(stage_user) + ":" + str(stage_password) + "#" + str(stage_host) + ":" + str(stage_port) + "/" + str(stage_database))
because
conn = psycopg2.connect(**params)
only allows to pass the arguments.
host =
database =
user =
password =
port =
Before I tried the above I tried for ex.
cur.copy_to(open("sql_tmp_export.csv", "w", encoding="utf-8", errors="ignore"), "table", sep=";", columns=("no","description"))
,
conn.decode("win1250").encode("utf8")
and
conn.set_client_encoding("win1250")
but I receive an encoidng issue all the time. Based on the doc of postgres switching between utf8 and win1250 should never be an problem.
On the ETL tool I'd a similar issue but was able to solve it via sending an
"set client_encoding=\"windows-1250\"
after esthablishing the connection to the database.
But if I try this in psycopg2 to
cur.execute("set client_encoding=\"windows-1250\;select * from table")
I stil get the encoding issue.
Any clue if I have an option to pass the driver on builing up a psycopg2 connection? I think this should solve my issue.
My real issue (getting data from db) wasn't solved because of follow up issues. If you want to get into, I'm happy to discuss on my next question: Downloading a postgreSQL pg_dump file from a remote server using Python
But I was able to solve this question. If you want to use an ANSI you've to install the last ODBC driver from https://www.postgresql.org/ftp/odbc/versions/msi/
Then you can swith the psycopg2 connection to an pyodbc connection.
import pyodbc
conn_str = (
"DRIVER={PostgreSQL Ansi(x64)};"
"DATABASE="+database+";"
"UID="+user+";"
"PWD="+password+";"
"SERVER="+host+";"
"PORT="+port+";"
)
conn = pyodbc.connect(conn_str)
cur = conn.execute("SELECT 1")
row = cur.fetchone()
print(row)
cur.close()
conn.close()
My general problem has been fixed now as well. But the solution was strange. If someone stucks on something similar, I simply run the same script twice but first of all with limit and offset.
def any_postrgres_method_to_load_data_from_db:
conn = some_lib.conect(var1, var2)
cur = conn.cursor()
sql_pre_statement = """\
set client_encoding = "Windows-1250"
"""
cur.execute(sql_pre_statement)
sql_statement = """\
select * from n
"""
cur.execute(sql_statement)
df = pandas.read_sql_query(sql, conn)
df.to_csv("sql_tmp_export.csv", index=False)
The script above returned several encoding issues.
After running the script slightly adjusted as shown below ones, I was able to run the original one working.
def any_postrgres_method_to_load_data_from_db:
conn = some_lib.conect(var1, var2)
cur = conn.cursor()
sql_pre_statement = """\
set client_encoding = "Windows-1250"
"""
cur.execute(sql_pre_statement)
sql_statement = """\
select * from n offset 500 limit 1000
"""
cur.execute(sql_statement)
df = pandas.read_sql_query(sql, conn)
df.to_csv("sql_tmp_export.csv", index=False)
I can't really explain this. I've just the feeling that there was something strange in the cache on the remote db.
I am trying execute below block of code with cx_oracle by bind variables, but getting below mentioned error everytime. Not sure what is missing.
Anyone has idea on this
Code :
a = input("Please enter your name ::")
conn = cx_Oracle.connect('hello/123#oracle')
cur = conn.cursor()
text1 = "select customer from visitors where name = :myvalue;"
cur.execute(text1,myvalue=str(a))
ERROR observed :
cx_Oracle.DatabaseError: ORA-00933: SQL command not properly ended
Remove the semi-colon at the end of your SQL statement.
I'm trying to run a postgres db in a docker container, running a small python program with a class to call the db.
When I run code with a query it seems to work fine, but gives no results.
I can see that I in fact have hit the database because one of the tables have an id constraint, resulting in an error when I try to insert something that already exists.
Using the db from TablePlus works fine.
Code:
import psycopg2
class postgres():
def __init__(self, db="foo", user="bar", password='baz', host='127.0.0.1', port=5432):
self.conn = psycopg2.connect(
database=db, user=user, password=password, host=host, port=port)
self.cur = self.conn.cursor()
def query(self, query):
self.cur.execute(query)
# self.cur.execute("CREATE TABLE test (id serial PRIMARY KEY, num integer, data varchar);")
def close(self):
self.cur.close()
self.conn.close()
db = postgres()
db.query("INSERT INTO test (id) values('test2')")
db.close()
results in:
"""
Traceback (most recent call last):
File "/Users/myname/projects/myproject/dataGathering/postgres.py", line 21, in
db.query("INSERT INTO test (id) values('test2')")
File "/Users/myname/projects/myproject/dataGathering/postgres.py", line 11, in query
self.cur.execute(query)
psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "id_pkey"
DETAIL: Key (id)=(test) already exists.
"""
Inserting something without a conflicting id raises no error but neither gives any results in the db. SELECT queries have the same faith.
You are not doing:
conn.commit()
So your changes are not committed.
From here:
https://www.psycopg.org/docs/connection.html
" close()
Close the connection now (rather than whenever del is executed). The connection will be unusable from this point forward; an InterfaceError will be raised if any operation is attempted with the connection. The same applies to all cursor objects trying to use the connection. Note that closing a connection without committing the changes first will cause any pending change to be discarded as if a ROLLBACK was performed (unless a different isolation level has been selected: see set_isolation_level())."
def depositamt():
conn = sqlite3.connect("atm.db")
cursor = conn.cursor()
cursor.execute('''UPDATE data SET balance = balance + '%s' WHERE username = '%s' '''%(depositamount.get(),username.get()))
messagebox.showinfo("Successful", "deposit amount successfully")
conn.commit()
cursor.close()
conn.close()
This function gets the username from username Entry box and updates the record. But the database is not being updated.
Are you sure about your syntax?
It is not the syntax I am familiar with. Assuming both functions work as expected and username is already present into your base I would use the following statement:
cursor.execute("UPDATE data SET balance=? WHERE username=?", (depositamount.get(), username.get()))
Regards.
I have been playing around with pyodbc and for some reason when trying to connect if I get the Server property from a input() it cannot find the server, but if I take the same server name and declare it as a variable before hand it works fine with the exact same code. Does anyone know what is going on here?
Code with input()
import pyodbc
driver = '{ODBC Driver 17 for SQL Server}'
instance = input("Please Enter your SQL Instance: ")
connectionstring = f'Driver={driver}; Server={instance}; Trusted_Connection=yes'
conn = pyodbc.connect(connectionstring)
cursor = conn.cursor()
cursor.execute('SELECT name FROM sys.databases')
for row in cursor:
print(row)
Output
Code output server name from Input()
Code with Variable
import pyodbc
driver = '{ODBC Driver 17 for SQL Server}'
instance = 'DESKTOP-J7PBL8S\\NORTHWIND'
connectionstring = f'Driver={driver}; Server={instance}; Trusted_Connection=yes'
conn = pyodbc.connect(connectionstring)
cursor = conn.cursor()
cursor.execute('SELECT name FROM sys.databases')
for row in cursor:
print(row)
Output
Code output server name from variable
I have tried using str() on the input with no luck. Not really sure why it doesn't like when I get the server name from an input because it returns a string and the connection string is exactly the same as the code that works.
I am kind of curious why this is the case not really looking to use input() on any real project
I tried:
ins1 = input("Please Enter your SQL Instance: ")
ins2 = 'DESKTOP-J7PBL8S\\NORTHWIND'
print(ins1)
print(ins2)
Then pass the DESKTOP-J7PBL8S\\NORTHWIND as input and the output was:
DESKTOP-J7PBL8S\\NORTHWIND
DESKTOP-J7PBL8S\NORTHWIND
So I think we found the problem.
The input() returns the raw string without considering \ special meaning.
So if you use just one \-> DESKTOP-J7PBL8S\NORTHWIND, you should be able to connect to the intended server.