I'm reading the psycopg2 documentation & wondering how to parametrize SQL identifiers of tables with a name? Here is an example:
import psycopg2
conn = psycopg2.connect()
cursor = conn.cursor()
cursor.execute(
"SELECT * FROM %(my_table)s LIMIT %(my_limit)s;"
vars={
"my_limit": 42, # parametrizing literals works fine.
"my_table": sql.Identifier("foo"), # how to do same with named identifiers?
}
)
psycopg2.ProgrammingError: can't adapt type 'Identifier'
I know I could use positional parameters %s or {} but I would like the query to mix and match identifiers with literals with a named mapping.
This did it for me:
import psycopg2
from psycopg2 import sql
conn = psycopg2.connect()
cursor = conn.cursor()
cursor.execute(sql.SQL(
"SELECT * FROM {my_table} LIMIT {my_limit};"
).format(
my_limit = sql.Literal(42),
my_table = sql.Identifier("foo"),
).as_string(conn)
)
Related
I have been having trouble using WHERE $VARIABLE IN clauses in psycopg2:
from app.commons.database import conn
from psycopg2 import sql
from psycopg2.extras import DictCursor
query = '''
SELECT
*
FROM
{}.{}
WHERE
{} in %s
'''.format(
sql.Identifier('information_schema'),
sql.Identifier('tables'),
sql.Identifier('table_schema')
)
data = (
'information_schema',
'pg_catalog'
)
with conn.cursor(cursor_factory=DictCursor) as cursor:
cursor.execute(query, data)
print(cursor.fetchall())
raises
TypeError: not all arguments converted during string formatting
I've read the seemingly hundreds of posts on this same topic, and the overwhelming answer has been: "you need to use tuples when submitting data as the second argument to cursor.execute". I've been doing that and still can't seem to determine where the gap is.
Check out the psycopg2 documentation on Lists adaptation
You are getting that error because psycopg2 is trying to substitute the two parameters, but you only gave it one parameter. Try changing to this:
from app.commons.database import conn
from psycopg2 import sql
from psycopg2.extras import DictCursor
query = '''
SELECT
*
FROM
{}.{}
WHERE
{} =ANY(%s)
'''.format(
sql.Identifier('information_schema'),
sql.Identifier('tables'),
sql.Identifier('table_schema')
)
data = [
'information_schema',
'pg_catalog'
] # A list now, instead of a tuple
with conn.cursor(cursor_factory=DictCursor) as cursor:
cursor.execute(query, (data, )) # A tuple, containing your list
print(cursor.fetchall())
Been trying to connect our ERP ODBC by using PYODBC, Although I got the syntax correct the only error I'm getting at this point is this 'TypeError: the first argument to execute must be a string or unicode query'
I've tried adding .decode('utf-8').
import pyodbc
import pandas as pd
conn = pyodbc.connect(
'DRIVER={SQL Server};'
'SERVER=192.168.1.30;'
'DATABASE=Datamart;'
'Trusted_Connection=yes;')
cursor = conn.cursor()
for row in cursor.tables(tableType='TABLE'):
print(row)
sql = """SELECT * FROM ETL.Dim_FC_UPS_Interface_Detail"""
cursor.execute(row, sql)
df = pd.read_sql(sql, conn)
df.head()
I think your ordering of commands is off a bit for use of pyodbc cursor execute function. See the docs.
cursor = conn.cursor()
sql = """SELECT * FROM ETL.Dim_FC_UPS_Interface_Detail"""
cursor.execute(sql)
for row in cursor:
print(row)
Can't execute a psycopg2 insert query (Postgres db), which uses the best practice %s sign for inserting and contains a LIKE statement which includes a % sign.
% sign in LIKE statement is interpreted as an insert placeholder.
'IndexError: tuple index out of range' is thrown.
Tried escaping % with backslash, didn't work out.
with psycopg2.connect(some_url) as conn:
with conn.cursor() as cur:
query = """
SELECT id
FROM users
WHERE surname IN %s AND named LIKE '%john'
"""
cur.execute(query, (tuple(["smith", "mcnamara"]),))
data = cur.fetchall()
Try using a placeholder also for the LIKE expression, and then bind a literal with a wildcard to it:
query = """
SELECT id
FROM users
WHERE surname IN %s AND named LIKE %s"""
cur.execute(query, (tuple(["smith", "mcnamara"]), "%John",))
data = cur.fetchall()
try this one:
with psycopg2.connect(some_url) as conn:
with conn.cursor() as cur:
query = """
SELECT id
FROM users
WHERE surname IN %s AND named LIKE '%sjohn'
"""
cur.execute(query, (tuple(["smith", "mcnamara"]), '%'))
data = cur.fetchall()
Why following code is giving syntax error "sqlite3.OperationalError: near "?": syntax error"
import sqlite3
connection = sqlite3.connect('data.db')
cursor = connection.cursor()
table = "device_store"
uuid = "bbebe39e-fe2e-4817-b022-a3ef13bd6283"
page = 1
POSTS_PER_PAGE = 10
query = "SELECT * FROM ? WHERE uuid=? LIMIT ? OFFSET ?"
result = cursor.execute(query, (table, uuid, POSTS_PER_PAGE, 0))
rows = result.fetchall()
connection.close()
print("==>> Printing rows <<==")
print(rows)
The error is caused by the placeholder in FROM ?, not the others. Table names can't be passed as parameters, they have to be hardcoded in the statement.
i have the following function which extracts data from table, but i want to pass the table name in function as parameter...
def extract_data(table):
try:
tableName = table
conn_string = "host='localhost' dbname='Aspentiment' user='postgres' password='pwd'"
conn=psycopg2.connect(conn_string)
cursor = conn.cursor()
cursor.execute("SELECT aspects_name, sentiments FROM ('%s') " %(tableName))
rows = cursor.fetchall()
return rows
finally:
if conn:
conn.close()
when i call function as extract_data(Harpar) : Harpar is table name
but it give an error that 'Harpar' is not defined.. any hepl ?
Update: As of psycopg2 version 2.7:
You can now use the sql module of psycopg2 to compose dynamic queries of this type:
from psycopg2 import sql
query = sql.SQL("SELECT aspects_name, sentiments FROM {}").format(sql.Identifier(tableName))
cursor.execute(query)
Pre < 2.7:
Use the AsIs adapter along these lines:
from psycopg2.extensions import AsIs
cursor.execute("SELECT aspects_name, sentiments FROM %s;",(AsIs(tableName),))
Without the AsIs adapter, psycopg2 will escape the table name in your query.