I'm Executing select query to postgresql database and after fetching those results I'm appending those results to list and then I'm giving that list as the input to another postgresql select query.
But due to conversion of those values to list it converts values with apostrophe(special character) cat's to double quotes "cat's". while executing second select query the value with double quotes is not been fetched because value with double quotes is not present in the database it is without double quotes cat's.
And there it gives me error that value is not present.
I have tried JSON dumps method but its isn't working because I cannot convert JSON list to tuple and give it as the input to postgresql select query
select_query = """select "Unique_Shelf_Names" from "unique_shelf" where category = 'Accessory'"""
cur.execute(select_query)
count = cur.fetchall()
query_list = []
for co in count:
for c in co:
query_list.append(c)
output of query_list:
query_list = ['parrot', 'dog', "leopard's", 'cat', "zebra's"]
Now this querylist is been converted to tuple and given as the input to another select query.
list2 = tuple(query_list)
query = """select category from "unique_shelf" where "Unique_Shelf_Names" in {} """.format(list2)
cur.execute(query)
This is where it gives me error "leopard's" doesn't exist but in database leopard's exists.
I want all the values in the query_list to be double quotes so this error doesn't arises.
Do not use format to construct the query. Simply use %s and pass the tuple into execute
query = """select category from "unique_shelf" where "Unique_Shelf_Names" in %s """
cur.execute(query,(list2,))
Tuples adaptation
Related
I want get variable with sql query:
Dt = spark.sql("""select max(dt) from table""")
Script = """select * from table where dt > """ + dt
Spark.sql(script)
but when I try to substitute a variable in the request I get error:
"Can only concatenate str (not dataframe) to str"
How do I get the variable as a string and not a dataframe?
To get the result in a variable, you can use collect() and extract the value. Here's an example that pulls the max date-month (YYYYMM) from a table and stores it in a variable.
max_mth = spark.sql('select max(mth) from table').collect()[0][0]
print(max_mth)
# 202202
You can either cast the value to string in the sql statement, or use str() on the variable while using to convert the integer value to string.
P.S. - the [0][0] is to select the first row-column
I want replace parameters in SQL scrip in string format using references from list with dictionary inside. This is example SQL script:
sql = "SELECT DISTINCT id, channel AS value FROM ${source} WHERE channel IN (${channel});"
All of the queries are in this structure:
{'name': 'example', 'description': 'example1', 'sql': 'SELECT DISTINCT id, channel AS value FROM ${source} WHERE channel IN (${channel});'}
The dictionary looks like this:
my_dict = [{"source":"table_name","channel":["abc","abcd"]}]
The desired output is:
SELECT DISTINCT pnr, channel AS value FROM table_name WHERE channel IN ("abc", "abcd");
Note: It is possible the channel parameter to have single value: my_dict = [{"source":"table_name","channel":["abc"]}] This is very important to have in mind.
I was thinking to .split the query and to replace the parameters, but for some reason it is not working and the channels are without quotes. I need them in quotes, so the query can be executed later.
queries = sql.split()
final_string = ' '.join(str(my_dict.get(word, word)) for word in queries)
Another solution I found, but could not implemented for my case : Replacing a value in string with a value in a dictionary in python
What you have is not a dictionary, but a list. If you are interested in the dictionary at the first position of the list you could use an f-string and do:
>>> sql = f"SELECT DISTINCT id, channel AS value FROM {my_dict[0]['source']} WHERE channel IN {tuple(my_dict[0]['channel'])};"
>>> sql
"SELECT DISTINCT id, channel AS value FROM table_name WHERE channel IN ('abc', 'abcd');"
Sorry for this but I'm real new to sqlite: i've created a database from an excel sheet I had, and I can't seem to fetch the values of the column I need
query = """ SELECT GNCR from table"""
cur.execute(query)
This actually works, but
query = """ SELECT ? from table"""
cur.execute(query, my_tuple)
doesn't
Here's my code:
def print_col(to_print):
db = sqlite3.connect('my_database.db')
cur = db.cursor()
query = " SELECT ? FROM my_table "
cur.execute(query, to_print)
results = cur.fetchall()
print(results)
print_col(('GNCR',))
The result is:
[('GNCR',), ('GNCR',), ('GNCR',), ('GNCR',), [...]]
instead of the actual values
What's the problem ? I can't figure it out
the "?" character in query is used for parameter substitution. Sqlite will escape the parameter you passed and replace "?" with the send text. So in effect you query after parameter substitution will be SELECT 'GNCR' FROM my_table where GNCR will be treated as text so you will get the text for each row returned by you query instead of the value of that column.
Basically you should use the query parameter where you want to substitute the parameter with escaped string like in where clause. You can't use it for column name.
Postgresql select query containing tuple with single quotes as well as double quotes when giving this tuple as the input to select query it genrates error stating that specific value is not present in the database.
I have treid converting that list of values to JSON list with double quotes but that doesn't help either.
list = ['mango', 'apple', "chikoo's", 'banana', "jackfruit's"]
query = """select category from "unique_shelf" where "Unique_Shelf_Names" in {}""" .format(list)
ERROR: column "chikoo's" doesn't exist
Infact chikoo's does exist
But due to double quotes its not fetching the value.
Firstly please don't use list as a variable name, list is a reserved keyword and you don't wanna overwrite it.
Secondly, using "" around tables and columns is bad practice, use ` instead.
Thirdly, when you format an array, it outputs as
select category from `unique_shelf`
where `Unique_Shelf_Names` in (['mango', 'apple', "chikoo's", 'banana', "jackfruit's"])
Which is not a valid SQL syntax.
You can join all values with a comma
>>>print("""select category from `unique_shelf` where `Unique_Shelf_Names` in {})""".format(','.join(l)))
select category from `unique_shelf`
where `Unique_Shelf_Names` in (mango,apple,chikoo's,banana,jackfruit's)
The issue here is that the values inside the in bracket are not quoted. We can do that by formatting them beforehand using double quotes(")
l = ['mango', 'apple', "chikoo's", 'banana', "jackfruit's"]
list_with_quotes = ['"{}"'.format(x) for x in l]
query = """
select category from `unique_shelf`
where `Unique_Shelf_Names` in ({})""" .format(','.join(list_with_quotes))
This will give you an output of
select category from `unique_shelf`
where `Unique_Shelf_Names` in ("mango","apple","chikoo's","banana","jackfruit's")
I have a line of code here:
query = """SELECT v.retailiqpo_ordernumber
FROM public.vmi_purchase_orders v
WHERE v.vendor_account = {}""".format(str(primary_account_number))
I tried to load in the string value of the number, but psycopg2 still throws this error.
psycopg2.ProgrammingError: operator does not exist: character varying = integer
What options do I have to ensure Psycopg2 sees this as a string? Or should I just change the overall structure of the database to just integers?
It's (almost) always better to let psycopg2 interpolate query parameters for you. (http://initd.org/psycopg/docs/usage.html#the-problem-with-the-query-parameters)
query = """SELECT v.retailiqpo_ordernumber
FROM public.vmi_purchase_orders v
WHERE v.vendor_account = %s"""
cur.execute(query, (str(primary_account_number),))
This way psycopg2 will deal with the proper type formatting based on the type of the python value passed.
Use
query = """
SELECT v.retailiqpo_ordernumber
FROM public.vmi_purchase_orders v
WHERE v.vendor_account = '{}'
""".format(primary_account_number)
That way the number inside your query is passed as a string - if your c.vendor_account is of a stringtype (varchar i.e.). The important part are the ' before/after {} so the query string sees it as string.
As Jon Clements pointed out, it is better to let the api handle the conversion:
query = """
SELECT v.retailiqpo_ordernumber
FROM public.vmi_purchase_orders v
WHERE v.vendor_account = %s
"""
cursor.execute(query, (str(primary_account_number),)
Doku: Psycopg - Passing parameters to sql queries