I am stuck to find out better one between str() and format() in python
"SELECT schools.deis_income , schools.school_name,SUM(money.coin_in_amount) AS coinamount, SUM(money.note_in_amount) AS noteamount , SUM(money.coffee_coin_in_amount) AS coffeeamount , SUM(money.coin_out_amount) AS coinoutamount, SUM(money.note_out_amount) AS noteoutamount FROM money_transactions AS money JOIN school_admin_details AS sa on sa.id = money.school_admin_id JOIN schools ON schools.id=sa.school_id WHERE sa.school_id ={school_id} AND money.transaction_time BETWEEN '{start_date}' AND '{end_date}' GROUP BY schools.id".format(school_id=school_id,start_date=start_date,end_date=end_date)
I use format function here. can I use str() ?
please tell me which one give me quick result, str() or format() ???
If your question is which of this:
foo = "some text " + str(some_var) + " and some other text"
or this:
foo = "some text {} and some other text".format(var)
is "better", the general consensus is very clear: string formatting is much easier to read and maintain and the one pythonic way to go.
Now for your particular example, the answer is that both are totally wrong - unless you're ok to give full access to your database to even the most inept script kiddie. For SQL queries, the proper solution is to use prepared statements, where your db connector will take care of proper formatting and sanitizing of the values:
# assumes MySQL - for other vendors check your own
# db-api connector's doc for the correct placeholder
query = "SELECT somefield FROM mytable where somedate > %(somedate)s and something_else = %(someval)s"
cursor.execute(query, {"somedate": some_date, "someval": 42})
Related
I have had to look up hundreds (if not thousands) of free-text answers on google, making notes in Excel along the way and inserting SAS-code around the answers as a last step.
The output looks like this:
This output contains an unnecessary number of blank spaces, which seems to confuse SAS's search to the point where the observations can't be properly located.
It works if I manually erase superflous spaces, but that will probably take hours. Is there an automated fix for this, either in SAS or in excel?
I tried using the STRIP-function, to no avail:
else if R_res_ort_txt=strip(" arild ") and R_kom_lan=strip(" skåne ") then R_kommun=strip(" Höganäs " );
If you want to generate a string like:
if R_res_ort_txt="arild" and R_kom_lan="skåne" then R_kommun="Höganäs";
from three variables, let's call them A B C, then just use code like:
string=catx(' ','if R_res_ort_txt=',quote(trim(A))
,'and R_kom_lan=',quote(trim(B))
,'then R_kommun=',quote(trim(C)),';') ;
Or if you are just writing that string to a file just use this PUT statement syntax.
put 'if R_res_ort_txt=' A :$quote. 'and R_kom_lan=' B :$quote.
'then R_kommun=' C :$quote. ';' ;
A saner solution would be to continue using the free-text answers as data and perform your matching criteria for transformations with a left join.
proc import out=answers datafile='my-free-text-answers.xlsx';
data have;
attrib R_res_ort_txt R_kom_lan length=$100;
input R_res_ort_txt ...;
datalines4;
... whatever all those transforms will be performed on...
;;;;
proc sql;
create table want as
select
have.* ,
answers.R_kommun_answer as R_kommun
from
have
left join
answers
on
have.R_res_ort_txt = answers.res_ort_answer
& have.R_kom_lan = abswers.kom_lan_answer
;
I solved this by adding quotes in excel using the flash fill function:
https://www.youtube.com/watch?v=nE65QeDoepc
I have been trying to update some columns of a database table using cx_Oracle in Python. I created a list named check_to_process which is a result of another sql query. I am creating log_msg in the program based on success or failure and want to update same in the table only for records in check_to_process list. When I update the table without using bind variable <MESSAGE = %s>, it works fine. But when I try to use bind variable to update the columns it gives me error :
cursor.executemany("UPDATE apps.SLCAP_CITI_PAYMENT_BATCH SET MESSAGE = %s, "
TypeError: an integer is required (got type str)
Below is the code, I am using:
import cx_Oracle
connection = cx_Oracle.connect(user=os.environ['ORA_DB_USR'], password=os.environ['ORA_DB_PWD'], dsn=os.environ['ORA_DSN'])
cursor = connection.cursor()
check_to_process = ['ACHRMUS-20-OCT-2021 00:12:57', 'ACHRMUS-12-OCT-2021 16:12:01']
placeholders = ','.join(":x%d" % i for i,_ in enumerate(check_to_process))
log_msg = 'Success'
cursor.executemany("UPDATE apps.SLCAP_CITI_PAYMENT_BATCH SET MESSAGE = %s, "
"PAYMENT_FILE_CREATED_FLAG='N' "
"WHERE PAYMENT_BATCH_NAME = :1",
[(i,) for i in check_to_process], log_msg, arraydmlrowcounts=True)
Many thanks for suggestions and insights!
Your code has an odd mix of string substitution (the %s) and bind variable placeholders (the :1). And odd code that creates bind variable placeholders that aren't used. Passing log_msg the way you do isn't going to work, since executemany() syntax doesn't support string substitution.
You probably want to use some kind of IN list, as shown in the cx_Oracle documentation Binding Multiple Values to a SQL WHERE IN Clause. Various solutions are shown there, depending on the number of values and frequency that the statement will be re-executed.
Use only bind variables. You should be able to use execute() instead of executemany(). Effectively you would do:
cursor.execute("""UPDATE apps.SLCAP_CITI_PAYMENT_BATCH
SET MESSAGE = :1
WHERE PAYMENT_BATCH_NAME IN (something goes here - see the doc)""",
bind_values)
The bottom line is: read the documentation and review examples like batch_errors.py. If you still have problems, refine your question, correct it, and add more detail.
In python3 and postgresql12, is there a safety difference between parametizing SQL queries the "proper" way or just escaping potentially dangerous content using psycopg2.quote_ident()?
For example, consider these two options.
Option 1:
name = get_unsafe_input_from_web_form()
cursor.execute("SELECT * FROM students WHERE name = %s;", (name,))
Option 2:
from psycopg2.extensions import quote_ident
name = get_unsafe_input_from_web_form()
cursor.execute(f"SELECT * FROM students WHERE name = {quote_ident(name, cursor)};"
The documentation is not particularly explicit. Is Option 2 totally equivalent in terms of safety against injection attacks?
quote_ident() would be the wrong thing to use, as it is for identifiers e.g table, column names. You would want quote_literal() which does not exist in psycopg2.extensions. I would stick with the first option, but using the psycopg2.sql module:
https://www.psycopg.org/docs/sql.html
Safety-wise, both parameterized queries and quote_ident can safely handle untrusted input and will not open you to SQL injection issues. But you can't use quote_ident for values as you're trying to do in your example. The string you're passing to cursor.execute() will end up being (for name foobar) SELECT * FROM students WHERE name = "foobar";, which will try to find rows where the name column is equal to the foobar column, not where name is equal to the string 'foobar'.
I need to create a dynamic query based on two string parameters:
description = "This is the description"
comment = "This is the comment"
query = "insert into case(desc, comm) value(description, comment)"
Note:
there might be single quote and double quotes in both description and comment.
How do I use formatted %s to generate the query string?
Thank you very much.
UPDATE:
Thanks to Green Cloak Guy (his/her answer has minors to be corrected), the right query is:
query = f"insert into case(description, comment) value(\'{description}\', \'{comment}\')"
Use an f-string.
query = f"insert into case({description}, {comment}) value({description}, {comment})"
Don't use any type of string formatting to do actual database queries - that leads to you having a SQL Injection problem. Use a database library instead, that properly sanitizes the data.
But if all you need to do is parse some variables into a string, this is more flexible than the % formatting that other languages tend to use (and that's technically still available in python via "some_string %s %s %s" % (str1, str2, str3)")
A possible solution for this question is here:
https://stackoverflow.com/a/6223961/12343395
It will probably work with a lot of work around.
But I have stored my table names in string format and want to call them as needed.
I am using Pandas read_sql_query. So as in params, I am passing, the table name and a few parameters in the WHERE section.
The WHERE section is fine, since the parameters are originally strings. But in the FROM section,
I really want the schema.table as a non-string.
Here is a snippet.
SELECT "rainfall(mm)","tmin(C)","tmax(C)","TimeStamp"
FROM crop_tables[choose_crop][0]
WHERE "District_Name" = %s AND "Season" = %s
ORDER BY "TimeStamp" ASC
where crop_tables[choose_crop][0] is 'sagita_historic.soyabean_daily_analyses' in this case.
But FROM will throw an error since it doesn't accept strings. So in essence, I wish to strip the 'sagita_historic.soyabean_daily_analyses' as a non-string.
Is it possible to do so?
Thank you.
Not sure I fully understand but maybe this will do?
SELECT "rainfall(mm)","tmin(C)","tmax(C)","TimeStamp"
FROM f"{crop_tables[choose_crop][0]}"
WHERE "District_Name" = %s AND "Season" = %s
ORDER BY "TimeStamp" ASC