I am trying to extract data from a SQL Server Database using pypyodbc.
But it appears my code breaks when I try to construct the SELECT Statement using
myCursor.execute(SQLCommand,values)
Can anyone spot an issue and point me in the right direction?
import pypyodbc
try:
myConnection = pypyodbc.connect('Driver={SQL Server};'
'Server=THINKPAD\STEVE_DEVELOPER;'
'Database=PythonTest;'
'uid=sa; pwd=passwordCC')
myCursor = myConnection.cursor()
print("Connection Made")
SQLCommand =("SELECT First_Name, Date FROM [PythonTest].[dbo].[Names] WHERE First_Name =?")
values = ['Mike']
print("SQL command elements Created")
#After this is where it falls over
myCursor.execute(SQLCommand,values)
print("SQL statement constructed ")
results = myCursor.fetchone()
print(results[0])
print("Sucessfully retreive record")
myconnection.close()
except:
print('Record NOT sucessfully retreived')
Cheers
Steve
In Python, exceptions are your friend. The traceback tells you where things went wrong, and the exception usually (hopefully) tells you what went wrong.
Suppressing all exceptions using except: is (almost) always a bad idea - if you catch an exception, you should know what you're expecting to catch and how to deal with it; if you don't, you usually want to let it go to the next handler out, which will either handle it or show the traceback (or both).
Related
This is a question about code structure, rather than syntax. I'd like to know what is best practice and why.
Imagine you've got a python programme. You've got a main class like so:
import my_db_manager as dbm
def main_loop():
people = dbm.read_db()
#TODO: Write rest of programme doing something with the data from people...
if __name__ == '__main__':
main_loop()
Then you've got several separate .py files for managing interactions with various tables in a database. One of these .py files, my_db_manager looks like this:
def read_db():
people = []
db = connection_manager.get_connection()
cursor = db.cursor()
try:
#Database reading statement
sql = 'SELECT DISTINCT(name) FROM people'
cursor.execute(sql)
results = cursor.fetchall()
people = [x[0] for x in results]
except Exception as e:
print(f'Error: {e}')
finally:
return people
In the example above the function read_db() is called from main_loop() in the main class. read_db() contains try/except clauses to manage errors in interacting with the database. While this works fine, The try/except clauses could instead be placed in main_loop() when calling read_db(). They could equally be located in both places. What is 'best practice' when using try/except? using try/except in the db_manager or using it in the main_loop() where you're managing the programmes logic flow, or using it in both places? Bear in mind I'm giving the above specific example but I'm trying to extrapolate a general rule for applying try/except when writing python.
The best way to write try-except -- in Python or anywhere else -- is as narrow as possible. It's a common problem to catch more exceptions than you meant to handle! (Credits: This Answer)
In your particular case, of course inside the function. It:
a) creates abstraction from db errors for the main thread
b) eliminates the suspicion of db errors in case some other thing caused exception inside of main thread (tho you can see but still it plucks out every last chance)
c) enables you to deal with all database related errors at one place. Efficiently and creatively. How are you gonna make list people outside of function in the main thread. It will double the mess.
Finally you should stick to this minimalism even inside the function although while catering every place where exception could occur as:
def read_db():
#Database reading statement
sql = 'SELECT DISTINCT(name) FROM people'
try:
db = connection_manager.get_connection()
cursor = db.cursor()
cursor.execute(sql)
results = cursor.fetchall()
except Exception as e:
print(f'Error: {e}')
return []
else:
return [x[0] for x in results]
I'm trying to understand if it's possible to set a loop inside of a Try/Except call, or if I'd need to restructure to use functions. Long story short, after spending a few hours learning Python and BeautifulSoup, I managed to frankenstein some code together to scrape a list of URLs, pull that data out to CSV (and now update it to a MySQL db). The code is now working as planned, except that I occasionally run into a 10054, either because my VPN hiccups, or possibly the source host server is occasionally bouncing me (I have a 30 second delay in my loop but it still kicks me on occasion).
I get the general idea of Try/Except structure, but I'm not quite sure how I would (or if I could) loop inside it to try again. My base code to grab the URL, clean it and parse the table I need looks like this:
for url in contents:
print('Processing record', (num+1), 'of', len(contents))
if url:
print('Retrieving data from ', url[0])
html = requests.get(url[0]).text
soup = BeautifulSoup(html, 'html.parser')
for span in soup('span'):
span.decompose()
trs = soup.select('div#collapseOne tr')
if trs:
print('Processing')
for t in trs:
for header, value in zip(t.select('td')[0], t.select('td:nth-child(2)')):
if num == 0:
headers.append(' '.join(header.split()))
values.append(re.sub(' +', ' ', value.get_text(' ', strip=True)))
After that is just processing the data to CSV and running an update sql statement.
What I'd like to do is if the HTML request call fails is wait 30 seconds, try the request again, then process, or if the retry fails X number of times, go ahead and exit the script (assuming at that point I have a full connection failure).
Is it possible to do something like that in line, or would I need to make the request statement into a function and set up a loop to call it? Have to admit I'm not familiar with how Python works with function returns yet.
You can add an inner loop for the retries and put your try/except block in that. Here is a sketch of what it would look like. You could put all of this into a function and put that function call in its own try/except block to catch other errors that cause the loop to exit.
Looking at requests exception hierarchy, Timeout covers multiple recoverable exceptions and is a good start for everything you may want to catch. Other things like SSLError aren't going to get better just because you retry, so skip them. You can go through the list to see what is reasonable for you.
import itertools
# requests exceptions at
# https://requests.readthedocs.io/en/master/_modules/requests/exceptions/
for url in contents:
print('Processing record', (num+1), 'of', len(contents))
if url:
print('Retrieving data from ', url[0])
retry_count = itertools.count()
# loop for retries
while True:
try:
# get with timeout and convert http errors to exceptions
resp = requests.get(url[0], timeout=10)
resp.raise_for_status()
# the things you want to recover from
except requests.Timeout as e:
if next(retry_count) <= 5:
print("timeout, wait and retry:", e)
time.sleep(30)
continue
else:
print("timeout, exiting")
raise # reraise exception to exit
except Exception as e:
print("unrecoverable error", e)
raise
break
html = resp.text
etc…
I've done a little example by myself to graphic this, and yes, you can put loops inside try/except blocks.
from sys import exit
def example_func():
try:
while True:
num = input("> ")
try:
int(num)
if num == "10":
print("Let's go!")
else:
print("Not 10")
except ValueError:
exit(0)
except:
exit(0)
example_func()
This is a fairly simple program that takes input and if it's 10, then it says "Let's go!", otherwise it tells you it's not 10 (if it's not a valid value, it just kicks you out).
Notice that inside the while loop I put a try/except block, taking into account the necessary indentations. You can take this program as a model and use it on your favor.
So I've queried data from oracle database using cursor.execute(). A relatively simple select query. It works.
But when I try to fetch data from it, python crashes.
The same occurs for fetchall(), fetchmany() and fetchone().
When the query first broke in fetchmany() I decided to loop through fetchone() and it worked for the first two rows then broke at the third.
I'm guessing it is because there's too much data in third row.
So, is there any way to bypass this issue and pull the data?
(Please ignore the wrong indents could not copy properly in my phone)
EDIT:
I removed four columns with type "ROWID". There was no issue after that. I was easily able to fetch 100 rows in one go.
So to confirm my suspicion I went ahead and created another copy with only those rowed columns and it crashes as expected.
So is there any issue with ROWID type?
Test table for the same.
Insert into TEST_FOR_CX_ORACLE (Z$OEX0_LINES,Z$OEX0_ORDER_INVOICES,Z$OEX0_ORDERS,Z$ITEM_ROWID) values ('ABoeqvAEyAAB0HOAAM','AAAL0DAEzAAClz7AAN','AAAVeuABHAAA4vdAAH','ABoeo+AIVAAE6dKAAQ');
Insert into TEST_FOR_CX_ORACLE (Z$OEX0_LINES,Z$OEX0_ORDER_INVOICES,Z$OEX0_ORDERS,Z$ITEM_ROWID) values ('ABoeqvABQAABKo6AAI','AAAL0DAEzAAClz7AAO','AAAVeuABHAAA4vdAAH','ABoeo+AIVAAE6dKAAQ');
Insert into TEST_FOR_CX_ORACLE (Z$OEX0_LINES,Z$OEX0_ORDER_INVOICES,Z$OEX0_ORDERS,Z$ITEM_ROWID) values ('ABoeqvABQAABKo6AAG','AAAL0DAEzAAClz7AAP','AAAVeuABHAAA4vdAAH','ABoeo+AHIAAN+OIAAM');
Insert into TEST_FOR_CX_ORACLE (Z$OEX0_LINES,Z$OEX0_ORDER_INVOICES,Z$OEX0_ORDERS,Z$ITEM_ROWID) values ('ABoeqvAEyAAB0HOAAK','AAAL0DAEzAACl0EAAC','AAAVeuABHAAA4vdAAH','ABoeo+AHIAAN+OIAAM');
Script:
from cx_Oracle import makedsn,connect,Cursor
from pandas import read_sql_table, DataFrame, Series
from time import time
def create_conn( host_link , port , service_name , user_name , password ):
dsn=makedsn(host_link,port,service_name=service_name)
return connect(user=user_name, password=password, dsn=dsn)
def initiate_connection(conn):
try:
dbconnection = create_conn(*conn)
print('Connected to '+conn[2]+' !')
except Exception as e:
print(e)
dbconnection = None
return dbconnection
def execute_query(query,conn):
dbconnection=initiate_connection(conn)
try:
cursor = dbconnection.cursor()
print ('Cursor Created!')
return cursor.execute(query)
except Exception as e:
print(e)
return None
start_time = time()
query='''SELECT * FROM test_for_cx_oracle'''
try:
cx_read_query = execute_query(query,ecspat_c)
time_after_execute_query = time()
print('Query Executed')
columns = [i[0] for i in cx_read_query.description]
time_after_getting_columns = time()
except Exception as e:
print(e)
print(time_after_execute_query-start_time,time_after_getting_columns-time_after_execute_query)
Unfortunately, this is a bug in the Oracle Client libraries. You will see it if you attempt to fetch the same rowid value multiple times in consecutive rows. If you avoid that situation all is well. You can also set the environment variable ORA_OCI_NO_OPTIMIZED_FETCH to the value 1 before you run the query to avoid the problem.
This has been reported earlier here: https://github.com/oracle/python-cx_Oracle/issues/120
I'm playing around with trying to make a bot in the slack channel so that I can understand how the whole process works. I have 2 questions. [1st question solved]First, I know how to send a message if someone said anything but I can't figure out how to check what was specifically said. Here's my code:
import os
import time
from slackclient import SlackClient
BOT_TOKEN = os.getenv('SLACK_BOT_USER_TOKEN')
print(BOT_TOKEN)
CH_NM = "bot_testing"
def main():
sc = SlackClient(BOT_TOKEN)
SM = sc.rtm_send_message
if sc.rtm_connect():
print("Bot running")
SM(CH_NM, "Started")
while True:
for slack_message in sc.rtm_read():
message = slack_message.get("text")
user = slack_message.get("user")
if 'hello' in message:
SM(CH_NM, "Hey, <#{}>!".format(user))
if not message or not user:
continue
SM(CH_NM, """<#{}> wrote something...""".format(user))
time.sleep(.5)
if __name__ == '__main__':
main()
The main line I'm having trouble with is
if 'hello' in message:
SM(CH_NM, "Hey, <#{}>!".format(user))
because I can't iterate through 'message' since it is 'NoneType'. How then would I go about checking if it contained a specific string?
Second question, I see that there are all sorts of event types, but mine only seems to work for the "message" type of events. If I wanted to return something every time a specific user started typing for example what would I add to make that work? I already tried adding typing = slack_message.get("user_typing") but understandably it doesn't work since 'user_typing' is an event type, not part of the message event type I'm pulling 'text' and 'user' out of.
So you know, I'm using python 3.6, Windows 10, powershell.
The issue was that I had my if not message or not user: continue line below the if 'hello' in message: line, so it would error out due to any other event that is happening that isn't a message. 1st Issue solved.
can someone help me solve this problem?
I'm using Blender 2.74 and Python 3.4 with the correct connector for MySQL. (By the way, I'm just a beginner in using Blender and Python.)
What I want is to make a login UI and save the inputted name into the database, but my code seems a bit off or wrong. When I try to run the code, it didn't save the value in the variable, but when i try to run it in python IDE (PyCharm) it worked.
Here's the code:
import sys
sys.path.append('C:\Python34\Lib\site-packages')
sys.path.append('C:\Python34\DLLs')
import mysql.connector
import bge
bge.render.showMouse(1)
cont = bge.logic.getCurrentController()
own = cont.owner
sensor = cont.sensors ["enter"]
pname = own.get("prpText")
enter = cont.sensors ["enter"]
numpadenter = cont.sensors ["numpadenter"]
if enter.positive or numpadenter.positive:
db = mysql.connector.connect(user='root', password='', host='localhost', database='dbname')
cursor = db.cursor()
cursor.execute("INSERT INTO tblname VALUE(%s", (pname))
#this are the other codes that i have tried so far:
#add_player = ("INSERT INTO storymode " "(PlayerName) " "VALUES (%s)")
#data_player = (pname)
#cursor.execute(add_player, data_player)
#cursor.execute("INSERT INTO storymode" "(PlayerName)" "VALUES (%(pname)s)")
db.commit()
db.close()
The Error is:
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your mysql server version for the right syntax to use near '%s' at line 1.
Can someone tell what i need to do here? Do I need some add-ons for it to work?
Thank you very much for reading my post and for the people who will give their opinions.
Looks like you're missing a closing parenthesis and an 'S' in you're sql INSERT statement?
INSERT INTO tblname VALUE (%s
needs to be
INSERT INTO tblname VALUES (%s)