I want to wrap my database-operations into sessions (transactions). For a good user experience I want to try to handle as many possible errors inside my code. My basic structure of a database transaction is in the code block.
There are two kinds of error:
My code can try to handle them, for ex. a user try to insert a key that already exists.
Some other exception raise, that was not handled by the code.
In both situations, I want to show the user a message.
How can I give out the error, so that I can display it after the session was closed?
# Transaction
session = Session()
try:
db_operation1(session, ...)
db_operation2(session, ...)
...
session.commit()
except:
session.rollback()
finally:
session.close()
# Show error
# ???
Edit:
After some investigation, I change my code to this:
# Transaction
session = Session()
try:
db_operation1(session, ...)
db_operation2(session, ...)
...
session.commit()
except Exception as e:
session.rollback()
# Show Error
print(str(e))
finally:
session.close()
But in the case, my code can handle the exception, I need some more user friendly message.
Edit2:
As snakecharmerb points me to, it is better to handle the exception outside the session-scope, I change the code again:
try:
# Transaction
session = Session()
try:
db_operation1(session, ...)
db_operation2(session, ...)
...
session.commit()
except:
session.rollback()
raise
finally:
session.close()
except Exception as e:
# Show Error
print(str(e))
Related
I'd like to catch any type of error in Python3.
I'm trying something like that:
try:
fields = line.split(' ')
...
<PostgreSQL query execution>
except psycopg2.Error:
conn.rollback()
QUERY = "UPDATE table SET error='sql'"
cur = conn.cursor()
cur.execute(QUERY)
conn.commit()
cur.close()
continue
except:
conn.rollback()
e="generic"
QUERY = "UPDATE table SET error='generic'"
cur = conn.cursor()
cur.execute(QUERY)
conn.commit()
cur.close()
continue
But I noted that, for example, an IndexError is not caught and the script fails.
What's wrong?
Thanks!
This happens when a new exception occurs in the except block.
For example:
try:
print('foo')
raise ValueError
except:
print('noes!')
print(1/0)
Will exit with divide by zero exception.
In order to see if it is so, we need more of actual code from you, particularly both of the except blocks.
If you want to make sure you catch "any" exception, make sure your except blocks are unexceptional.
How can I keep sensitive data out of logs?
Currently I have an exception which is raised in a method A. In method B this exception is re-raised adding further information. In method C I want to log the exception with a further information.
My first attempt was to add a string replace method before logging the exception, but this does not affect the whole traceback. Especially because I call the Python library request in methodA. The first exception takes place in this library.
first excepton in requests: urllib3.exceptions.MaxRetryError
second exception in requests: requests.exceptions.ProxyError
Both exceptions within the request library contain already the sensitive data in the traceback.
def methodA():
try:
connect_to_http_with_request_lib()
except requests.exceptions.ProxyError as err:
raise MyExceptionA(f"this log contains sensitive info in err: {err}")
def methodB():
try:
methodA()
except MyExceptionA as err:
raise MyExceptionB (f"add some more info to: {err}")
def methodC():
try:
methodB()
return True
except MyExceptionB as err:
err = re.sub(r"(?is)password=.+", "password=xxxx", str(err))
logger.exception(f("methodB failed exception {err}")
return False
How can I parse the whole traceback before logging the exception in order to mask out sensitive data?
I use loguru as logging library.
The Django framework seems to address the same problem with their own methods. See here
I have to define an attribute in a class and I would like to manage error in the most pythonic way.
Here is the code I have tried so far. I can't figure out why I can not "reach" the exception in the following code.
# global variable to be used in the example
my_dict = {"key1": {"property": 10}, "key2": {}}
class Test(object):
#property
def my_attribute(self):
try:
return self._my_attribute
except AttributeError:
self._my_attribute = {}
for key, value in my_dict.items():
print(key)
self._my_attribute[key] = value['property']
except Exception:
print('error')
# I would like to manage my error here with a log or something
print("I am not reaching here")
finally:
return self._my_attribute
if __name__ == '__main__':
Test().my_attribute
I expected to reach the Exception case in the second iteration of the for loop since it is a KeyError ("key2" has no "property"). But it just passes by it. In this example, if the script is run, it does not print "I am not reaching here". Could anyone explain why I am seeing this wrong? Thanks!
The potential KeyError in self._my_attribute[key] = value['property'] is not covered by the except Exception block. Once it is raised the finally block is executed (as a matter of fact the finally block is always executed, regardless of an exception being raised or even handled). This can be easily verified by using a step-by-step debugger or with a simple print('finally') inside the finally block.
This is (among other reasons) why try blocks should be as minimal as possible. If you know that line might raise a KeyError then explicitly try-except it:
for key, value in my_dict.items():
print(key)
try:
self._my_attribute[key] = value['property']
except KeyError as e:
print('Key ', e, 'does not exist')
I want to loop through a certain line of code until any exception or keyboard interruption occurs. But I can not reach the exception block whenever any exception occurs or due to keyboard interruption.
How can I modify my code so that I could actually reach in case of exception being thrown?
def run():
lidar = RPLidar(PORT_NAME)
iterator = lidar.iter_scans(50000)
time.sleep(2)
environment(iterator)
while True:
try:
print('Hi')
update_line(iterator)
except Exception or KeyboardInterrupt:
print("exception occur. Run again")
#lidar = RPLidar(PORT_NAME)
lidar.stop_motor()
lidar.stop()
lidar.disconnect()
break
if __name__ == '__main__':
run()
I'm surprised that code actually runs. When you say except Exception or KeyboardInterrupt you are saying only take the first thing here that evaluates to True. Since bool(Exception) is True you are only going to catch Exceptions. To catch multiple types of exceptions you would write it like this:
try:
except (Exception, KeyboardInterrupt):
It might not be triggering or non-keyboard exceptions because the exception you are trying to catch derives from BaseException and not Exception. To fix that change Exception to BaseException.
def run():
while True:
try:
print('Hi')
function_doesnt_exist(iterator)
except Exception or KeyboardInterrupt:
print("exception occur. Run again")
break
if __name__ == '__main__':
run()
When I intentionally call function that doesn't exist in your while loop it calls exception : exception occur. Run again
Perhaps you didn't generate an error properly therefore no exception is called
Also , Exception or KeyboardInterrupt means Exception since Exception includes KeyboardInterrupt and u handle them in same manner
so If you just want to catch Keyboard Interrupt then go for:
except KeyboardInterrupt:
pass
Or if you want to handle general exception and Keyboard one different do something like this:
except KeyboardInterrupt:
print("don't press ctrl+C")
pass
except:
print("exception occured")
pass
Hello I have the following for my async loop
async def start_process(restore_items, args, loop):
with GlacierRestorer(args.temp_dir, args.error_log_bucket, loop) as restorer:
restorer.initiate_restore_all(restore_items)
tasks = []
semaphore = asyncio.BoundedSemaphore(4)
for item in restore_items:
tasks.append(asyncio.ensure_future(restorer.transfer(item, semaphore)))
await asyncio.gather(*tasks)
def main():
args = get_args()
restore_items = get_restore_items(args)
for item in restore_items:
print(item.source, ':', item.destination)
try:
loop = asyncio.get_event_loop()
loop.run_until_complete(start_process(restore_items, args, loop))
except KeyboardInterrupt:
pass
My job and files get larger I see that I keep getting an
socket.send() exception
After reading the documentation it seems to be coming from loop.run_until_complete
The exception doesn't come the program to crash, but eventually bogs it down so much it gets stuck printing the exception.
How do I modify the current code to fix this?
run_until_complete only propagates the exception raised inside start_process. This means that if an exception happens at any point during start_process, and start_process doesn't catch it, run_until_complete(start_process()) will re-raise the same exception.
In your case the exception likely originally gets raised somewhere in restorer.transfer(). The call to gather returns the results of the coroutines, which includes raising an exception, if one occurred.
The exception doesn't come the program to crash, but eventually bogs it down so much it gets stuck printing the exception. How do I modify the current code to fix this?
Ideally you would fix the cause of the exception - perhaps you are sending in too many requests at once, or you are using the GlacierRestorer API incorrectly. But some exceptions cannot be avoided, e.g. ones caused by a failing network. To ignore such exceptions, you can wrap the call to restorer.transfer in a separate coroutine:
async def safe_transfer(restorer, item, semaphore):
try:
return await restorer.transfer(item, semaphore)
except socket.error as e:
print(e) # here you can choose not to print exceptions you
# don't care about if doing so bogs down the program
In start_process you would call this coroutine instead of restorer_transfer:
coros = []
for item in restore_items:
coros.append(safe_transfer(restorer, item, semaphore))
await asyncio.gather(*coros)
Note that you don't need to call asyncio.ensure_future() to pass a coroutine to asyncio.gather; it will be called automatically.