I am extracting data from netCDF files with Python code. I need to check if the netCDF files are in agreement with the CORDEX standards (CORDEX is a coordinated effort to carry modelling experiments with regional climate models). For this I need to access an attribute of the netCDF file. If the attribute is not found, then the code should go to the next file.
A snipet of my code is as follows:
import netCDF4
cdf_dataset = netCDF4.Dataset(file_2read)
try:
cdf_domain = cdf_dataset.CORDEX_domain
print(cdf_domain)
except:
print('No CORDEX domain found. Will exit')
....some more code....
When the attribute "CORDEX_domain" is available everything is fine. If the attribute is not available then the following exception is raised.
AttributeError: NetCDF: Attribute not found
This is a third party exception, which I would say should be handled as a general one, but it is not, as I am not able to get my "print" inside the "except" statement to work or anything else for that matter. Can anyone point me the way to handle this? Thanks.
There is no need for a try/except block; netCDF4.Dataset has a method ncattrs which returns all global attributes, you can test if the required attribute is in there. For example:
if 'CORDEX_domain' in cdf_dataset.ncattrs():
do_something()
You can do the same to test if (for example) a required variable is present:
if 'some_var_name' in cdf_dataset.variables:
do_something_else()
p.s.: "catch alls" are usually a bad idea..., e.g. Python: about catching ANY exception
EDIT:
You can do the same for variable attributes, e.g.:
var = cdf_dataset.variables['some_var_name']
if 'some_attribute' in var.ncattrs():
do_something_completely_else()
Related
I am connected to an Excel application and can execute the "Debug"->"Compile VBAProject" from my Python code using win32com like so (inspired by code from here):
from win32com import client
def compile(self):
self.__excel = client.GetActiveObject("Excel.Application")
compile_button = self.__excel.VBE.CommandBars.FindControl(1, 578)
compile_button.Execute()
If there is a compilation error in the Excel VBA code I get a popup message in Excel telling me the error just fine.
Now I would like to check from the Python code if there was a compilation error and raise an exception if there was. I don't necessarily need the compilation error to be part of the exception but if that were possible I would of course gladly take that, too.
Can this be done somehow?
I've been experimenting with all kinds of window counts before and after the compilation etc. but so far have not found a property of any object that would indicate that there was a popup or a compilation error.
Ok, I found a somewhat ugly but doable way - that I would like to document for others having the same issue:
You need to import a code file into the opened Excel file that has (at least) one function defined. Then you can call this function from your Python code and catch any exception. If there was an exception your code - including the imported file - did not compile, if there is none the compilation was pass.
Here's my code:
compile_code.bas
Public Sub compileCode()
' doesn't need to do anything, it just needs to be available!
End Sub
Python file
from win32com import client
def compile(self) -> bool:
self.__excel = client.GetActiveObject("Excel.Application")
self.__book = self.__excel.ActiveWorkbook
self.__book.VBProject.VBComponents.Import(<Path_to_compile_code.bas>)
try:
self.__excel.Application.Run("compileCode")
# if you reach here the code compiled
return True
except Exception:
return False
I tried running the example code as given in the readme file for geograpy3. However, I am getting answers like this. What can be done about it?
Your question raises a similar issue as https://github.com/somnathrakshit/geograpy3/issues/3
There is now a get_geoPlace_context function that will limit the search to the GPE label of NLTK thus ignoring PERSON and ORGANIZATION entries as the orginal function get_place_context would do:
see also test_extractor.py
def testGetGeoPlace(self):
'''
test geo place handling
'''
url='http://www.bbc.com/news/world-europe-26919928'
places=geograpy.get_geoPlace_context(url=url)
if self.debug:
print(places)
self.assertEqual(['Moscow', 'Donetsk', 'Brussels', 'Kharkiv', 'Russia'],places.cities)
I have a template engine named Contemplate which has implementations for php, node/js and python.
All work fine except lately the python implementation gives me some issues. Specificaly the problem appears when first parsing a template and creating the template python code which is then dynamically imported as a module. When template is already created everything works fine but when template needs to be parsed and saved to disk and THEN imported it raises an error eg
ModuleNotFoundError: No module named 'blah blah'
(note this error appears to be random, it is not always sure that it will be raised, many times it works even if template is created just before importing, other times it fails and then if ran again with template already created it succeeds)
Is there any way I can bypass this issue, maybe add a delay between saving a parsed template and then importing as module or somethig else?
The code to import the module (the parsed template which is now a python class) is below:
def import_tpl( filename, classname, cacheDir, doReload=False ):
# http://www.php2python.com/wiki/function.import_tpl/
# http://docs.python.org/dev/3.0/whatsnew/3.0.html
# http://stackoverflow.com/questions/4821104/python-dynamic-instantiation-from-string-name-of-a-class-in-dynamically-imported
#_locals_ = {'Contemplate': Contemplate}
#_globals_ = {'Contemplate': Contemplate}
#if 'execfile' in globals():
# # Python 2.x
# execfile(filename, _globals_, _locals_)
# return _locals_[classname]
#else:
# # Python 3.x
# exec(read_file(filename), _globals_, _locals_)
# return _locals_[classname]
# http://docs.python.org/2/library/imp.html
# http://docs.python.org/2/library/functions.html#__import__
# http://docs.python.org/3/library/functions.html#__import__
# http://stackoverflow.com/questions/301134/dynamic-module-import-in-python
# http://stackoverflow.com/questions/11108628/python-dynamic-from-import
# also: http://code.activestate.com/recipes/473888-lazy-module-imports/
# using import instead of execfile, usually takes advantage of Python cached compiled code
global _G
getTplClass = None
# add the dynamic import path to sys
basename = os.path.basename(filename)
directory = os.path.dirname(filename)
os.sys.path.append(cacheDir)
os.sys.path.append(directory)
currentcwd = os.getcwd()
os.chdir(directory) # change working directory so we know import will work
if os.path.exists(filename):
modname = basename[:-3] # remove .py extension
mod = __import__(modname)
if doReload: reload(mod) # Might be out of date
# a trick in-order to pass the Contemplate super-class in a cross-module way
getTplClass = getattr( mod, '__getTplClass__' )
# restore current dir
os.chdir(currentcwd)
# remove the dynamic import path from sys
del os.sys.path[-1]
del os.sys.path[-1]
# return the tplClass if found
if getTplClass: return getTplClass(Contemplate)
return None
Note the engine creates a __init__.py file in cacheDir if it is not there already.
If needed I can change the import_tpl function to sth else I dont mind.
Python tested is python 3.6 on windows but I dont think this is a platform-specific issue.
To test the issue you can download the github repository (linked above) and run the /tests/test.py test after clearing all cached templates from /tests/_tplcache/ folder
UPDATE:
I am thinking of adding a while loop with some counter in import_tpl that catches the error raised if any and retries a specified amount of times until it succeeds to import the module. But I am also wondering if this is a good solution or there is something else I am missing here..
UPDATE (20/02/2019):
Added a loop to retry a specified amount of times plus a small delay of 1 sec if initially failed to import template module (see online repository code), but still it raises same error sometimes when templates are first created before being imported. Any solutions?
Right, if you use a "while" loop with to handle exceptions would be one way.
while True:
try:
#The module importing
break
except ModuleNotFoundError:
print("NOPE! Module not found")
If it works for some other, an not other "module" files, the likely suspect is the template files the template files themselves.
How can I know if pickle.dump() successfully saved the pickle file?
In the docs, I do not see a return value that indicates success or failure.
I'm working with python 3, and currently in a jupyter notebook
If pickle.dump or pickle.dumps fails an error will be thrown. See the docs further down for what can and can't be pickled. You may also get an OSError (link) if some lower level system call fails
Note however that even if pickle.dump does not throw an error, you may still not be able to load the pickled data. It may for instance be the case that an object you pickled uses an import or references a function that was defined in the context of the pickling code, say a Jupyter notebook defines a custom function which is referenced by the pickled object. If you now ship that pickled object file to another machine it won't see the function that is referenced in the object and the unpickling will fail.
Similarly if there's an API change in a module that the picled object depends on, the import paths may have changed and the unpickling will again fail.
You may also want to have a look at dill which covers slightly more cases than pickle https://github.com/uqfoundation/dill
I have a model file in my pyramid app, and inside of that model file, I am doing automatic validation before an insert using formencode. A failed validation inside of my model file raises a formencode.Invalid exception.
I found the following documentation on how to set up a custom exception view, but I am unclear on a couple of things:
Where do I put the actual exception view code? This is clearly view code, so it should be in a view somewhere. Should it be in its own view file? I've pasted the code I need to place at the bottom.
How do I make the rest of my pyramid app aware of this code? The only obvious way that I see is to import the view file inside of my model files, but that gives me a bad taste in my mouth. I'm sure there must be another way to do it, but I'm not sure what that is.
Code to place:
from pyramid.view import view_config
from helloworld.exceptions import ValidationFailure
#view_config(context=ValidationFailure)
def failed_validation(exc, request):
response = Response('Failed validation: %s' % exc.msg)
response.status_int = 500
return response
1) Anywhere in your project directory. I made a new file called exceptions.py where I place all my HTTP status code and validation exceptions. I placed this file in the same directory as my views.py, models.py, etc.
2) That bad taste in your mouth is Python, because importing methods is the Pythonic way to go about using classes and functions in other files, rather than some sort of magic. Might be weird at first, but you'll quickly get used to it. Promise.
I want to note that in your models.py file, you're only going to be importing ValidationFailure from helloworld.exception and raising ValidationFailure wherever you want. You aren't importing the whole view function you've defined (failed_validation). That's why the context for that view function is ValidationFailure, so it knows to go there when you simply raise ValidationFailure