Related
I have read shapefile in a zip format from my S3 bucket successfully through geopandas, but I get error when trying to output the same geodataframe as a shapefile to the same S3 bucket.
The code below is how I read the zip file, and it works nicely:
## session for connecting to S3
session = boto3.session.Session(aws_access_key_id='MY-KEY-ID',
aws_secret_access_key='MY-KEY')
s3 = session.resource('s3')
bucket = s3.Bucket('my_bucket')
## read shapefile
TPG = bucket.Object(key='/shapefiles/grid.zip')
TPGrid = geopandas.read_file(TPG.get()['Body'])
But when I tried to output the same geodataframe like this:
TPGrid.to_file(filename='s3://my_bucket/output/TPGrid.zip', driver='ESRI Shapefile')
I will get error code:
ERROR:fiona._env:Only read-only mode is supported for /vsicurl
ERROR:fiona._env:Only read-only mode is supported for /vsicurl
ERROR:fiona._env:Only read-only mode is supported for /vsicurl
ERROR:fiona._env:Unable to open /vsis3/my_bucket/output/TPGrid.zip/TPGrid.shp or /vsis3/my_bucket/output/TPGrid.zip/TPGrid.SHP.
Traceback (most recent call last):
File "fiona/ogrext.pyx", line 1133, in fiona.ogrext.WritingSession.start
File "fiona/_err.pyx", line 291, in fiona._err.exc_wrap_pointer
fiona._err.CPLE_AppDefinedError: Unable to open /vsis3/my_bucket/output/TPGrid.zip/TPGrid.shp or /vsis3/my_bucket/output/TPGrid.zip/TPGrid.SHP.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/runscript.py", line 211, in <module>
runpy.run_path(temp_file_path, run_name='__main__')
File "/usr/local/lib/python3.6/runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "/usr/local/lib/python3.6/runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/tmp/glue-python-scripts-c8krhm5u/test_to_file_geo.py", line 40, in <module>
File "/glue/lib/installation/geopandas/geodataframe.py", line 1086, in to_file
_to_file(self, filename, driver, schema, index, **kwargs)
File "/glue/lib/installation/geopandas/io/file.py", line 328, in _to_file
filename, mode=mode, driver=driver, crs_wkt=crs_wkt, schema=schema, **kwargs
File "/glue/lib/installation/fiona/env.py", line 408, in wrapper
return f(*args, **kwargs)
File "/glue/lib/installation/fiona/__init__.py", line 274, in open
**kwargs)
File "/glue/lib/installation/fiona/collection.py", line 165, in __init__
self.session.start(self, **kwargs)
File "fiona/ogrext.pyx", line 1141, in fiona.ogrext.WritingSession.start
fiona.errors.DriverIOError: Unable to open /vsis3/my_bucket/output/TPGrid.zip/TPGrid.shp or /vsis3/my_bucket/output/TPGrid.zip/TPGrid.SHP.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/runscript.py", line 230, in <module>
raise e_type(e_value).with_traceback(new_stack)
File "/tmp/glue-python-scripts-c8krhm5u/test_to_file_geo.py", line 40, in <module>
File "/glue/lib/installation/geopandas/geodataframe.py", line 1086, in to_file
_to_file(self, filename, driver, schema, index, **kwargs)
File "/glue/lib/installation/geopandas/io/file.py", line 328, in _to_file
filename, mode=mode, driver=driver, crs_wkt=crs_wkt, schema=schema, **kwargs
File "/glue/lib/installation/fiona/env.py", line 408, in wrapper
return f(*args, **kwargs)
File "/glue/lib/installation/fiona/__init__.py", line 274, in open
**kwargs)
File "/glue/lib/installation/fiona/collection.py", line 165, in __init__
self.session.start(self, **kwargs)
File "fiona/ogrext.pyx", line 1141, in fiona.ogrext.WritingSession.start
fiona.errors.DriverIOError: Unable to open /vsis3/my_bucket/output/TPGrid.zip/TPGrid.shp or /vsis3/my_bucket/output/TPGrid.zip/TPGrid.SHP.
I have tried several ways, such as using '.csv' or '.shp', but not any one worked.
I am using python 3.6 and packages below, hope these information will help:
geopandas-0.9.0
shapely-1.7.1
fiona-1.8.20
GDAL-3.2.3
I kept fighting with this problem all day....
Any help will be highly appreciated.
I have to start using odoo 12 on my job but I can't start using it, I have already 5 days searching on google to find an answer.
I would be very grateful if you could please help me, with some of your wisdom :)
After I install odoo on my computer and visit localhost:8069 for first time it asks me to create my database but after I do this it doesn't load the login page instead it gives me an 500 internal server error with this console log every time I refresh the page:
2020-10-12 18:31:41,068 21425 ERROR prueba werkzeug: Error on request:
Traceback (most recent call last):
File "/opt/odoo/odoo12/entvirt/lib/python3.8/site-packages/werkzeug/serving.py", line 205, in run_wsgi
execute(self.server.app)
File "/opt/odoo/odoo12/entvirt/lib/python3.8/site-packages/werkzeug/serving.py", line 193, in execute
application_iter = app(environ, start_response)
File "/opt/odoo/odoo12/entvirt/src/odoo/service/server.py", line 434, in app
return self.app(e, s)
File "/opt/odoo/odoo12/entvirt/src/odoo/service/wsgi_server.py", line 142, in application
return application_unproxied(environ, start_response)
File "/opt/odoo/odoo12/entvirt/src/odoo/service/wsgi_server.py", line 117, in application_unproxied
result = odoo.http.root(environ, start_response)
File "/opt/odoo/odoo12/entvirt/src/odoo/http.py", line 1320, in __call__
return self.dispatch(environ, start_response)
File "/opt/odoo/odoo12/entvirt/src/odoo/http.py", line 1293, in __call__
return self.app(environ, start_wrapped)
File "/opt/odoo/odoo12/entvirt/lib/python3.8/site-packages/werkzeug/wsgi.py", line 599, in __call__
return self.app(environ, start_response)
File "/opt/odoo/odoo12/entvirt/src/odoo/http.py", line 1488, in dispatch
result = ir_http._dispatch()
File "/opt/odoo/odoo12/entvirt/src/addons/web_editor/models/ir_http.py", line 22, in _dispatch
return super(IrHttp, cls)._dispatch()
File "/opt/odoo/odoo12/entvirt/src/odoo/addons/base/models/ir_http.py", line 212, in _dispatch
return cls._handle_exception(e)
File "/opt/odoo/odoo12/entvirt/src/odoo/addons/base/models/ir_http.py", line 182, in _handle_exception
return request._handle_exception(exception)
File "/opt/odoo/odoo12/entvirt/src/odoo/http.py", line 776, in _handle_exception
return super(HttpRequest, self)._handle_exception(exception)
File "/opt/odoo/odoo12/entvirt/src/odoo/http.py", line 314, in _handle_exception
raise pycompat.reraise(type(exception), exception, sys.exc_info()[2])
File "/opt/odoo/odoo12/entvirt/src/odoo/tools/pycompat.py", line 87, in reraise
raise value
File "/opt/odoo/odoo12/entvirt/src/odoo/addons/base/models/ir_http.py", line 208, in _dispatch
result = request.dispatch()
File "/opt/odoo/odoo12/entvirt/src/odoo/http.py", line 835, in dispatch
r = self._call_function(**self.params)
File "/opt/odoo/odoo12/entvirt/src/odoo/http.py", line 346, in _call_function
return checked_call(self.db, *args, **kwargs)
File "/opt/odoo/odoo12/entvirt/src/odoo/service/model.py", line 98, in wrapper
return f(dbname, *args, **kwargs)
File "/opt/odoo/odoo12/entvirt/src/odoo/http.py", line 342, in checked_call
result.flatten()
File "/opt/odoo/odoo12/entvirt/src/odoo/http.py", line 1270, in flatten
self.response.append(self.render())
File "/opt/odoo/odoo12/entvirt/src/odoo/http.py", line 1263, in render
return env["ir.ui.view"].render_template(self.template, self.qcontext)
File "/opt/odoo/odoo12/entvirt/src/odoo/addons/base/models/ir_ui_view.py", line 1324, in render_template
return self.browse(self.get_view_id(template)).render(values, engine)
File "/opt/odoo/odoo12/entvirt/src/addons/web_editor/models/ir_ui_view.py", line 29, in render
return super(IrUiView, self).render(values=values, engine=engine, minimal_qcontext=minimal_qcontext)
File "/opt/odoo/odoo12/entvirt/src/odoo/addons/base/models/ir_ui_view.py", line 1333, in render
return self.env[engine].render(self.id, qcontext)
File "/opt/odoo/odoo12/entvirt/src/odoo/addons/base/models/ir_qweb.py", line 59, in render
result = super(IrQWeb, self).render(id_or_xml_id, values=values, **context)
File "/opt/odoo/odoo12/entvirt/src/odoo/addons/base/models/qweb.py", line 275, in render
self.compile(template, options)(self, body.append, values or {})
File "<decorator-gen-54>", line 2, in compile
File "/opt/odoo/odoo12/entvirt/src/odoo/tools/cache.py", line 93, in lookup
value = d[key] = self.method(*args, **kwargs)
File "/opt/odoo/odoo12/entvirt/src/odoo/addons/base/models/ir_qweb.py", line 114, in compile
return super(IrQWeb, self).compile(id_or_xml_id, options=options)
File "/opt/odoo/odoo12/entvirt/src/odoo/addons/base/models/qweb.py", line 338, in compile
raise QWebException("Error when compiling AST", e, path, node and etree.tostring(node[0], encoding='unicode'), name)
odoo.addons.base.models.qweb.QWebException: Name node can't be used with 'None' constant
Traceback (most recent call last):
File "/opt/odoo/odoo12/entvirt/src/odoo/tools/cache.py", line 88, in lookup
r = d[key]
File "/opt/odoo/odoo12/entvirt/src/odoo/tools/func.py", line 69, in wrapper
return func(self, *args, **kwargs)
File "/opt/odoo/odoo12/entvirt/src/odoo/tools/lru.py", line 44, in __getitem__
a = self.d[obj].me
KeyError: ('ir.qweb', <function IrQWeb.compile at 0x7f3b32b84280>, 173, ('en_US', None, None, None, None, None))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/odoo/odoo12/entvirt/src/odoo/addons/base/models/qweb.py", line 330, in compile
unsafe_eval(compile(astmod, '<template>', 'exec'), ns)
ValueError: Name node can't be used with 'None' constant
Error when compiling AST
ValueError: Name node can't be used with 'None' constant
Template: 173
Path: /templates/t/t/form/input[2]
Node: <input type="hidden" name="redirect" t-att-value="redirect"/> - - -
This is a problem with python 3.8.5.Try applying this fix https://github.com/odoo/odoo/pull/55305/commits/5baf0f2130b8d27d50aa60b54d68a5fc57b127a0
I'm getting the following error. Which is about permissions. The error occurs when I call a custom method through an action. The custom method will create an .xml using ElementTree library.
The code inside the method is about 1500+ lines so putting it in here wouldn't be optimal. But I believe the method is rejected when trying the following
tree.write("20100066603-01-F001-1.xml", "ISO-8859-1", True)
This is the error. Odoo returns
Error:
Odoo Server Error
Traceback (most recent call last):
File "/usr/lib/python3.6/xml/etree/ElementTree.py", line 789, in _get_writer
write = file_or_filename.write
AttributeError: 'str' object has no attribute 'write'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/odoo/tools/safe_eval.py", line 350, in safe_eval
return unsafe_eval(c, globals_dict, locals_dict)
File "", line 2, in <module>
File "/usr/lib/python3/dist-packages/odoo/addons/myModule/models/models.py", line 1141, in x_nc_create_xml
tree.write("20100066603-01-F001-1.xml", "ISO-8859-1", True)
File "/usr/lib/python3.6/xml/etree/ElementTree.py", line 759, in write
with _get_writer(file_or_filename, enc_lower) as write:
File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
return next(self.gen)
File "/usr/lib/python3.6/xml/etree/ElementTree.py", line 796, in _get_writer
errors="xmlcharrefreplace")
PermissionError: [Errno 13] Permission denied: '20100066603-01-F001-1.xml'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/odoo/http.py", line 650, in _handle_exception
return super(JsonRequest, self)._handle_exception(exception)
File "/usr/lib/python3/dist-packages/odoo/http.py", line 310, in _handle_exception
raise pycompat.reraise(type(exception), exception, sys.exc_info()[2])
File "/usr/lib/python3/dist-packages/odoo/tools/pycompat.py", line 87, in reraise
raise value
File "/usr/lib/python3/dist-packages/odoo/http.py", line 692, in dispatch
result = self._call_function(**self.params)
File "/usr/lib/python3/dist-packages/odoo/http.py", line 342, in _call_function
return checked_call(self.db, *args, **kwargs)
File "/usr/lib/python3/dist-packages/odoo/service/model.py", line 97, in wrapper
return f(dbname, *args, **kwargs)
File "/usr/lib/python3/dist-packages/odoo/http.py", line 335, in checked_call
result = self.endpoint(*a, **kw)
File "/usr/lib/python3/dist-packages/odoo/http.py", line 936, in __call__
return self.method(*args, **kw)
File "/usr/lib/python3/dist-packages/odoo/http.py", line 515, in response_wrap
response = f(*args, **kw)
File "/usr/lib/python3/dist-packages/odoo/addons/web/controllers/main.py", line 1231, in run
result = request.env['ir.actions.server'].browse([action_id]).run()
File "/usr/lib/python3/dist-packages/odoo/addons/base/ir/ir_actions.py", line 554, in run
res = func(action, eval_context=eval_context)
File "/usr/lib/python3/dist-packages/odoo/addons/base/ir/ir_actions.py", line 430, in run_action_code_multi
safe_eval(action.sudo().code.strip(), eval_context, mode="exec", nocopy=True) # nocopy allows to return 'action'
File "/usr/lib/python3/dist-packages/odoo/tools/safe_eval.py", line 373, in safe_eval
pycompat.reraise(ValueError, ValueError('%s: "%s" while evaluating\n%r' % (ustr(type(e)), ustr(e), expr)), exc_info[2])
File "/usr/lib/python3/dist-packages/odoo/tools/pycompat.py", line 86, in reraise
raise value.with_traceback(tb)
File "/usr/lib/python3/dist-packages/odoo/tools/safe_eval.py", line 350, in safe_eval
return unsafe_eval(c, globals_dict, locals_dict)
File "", line 2, in <module>
File "/usr/lib/python3/dist-packages/odoo/addons/myModule/models/models.py", line 1141, in x_nc_create_xml
tree.write("20100066603-01-F001-1.xml", "ISO-8859-1", True)
File "/usr/lib/python3.6/xml/etree/ElementTree.py", line 759, in write
with _get_writer(file_or_filename, enc_lower) as write:
File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
return next(self.gen)
File "/usr/lib/python3.6/xml/etree/ElementTree.py", line 796, in _get_writer
errors="xmlcharrefreplace")
ValueError: <class 'PermissionError'>: "13
Permission denied" while evaluating
'if records: \n action = records.x_nc_create_xml()'
I have already tried the same code on PyCharm and it worked. So I believe, this is an issue about permissions on Odoo.
How do I set permission for my module to create,edit and delete files it creates?
PD:If this is unlikely to achieve on Odoo.sh, then how can I do it on a Local installation? I currently have both.
use command chmod 777 * this will allow all kind of access to all of your custom modules and if you want to give permission to only one module use name of module instead of * like this chmod 777 module_name
Trying to make it work .Df is dataframe and i need to add new column with UDF :
df = spark.sql("select 'foo' as tweet, 'bar' as medicine_name")
df.createOrReplaceTempView("medicine")
def similarity(x):
return 'df_' + x
similarity_udf = udf(lambda z: similarity(z), returnType=StringType())
def main():
distinct_df = spark.sql("select distinct medicine_name as medicine_name from medicine where medicine_name is not null")
distinct_df.createOrReplaceTempView("distinctDF")
df.show()
result_df = df.withColumn('m_name', similarity_udf('tweet'))
result_df.show()
but getting error
Traceback (most recent call last):
File "PycharmProjects/TestCase/main.py", line 85, in <module>
main()
File "PycharmProjects/TestCase/main.py", line 50, in main
result_df.show()
File "PycharmProjects\TestCase\venv\lib\site-packages\pyspark\sql\dataframe.py", line 350, in show
print(self._jdf.showString(n, 20, vertical))
File "PycharmProjects\TestCase\venv\lib\site-packages\py4j\java_gateway.py", line 933, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "PycharmProjects\TestCase\venv\lib\site-packages\pyspark\sql\utils.py", line 63, in deco
return f(*a, **kw)
File "PycharmProjects\TestCase\venv\lib\site-packages\py4j\protocol.py", line 312, in get_return_value
format(target_id, ".", name), value)
py4j.protocol.Py4JJavaError: An error occurred while calling o77.showString.
: java.lang.IllegalArgumentException
Can anybody provide what is the problem?
I am trying to parallelize the tasks of correcting texts on many documents with Python, so I naturally found "joblib". I want each task to be to correct a given document. Here is the structure of the code:
if __name__ == '__main__':
lexicon = build_compact_lexicon()
from joblib import Parallel, delayed
import multiprocessing
num_cores = multiprocessing.cpu_count()
results = Parallel(n_jobs=num_cores)(delayed(find_errors)('GDL', i, 1, lexicon) for i in range(1798, 1820))
I am using the function find_errors summed up here :
def find_errors(newspaper, year, month, lexicon):
# parse the input newspaper text data using etree parser from LXML
# detect errors in the text
return found_errors_type1, found_errors_type2, found_errors_type3
This does raise me a few errors
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/site-packages/joblib/parallel.py", line 130, in __call__
return self.func(*args, **kwargs)
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/site-packages/joblib/parallel.py", line 72, in __call__
return [func(*args, **kwargs) for func, args, kwargs in self.items]
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/site-packages/joblib/parallel.py", line 72, in <listcomp>
return [func(*args, **kwargs) for func, args, kwargs in self.items]
File "hellowordParallel.py", line 85, in find_errors
tree = etree.parse(xml_file_path)
File "src/lxml/lxml.etree.pyx", line 3427, in lxml.etree.parse (src/lxml/lxml.etree.c:79801)
File "src/lxml/parser.pxi", line 1805, in lxml.etree._parseDocument (src/lxml/lxml.etree.c:116293)
TypeError: cannot parse from 'NoneType'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/tokenize.py", line 392, in find_cookie
line_string = line.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 24: invalid start byte
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/site-packages/joblib/parallel.py", line 139, in __call__
tb_offset=1)
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/site-packages/joblib/format_stack.py", line 373, in format_exc
frames = format_records(records)
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/site-packages/joblib/format_stack.py", line 274, in format_records
for token in generate_tokens(linereader):
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/tokenize.py", line 514, in _tokenize
line = readline()
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/site-packages/joblib/format_stack.py", line 265, in linereader
line = getline(file, lnum[0])
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/linecache.py", line 16, in getline
lines = getlines(filename, module_globals)
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/linecache.py", line 47, in getlines
return updatecache(filename, module_globals)
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/linecache.py", line 136, in updatecache
with tokenize.open(fullname) as fp:
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/tokenize.py", line 456, in open
encoding, lines = detect_encoding(buffer.readline)
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/tokenize.py", line 433, in detect_encoding
encoding = find_cookie(first)
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/tokenize.py", line 397, in find_cookie
raise SyntaxError(msg)
File "<string>", line None
SyntaxError: invalid or missing encoding declaration for '/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/site-packages/lxml/etree.so'
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "hellowordParallel.py", line 160, in <module>
results = Parallel(n_jobs=num_cores)(delayed(find_errors)('GDL', i, 1, lexicon) for i in range(1798, 1820))
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/site-packages/joblib/parallel.py", line 810, in __call__
self.retrieve()
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/site-packages/joblib/parallel.py", line 727, in retrieve
self._output.extend(job.get())
File "/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/multiprocessing/pool.py", line 608, in get
raise self._value
SyntaxError: invalid or missing encoding declaration for '/home/mbl/anaconda3/envs/OCR_Correction/lib/python3.5/site-packages/lxml/etree.so'
I don't understand if this is due do something related with configs or if my function doesn't fit in a parallel implementation... (I guess it should...)
Did it happen to some of you before?
Hope my question is clear and there is enough information for someone to give me some help!