I am trying to import EF database (not just the method) into Brightway2.
EF database is available for Simapro, for GaBi, and for openLCA.
I was able to download the ef_secondarydata_202202.zolca file from openLCA Nexus and import it into openLCA.
But from there, I don't see any path to bring it to Brightway.
It is possible to export processes and methods in EcoSpold1 format from openLCA.
But trying to import them into Brightway using
ef = SingleOutputEcospold1Importer('C:\\...\\EcoSpold01', 'EF', use_mp=False)
ef.apply_strategies()
I get the following error :
Couldn't apply strategy link_technosphere_by_activity_hash:
Object in source database can't be uniquely linked to target database.
Problematic dataset is:
{'categories': ('End-of-life treatment',),
'filename': '(missing)',
'location': 'EU-28+EFTA',
'name': 'Electronics scrap',
'unit': 'kilogram'}
Possible targets include (at least one not shown):
[{'categories': ['End-of-life treatment'],
'filename': 'S:\\...\\EcoSpold01\\process_2fe13aaf-f0b4-4be2-9e8d-fdc27b30ac94.xml',
'location': 'EU-28+EFTA',
'name': 'Electronics scrap',
'unit': 'kilogram'},
+ many many others...
]
Applied 11 strategies in 94.26 seconds
Furthermore all (or nearly all) exchanges are unlinked :
>>> ef.statistics()
3185 datasets
4647753 exchanges
4647752 unlinked exchanges
Type biosphere: 4101 unique unlinked exchanges
Type production: 2149 unique unlinked exchanges
Type technosphere: 133 unique unlinked exchanges
(3185, 4647753, 4647752)
Of course, after that write_database() fails.
I checked the list of elementary fluxes, they are very different between EF and Biosphere3. To begin with, there are 7741 fluxes in EF vs 4709 in Biosphere3, and looking for similar names I didn't find many...
I am using :
Windows 10
brightway2 2.4.2 py_6 cmutel
bw2io 0.8.8 py_0 cmutel
OpenLCA 1.11.0 Windows 64 bit
To summarize : how would you import EF database into Brightway2 ?
I did not find the answer in SO existing questions.
Related
Short version:
I am trying to upload US LCI database to Brightway2 and I am failing miserably. Has anyone succeeded? If so, could you share it with me? :D
Long version:
I am following the notebook IO - Importing the US LCI database notebook and I am having a lot of problems. I am aware that, as the notebook indicates, it is a work in progress. Anyhow, I wanted to give it a try:
I tried uploading every ecospold version database found here, following the method from the notebook. The only one that gave me a similar results was version FY20.Q3.02. However, right off the bat I get the following differences/errors:
Same as the notebook, I get this error: Couldn't apply strategy link_technosphere_by_activity_hash: Object in source database can't be uniquely linked to target database. And two activities that are linked. When I follow the instructions of ignoring these datasets, it throws me that error over and over again.
Trying to move on with the tutorial, I get more errors and at the end I end up with all exchanges unlinked:
633 datasets
37513 exchanges
37505 unlinked exchanges
Finally, after running the code in line [15]:
import functools
f = functools.partial(link_iterable_by_fields,
other=Database(config.biosphere),
kind='biosphere'
)
sp.apply_strategy(f)
sp.statistics(f)
I end up with:
0 datasets
0 exchanges
0 unlinked exchanges
Which is hilarious and sad at the same time. Since I am new with Python and BW, my troubleshooting is clumpsy and probably erroneous (I promise I googled a lot and went through the code). And concluded I am failing and it is time to ask questions:
Has anybody succeeded uploading the US LCI database to Brightway2?
If so, how? Which file did you use?
Thank you!!!!
This is an excellent question. I have added text to the offending notebook to note that it is obsolete.
In general, I think trying to import the ecospold files is a fools errand, as though they are labeled ecospold2, they are actually ecospold1 (which is a totally different format):
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ecoSpold xmlns="http://www.EcoInvent.org/EcoSpold01">
The most recent export also raises an error when I try the ecospold1 importer:
AttributeError: no such child: {http://www.EcoInvent.org/EcoSpold01}modellingAndValidation
This is a required attribute in ecospold1.
I think the best way forward would be to consume the JSON-LD directly. Note that it is important not to run bw2setup(), as you would also want to use their list of elementary flows and LCIA methods. Currently the experimental JSON-LD importer fails because the provided datasets need allocation, but don't provide a set of consistent allocation methods. When I use the git checkout of bw2io and do the following:
uslci = JSONLDImporter(
"/Users/cmutel/Downloads/National_Renewable_Energy_Laboratory-USLCI_Database/",
"US LCI",
preferred_allocation="CAUSAL_ALLOCATION"
)
uslci.apply_strategies()
I get the following error:
UnallocatableDataset: We currently only support exchange-specific CAUSAL_ALLOCATION
This is fixable, but someone would need to step through this and fix the allocation procedure, and I don't have the time to do that now.
I try to retrieve historical financial Data from iex or morningstar. For this I use the Following Code.
import pandas as pd
pd.core.common.is_list_like = pd.api.types.is_list_like
import pandas_datareader.data as web
import datetime
start = datetime.datetime(2019, 1, 1)
end = datetime.datetime(2019, 1, 10)
facebook = web.DataReader("FB", 'morningstar', start, end)
print(facebook.head())
Unfortunatly I get the error message:
NotImplementedError: data_source='morningstar' is not implemented
or
ValueError: The IEX Cloud API key must be provided either through the
api_key variable or through the environment variable IEX_API_KEY
depending on which of both sources I use.
I tried to
pip uninstall pandas-datareader
pip install pandas-datareader
several times and also restarted the kernel but nothing changes. Was there any change to this APIs or am I doing anything wrong?
From the documentation:
You need to obtain the IEX_API_KEY from IEX and pass it to os.environ["IEX_API_KEY"]. (https://pandas-datareader.readthedocs.io/en/latest/remote_data.html#remote-data-iex)
I don't know if the IEX API still works.
The morningstar is not implemented. The following data sources (at the time of writing) are:
Tiingo
IEX
Alpha Vantage
Enigma
Quandl
St.Louis FED (FRED)
Kenneth French’s data library
World Bank
OECD
Eurostat
Thrift Savings Plan
Nasdaq Trader symbol definitions
Stooq
MOEX
You must provide an API Key when using IEX. You can do this using
os.environ["IEX_API_KEY"] = "pk_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
or by exporting the key before starting the IPython session.
You can visit iexcloud.io, after creating a student account you will get an API key for free.
The import of Ecoinvent 2.2 and 3.x does not happen. I do not understand what is the issue here.
I have downloaded both 2 and 3 versions from Ecoinvent.org but both the Ecospold1 and 2 importers show me the same (or the lack of) results.
`from brightway2 import *
ei33cutoff = SingleOutputEcospold2Importer(
r"C:\Users\HS254581\Documents\test\ecoinvent33cutoff\datasets",
"ecoinvent 3.3 cutoff"
)
ei33cutoff.apply_strategies()
ei33cutoff.statistics()`
All I get is
"Extracting XML data from 13831 datasets"
WITHOUT any error message. I have kept it running for hours sometimes, ultimately I have to quit.
I can however import Forwast database, do some LCA calculations here and some CSV files from Simapro.
I read most of the questions answered here and on github, but cannot find a solution to this problem.
Send help!
P.S. I am using Python 3.7 with Spyder. Here I need to activate bw2 environment in Anaconda prompt and set Python interpreter path to default. Have installed kernels too. This is the only way it seems to work. Also updated conda and brightway packages today just in case.
I was trying to use the following code (Figure.1) to import Ecoinvent v2.2 into Brightway. I followed code from: https://github.com/PoutineAndRosti/Brightway-Seminar-2017/blob/master/Day%201%20AM/2%20-%20BW%20structure%20and%20first%20LCAs.ipynb
I obtained all XML (ecospold files) downloaded from Simapro (which is connected to ecoinvent database) and save all datafiles into the folder: C:\bw2-python\ecoSpold1.
However, when I run the next step, I ran into the following errors:
Figure 2
I am not sure what is wrong here. Any suggestion would be very helpful!
I think the ecospold files obtained from simapro and from the ecoinvent website are not the same. Simapro codes things a bit differently, which I think affects the naming of exchanges (that is why you got an invalid exchange). You either download the ecospold files from ecoinvent or use the tools see notebooks here and here to read exported csv files (the format prefered by simapro to export datasets).
I want to calculate distance between sensors deployed in georaphical area using longitude and latitude in sparql query issued in apache jena 2.11.(Sensor description and observation are stored as RDF triple in sensor.n3, eclipse as IDE and Fedora 19, TDB as triple store)
I found that "Spatial searches with SPARQL" will help in this regard. But when I import package given at http://jena.apache.org/documentation/query/spatial-query.html import org.apache.jena.query.spatial.EntityDefinition in eclipse I get the error The import org.apache.jena.query cannot be resolved. When browsed the folder ../apache-jena-2.11.1/javadoc-arq/org/apache/jena directory it contains only
(altas, common, web, riot) there is no query folder which is the reason why import is highlighted in red.
I have one more doubt whether Apache Solr need to be installed ( I have downloaded solr 4.10.1) or just use build path to import external jar.
You need to separately download jena-spatial. (Use maven to manage your dependencies.) You can use lucene instead of Solr. Again, maven will load the dependencies. AndyS