I am currently trying to import the inventory from excel. I matched the names with ecoinvent 3.8 database and excel template to brightway2 format.
However, all my biosphere flows are unlinked together with some exchanges from ecoinvent. The differences are
I could not understand what reference product meant and which location must be inserted for biosphere flows.
Could someone please help me understand where the difference is coming from and how I should go about it? Thank you very much in advance!
By default, the linking algorithm matches on all these fields:
"name"
"categories"
"unit"
"reference product"
"location"
It looks to me that your biosphere flows aren't linking because they are missing the category field (there are multiple biosphere flows, one for each category). You can fix this by either adding the biosphere flow codes (then you match only on code, not other fields), or by including the categories. Note that categories are stored as a tuple, and brightway uses :: to separate the tuple elements, so ('foo', 'bar') would become foo::bar in your CSV or Excel.
Your technosphere flows are linking because you don't specify the reference product. This fails to match because the background database does specify the reference product, so these attributes aren't the same. You can try to fix that by doing my_importer.match_database("ecoinvent 3.8_cutoff", fields=["name", "unit", "location"]), i.e. not matching on the reference product.
Related
I would like to import a customer list with countries as string.
I have imported entities with the countries ISO code before, but I am struggling with just the translated names as the identifier for the country.
The field which I use in my import profile is defaultShippingAddress.country.translations.en-GB.name. This field works perfectly fine on export, but fails on import - even with the previously exported file. I noticed, that the sheet containing the failed imports has Ids in the country column, that do not seem to be valid.
Is there a correct way to import entities with countries by the country name? Or do I have to use an Id / Iso code?
This is not possible. When you pass in an association just some fields, it will create that entity and assign it. You need to resolve the ID beforehand and reference only the ID.
There is an issue open to allow referencing using unique fields, to solve issues like this. But I guess this will be not solved in near future
I was able to resolve the ID's by exporting the SW Countries into a CSV:
SELECT country_translation.country_id, country_translation.name FROM country_translation LEFT JOIN language ON language.id = country_translation.language_id WHERE language.name = 'English';
And then use VLOOKUP in Excel to match the ID's to the countries.
https://support.microsoft.com/en-us/office/vlookup-function-0bbc8083-26fe-4963-8ab8-93a18ad188a1
I work for a home based education program and we are trying to find a way to make comparing our two tracking systems easier. Unfortunately due to limitations within the software we are using for this, we cannot combine it all into one place. I have two different data sets that share some fields but are for two different things. The first set is for documenting that a visit to the home occurred or not. The second set documents what happened during that visit. Both datasets have include a date field and a field for the child's name.
table_1 table2
I've included a sample version of each table. There are other columns but they are unique to each data set. The ultimate goal is to be able to ensure that the information is being documented into both systems. I had thought to utilize the name or date field to link the data sets together to identify discrepancies but haven't had much success as of yet.
Basically I need to make sure that if a visit occurred on 4/5/21 than it was documented to both places and if it wasn't, identify which area is missing the information. It would be nice to be able to also then summarize the information by child.
I've tried Power BI and Power Query in Excel but to be honest I'm not even sure how to phrase my question in order to find the proper way to do this from Google.
I am a bit confused about terminology. In SharePoint we have a List and it consists of List columns but I have read in one of the site that this can be meta data too.
Taking an example if we have a SharePoint list and if we have a List Columns with name, Job description, age and Income, will this be termed as column, field or metadata?
If this is metadata then how can you define fields/columns/site column?
You may know that Meta Data is nothing but data about data.
In share point context Meta Data can be used as list column also. Consider any excel documents which are classified using Phase , Category, Document Type etc. So when ever you see these columns you can identify what kind of document it is.
As per my knowledge key words are nothing but taxonomy. By using these keywords you can classify the items.
Site Column is different. Site Column can be used as list column in any list. For eg. Consider the Age. In your site you are using Age column repeatedly in your lists/libraries. So instead of creating Age column in each list you can create only one site column and you can add to required list. In simple words reusable purpose.
Fields are nothing but a columns/site columns. These are related to list.
As Mihir states, the word "metadata" means "data about data". So to have metadata you need to have data. In SharePoint the most relevant example would be documents. Documents are data and you can have data (metadata) about these documents. Mihirs Excel example is an example of this.
Say, you want a document library in SharePoint and these documents have a document type chosen from a list of choices.
To achieve this you need to be able to put a document type on the documents. Since Document libraries in SharePoint is a special kind of list, you can create columns=fields on the list - and the data in these columns becomes metadata for your document.
So on the library you create a field, DocumentType, e.g. as a choice field. The field is setup with possible choices for the document type. So the list fields/columns are used to define metadata on the document(s).
When it comes to pure SP lists it becomes more difficult to talk about metadata. For what is it data about? In case of an employee I don't think you can define it as metadata, expect if you treat the physical person as "the data". However, I don't think it technical make a difference. You have your datamodel of the employees and implement it using SP lists - that's what they are there for!
Ok - so here's the preface. I realize that SharePoint isn't really the best solution for this, and if I have to use SharePoint, then coding this would be preferable. However, my wings are clipped due to IT policy, and they have no time to do this "for me".
I am developing a document control system for an Environmental Management System. The basic concept is that there is a list of 'General Responsibilities', which identifies a regulatory agency, some category info, links to laws, etc... There's a second list 'EMS Responsibilities', which basically just links (via a lookup) a particular site to a general responsibility, to identify which sites have which responsibilities. We have many sites, and most have most responsibilities so it doesn't make sense to combine the two lists and repeat all this stuff 10x. Finally, I have a document library (for simplicty sake I am ignoring the draft/published libraries aspect) which contains the actual documents. These documents should reference an EMS Responsibility, which in turn references a General Responsibility. The categorization from 'General Responsibilities' needs to propagate all the way to the document library.
Currently, I use workflow to automatically copy the secondary lookup columns referencing the 'General Responsibility' to a 'Single Line of Text' column in 'EMS Responsibilities' so that it is available to the lookup in 'EMS Documents'. However, despite the values being present in both responsibility lists, the values do not propagate to my final list.
This workflow-based approach is stupid. The lookup columns are stupid. There's no way to get this to work elegantly using out of the box components. I want to believe that the reason this isn't working is because I am missing something... but I have searched for hours and can't find any more effective relational capabilities.
First of all, any theories as to why the values do not end up in the final list despite referencing a single line of text column that is filled?
Second, is there a better overall approach that doesn't rely so much on workflows copying data back and forth, and these pathetic lookup columns?
Thanks in advance!
Given the requirements you gave above together with your answer to my question I believe you won't need a workflow to link all three items together. I have two options for you:
1st Option:
Create a General Responsibility list. This list should contain at least two columns. The ID and Title column.
Create an EMS Responsibilities list. This list should contain at least three columns. The ID, the Title column and a lookup column linked to the General Responsibility list.
Create a Document Library. Add a lookup column linked to EMS Responsibilities.
Since the lookup list you are using is already linked to General Responsibilities then there is no need to have another column solely dedicated to point to General Responsibilities. If however, you need to have specific columns in your document library so that it explicitly has columns for both General Responsibilities and EMS Responsibilities use option 2 below.
Option 2
Create a General Responsibility list. This list should at least contain the ID and Title column.
Create an EMS Responsibility list. This list should at least contain the ID, lookup column linked to General Responsibility and Title columns.
Create a document library. You should add two lookup columns. One pointing to General Responsibility the other to EMS Responsibility.
(now the fun part starts)
Follow the guide to edit both your NewForm and EditForm aspx in this link so that your lookup dropdowns will be a cascading dropdown. This will make sure that items from EMS dropdown will only be populated once you select a value from the General dropdown.
Let me know if you need any more clarifications.
I got a question about the behavior of lookup fields when importing data. I wonder how the lookup fields behave when the list they point to is being replaced/imported. To explain the issue, I will provide a quick example below:
As example, assume we have these two sharepoint lists:
Product Types
-------------
+ Type Name
+ Code Nr
+ etc
Products
--------
+ Product Name
+ Product Type (Lookup field to list "Product Types")
+ etc
In my scenario, the Products List contains production data on the production Sharepoint platform. It is filled with data by the business users.
However the Product Types list contains rather static data and is maintained by the developer.
Now after a development cycle, the developer wants to deploy his new webparts and his new data (product types list). The developer performs the following procedure:
On the dev machine: Export "product type" list using stsadm
On the production machine: Delete all items in the "product type" list
On the production machine: Import the "product type" list using stsadm
This means we basically replace the "product type" list on the production server while keeping the "product" list as it is.
Now the question:
Is this safe? Will the lookup references break under certain circumstances?
Any downside of this import/export procedure?
What happens if someone accesses a "product" during the import? Will the (now invalid) reference clear its own content (become a null value).
What happens if the schema of the "product type" list changes (new column)? Will this cause any troubles?
Thanks for all feedback and suggestions!
Update 1
The imported "product type" items have the same IDs as previously deleted ones.
Update 2
Started a bounty to get some more feedback/opinions.
We have had this exact same scenario before. This is a little tricky, depending upon how you will approach it.
1) Delete and Recreate Product Type list through UI
If you delete and recreate the lookup List(Product Type in your case) through UI, then you will lose the connections because the List's id GUID will change upon recreation. So do not go that route.
2) Delete and Recreate Product Type through a Feature
If you had created the Product Type list through a feature.xml file using the <ListInstance> element, then if you delete that list and then recreate it using the same feature (basically Id attribute of ListInstance remains the same, number of list items, i.e. the number of <Row> elements, may change), the association would be maintained. So if you were adding 5 more product types, then if you had created the list using a feature, you could just delete the list and provision the new one using the same feature with extra info for new items and everything would just work!
As a side note, this is the better approach because if you have to do the upgrade on a lot of servers, then rather than doing list export import via stsadm, feature deactivation and activation is a much more recommended solution. This is how we did it.
3)Deleting all list items from Product Type and adding new ones (list is never deleted)
If you are linking the lookup field (in Product List) to the ID field of the lookup list(Product Type), you have to remember that ID is auto-incrementing, so if you delete all items and then add new ones, then their ID's would be different. Say you had 5 items with ID's (ID field is not shown in UI while editing in Datasheet view) 1-5 in the list. If you delete them and add new items, their ID's would start from 6 and not 1 again. So if your lookup field had link to the item with ID 1 in it, then this method is not going to work because there is no item with ID 1 in the Product Type list anymore. So you might want to really try this out before going to production with this method.
4) Editing the list in place
If the list is not extraordinarily huge, and you only have to make this change to one or two instances, could you not just edit the list directly in the datasheet view on the prod server? When editing in datasheet view, do not delete the item, but just overwrite the values of its columns. And you can add more items if you want. This will make sure your ID's are valid.
I have mostly talked about adding new items to the list. Now if you were deleting existing items, then your lookup fields will be affected because assuming you linked the field by ID, the ID is not present anymore since the item has been deleted. Basically, any method you use, maintaining your ID's is critical.
Now regarding your doubts/questions:
I am not too sure about stsadm export import for a list (never done it myself), but stsadm can be tricky as some operations will work on certain scopes only. So you better try out your exact scenario on a dev env.
What happens during an import is tricky again depending on the exact timing. I am sure SP has its own concurrency mechanisms, but you cannot have a definitive answer as it might probably be different based on the stage of the import. If possible, recommended approach is to do the import during a planned downtime.
Regarding changing schema of the list, a change in the schema of a list will not affect the existing list instances (for the most part). If you do this through UI, I believe SP makes changes to the content DB directly. I am not certain how you intend to do this, but if you were to add a column to an existing list using a feature, the way to do this is during feature activation by adding a new content type to the list and adding your new column to this content type. This way you add the column but do not affect the existing list items.
Good luck...
There are two components to a particular lookup: the field, and the field value. The field value only contains the ID of the item(s) it refers to, and the display field. This information is meaningless without the field, which specifies what list to look at and what field to use as the display field.
The primary reason that a Lookup will break occurs on the field scope: either the list it referred to no longer exists, or the list does not contain the required field. These would generally happen if you deleted and recreated the list, but you aren't doing that. If you do break a lookup's list reference, then the only thing you can do is re-create the lookup, because you cannot configure the list reference for a lookup field once it is created.
The downside of your import/export procedure is that you lose the validity of all currently existing lookup values. A lookup maintains its integrity based on the ID of the item it references. So when the display field changes, it still refers to the same item. If you delete the item, then the lookup no longer references it, even if you create a new item that has the same value for the display field. So you would have to reassign all of the products to the new product types.
It should be noted that if you were to revert the deletion of that item, it would return to being on the lookup! The reference to that ID is kept until the actual lookup value is updated (such as by editing the Product).
All of your now invalid references will be null for purposes of interaction. You won't see anything on display forms, and you won't have the options when you try to update the product. When you do update the product, you update it to what you just set it to, which since you can't set the non-existent IDs, means that there are no more references to those IDs.
Any changes to the Product Type list's schema that do not affect the display field specified for the lookup will not have any effect on the lookup integrity. If you do change the display field in any fashion, and of course if you delete it, then it will break in the same fashion as with the list reference. However, you can set the display field, both in the UI and in the object model, so it is easy to fix this.