I am building a search engine, and have a not so unique ID for a lot of different names... So, for example, there could be an id of B0051QVF7A which would have multiple names like "Kindle" "Amazon Kindle" "Amazon Kindle 3G" "Kindle Ebook Reader" "New Kindle" etc.
The problem, and question i have, is that i am trying to enter this data from a DB of 11 ish million rows. each is being read one at a time. So i dont have all the names of each ID. I am adding new documents to the list each time.
What i am trying to find out is how do i add names to an existing Document? if i am reading documentation correctly, it seems to overwrite the whole document, not add extra info to the field... i just want to add an extra name to the document multivalue field...
I know this could cause some weird and wonderful "issues" if a name is removed (in the example above, "New Kindle" could be removed when a newer Kindle gets released) but i am thinking of recreating the index every now and again, to clear out issues like that (once a month or so. Its taking about 45min currently to create the index).
So, how do you add a value to a multivalue field in solr for an existing document?
Since according to the question linked to by #Mauricio Scheffer's comment... Solr does not currently support updating a single field value in an existing document. I see that there might be a couple of options here...
In your process that is pulling data from the database, when it finds a new name, it will need to pull all fields for the existing document from Solr, add the new value and resend the complete document to Solr (you may already be doing this).
Add some additional logic to your code that reads from the database, to gather all of the unique names for each document prior to inserting documents into the index. However, given that you have ~11 million records, there could be a resource constraint that would prevent this from being feasible.
Related
I am trying to create a collection name based on the date like i have a collection name like this Change 06-05-2020 and i want to overwrite this collection daily and make the collection name like this Change 07-05-2020 and so on is it possible to do it i ma creating the collection in this way.and basically i am trying to store the daily updated data in a particular way so i can track that information.so can i update the collection name dynamically daily?
await growthfilemsdb.collection(`Change${getISO8601Date()}`).doc(change.after.data().officeId).set(change.after.data(),{merge:false})
It's not possible to change the name of a collection. What you can do instead is simply copy all the documents from the old collection to a new one with a new name.
However, it's usually not a good idea to make the names of your collection dynamic like this. Instead, consider putting the date as a field inside the document, and using that to filter the results of queries or delete old documents.
I'm building a mongoDB database that holds sales data from multiple different systems. Each system is integrated via an node/mongoose/Express API that I'm creating for the database. Typically, you'd check the id to determine if a record already exists, and insert it if it doesn't. But since the ID from these different sources could technically overlap, I need a system to make sure that a source can only update records that originally came from that source. So I've added a column called "external_ID" where the record id from the source is saved, and another column called "integration ID", which will be unique to the specific system that sends data. But for that idea to work, I'd need to update only if those two columns matches, and otherwise insert a new record. Is that possible with MongoDB, or am I approaching this wrong?
Thank you so much.
Use upsert on update(). It will creates a new document when no document matches the query criteria.
db.collection.update(<query>, <update>, { upsert: true })
You can find more detail at Upsert Behavior documentation
I am running into an issue when adding a new field to an entity. In my entity I had 526 fields, I tried to add a new one but I got the error message "Attribute xxx cannot be created because we have hit a maximum number of attributes allowed for an entity (1000)". However I only have 526 fields in the entity. Furthermore, I deleted one field I don't need anymore and I tried to add my new field but I still have the same error message even though I've deleted one field. Could anyone help me out please. I do appreciate it. Thanks !
Walloud
You will be hitting the column limit, even with only 526 fields.
You have to remember that CRM will add its own fields in addition to the ones that you have, such as ones for base currency. This can take you over the limit.
Also consider that the Filtered Views that CRM automatically creates have extra columns for lookup (name), optionset (name) and date (local) fields which can also mean you exceed the limit there.
I think your only options are to delete more fields in order to free up the room, or re-architect your entity design to split it over multiple related entities instead
Looks like you are facing the SQL Server 8k row limit
you can check the field names and verify this.
more information here:
http://mileyja.blogspot.com/2013/09/dealing-with-8k-sql-server-row-limits.html
You need to remember that some fields, such as currency will have two separate fields in SQL thus increment the column count by two for each one you create. If you have 500+ fields on a single entity then it is time to normalize the data and start breaking it out into related entities. It will be better for performance and your data in the long run.
http://en.wikipedia.org/wiki/Database_normalization
Cheers!
i am looking to create an invoice in either MS-excel or MS-word. This invoice will contain several fields like invoice no., customer name, product info, quantity, Amount, Date, Address of customer, phone no. etc. The function of the invoice will be, to generate a unique invoice number, every time i open it, and then the vendor will mention the customer's info, product's info and click on submit button or save it. The info mentioned in the invoice will automatically be saved in the MS-Access database whenever submit button is clicked or the document is saved.
Thus, All the records of the customer will be saved in the MS-Access database. whenever i need to search for a particular customer, i should be able to search it from either invoice no. or any unique field for that particular invoice. I hope my query is explained clearly. please let me know the easiest way to do it. I do not have the vast knowledge about this subject, so give me suggestions that are understandable by a Novice.
I think you are starting from the wrong end. Use an Access form to get the data and then run a mailmerge, the easiest way is to output a text file from Access as the data file and use a Word template for the merge.
An autonumber may suit for invoice number as long as all you need is a unique number. If you need documented sequential numbers, you will have to create then yourself. How you do it will depend on the number of users working at the same time.
I can tell you now, generating Word files with Access is a bit of a pain in the ass. If you really want to do formatting it gets hard (is my experience).
I ended up generating HTML files in which I could control everything, and opening them as .docs. But if you are really new to this, I suggest you start with some VBA tutorials, where they explain to you how you get records from you database and loop through them to generate output. And then you can start looking at file writing functions in VBA.
Can't find any tutorials real quick (my girlfriend is getting angry as we speak), but here is a sample:
http://www.access-programmers.co.uk/forums/showthread.php?t=25354
Just look around in fora, look for file generation and looping through records.
Hi just reading your post, like Remou l would strongly suggest you use Access to enter and store the data. It is possible to get a user to enter data into a spreadsheet and write the data back to Access DB. Not something l would recommend for the novice, here is a link to some code on how it could be done
Returning to your first question, of creating the invoice have you considered generating the invoices from Access using a report? They can be printed to PDF, or exported to various electronic formats. Or is there specific reason to use Word/Excel? If are going down the route of using Word to generate the invoice then use a template as Remou suggested. See this link for some samples see the section titled Access > Word. I have used the examples as a basis for Access to Word. A number of the examples though use a tecnology called DAO, which l understand will not be included in any operating system after Windows 7. Just something to be aware of.
Searching for a record in a database table this link has one possible solution . Also the author has included a example database.
I got a question about the behavior of lookup fields when importing data. I wonder how the lookup fields behave when the list they point to is being replaced/imported. To explain the issue, I will provide a quick example below:
As example, assume we have these two sharepoint lists:
Product Types
-------------
+ Type Name
+ Code Nr
+ etc
Products
--------
+ Product Name
+ Product Type (Lookup field to list "Product Types")
+ etc
In my scenario, the Products List contains production data on the production Sharepoint platform. It is filled with data by the business users.
However the Product Types list contains rather static data and is maintained by the developer.
Now after a development cycle, the developer wants to deploy his new webparts and his new data (product types list). The developer performs the following procedure:
On the dev machine: Export "product type" list using stsadm
On the production machine: Delete all items in the "product type" list
On the production machine: Import the "product type" list using stsadm
This means we basically replace the "product type" list on the production server while keeping the "product" list as it is.
Now the question:
Is this safe? Will the lookup references break under certain circumstances?
Any downside of this import/export procedure?
What happens if someone accesses a "product" during the import? Will the (now invalid) reference clear its own content (become a null value).
What happens if the schema of the "product type" list changes (new column)? Will this cause any troubles?
Thanks for all feedback and suggestions!
Update 1
The imported "product type" items have the same IDs as previously deleted ones.
Update 2
Started a bounty to get some more feedback/opinions.
We have had this exact same scenario before. This is a little tricky, depending upon how you will approach it.
1) Delete and Recreate Product Type list through UI
If you delete and recreate the lookup List(Product Type in your case) through UI, then you will lose the connections because the List's id GUID will change upon recreation. So do not go that route.
2) Delete and Recreate Product Type through a Feature
If you had created the Product Type list through a feature.xml file using the <ListInstance> element, then if you delete that list and then recreate it using the same feature (basically Id attribute of ListInstance remains the same, number of list items, i.e. the number of <Row> elements, may change), the association would be maintained. So if you were adding 5 more product types, then if you had created the list using a feature, you could just delete the list and provision the new one using the same feature with extra info for new items and everything would just work!
As a side note, this is the better approach because if you have to do the upgrade on a lot of servers, then rather than doing list export import via stsadm, feature deactivation and activation is a much more recommended solution. This is how we did it.
3)Deleting all list items from Product Type and adding new ones (list is never deleted)
If you are linking the lookup field (in Product List) to the ID field of the lookup list(Product Type), you have to remember that ID is auto-incrementing, so if you delete all items and then add new ones, then their ID's would be different. Say you had 5 items with ID's (ID field is not shown in UI while editing in Datasheet view) 1-5 in the list. If you delete them and add new items, their ID's would start from 6 and not 1 again. So if your lookup field had link to the item with ID 1 in it, then this method is not going to work because there is no item with ID 1 in the Product Type list anymore. So you might want to really try this out before going to production with this method.
4) Editing the list in place
If the list is not extraordinarily huge, and you only have to make this change to one or two instances, could you not just edit the list directly in the datasheet view on the prod server? When editing in datasheet view, do not delete the item, but just overwrite the values of its columns. And you can add more items if you want. This will make sure your ID's are valid.
I have mostly talked about adding new items to the list. Now if you were deleting existing items, then your lookup fields will be affected because assuming you linked the field by ID, the ID is not present anymore since the item has been deleted. Basically, any method you use, maintaining your ID's is critical.
Now regarding your doubts/questions:
I am not too sure about stsadm export import for a list (never done it myself), but stsadm can be tricky as some operations will work on certain scopes only. So you better try out your exact scenario on a dev env.
What happens during an import is tricky again depending on the exact timing. I am sure SP has its own concurrency mechanisms, but you cannot have a definitive answer as it might probably be different based on the stage of the import. If possible, recommended approach is to do the import during a planned downtime.
Regarding changing schema of the list, a change in the schema of a list will not affect the existing list instances (for the most part). If you do this through UI, I believe SP makes changes to the content DB directly. I am not certain how you intend to do this, but if you were to add a column to an existing list using a feature, the way to do this is during feature activation by adding a new content type to the list and adding your new column to this content type. This way you add the column but do not affect the existing list items.
Good luck...
There are two components to a particular lookup: the field, and the field value. The field value only contains the ID of the item(s) it refers to, and the display field. This information is meaningless without the field, which specifies what list to look at and what field to use as the display field.
The primary reason that a Lookup will break occurs on the field scope: either the list it referred to no longer exists, or the list does not contain the required field. These would generally happen if you deleted and recreated the list, but you aren't doing that. If you do break a lookup's list reference, then the only thing you can do is re-create the lookup, because you cannot configure the list reference for a lookup field once it is created.
The downside of your import/export procedure is that you lose the validity of all currently existing lookup values. A lookup maintains its integrity based on the ID of the item it references. So when the display field changes, it still refers to the same item. If you delete the item, then the lookup no longer references it, even if you create a new item that has the same value for the display field. So you would have to reassign all of the products to the new product types.
It should be noted that if you were to revert the deletion of that item, it would return to being on the lookup! The reference to that ID is kept until the actual lookup value is updated (such as by editing the Product).
All of your now invalid references will be null for purposes of interaction. You won't see anything on display forms, and you won't have the options when you try to update the product. When you do update the product, you update it to what you just set it to, which since you can't set the non-existent IDs, means that there are no more references to those IDs.
Any changes to the Product Type list's schema that do not affect the display field specified for the lookup will not have any effect on the lookup integrity. If you do change the display field in any fashion, and of course if you delete it, then it will break in the same fashion as with the list reference. However, you can set the display field, both in the UI and in the object model, so it is easy to fix this.