I use NodeJS and I have a MongoDB collection with a lot of entries. 99% of time the last entry is selected, sometimes the entry before. Since MongoDB has to get through all entries one by one, it would be more useful to sort the entries the other way round:
Instead of this:
{
_id: "foo",
name: "name"
},
{
_id: "bar",
name: "name"
}
// <- new entry will be inserted here
I want to use this:
// <- new entry will be inserted here
{
_id: "foo",
name: "name"
},
{
_id: "bar",
name: "name"
},
So that in most cases the entry I search for is the first or the second item.
Is that possible or even necessary (does it make any difference in speed)?
I could also reverse the items and then iterate through them, but I don't think that this would be faster.
You schould not be concerned about position of the item in the collection. Each collection has an index on _id field so if you sort your collection by this field and take first (second or third) element you will get in in no time ( porbably 0 ms)
Related
My question builds off this: Get only the last element of array mongoose. However, I'm using Mongoose and want to also update a field in the last element of the array, not just retrieve it.
How can I modify the last element/object in the rounds array to change from status: started to status: terminated?
Document to modify
{
"_id" : "8844d3f2d25f45df8105db1ab058d7d6",
"rounds" : [
{ "status" : "offline" },
{ "status" : "paused" },
{ "status" : "started" }
],
"updatedAt" : ISODate("2020-07-05T21:21:58.823Z")
}
Seems like using negative indices would make this easy, allowing the n-1 element to be modified. I've seen many questions mention aggregation, but none modify the underlying data. I want to combine aggregation with updating a documenting.
Is it possible to use $set with a negative index?
As of v4.2 MongoDB does not have a straightforward way to update an array element by index.
However starting from v4.2 you can use updates with aggregation pipeline, which allow you to construct a new array based on the current array and replace the current array.
Instead of modifying the last element, you can construct a new array consisting of the first top n - 1 elements combining with a new element { status: "terminated" }.
You can achieve this by using $slice to get the top n - 1 elements and combine with the new element using $concatArrays
This assumes the elements only contains the field status you you want to update a field of an array elements that contains multiple fields, you'll need to merge the updated field with the current element using $mergeObjects
Model.updateOne({ // or updateMany with your condition
_id: "8844d3f2d25f45df8105db1ab058d7d6"
}, [{
$set: {
rounds: {
$concatArrays: [ // combine arrays
{
$slice: [ // get the top n - 1 elements
"$rounds",
{ $subtract: [{ $size: "$rounds" }, 1] }
]
},,
[{ status: "terminated" }] // a new element to replace the last element
]
}
}
}])
I have a collection in my database that contains a field which is composed of 3 arrays, like this :
use_homepage: {
home: [Array],
hidden: [Array],
archive: [Array]
}
This field represents the homepage of a user.
Each array contains an ObjectID that identifies projects shown on the user homepage.
I would like to check if my project id is in use_homepage.home or use_homepage.hidden, and if it is, remove the id from the array that match.
Can I do this with 1 (or 2 max) requests or do I have to make a request each time I have to check in another array ?
In case you expect to update one document at most, you can try this:
db.entities.findAndModify({
query: { $or : [
{ home: ObjectId('<HERE YOUR ID TO BE FOUND>') },
{ hidden: ObjectId('<HERE YOUR ID TO BE FOUND>') }
]},
update: { $pull: {
home: ObjectId('<HERE YOUR ID TO BE DELETED>'),
hidden: ObjectId('<HERE YOUR ID TO BE DELETED>')
}
}
});
As you can see, in general, you can search for some value and delete some other value.
The statement returns the original matching document (i.e. before the deletion is performed). If you want the modified document you can add the following attribute:
new: true
In case you search for many documents to update, this solution does not work, since findAndModify() works just on the first document matching the query condition.
Finaly, i used to make 2 requests to do the job :
db.User.find({"use_homepage.home": id}, {_id: 1}).toArray(function(err, result) {
// If some users have the id in the array home
db.User.updateMany({_id: {$in: users_match_ids}}, {
$pull: {"use_homepage.home": id}
}
});
// Do the same with 'hidden' array
If anyone see this post and have a better solution, I take it :)
I need to add to a list of items in Mongo so if I have
items:{item: "apple"}
what would I use to add another item in an object instead of changing that initial object? So I can end up with.
items: {item: "apple"},{item:"orange"},{item:"blueberry"}
Can I use findOneAndUpdate? Or will this over-write the original data. I am having a hard time finding the distinction in the documents.
In closing, what method is used for updating and overwriting and what is used for appending to objects and arrays?
You can use the $addToSet operator.
For example:
db.yourCollection.update(
{ _id: 1 },
{ $addToSet: { items: {item : "orange" } } }
)
The code abode will add the item : "orange" to the items list of the document with id=1
We have been using MongoDB for some time now and there is one thing I just cant wrap my head around. Lets say I have a a collection of Users that have a Watch List or Favorite Items List like this:
usersCollection = [
{
_id: 1,
name: "Rob",
itemWatchList:[
"111111",
"222222",
"333333"
]
}
];
and a separate Collection of Items
itemsCollection = [
{
_id:"111111",
name: "Laptop",
price:1000.00
},
{
_id:"222222",
name: "Bike",
price:123.00
},
{
_id:"333333",
name: "House",
price:500000.00
}
];
Obviously we would not want to insert the whole item obj inside the itemWatchList array because the items data could change i.e. price.
Lets say we pull that user to the GUI and want to diplay a grid of the user itemWatchList. We cant because all we have is a list of ID's. Is the only option to do a second collection.find([itemWatchList]) and then in the results callback manipulate the user record to display the current items? The problem with that is what if I return an array of multiple Users each with an array of itemWatchList's, that would be a callback nightmare to try and keep the results straight. I know Map Reduce or Aggregation framework cant traverse multiple collections.
What is the best practice here and is there a better data structure that should be used to avoid this issue all together?
You have 3 different options with how to display relational data. None of them are perfect, but the one you've chosen may not be the best option for your use case.
Option 1 - Reference the IDs
This is the option you've chosen. Keep a list of Ids, generally in an array of the objects you want to reference. Later to display them, you do a second round-trip with an $in query.
Option 2 - Subdocuments
This is probably a bad solution for your situation. It means putting the entire array of documents that are stored in the items collection into your user collection as a sub-document. This is great if only one user can own an item at a time. (For example, different shipping and billing addresses.)
Option 3 - A combination
This may be the best option for you, but it'll mean changing your schema. For example, lets say that your items have 20 properties, but you really only care about the name and price for the majority of your screens. You then have a schema like this:
usersCollection = [
{
_id: 1,
name: "Rob",
itemWatchList:[
{
_id:"111111",
name: "Laptop",
price:1000.00
},
{
_id:"222222",
name: "Bike",
price:123.00
},
{
_id:"333333",
name: "House",
price:500000.00
}
]
}
];
itemsCollection = [
{
_id:"111111",
name: "Laptop",
price:1000.00,
otherAttributes: ...
},
{
_id:"222222",
name: "Bike",
price:123.00
otherAttributes: ...
},
{
_id:"333333",
name: "House",
price:500000.00,
otherAttributes: ...
}
];
The difficulty is that you then have to keep these items in sync with each other. (This is what is meant by eventual consistency.) If you have a low-stakes application (not banking, health care etc) this isn't a big deal. You can have the two update queries happen successively, updating the users that have that item to the new price. You'll notice this sort of latency on some websites if you pay attention. Ebay for example often has different prices on the search results pages than the actual price once you open the actual page, even if you return and refresh the search results.
Good luck!
I am maintaining a collection of unique values that has a companion collection that has instances of those values. The reason I have it that way is the companion collection has >10 million records where the unique values collection only add up to 100K and I use those values all over the place and do partial match lookups.
When I upload a csv file it is usually 10k to 500k records at a time that I insert into the companion collection. What is the best way to insert only values that dont already exist into the unique values collection?
Example:
//Insert large quantities of objects into mongo
var bulkInsert = [
{
name: "Some Name",
other: "zxy",
properties: "abc"
},
{
name: "Some Name",
other: "zxy",
properties: "abc"
},
{
name: "Other Name",
other: "zxy",
properties: "abc"
}]
//Need to insert only values that do not already exist in mongo unique values collection
var uniqueValues = [
{
name:"Some Name"
},
{
name:"Other Name"
}
]
EDIT
I tried creating a unique index on the field, but once it finds a duplicate in the Array of documents that I am inserting, it stops the whole process and doesnt proceed to check any values after the break.
Figured it out. If your doing it from the shell, you need to use Bulk() and create insert jobs like this:
var bulk = db.collection.initializeUnorderedBulkOp();
bulk.insert( { name: "1234567890a"} );
bulk.insert( { name: "1234567890b"} );
bulk.insert( { name: "1234567890"} );
bulk.execute();
and in node, the continueOnError flag works on a straight collection.insert()
collection.insert( [{name:"1234567890a"},{name:"1234567890c"}],{continueOnError:true}, function(err, doc){}
Well, I think the solution here is quite simple if I understand correctly your issue.
Since the process is stopped when it finds a duplicated field you should basically check if the value doesn't already exists before to try to add it.
So, for each element in uniqueValues, make a find/findOne query, if it doesn't return any result then add the element, otherwise don't.