I have a collection, say, "Things":
{ id: 1
creator: 1
created: Today }
{ id: 2
creator: 2
created: Today }
{ id: 3
creator: 2
created: Yesterday }
I'd like to create a query that'll return each Thing created by a set of users, but only their most recently created thing.
What would this look like? I can get search my collection with an array of creators and it works just fine - how can I also only get the most recently created object per user?
Thing.find({ _creator : { "$in" : creatorArray })...
You cannot find, sort and pick the most recent in just a single find() query. But you can do it using aggregation:
Match all the records where the creator is amongst the one who we are looking
for.
Sort the records in descending order based on the created field.
Group the documents based on the creator.
Pick each creator's first document from the group, which will also be
his latest.
Project the required fields.
snippet:
Thing.aggregate([
{$match:{"creator":{$in:[1,2]}}},
{$sort:{"created":-1}},
{$group:{"_id":"$creator","record":{$first:"$$ROOT"}}},
{$project:{"_id":0,
"id":"$record.id",
"creator":"$record.creator",
"created":"$record.created"}}
], function(err,data){
})
Related
We have a production database with over 5 million customer customer records, each customer document has an embedded array of licenses they have applied for. And example customer document is as follows:
{
_id: ObjectId('...'),
phoneNumber: 'xxxx',
// Other customer fields
licenses: [
{
_id: ObjectId('...'),
state: 'PENDING',
expired: false,
createdAt: ISODate(''),
// Other license fields
},
// More Licenses for this customer
]
}
I have been tasked with changing the state of every PENDING license applied for during the month of September to REJECTED and sending an SMS to every customer whose pending permit just got rejected.
Using the model.where(condition).countDocuments() I have found that there is over 3 million customers (not licenses) matching the aforementioned criteria. Each customer has an average of 9 licenses.
I need assistance coming up with a strategy that won't slow down the system when performing this action. Furthermore, this is around 17GB of data.
Sending SMS is fine, I can queue details for SMS service. My challenge is processing the licenses while extracting relevant information for SMS.
First of all you have to create an index on the collection:
db.collection.createIndex( { "licenses.state": 1 } )
Then you shoud do something like that:
model.updateMany({}, {
'$set': {
'licenses.$[elem].state': 'REJECTED'
}
}, { arrayFilters: [{
'elem.createdAt': { $gte: ISODate(....) }
}],
multi: true
} ).then(function (doc)){}
If you have a replica set and your updates are on the primary instance you should not affect the secondary instances when reading on those once.
If you want to split the update on many batches you can use the _id (already indexed). Of course it depends on your _id format.
I am new to MongoDB and mongoose. I am trying to create a Node & MongoDB auction app. So since it is actually an online auction, users should be able to bid for items. I successfully completed the user registration, sign in page and authentication process, however, I am a bit stuck in the bidding page.
I created a Schema using mongoose and each item for auction is saved in the database. I want to add name and price of each user who bid for the item in the same object in MongoDB, like this:
{
name: "valuable vase from 1700s",
owner: "John Doe",
itemId: 100029,
bids: {
100032: 30000,
100084: 34000
}
}
So each user will have ids like 100032: 30000, and when they bid, their "account id: price" will be added under bids in the database object of the item.
I made some research and found some ways to solve the problem but I want to know if what I want to do is possible and if it is the right solution to do.
Thanks for giving me your time!
There are indeed couple of ways to achieve what you want.
In my opinion, a collection called ItemBids, where each document includes this data structure, will benefit you the most.
{
itemId: ObjectId # reference to the item document
accountId: ObjectId # reference to the account
bid: Number # the bid value
}
This pattern is suitable for your case because you can easily query the bids by whatever you want -
You can get all the account bids, you can get all the item bids, and you can sort them with native Mongo by the bid price.
Every time there's a bid, you insert a new document to this collection.
Another option is embedding an array of Bids objects in the item Object.
Each Bid object should include:
bids: [{
account: ObjectId("") # This is the account
price: Number
}]
The cons here are that querying it and accessing it will require more complex queries.
You can read more about the considerations
here:
https://docs.mongodb.com/manual/core/data-model-design
https://coderwall.com/p/px3c7g/mongodb-schema-design-embedded-vs-references
The way you decided to implement your functionality is a little bit complicated.
It is not impossible to do that but, the better way is to use array of objects instead of a single object like this:
{
name: '',
..
..
bids: [{
user: 100032,
price: 30000
}, {
user: 100084,
price: 34000
}]
}
I know there's a lot of threads about this subject - but i still didn't find anything that matches my situation.
There is a big database with companies and products schema's.
Each company uploads their products, (product has "company" field with the mongo id of the company)
My operation goes like this -
Company uploads array of products.
Each product has his own productId.
I need to insert all the products in to the DB but -
If i find product with the same productId - update it - if not - insert a new product.
Change some fields (change product "status" to "deleted") to all of the products that belongs to that company in the DB, that didn't match any of the product id's (from the array of products the user uploaded).
I'm searching for a better solution than the one i'm using, my process goes like this -
fetching all the company's products.
looping through all of the products - and updating a field (status = "deleted") building a mapping object for each product, so i can identify it and check if it exists in the uploaded array.
looping through the company's array (with async library), checking if product exists (in the mapping array), if exists - updating it, if not - create a new product.
It feels very sloppy and i'm sure there is a better way to do it.
the thing i tried to use is -
await Product.updateMany({ productStatus: { $ne: 'manual'} , company: {$eq: company._id}}, { $set: {status: 'deleted' }});
and then
Product.bulkWrite(
productsArray.map((row) =>
({
updateOne: {
filter: { productId: row.productId, company: row.company },
update: row,
upsert: true
}
})
)
)
It was super slow (my method was faster even thought it's more complicated)
If i find product with the same productId - update it - if not - insert a new product.
You can use upserts, which is a find-and-modify operation with upsert: true set. Pass the whole product document.
Change some fields (change product "status" to "deleted") to all of the products that belongs to that company in the DB, that didn't match any of the product id's (from the array of products the user uploaded).
You can use $nin to update the status on all products whose ids are not in the list of currently available products.
I have a collection in my database that contains a field which is composed of 3 arrays, like this :
use_homepage: {
home: [Array],
hidden: [Array],
archive: [Array]
}
This field represents the homepage of a user.
Each array contains an ObjectID that identifies projects shown on the user homepage.
I would like to check if my project id is in use_homepage.home or use_homepage.hidden, and if it is, remove the id from the array that match.
Can I do this with 1 (or 2 max) requests or do I have to make a request each time I have to check in another array ?
In case you expect to update one document at most, you can try this:
db.entities.findAndModify({
query: { $or : [
{ home: ObjectId('<HERE YOUR ID TO BE FOUND>') },
{ hidden: ObjectId('<HERE YOUR ID TO BE FOUND>') }
]},
update: { $pull: {
home: ObjectId('<HERE YOUR ID TO BE DELETED>'),
hidden: ObjectId('<HERE YOUR ID TO BE DELETED>')
}
}
});
As you can see, in general, you can search for some value and delete some other value.
The statement returns the original matching document (i.e. before the deletion is performed). If you want the modified document you can add the following attribute:
new: true
In case you search for many documents to update, this solution does not work, since findAndModify() works just on the first document matching the query condition.
Finaly, i used to make 2 requests to do the job :
db.User.find({"use_homepage.home": id}, {_id: 1}).toArray(function(err, result) {
// If some users have the id in the array home
db.User.updateMany({_id: {$in: users_match_ids}}, {
$pull: {"use_homepage.home": id}
}
});
// Do the same with 'hidden' array
If anyone see this post and have a better solution, I take it :)
Is there any recommended way to update multiple items in MongoDB with one query ? I know that this is possible:
db.collection('mycollection').update({active: 1}, {$set: {active:0}}, {multi: true});
But in my case I want to update several documents with "unique" changes.
e.g. I want to combine these two queries into one:
db.collection('mycollection').update({
id: 'my id'
}, {
$set: {
name: "new name"
}
});
db.collection('mycolleciont').update({
id: 'my second id'
}, {
$set: {
name: "new name two"
}
});
Why ? I have a system which gets daily updates imported. The updates are mostly large so its around 200,000 Updates a day so currently I am executing 200,000 times the update query which takes a long time.
If its necessary to know: I am using Mongo 3 and nodeJS.