I need to check the amount of occurrence for a specific product id in a high complexity array of objects structure.
I tried with the aggregation functionality of MongoDB, but I only got the occurrences per document rather than the occurrences cross all the documents. The data structure of each document looks like this:
{
"_id" : ObjectId("5c4d67905a07f5dec1fcc763"),
"updatedAt" : ISODate("2019-01-27T08:16:11.706Z"),
"createdAt" : ISODate("2019-01-27T08:10:56.553Z"),
"pickupTime" : ISODate("2019-01-27T08:20:00.000Z"),
"shop" : ObjectId("5c24b007b55ea3d7c599c95b"),
"owner" : ObjectId("5c242e361ee775cdd8b047b6"),
"total" : 350,
"status" : "completed",
"lineItems" : [
{
"product" : {
"options" : [],
"enabled" : false,
"description" : "",
"shop" : ObjectId("5c24b007b55ea3d7c599c95b"),
"price" : 350,
"name" : "Capuccino",
"createdAt" : ISODate("2019-01-01T21:56:31.928Z"),
"updatedAt" : ISODate("2019-01-08T12:14:53.322Z"),
"_id" : ObjectId("5c2be20f8e52115849726fdc")
},
"_id" : ObjectId("5c48efbc9efde32fab7ae47d"),
"status" : "pending",
"quantity" : 1,
"config" : []
}
],
"__v" : 0
}
As you can see each document is a order, each order has lineItems that is a array of objects that contain the quantity of the desired product, the status of the current product, some configuration and the product itself. The product field is a snapshot of the product in the moment of the order as they may be changed by the supplier (cost, name description, etc.)
I currently tried the next aggregate formula:
{
$group: {
_id: {
'product': '$lineItems.product._id'
},
count: {
$sum: 1
}
}
}
But only returns the the occurrences of a product in each order, rather than cross all the orders. I understood I need to use $reduce to reformat the data structure, but I couldn't find a way how to organice my current data structure in order to make it match. Also I got said that after counting per product I can use the $sort to put the highest values on the top and then limit the search to the amount of records to return
The desirable result is get the top 5 most sold products based on the orders data.
Thanks.
Related
I need to perform the upsert operation. Below is my document structure.
I want to update the spin_details array when there is a match found with package_id but with two conditions i.e. user_id & spin_details.package_id.
If there is a match with user_id but there is no match with spin_details.package_id then some package information has to be pushed into the spin_details array. If there is no match with user_id(only) itself then it should be able to insert a new document.
{
"_id" : ObjectId("6234ffa6bd36b0e5a05ac913"),
"user_id" : ObjectId("6230e5e2b1530b407cedeb1d"),
"__v" : 0,
"is_active" : true,
"spin_details" : [
{
"package_id" : ObjectId("6230e5e2b1530b407cedeb9d"),
"spin_count" : 10,
"_id" : ObjectId("6234ffa6f390e1fafa8e215b")
},
{
"package_id" : ObjectId("6230e5e2b1530b407cedeb2a"),
"spin_count" : 25,
"_id" : ObjectId("6234ffa6f390e1fafa8e409b")
}
]
}
I can do this using multiple different queries and then based on the result value. How can I do this with a single mongoose query for this situation?
I have a collection with multiple documents with this format:
{
"id" : "65451735",
"recommendations" : [
{
"lastModifiedDate" : ISODate("2018-12-07T00:00:00.000Z"),
"products" : [
{
"productName" : "name1",
"productId" : "productid1"
},
{
"productName" : "name2",
"productId" : "productid1"
}
],
"recommendationsValueGlobal" : 0.0
}
]
}
I have with filter with "id" and "productId".
There's 2 issues regarding the productId:
1.- it can accept comma separated values of multiple product ids
2.- the productId is really deeply nested, I've never searched through an object this deep in the document. There can be multiple recommendations, and multiple products
I need to return all documents where id matches and only include any of the products that have the Ids given, all other should not be displayed.
I know how to search on nested object inside an array, but this is way too much, specially with comma separated values. Any suggestion?
EDIT: Also the Id is not unique and can be repeated in multiple documents
How would I go about displaying the best reviews and the worst reviews at the top of the page.
I think the user's "useful" and "notUseful" votes should have an effect on the result.
I have reviews and if people click on the useful and notUseful buttons their Id gets added to the appropriate array (useful or notUseful).
you can tell what a positive or a negative score is by the "overall" score. that is 1 through 5. so 1 would be the worst and 5 would be the best.
I guess If someone gave a review with a 5 overall score but only got one useful but someone gave a score with a 4 overall and 100 people clicking on "useful" the one with 100 people should be shown as the best positive?
I only want to show 2 reviews at the top of the page the best and the worst worst review if there are ties with the overall scores the deciding factor should be the usefulness. so if there are 2 reviews with the same overall score and one of them has 5 usefuls and 10 notUsefuls that would be -5 usefuls and in the other review someone has 5 usefuls and and 4 notUsefuls that would be 1 usefuls so that would be shown to break the tie.
I'm hopping to do it with one mongoose query and not aggregation but I think the answer will be aggregation.
I guess there could be a cut off like scores greater than 3 is a positive review and lower is negative review.
I use mongoose.
Thanks in advance for your help.
some sample data.
{
"_id" : ObjectId("5929f89a54aa92274c4e4677"),
"compId" : ObjectId("58d94c441eb9e52454932db6"),
"anonId" : ObjectId("5929f88154aa92274c4e4675"),
"overall" : 3,
"titleReview" : "53",
"reviewText" : "53",
"companyName" : "store1",
"replies" : [],
"version" : 2,
"notUseful" : [ObjectId("58d94c441eb9e52454932db6")],
"useful" : [],
"dateCreated" : ISODate("2017-05-27T22:07:22.207Z"),
"images" : [],
"__v" : 0
}
{
"_id" : ObjectId("5929f8dfa1435135fc5e904b"),
"compId" : ObjectId("58d94c441eb9e52454932db6"),
"anonId" : ObjectId("5929f8bab0bc8834f41e9cf8"),
"overall" : 3,
"titleReview" : "54",
"reviewText" : "54",
"companyName" : "store1",
"replies" : [],
"version" : 1,
"notUseful" : [ObjectId("5929f83bf371672714bb8d44"), ObjectId("5929f853f371672714bb8d46")],
"useful" : [],
"dateCreated" : ISODate("2017-05-27T22:08:31.516Z"),
"images" : [],
"__v" : 0
}
{
"_id" : ObjectId("5929f956a692e82398aaa2f2"),
"compId" : ObjectId("58d94c441eb9e52454932db6"),
"anonId" : ObjectId("5929f93da692e82398aaa2f0"),
"overall" : 3,
"titleReview" : "56",
"reviewText" : "56",
"companyName" : "store1",
"replies" : [],
"version" : 1,
"notUseful" : [],
"useful" : [],
"dateCreated" : ISODate("2017-05-27T22:10:30.608Z"),
"images" : [],
"__v" : 0
}
If I am reading your question correctly then it appears you want a calculated difference of the "useful" and "nonUseful" votes to also be taken into account when sorting on the "overall" score of the documents.
The better option here is include that calculation in your stored documents, but for totality we will cover both options.
Aggregation
Without changes to your schema and other logic, then aggregation is indeed required to do that calculation. This is best presented as:
Model.aggregate([
{ "$addFields": {
"netUseful": {
"$subtract": [
{ "$size": "$useful" },
{ "$size": "$notUseful" }
]
}
}},
{ "$sort": { "overall": 1, "netUseful": -1 } }
],function(err, result) {
})
So you are basically getting the difference between the two arrays, where more "useful" responses have a positive impact boosting the ranking ans more "notUseful" will reduce that impact. Depending on the MongoDB version you have available you use either $addFields with only the additional field or $project with all the fields you need to return.
The $sort is then performed on the combination of the "overall" score in ascending order as per your rules, and the new field of "netUseful" in descending order ranking "positive" to "negative".
Re-Modelling
Foregoing aggregation altogether, you get a faster result from the plain query. But this of course means maintaining that "score" in the document as you add members to the array.
In basic options, you are using the $inc update operator along with $push to change the score.
So for a "useful" entry, you would do something like this:
Model.update(
{ "_id": docId, "useful": { "$ne": userId } },
{
"$push": { "useful": userId },
"$inc": { "netUseful": 1 }
},
function(err, status) {
}
)
And for a "notUseful" you do the opposite by "decrementing" with a negative value to $inc:
Model.update(
{ "_id": docId, "nonUseful": { "$ne": userId } },
{
"$push": { "nonUseful": userId },
"$inc": { "netUseful": -1 }
},
function(err, status) {
}
)
To cover all cases including where a vote is "changed" from "useFul" to "nonUseful" then you would expand on the logic and implement the appropriate reverse actions with $pull. But this should give the general idea.
N.B The reason we do not use the $addToSet operation here is because we want to make sure the user id is not present in the array when "incrementing" or "decrementing". Thus instead the $ne operator is used to test the value does not exist. If it does, then we do not attempt to modify the array or affect the "netUseful" value. The same applies to the reverse case of "removing" the user from those votes.
Since the calculation is always maintained with each update, you simply perform as query with a standard .sort()
Model.find().sort({ "overall": 1, "netUseful": -1 }).exec(function(err,results) {
})
So by moving the "cost" into the maintenance of the "votes", you remove the overhead of running the aggregation later. For my money, where this is a regular operation and the "sort" does not rely on other run-time parameters which force the calculation to be dynamic, then you use the stored result instead.
I have a collection data which has around 300k entries and its document looks like
{
"_id" : ObjectId("5xxx85"),
"user_id" : "1",
"name" : "test",
"user_private" : "0"
}
now i want to update all the documents in this collection and new document will look like
{
"_id" : ObjectId("5xxx85"),
"rid" : "1",
"user_name" : "test",
"is_private" : "private",
"is_moderator" : "true",
"amount" : "11111"
}
i.e i need to add new fields, update field names and check if user_private = 0 then put is_private as private or else put is_private as not_private.
I am a bit new so I am not able to get how can i do this efficiently as entries are around 300k.
Please suggest some ways, pseudo code will be really helpful
To update a document a filter criteria. Check pseudo code below and follow link to read more.
You'll need to have an existing value for user_private
db.messages.updateMany([
{ "user_private" : 0 }, // filter.
{ $set: {
"user_private" : "private",
"is_moderator" : "true"
}
}, // update.
{
upsert: true
}
]);
upserts - Creates a new document if no documents match the filter or Updates documents that match the filter based on the filter and update parameters provided.
So I have this document within the course collection
{
"_id" : ObjectId("53580ff62e868947708073a9"),
"startDate" : ISODate("2014-04-23T19:08:32.401Z"),
"scoreId" : ObjectId("531f28fd495c533e5eaeb00b"),
"rewardId" : null,
"type" : "certificationCourse",
"description" : "This is a description",
"name" : "testingAutoSteps1",
"authorId" : ObjectId("532a121e518cf5402d5dc276"),
"steps" : [
{
"name" : "This is a step",
"description" : "This is a description",
"action" : "submitCategory",
"value" : "532368bc2ab8b9182716f339",
"statusId" : ObjectId("5357e26be86f746b68482c8a"),
"_id" : ObjectId("53580ff62e868947708073ac"),
"required" : true,
"quantity" : 1,
"userId" : [
ObjectId("53554b56e3a1e1dc17db903f")
]
},...
And I want to do is create a query that returns all courses that have a specific userId in the userId array that is in the steps array for a specific userId. I've tried using $elemMatch like so
Course.find({
"steps": {
"$elemMatch": {
"userId": {
"$elemMatch": "53554b56e3a1e1dc17db903f"
}
}
}
},
But It seems to be returning a empty document.
I think this will work for you, you have the syntax off a bit plus you need to use ObjectId():
db.Course.find({ steps : { $elemMatch: { userId:ObjectId("53554b56e3a1e1dc17db903f")} } })
The $elemMatch usage is not necessary unless you actually have compound sub-documents in that nested array element. And also is not necessary unless the value being referenced could possibly duplicate in another compound document.
Since this is an ObjectId we are talking about, then it's going to be unique, at least within this array. So just use the "dot-notation" form:
Course.find({
"steps.userId": ObjectId("53554b56e3a1e1dc17db903f")
},
Go back and look at the $elemMatch documentation. In this case, the direct "dot-notation" form is all you need