I have a car model given as below
{
"_id": "54b8a71843286774060b8bed",
"name": "Car1",
"active": true,
"model": [
{
"name": "Model1",
"active": true,
"_id": "54b8a71843286774060b8bee",
"available": [
{
"Day": "Mon",
"quantity": "6"
},
{
"Day": "Tue",
"quantity": "6"
},
{
"Day": "Wed",
"quantity": "6"
},
{
"Day": "Thurs",
"quantity": "6"
},
{
"Day": "Fri",
"quantity": "0"
}
]
},
{
"name": "Model2",
"active": true,
"_id": "54b8a71843286774060b8bef",
"available": [
{
"Day": "Mon",
"quantity": "6"
},
{
"Day": "Tue",
"quantity": "6"
},
{
"Day": "Wed",
"quantity": "6"
},
{
"Day": "Thurs",
"quantity": "6"
},
{
"Day": "Fri",
"quantity": "6"
}
]
},
{
"name": "Model3",
"active": true,
"_id": "54b8a71843286774060b8beg",
"available": [
{
"Day": "Mon",
"quantity": "6"
},
{
"Day": "Tue",
"quantity": "6"
},
{
"Day": "Wed",
"quantity": "6"
},
{
"Day": "Thurs",
"quantity": "6"
},
{
"Day": "Fri",
"quantity": "0"
}
]
}
]
}
I am trying to search availability of car on given days.
Like if I select Friday then it should return me cars whose quantity more than 0 on Friday but currently it is returning all the cars having quantity 0 as well.
I have written query as below
Car.find({
'active': true,
'model.available': {
$elemMatch: {
quantity: {$gte : 1}
}
}
})
But it returning documents those are having quantity 0 also.
For this, you'll need the aggregation pipeline.
The following code snippet does this:
Find all documents with at least one matching model.
Split up the documents: a document with an array of 3 models in it gets turned into three documents with one model each:
{name: "Car1": 1, models: [{name: "Model1"}, {name: "Model2"}, {name: "Model3"}]}
Becomes:
{name: "Car1", models: {name: "Model1"}} & {name: "Car1", models: {name: "Model2"}} & {name: "Car1", models: {name: "Model3"}}.
The split up documents are filtered (again) on quantity and day.
Optionally, glue the documents back together again. You might not need that in your application.
db.cars.aggregate([
// First, we do your query, which will return
// every document that has _at least one_
// model that is available.
{$match: {
'active': true,
'model.available': {$elemMatch: {
'quantity': {$gte: 1},
'Day': 'Fri'
}}
}},
// We split up the found documents,
// every document will now have exactly
// one 'model' in it.
{$unwind: "$model"},
// We now filter the split documents.
{$match: {
'model.available': {$elemMatch: {
'quantity': {$gte: 1},
'Day': 'Fri'
}}
}},
// If you want, you can now glue the
// models back together again.
{$group: {
_id: "$_id", // Group all documents with the same _id
name: {$first: "$name"},
active: {$first: "$active"},
model: {$push: "$model"} // Make an array of models
}}
])
Important note: For $gte to work, you'll need to store your quantity as a Number, not a String. Since your example has the numbers stored as strings, you might want to double check them in your database.
Related
I have a large collection of documents that look as follows:
{ "_id": "5a760191813a54000b8475f1", "orders": [{ "row": "3", "seat": "11" }, { "row": "3", "seat": "12" }], "product_id": "5a7628bedbcc42000aa7f614" },
{ "_id": "5a75f6f17abe45000a3ba05e", "orders": [{ "row": "3", "seat": "12" }, { "row": "3", "seat": "13" }], "product_id": "5a7628bedbcc42000aa7f614" },
{ "_id": "5a75ebdf813a54000b8475e7", "orders": [{ "row": "5", "seat": "16" }, { "row": "5", "seat": "15" }], "product_id": "5a75f711dbcc42000c459efc" }
I need to be able to find any documents where the product_id and items in the orders array are duplicates. I can't quite seem to wrap my head around accomplishing this. Any pointers?
I don't know what output you want, but this has the information about the duplicates, maybe you want to add unwind on duplicates also.
Result documents
product_id
order (that found duplicated)
duplicates (the documents that had that order as duplicate)
For your data would print
[{
"duplicates": [
"5a760191813a54000b8475f1",
"5a75f6f17abe45000a3ba05e"
],
"order": {
"row": "3",
"seat": "12"
},
"product_id": "5a7628bedbcc42000aa7f614"
}]
Query
(run it on your driver, MongoPlayground doesn't keep the order of fields and can show wrong results)
aggregate(
[{"$unwind" : {"path" : "$orders"}},
{
"$group" : {
"_id" : {
"orders" : "$orders",
"product_id" : "$product_id"
},
"duplicates" : {
"$push" : "$_id"
}
}
},
{"$match" : {"$expr" : {"$gt" : [ {"$size" : "$duplicates"}, 1 ]}}},
{
"$project" : {
"_id" : 0,
"order" : "$_id.orders",
"product_id" : "$_id.product_id",
"duplicates" : 1
}
}
])
Data (i added some more data)
[
{
"_id": "5a760191813a54000b8475f1",
"orders": [
{
"row": "3",
"seat": "11"
},
{
"row": "3",
"seat": "12"
}
],
"product_id": "5a7628bedbcc42000aa7f614"
},
{
"_id": "5a75f6f17abe45000a3ba05g",
"orders": [
{
"row": "3",
"seat": "12"
},
{
"row": "3",
"seat": "13"
}
],
"product_id": "5a7628bedbcc42000aa7f614"
},
{
"_id": "5a75f6f17abe45000a3ba05e",
"orders": [
{
"row": "3",
"seat": "12"
},
{
"row": "3",
"seat": "13"
}
],
"product_id": "5a7628bedbcc42000aa7f614"
},
{
"_id": "5a75ebdf813a54000b8475e7",
"orders": [
{
"row": "5",
"seat": "16"
},
{
"row": "5",
"seat": "15"
}
],
"product_id": "5a75f711dbcc42000c459efc"
}
]
Results
[{
"duplicates": [
"5a75f6f17abe45000a3ba05g",
"5a75f6f17abe45000a3ba05e"
],
"order": {
"row": "3",
"seat": "13"
},
"product_id": "5a7628bedbcc42000aa7f614"
},
{
"duplicates": [
"5a760191813a54000b8475f1",
"5a75f6f17abe45000a3ba05g",
"5a75f6f17abe45000a3ba05e"
],
"order": {
"row": "3",
"seat": "12"
},
"product_id": "5a7628bedbcc42000aa7f614"
}]
You could use below query. $unwind the orders array, $group by order row and product and collect matching ids and count. Keep the documents where count is greater than 1. $lookup to pull in the matching documents by id and $replaceRoot to flatten the documents.
db.collection.aggregate([
{
"$unwind": "$orders"
},
{
"$group": {
"_id": {
"order": "$orders",
"product_id": "$product_id"
},
"count": {
"$sum": 1
},
"doc_ids": {
"$push": "$_id"
}
}
},
{
"$match": {
"count": {
"$gt": 1
}
}
},
{
"$lookup": {
"from": "collection",
"localField": "doc_ids",
"foreignField": "_id",
"as": "documents"
}
},
{
"$unwind": "$documents"
},
{
"$replaceRoot": {
"newRoot": "$documents"
}
}
])
https://mongoplayground.net/p/YbztEGttUMx
While this can be done purely in Mongo I do not recommend it as it's very very very memory inefficient. you basically have to hold the entire collection in memory the entire time while you do certain manipulations on it.
I will however show the pipeline for this because we will use it with the second more scaleable approach.
We want to $group based on orders and product_id, however there are 2 issues standing in our way.
The orders field might not be sorted the same in all documents, because Mongo does not support "nested" sorting we have to $unwind the array, $sort it and restore the original structure. ( mind you you're sorting the entire collection here in memory ). This step which is one of the pain points of this pipeline can be skipped if you can ensure sort order is maintained in the orders array.
Mongo is inconsistent when $grouping an array of objects. full disclosure I'm not entirely sure what's going on in there but I'm guessing there are some "shortcuts" done for efficiency which affects the stability somehow. So our approach will be to convert these objects into a string (concating the "row" and "seat" together).
db.collection.aggregate([
{
"$unwind": "$orders"
},
{
$sort: {
"orders.row": 1,
"orders.seat": 1
}
},
{
$group: {
_id: "$_id",
tmpOrders: {
$push: {
$concat: [
"$orders.row",
"$orders.seat"
]
}
},
product_id: {
$first: "$product_id"
}
}
},
{
$group: {
_id: {
orders: "$tmpOrders",
product: "$product_id"
},
dupIds: {
$push: "$_id"
}
}
},
{
$match: {
"dupIds.0": {
$exists: true
}
}
},
{
$project: {
_id: 0,
dups: "$dupIds",
}
}
])
Mongo Playground
Now as I said this approach is not scaleable, and on large collections will take a very long time to run. So I recommend utilizing indexes and iterating over product_id's and executing each pipeline separately.
// wraps the native Promise, not required.
import Bluebird = require('bluebird');
// very fast with index.
const productIds = await collection.distinct('product_id')
await Bluebird.map(productIds, async (productId) => {
const dups = await collection.aggregate([
{
$match: {
product_id: productId
}
}
... same pipeline ...
])
if (dups.length) {
// logic required.
}
// can control concurrency based on db workload.
}, { concurrency: 5})
Make sure with this approach you have an index built on product_id so it will work efficiently.
My product document looks thus:
{
"_id": {
"$oid": "60999af1160b0eebed51f203"
},
"name": "Olive Skin care On1",
"business": {
"$oid": "609fa1d25adc157a33c59098"
},
"ratings": [{
"_id": {
"$oid": "60bdb541d6212ec44e62273c"
},
"user": {
"$oid": "5fdce4bd75dbe4864fcd5001"
},
"rating": 5
}]
}
I have this mongoose query to get product details alongside the product rating. Some products have ratings field while others do not. When I make a query as shown here, it returns a response as expected with calculated average rating. The response looks thus:
[
{
"_id": "609a657f2bf43c290fb22df8",
"name": "Olive babay Oil",
"business": "6079ed084d9ab0c3171317ea",
"averageRating": 5
}
]
Here is the query:
const productArray = await Product.aggregate([
{
$match: {
_id: mongooseProductId,
},
},
{ $unwind: "$ratings" },
{
$project: {
averageRating: { $avg: "$ratings.rating" },
name: 1,
business: 1,
},
},
]);
However if the same product above is modified by removing the ratings field, the query below will return an empty array.
How do I write my query to ensure that whether the ratings field exists or not, I do not get an empty array provided that the matching criteria is met.
Meaning that I can get an expected response like this when the ratings field doesn't exist on my product document:
[
{
"_id": "609a657f2bf43c290fb22df8",
"name": "Olive babay Oil",
"business": "6079ed084d9ab0c3171317ea",
"averageRating": null
}
]
And this when the rating field exists:
[
{
"_id": "609a657f2bf43c290fb22df8",
"name": "Olive babay Oil",
"business": "6079ed084d9ab0c3171317ea",
"averageRating": 5
}
]
Based on #turivishal's comment. The query below solved the problem.
const productArray = await Product.aggregate([
{
$match: {
_id: mongooseProductId,
},
},
{ $unwind:{ path: "$ratings", preserveNullAndEmptyArrays: true } },
{
$project: {
averageRating: { $avg: "$ratings.rating" },
name: 1,
business: 1,
},
},
]);
I have a collection in my MongoDB database that stores durations for people who are in groups, it looks a like this:
[{
"_id": "5c378eecd11e570240a9b0ac",
"state": "DRAFT",
"groupId": "5c378eebd11e570240a9ae49",
"personId": "5c378eebd11e570240a9aee1",
"date": "2019-01-07T00:00:00.000Z",
"duration": 480,
"__v": 0
},
{
"_id": "5c378eecd11e570240a9b0bb",
"state": "DRAFT",
"groupId": "5c378eebd11e570240a9ae58",
"personId": "5c378eebd11e570240a9aeac",
"date": "2019-01-07T00:00:00.000Z",
"duration": 480,
"__v": 0
},
{
"_id": "5c378eecd11e570240a9b0c5",
"state": "DRAFT",
"groupId": "5c378eebd11e570240a9ae3e",
"personId": "5c378eebd11e570240a9aef6",
"date": "2019-01-07T00:00:00.000Z",
"duration": 480,
"__v": 0
}]
I would like to be able to run an aggregate query which returns a collection of personIds and the duration grouped per day with the corresponding groupId, which would look like this:
[{
"personId": "5c378eebd11e570240a9aee1",
"time": [{
"date": "2019-01-07T00:00:00.000Z",
"entries": [{
"groupId": "5c378eebd11e570240a9ae49",
"duration": 480,
"state": "DRAFT"
}]
}]
}, {
"personId": "5c378eebd11e570240a9aeac",
"time": [{
"date": "2019-01-07T00:00:00.000Z",
"entries": [{
"groupId": "5c378eebd11e570240a9ae58",
"duration": 480,
"state": "DRAFT"
}]
}]
}, {
"personId": "5c378eebd11e570240a9aef6",
"time": [{
"date": "2019-01-07T00:00:00.000Z",
"entries": [{
"groupId": "5c378eebd11e570240a9ae3e",
"duration": 480,
"state": "DRAFT"
}]
}]
}]
So far, I have written the following aggregation (I'm using Mongoose, hence the syntax):
Time.aggregate()
.match({ date: { $gte: new Date(start), $lte: new Date(end) } })
.group({
_id: '$personId',
time: { $push: { date: '$date', duration: '$duration', state: '$state' } },
})
.project({ _id: false, personId: '$_id', time: '$time' })
Which returns the following:
[{
"personId": "5c378eebd11e570240a9aed1",
"time": [{
"date": "2019-01-11T00:00:00.000Z",
"duration": 480,
"state": "DRAFT"
}, {
"date": "2019-01-11T00:00:00.000Z",
"duration": 480,
"state": "DRAFT"
}
// ...
}]
Hopefully you can see that the durations are being grouped by personId but I've not been able to figure out how to apply another grouping to the time array as the dates are duplicated if a personId has more than one duration for a given date.
Is it possible to group by and ID, push to an array and then group the values in that array as an aggregation or will my application need to map/reduce the results instead?
I would suggest running two $group operations in a row:
db.time.aggregate({
// first, group all entries by personId and date
$group: {
_id: {
personId: "$personId",
date: "$date"
},
entries: {
$push: {
groupId: "$groupId",
duration: "$duration",
state: "$state"
}
}
}
}, {
// then, group previously aggregated entries by personId
$group: {
_id: "$_id.personId",
time: {
$push: {
date: "$_id.date",
entries: "$entries"
}
}
}
}, {
// finally, rename _id to personId
$project: {
_id: 0,
personId: "$_id",
time: "$time"
}
})
In Mongoose it should be something like that:
Time.aggregate()
.match({
date: {
$gte: new Date(start),
$lte: new Date(end)
}
})
.group({
_id: {
personId: '$personId',
date: '$date'
},
entries: {
$push: {
groupId: '$groupId',
duration: '$duration',
state: '$state'
}
}
})
.group({
_id: '$_id.personId',
time: {
$push: {
date: '$_id.date',
entries: '$entries'
}
}
})
.project({
_id: false,
personId: '$_id',
time: '$time'
})
db.getCollection("dummyCollection").aggregate(
[
{
"$group" : {
"_id" : "$personId",
"time" : {
"$push" : {
"date" : "$date",
"duration" : "$duration",
"state" : "$state"
}
}
}
},
{
"$project" : {
"_id" : false,
"personId" : "$_id",
"time" : "$time"
}
},
{
"$unwind" : "$time"
},
{
"$group" : {
"_id" : "$time.date",
"time" : {
"$addToSet" : "$time"
}
}
}
]
);
Use $addToSet which returns an array of all unique values that results from applying an expression to each document in a group of documents that share the same group by key.
I have a model that looks like:
fname: String,
lname: String,
rating: [{
rating: {
type: Number,
enum: RATING,
default: 5
},
date: {
type: Date,
default: Date.now
}
}]
I need to perform updates on this Model by adding new object inside the rating array, with new ratings and dates. I would like to use the bulkwrite method on Model.collection to do this because I need to enable bulk updates so that I don't have to update them one by one.
I created an array bulkUpdateOperations = [] and did the following in a loop:
bulkUpdateOperations.push({
'updateOne': {
'filter': {'_id': item.id},
'update': {$push: {rating: {'rating': item.rating, 'date': Date.now}}}
}
});
Person.collection.bulkWrite(bulkUpdateOperations, {orderd: true, w: 1}, callbackfunc);
But nothing gets updated. I get the following response:
...
...
...
insertedCount: 0,
matchedCount: 0,
modifiedCount: 0,
deletedCount: 0,
upsertedCount: 0,
upsertedIds: {},
insertedIds: {},
n: 0 }
I would be very thankful if someone helped me through this problem.
EDIT
Here is the array I'm sending as POST body to update the records:
[{
"id": "5b7d4d348151700014d25bdd",
"rating": 1
},{
"id": "5b771d10c1e03e1e78b854c2",
"rating": 1
},{
"id": "5b771d7ac1e03e1e78b854c8",
"rating": 1
},{
"id": "5b7bd75a33f88c1af8585be0",
"rating": 1
},{
"id": "5b814a2322236100142ac9f6",
"rating": 1
}]
And here is a sample collection in the DB
{
"_id": {
"$oid": "5b7d4d348151700014d25bdd"
},
"status": "ACTIVE",
"fname": "mr. client",
"lname": "good client",
"contact_info": {
"_id": {
"$oid": "5b7d4d348151700014d25bde"
},
"mobile_no": "0011223344",
"phone_no": "11223344"
},
"handlers": [
{
"_id": {
"$oid": "5b7d4d348151700014d25bdf"
},
"date": {
"$date": "2018-08-22T11:47:00.544Z"
},
"id": {
"$oid": "5b7d45fbfb6fc007d8bdc1f4"
}
}
],
"onboarding_date": {
"$date": "2018-08-22T11:47:00.551Z"
},
"rating": [
{
"rating": 5,
"_id": {
"$oid": "5b814a8e22236100142ac9fc"
},
"date": {
"$date": "2018-08-25T12:22:59.584Z"
}
},
{
"rating": 3,
"_id": {
"$oid": "5b814a8e22236100142ac9fb"
},
"date": {
"$date": "2018-08-25T12:24:46.368Z"
}
}
],
"__v": 0
}
EDIT
Adding upsert: true as a filter for updateOne creates a new document with only rating as its value.
SOLUTION
replace
'filter': {'_id': item.id},
by
'filter': {'_id': mongoose.Types.ObjectId(item.id)},
changing
bulkUpdateOperations.push({
'updateOne': {
'filter': {'_id': item.id},
'update': {$push: {rating: {'rating': item.rating, 'date': Date.now}}}
}
});
to
bulkUpdateOperations.push({
'updateOne': {
'filter': {'_id': mongoose.Types.ObjectId(item.id)},
'update': {$push: {rating: {'rating': item.rating, 'date': Date.now}}}
}
});
worked. Notice the type cast I had to manually perform in
'filter': {'_id': mongoose.Types.ObjectId(item.id)},
I thought mongoose would automatically cast the string to an ObjectId type, but maybe because I'm dropping down a level of abstraction by using Person.collection, mongoose did not auto-cast the itemId.
Please feel free to update this answer if anyone can confirm why I had to cast the string manually.
Suppose we have 10 collections, then we have to find the count on the basis of tag_id. For example, if tag_id contains 0 and 1, then we have to count all the data, as well as counting the data that don't have tag_id, or where tag_id is null. Then if it has unread : false then the output comes, count of all the unread.
Find the counts of tag_id and counts of unread when false.
{
"_id": ObjectId("5912c7240520df77f0c2c18a"),
"email_id": "54",
"unread": "false",
"__v": NumberLong(0),
"tag_id": ["0"
]
}, {
"_id": ObjectId("5912c71e0520df77f0c2c189"),
"email_id": "55",
"unread": "false",
"__v": NumberLong(0),
"tag_id": [
"1"
]
}, {
"_id": ObjectId("5912c71d0520df77f0c2c186"),
"email_id": "51",
"unread": "false",
"__v": NumberLong(0),
"tag_id": [
"2", "1"
]
}
expected result:
{
"data": [{
"tag_id": "1",
"count_email": 1,(count of email on the basis of tag_id)
"unread": 9(count the unread on the basis of output of tag_id)
}, {
"tag_id": "3",
"count_email": 45,
"unread": 3
}, {
"tag_id": "2",
"count_email": 5,
"unread": 4
}, {
"id": null,
"count_email": 52,
"unread": 35
}]
}
Try the following code
Refer - https://docs.mongodb.com/manual/reference/operator/aggregation/eq/
https://docs.mongodb.com/manual/reference/operator/aggregation/cond/
https://docs.mongodb.com/manual/reference/operator/aggregation/group/
DB.aggregate([
{$project:
{
tag_id: '$tag_id',
unreadcount: { $cond: [ { $eq: [ '$unread', 'true' ] }, 1, 0 ] }
}},
{ $group: {
_id: '$tag_id',
unread: { $sum: '$unreadcount'},
}}
], function (err, results) {
console.log(results);
})