So what I want to do is group all documents having same hash whose count is more than 1 and only keep the oldest record according to startDate
My db structure is as follows:
[{
"_id": "82bacef1915f4a75e6a18406",
"Hash": "cdb3d507734383260b1d26bd3edcdfac",
"duration": 12,
"price": 999,
"purchaseType": "Complementary",
"startDate": {
"$date": {
"$numberLong": "1656409841000"
}
},
"endDate": {
"$date": {
"$numberLong": "1687859441000"
}
}
}]
I was using this query which I created
db.Mydb.aggregate([
{
"$group": {
_id: {hash: "$Hash"},
dups: { $addToSet: "$_id" } ,
count: { $sum : 1 }
}
},{"$sort":{startDate:-1}},
{
"$match": {
count: { "$gt": 1 }
}
}
]).forEach(function(doc) {
doc.dups.shift();
db.Mydb.deleteMany({
_id: {$in: doc.dups}
});
})
this gives a result like this:
{ _id: { hash: '1c01ef475d072f207c4485d0a6448334' },
dups:
[ '6307501ca03c94389f09b782',
'6307501ca03c94389f09b783',
'62bacef1915f4a75e6a18l06' ],
count: 3 }
The problem with this is that the _id's in dups array are random everytime I run this query i.e. not sorted according to startDate field.
What can be done here?
Any help is appreciated. Thanks!
After $group stage, startDate field will not pre present in the results, so you can not sort based on that field. So, as stated in the comments, you should put $sort stage first in the Aggregation pipeline.
db.Mydb.aggregate([
{
"$sort": { startDate: -1}
},
{
"$group": {
_id: {hash: "$Hash"},
dups: { $addToSet: "$_id" } ,
count: { $sum : 1 }
},
{
"$match": { count: { "$gt": 1 }
}
]
Got the solution. I was using $addToSet in the group pipeline stage which does not allow duplicate values. Instead, I used $push which allows duplicate elements in the array or set.
Related
I have an aggregation query like this:
...
{
'$unwind': {
path: '$modifier'
}
}, {
'$group': {
'_id': {
'date': {
'$dateToString': {
'format': '%d/%m/%Y',
'date': '$updatedTime'
}
}
},
'$$modifier': { '$sum': 1 }
}
},
...
and I would like to use modifier values, a result of the previous stage ($unwind), as a field in the next stage ($group). The detail is in the picture below. How should I accomplish it?
MongoDB aggregation question detailed picture
Current:
This is the output of $unwind stage:
updatedTime:2020-03-27T11:02:43.608+00:00
modifier:"james#email.com"
updatedTime:2020-03-27T11:02:43.608+00:00
modifier:"eric#email.com"
This is the output of $group stage :
_id: { date:"27/03/2020" }
modifier:1
Expected:
the output of $unwind stage:
updatedTime:2020-03-27T11:02:43.608+00:00
modifier:"james#email.com"
updatedTime:2020-03-27T11:02:43.608+00:00
modifier:"eric#email.com"
This is the output of $group stage:
_id: { date:"27/03/2022" }
james#email.com:1
eric#email.com:1
total:2
Notice that "james#email.com" and "eric#email.com" come from $unwind stage which is before $group stage. total is the total of modifier (examples: 'james#email.com' and 'eric#email.com') values.
Any suggestion is appreciated.
It's not straight but You need to group by both the properties date, email and do another group by only date and construct the array of modifiers and do replace that in root,
$group by updatedTime and modifier and get total count
$group by only date property and construct the array of object of modifier and count in key-value pair
$arrayToObject convert that key-value pair into object
$mergeObject to merge required properties like we added date property and above array to object operation result
$replaceRoot to replace above merged object in root of the document
{
"$group": {
"_id": {
"date": {
"$dateToString": {
"format": "%d/%m/%Y",
"date": "$updatedTime"
}
},
"modifier": "$modifier"
},
"count": { "$sum": 1 }
}
},
{
"$group": {
"_id": "$_id.date",
"modifiers": {
"$push": {
"k": "$_id.modifier",
"v": "$count"
}
},
"total": { "$sum": "$count" }
}
},
{
"$replaceRoot": {
"newRoot": {
"$mergeObjects": [
{
"date": "$_id",
"total": "$total"
},
{ "$arrayToObject": "$modifiers" }
]
}
}
}
I have made several efforts to select a single specific document that contains the minimum value from the database.
let Lowestdate = await BTCMongo.aggregate(
[
// { "$sort": { "name": 1,
{
$match : { "createdAt" : { $gte: new Date(last),$lte: new Date(NEW) } } },
{
$group:
{
_id:null,
minFee_doc:{$min: "$$ROOT"},
minFee: { $min:{$toDouble:"$one"}},
firstFee: { $first: "$one" },
lastFee: { $last: "$one" },
maxFee: { $max: {$toDouble:"$one"}},
}
},
]
).then(result => {}):
with minFee_doc:{$min: "$$ROOT"}, I have been trying to return the document containing the minimum $min but it keeps returning document containing $first
How do i select the document with minimum value?
Note : i will like to return the whole document including the "CreatedAt" "UpdatedAt", and _id. of the document containing the minimum value
Expected Result should look like:
{
"minFee_doc": {
"_id": "61e84c9f622642463640e05c",
"createdAt": "2022-01-19T17:38:39.034Z",
"updatedAt": "2022-04-24T14:48:38.100Z",
"__v": 0,
"one": 2
},
"minFee": 2,
"firstFee": 3,
"lastFee": 5,
"maxFee": 6
}
Edit: also to provide a single document not multiple
$push all docs in $group then $set the array with $filter
db.collection.aggregate([
{
$match: {}
},
{
$group: {
_id: null,
minFee_doc: { $push: "$$ROOT" },
minFee: { $min: { $toDouble: "$one" } },
firstFee: { $first: "$one" },
lastFee: { $last: "$one" },
maxFee: { $max: { $toDouble: "$one" } }
}
},
{
$set: {
minFee_doc: {
$filter: {
input: "$minFee_doc",
as: "m",
cond: { "$eq": [ "$$m.one", "$minFee" ] }
}
}
}
}
])
mongoplayground
I have a mongoose document having the following Schema:
Products
{
"section":"",
"category":"Food & Drink",
"sub_category":"Main Dish",
"product_code":"ST",
"title":"Steak",
"description":"Served with sauted vegetables",
"tags":[
],
"warranty":"None",
"product_variants":[
{
"variant_code":"ST1",
"variant_title":"Rib Eye",
"images":[
],
"status":"Active",
"variant_details":[
{
"size":"6oz",
"local_price":800,
"local_discount":"0",
"foreign_price":0,
"foreign_discount":"0",
"inventory":[
{
"branch_id":{
},
"quantity":94
}
]
},
{
"size":"10oz",
"local_price":1000,
"local_discount":"0",
"foreign_price":0,
"foreign_discount":"0",
"inventory":[
{
"branch_id":{
},
"quantity":147
}
]
},
{
"size":"12oz",
"local_price":1200,
"local_discount":"0",
"foreign_price":0,
"foreign_discount":"0",
"inventory":[
{
"branch_id":{
},
"quantity":199
}
]
}
]
}
]
}
The above document shows only one object in the product_variants field but please note that there could be several objects as well. I need to sum the quantity for each size and product variant.
How would I do that using aggregate function? I am using mongoose in node js environment.
Query
(its based on the last comment in the previous answer, similar query but multiplies that quantity with the local price)
Test code here
db.collection.aggregate([
{
"$unwind": "$product_variants"
},
{
"$unwind": "$product_variants.variant_details"
},
{
"$unwind": "$product_variants.variant_details.inventory"
},
{
"$set": {
"total_local_price": {
"$multiply": [
"$product_variants.variant_details.inventory.quantity",
"$product_variants.variant_details.local_price"
]
}
}
},
{
$group: {
_id: null, // or "$_id" if you want only for 1 document
total_qty: {
$sum: "$total_local_price"
}
}
}
])
You can use this aggregation query:
Fisrt $project to get only the quantity values. It generates the following output:
"array": [
[
[
94
],
[
147
],
[
199
]
]
So next step is to use $unwind three times to flat the array.
And $group by _id using $sum
yourModel.aggregate([{
"$project": {
"array": "$product_variants.variant_details.inventory.quantity"
}
},
{
"$unwind": "$array"
},
{
"$unwind": "$array"
},
{
"$unwind": "$array"
},
{
"$group": {
"_id": "$_id",
"size": {
"$sum": "$array"
}
}
}])
Example here
Edit
As Takis _ suggested into the comments if you want to get all values from your entire collection (not only for each document) you can $group using null as this example
I have an aggregation query like this
nSkip=4, count=25, sortOn='first_name' , order=-1, toMatch='biker' // all variables are dynamic
query={status: true, roles: { $regex: toMatchRole, $options: "m" }} // also dynamic
User.aggregate([
{
$match: query
},
// after this I need the total number of documents that matched the criteria,
// before sorting or skipping or limiting in "total_count" variable
{
$sort: {
[sortOn]: order
}
},
{
$skip: nSkip
},
{
$limit: count
},
{
$project: {
last_name: 1,
first_name: 1,
email: 1
}
}
])
User Collection
{
_id:60befdcfa4198332b728f9cd",
status:false,
roles:["biker"],
email:"john#textmercato.com",
last_name:"aggr",
first_name:"john",
}
I am not sure how to achieve this without disturbing the rest of the stages in aggregation. Can someone please help me out.
You can use $group
{
"$group": {
"_id": null,
"data": { "$push": "$$ROOT" },
"total_count": { $sum: 1 }
}
},
{ $unwind: "$data" },
{
"$replaceRoot": {
"newRoot": {
"$mergeObjects": [ "$$ROOT", "$data" ]
}
}
}
and finally Project the total_count
Working Mongo playground
Please i am new to node js and MongoDB.
When i want to retrieve a post by id, i want to be able to retrieve the previous post and next post also.
this is my post, it only retrieves the current post by id.
Post.findById(req.params.postId)
.then((existingpost) => {
console.log(Post.find(req.params.postId))
if (existingpost) {
res.send(existingpost);
}
return res.status(404).send({
message: "Post does not exist with id " + req.params.postId,
});
})
.catch((err) => {
if (err.kind === "ObjectId") {
return res.status(404).send({
message: "Post does not exist with id " + req.params.postId,
});
}
return res.status(500).send({
message:
"Some error occurred while retrieving the post with postId " +
req.params.postId,
});
});
};
I currently receive the object with the id like this which is fine.
{
"_id": "6009f3e294d8a033402a76e7",
"title": "Covid 19 in Italy",
"author": "John Doe",
"createdAt": "2021-01-21T21:36:34.514Z",
"updatedAt": "2021-01-21T21:36:34.514Z",
"__v": 0
}
But i will love to receive the object of the current id, the previous object and the next object.
something like this.
[{
"_id": "3230g5e382d8a033402a76e7",
"title": "Effect of Covid on the Economy",
"author": "John Doe",
"createdAt": "2021-01-21T21:36:34.514Z",
"updatedAt": "2021-01-21T21:36:34.514Z",
"__v": 0
},
{
"_id": "6009f3e294d8a033402a76e7",
"title": "Covid 19 in Italy",
"author": "John Doe",
"createdAt": "2021-01-21T21:36:34.514Z",
"updatedAt": "2021-01-21T21:36:34.514Z",
"__v": 0
},
{
"_id": "4567hye294d8a033402a76e7",
"title": "Life after Covid",
"author": "John Doe",
"createdAt": "2021-01-21T21:36:34.514Z",
"updatedAt": "2021-01-21T21:36:34.514Z",
"__v": 0
}]
Since its a UUID, this approach might help you..
$sort to sort the documents by asc
$group and $unwind to get the index
$facet to categorize the incoming data into current and allDocs
We know current is only one object, so we do $unwind to deconstruct the array
We already know the index, so we use $filter to get prev, current and next using index
$unwind to deconstruct the array
$replaceRoot to make the objects to the root
Here is the script
db.collection.aggregate([
$sort: { createdAt: 1 } },
{
$group: {
_id: null,
data: { $push: "$$ROOT"}
}
},
{ $unwind: { path: "$data", includeArrayIndex: "index" } },
{
$facet: {
current: [
{ $match: { "data._id": "3230g5e382d8a033402a76e7" } }
],
allDocs: [
{ $match: {} }
]
}
},
{
$unwind: "$current"
},
{
$project: {
docs: {
$filter: {
input: "$allDocs",
cond: {
$or: [
{ $eq: [ "$$this.index", { $subtract: [ "$current.index", 1 ] } ] },
{ $eq: [ "$$this.index", "$current.index" ] },
{ $eq: [ "$$this.index", { $add: [ "$current.index", 1 ] } ] }
]
}
}
}
}
},
{ "$unwind": "$docs" },
{ "$replaceRoot": { "newRoot": "$docs.data" } }
])
Working Mongo playground
There are many ways to do this, this is one of the way. If you feel you have a lot of document, then try to avoid $unwind which is expensive, in that case you can try using createdDate instead of index
I am not sure is there any straight way to do this, you can try aggregation,
Using UUID and CreatedAt:
$facet to get all documents in all after sorting in ascending order by createdAt
$let to define vars states with start and total documents,
$cond check condition if index of input uuid is zero then return start: 0 and total: 2 documents we have to slice from all array, else get current index and subtract minus 1 and total: 3
in to return slice documents on the base of start and total
Post.aggregate([
{ $facet: { all: [{ $sort: { createdAt: 1 } }] } },
{
$project: {
result: {
$let: {
vars: {
states: {
$cond: [
{ $eq: [{ $indexOfArray: ["$all._id", req.params.postId] }, 0] },
{ start: 0, total: 2 },
{
start: {
$subtract: [{ $indexOfArray: ["$all._id", req.params.postId] }, 1]
},
total: 3
}
]
}
},
in: { $slice: ["$all", "$$states.start", "$$states.total"] }
}
}
}
}
])
Playground
Using ObjectID:
convert your string input id req.params.postId to object id using mongoose.Types.ObjectId
$facet to separate result,
first, $match to get current and next documents, $sort _id in descending order, $limit 2
second, $match to get previous document, $sort _id in descending order, $limit 1
$project to get result after concat both array first and second using $concatArrays
req.params.postId = mongoose.Types.ObjectId(req.params.postId);
Post.aggregate([
{
$facet: {
first: [
{ $match: { _id: { $gte: req.params.postId } } },
{ $sort: { _id: 1 } },
{ $limit: 2 }
],
second: [
{ $match: { _id: { $lt: req.params.postId } } },
{ $sort: { _id: -1 } },
{ $limit: 1 }
]
}
},
{ $project: { result: { $concatArrays: ["$first", "$second"] } } }
])
Playground