hope you are doing well and Happy New Year to everybody. Hope it will end up better than 2020 X). I am facing an issue here that I can not find a solution so far. Here is a data MongoDB model example :
`{ "_id" : ObjectId("5fe715bf43c3ca0009503f1d"),
"firstname" : "Nicolas ",
"lastname" : "Mazzoleni",
"email" : "nicolas.mazzoleni#orange.fr",
"items" : [
{
"item" : ObjectId("5f6c5422eeaa1364b0d7b267"),
"createdAt" : "2020-12-26T10:51:43.685Z"
},
{
"item" : ObjectId("5f6c5422eeaa1364b0d7b270"),
"createdAt" : "2021-01-04T09:21:46.260Z"
}
],
"createdAt" : ISODate("2020-12-26T10:51:43.686Z"),
"updatedAt" : ISODate("2021-01-04T09:21:46.260Z"),
"__v" : 0
}`
Let's assume that I have a lot of rows like this one above, with different 'items.createdAt' dates.
At the end, the goal is to do a query that would find an item where the ObjectId is equal to what I am looking for, and sort by the matching 'items.createdAt' date. Here is the query I use at the moment that is not sorting by the matching nested object :
`const items = await repo
.find({ 'items.item': ObjectId("5f6c5422eeaa1364b0d7b270") })
.sort({ 'items.createdAt': -1 })
.limit(5)`
PS : I also tried to use aggregations that ended up with this query
`const items = await repo.aggregate([
{ $unwind: '$items' },
{ $match: { 'items.item': ObjectId("5f6c5422eeaa1364b0d7b270") } },
{ $sort: { 'items.createdAt': -1 } },
{ $limit: 5 },
])`
This one is working like a charm, but it does not use indexes, and I am searching into millions of records, which makes this query way to long to execute.
Thank you for all your help ! Nicolas
Unfortunately, MongoDB can't sort nested array by simple Query. You need to use aggregate for that.
db.collection.aggregate([
{
"$match": {
"items.item": ObjectId("5f6c5422eeaa1364b0d7b270")
}
},
{
"$unwind": "$items"
},
{
"$sort": {
"items.createdAt": -1
}
},
{ "$limit" : 5 },
{
"$group": {
"items": {
"$push": "$items"
},
"_id": 1
}
])
Related
I have a test document like that:
{
"_id" : ObjectId("5fb6b0ed9cad6e97cfc24c2d"),
"dates" : [
{
"date" : ISODate("2020-01-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-02-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-03-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-04-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-05-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-06-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-07-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-08-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-09-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-10-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-11-01T00:00:00.000Z")
},
{
"date" : ISODate("2020-12-01T00:00:00.000Z")
}
]
}
Now I want to retrieve only dates $gt: 2020-14-01T00:00:00.000Z. I tried a lot of combinations but none of them worked for me.
This is one of the queries I tried (taken from Mongodb docs):
db.getCollection('things').find({_id: ObjectId("5fb6b0ed9cad6e97cfc24c2d"), "dates.date": { $gt: new Date("2020-04-01T00:00:00.000Z")} } )
But it return all the document, not the gt... i tried using new Date and new ISODate too, but same effect.
Thank you.
First of all, according to mongo documentation dates are in the format "<YYYY-mm-dd>".
Also more formats are allowed, but If you try to use 2020-14-01 as date it will fail (unless you convert string to date with an specific format) because month is 14.
But, answering the question, you need a query like this:
EDITED
db.collection.aggregate({
"$match": {
"_id": ObjectId("5fb6b0ed9cad6e97cfc24c2d"),
}
},
{
"$project": {
"dates": {
"$filter": {
"input": "$dates",
"as": "item",
"cond": {
"$gt": [
"$$item.date",
ISODate("2020-01-14T00:00:00.000Z")
]
}
}
}
}
})
First $match by _id to get only the document you want. And then using $project to create the fields you want to get. You can filter in the array those values whose field date is greater than your date using $filter and $gt
Note that I've used 2020-01-14 to avoid errors.
Example here.
Also another example using $dateFromString in this query.
Edit: You can use $dateFromString and specify a format. Check this example
Below is the sample document of a collection, say "CollectionA"
{
"_id" : ObjectId("5ec3f19225701c4f7ab11a5f"),
"workshop" : ObjectId("5ebd37a3d33055331eb4730f"),
"participant" : ObjectId("5ebd382dd33055331eb47310"),
"status" : "analyzed",
"createdBy" : ObjectId("5eb7aa24d33055331eb4728c"),
"updatedBy" : ObjectId("5eb7aa24d33055331eb4728c"),
"results" : [
{
"analyze_by" : {
"user_name" : "m",
"user_id" : "5eb7aa24d33055331eb4728c"
},
"category_list" : [
"Communication",
"Controlling",
"Leading",
"Organizing",
"Planning",
"Staffing"
],
"analyzed_date" : ISODate("2020-05-19T14:48:49.993Z"),
}
],
"summary" : [],
"isDeleted" : false,
"isActive" : true,
"updatedDate" : ISODate("2020-05-19T14:48:50.827Z"),
"createdDate" : ISODate("2020-05-19T14:47:46.374Z"),
"__v" : 0
}
I need to query all the documents to get the "results" array length and return a sum of all document's "results" length.
For example,
document 1 has "results" length - 5
document 2 has "results" length - 6
then output should be 11.
Can we write a query, instead of getting all, iterating and the adding the results length??
If I had understand clearly you would like to project the length of the result attribute.
So you should check the $size operator would work for you.
https://docs.mongodb.com/manual/reference/operator/aggregation/size/
You can use $group and $sum to calculate the total size of a field which contains the size of your results array. To create the field, You can use $size in $addFields to calculate the size of results in each document and put it the field. As below:
db.getCollection('your_collection').aggregate([
{
$addFields: {
result_length: { $size: "$results"}
}
},
{
$group: {
_id: '',
total_result_length: { $sum: '$result_length' }
}
}
])
You use an aggregation grouping query with $sum and $size aggregation operators to get the total sum of array elements size for all documents in the collection.
db.collection.aggregate( [
{
$group: {
_id: null,
total_count: { $sum: { $size: "$results" } }
}
}
] )
Aggregation using Mongoose's Model.aggregate():
SomeModel.aggregate([
{
$group: {
_id: null,
total_count: { $sum: { $size: "$results" } }
}
}
]).
then(function (result) {
console.log(result);
});
I am working in a mongoDB + node (with express and mongoose) API.
I have a document with more or less this structure:
// npcs
{
"_id" : ObjectId("5ea6c0f88e8ecfd3cdc39eae"),
"flavor" : {
"gender" : "...",
"description" : "...",
"imageUrl" : "...",
"class" : "...",
"campaign" : [
{
"campaignId" : "5eac9dfe8e8ecfd3cdc41aa0",
"unlocked" : true
}
]
},
},
// ...
And a second document in a separate table that is as follows:
// user
{
"_id" : ObjectId("5e987f8e4b88382a98c84042"),
"username" : "KuluGary",
"campaigns" : [
"5eac9dfe8e8ecfd3cdc41aa0",
"5eac9e458e8ecfd3cdc41ac1",
"5eac9e978e8ecfd3cdc41adb",
"5eac9eae8e8ecfd3cdc41ae3"
]
}
What I want to do is make a query in which I obtain all the NPCs that are a part of a campaign the user is part of, and are unlocked. The second part is fairly easy, just thought of once I retrieve the NPCs to filter those with unclocked false, but I'm having a hard time visualizing the query since I'm fairly unfamiliar with mongoDBs syntax and usage.
Any help would be greatly appreciated.
I understand you want to "join" a user with all relevant NPC's?
A simple aggregation with $lookup would work:
db.userCollection.aggregate([
{
$match: {
// match relevant users with whatever condition you want
}
},
{
$lookup: {
from: "npc_collection",
let: {campaigns: "$campaigns"},
pipeline: [
{
$match: {
$expr: {
$gt: [
{
$size: {
$filter: {
input: "$flavor.campaign",
as: "campaign",
cond: {
$and: [
{$setIsSubset: ["$flavor.campaign.campaignId", "$$campaigns"]},
{$eq: ["$$campaign.unlocked", true]}
]
}
}
}
},
0
]
}
}
}
],
as: "relevant_npcs"
}
}
])
Note that due to the need of an NPC to be active in a specific campaign and not just a unlocked in any we require the use of $filter.
I recommend that if you only want to lookup on one user you split this into 2 calls as i feel using $elemMatch would give better performance:
let campaigns = await db.userCollection.distinct("campaigns", {_id: userId})
let results = await db.npcCollection.find({"flavor.campaign": {$elemMatch: { campaignId: {$in: campaigns}, unlocked: true}}})
I have Post collection as like as following:
{ "_id" : ObjectId(..), "date" : ISODate("2014-03-01T08:00:00Z") }
{ "_id" : ObjectId(..), "date" : ISODate("2014-03-01T09:00:00Z") }
{ "_id" : ObjectId(..), "date" : ISODate("2014-03-15T09:00:00Z") }
{ "_id" : ObjectId(..), "date" : ISODate("2014-04-04T11:21:39.736Z") }
{ "_id" : ObjectId(..), "date" : ISODate("2014-04-04T21:23:13.331Z") }
I need to get total count and max date of post. So desired result for coeumtns above is the following:
{count: 5, date: ISODate("2014-04-04T21:23:13.331Z")}
How to get desired result with single query to MongoDB without handling and counting in application code?
EDIT: #chridam thanks for the response. I've accepted your answer as best one! Could help me with one more thing?
Let's say that posts are not exists yet, so I need to fetch result with zero count and current date as timestamp like the following:
{count: 0, [Date.now()]}
Is it possible with MongoDB ?
Use the $max and $sum operators as
Model.aggregate([
{
"$group": {
"_id": null,
"count": { "$sum": 1 },
"date": { "$max": "$date" }
}
}
]).exec(function (err, result) {
console.log(result);
})
EDIT: Addressing your further question with regards to an empty collection, the aggregate function will return an empty cursor since there wont be any documents to aggregate in the collection. So you would need to address this logic on the client i.e. check the results from the above aggregation, if the result is an empty array then create the placeholder doc as required:
Model.aggregate([
{
"$group": {
"_id": null,
"count": { "$sum": 1 },
"date": { "$max": "$date" }
}
}
]).exec(function (err, result) {
console.log(result);
if (!result.length) {
result = [{ count:0, date: new Date() }];
}
});
My structure.
User:
{
name: "One",
favoriteWorkouts: [ids of workouts],
workouts: [ { name: "My workout 1" },...]
}
I want to get list of favorits/hottest workouts from database.
db.users.aggregate(
{ $unwind : "$favorite" },
{ $group : { _id : "$favorite" , number : { $sum : 1 } } },
{ $sort : { number : -1 } }
)
This returns
{
"hot": [
{
"_id": "521f6c27145c5d515f000006",
"number": 1
},
{
"_id": "521f6c2f145c5d515f000007",
"number": 1
},...
]}
But I want
{
hot: [
{object of hottest workout 1, object of hottest workout 2,...}
]}
How do you sort hottest data and fill the result with object, not just ids?
You are correct to want to use MongoDB's aggregation framework. Aggregation will give you the output you are looking for if used correctly. If you are looking for just a list of the _id's of all users' favorite workouts, then I believe that you would need to add an additional $group operation to your pipeline:
db.users.aggregate(
{ $unwind : "$favoriteWorkouts" },
{ $group : { _id : "$favoriteWorkouts", number : { $sum : 1 } } },
{ $sort : { number : -1 } },
{ $group : { _id : "oneDocumentWithWorkoutArray", hot : { $push : "$_id" } } }
)
This will yield a document of the following form, with the workout ids listed by popularity:
{
"_id" : "oneDocumentWithWorkoutArray",
"hot" : [
"workout6",
"workout1",
"workout5",
"workout4",
"workout3",
"workout2"
]
}