I am working in a mongoDB + node (with express and mongoose) API.
I have a document with more or less this structure:
// npcs
{
"_id" : ObjectId("5ea6c0f88e8ecfd3cdc39eae"),
"flavor" : {
"gender" : "...",
"description" : "...",
"imageUrl" : "...",
"class" : "...",
"campaign" : [
{
"campaignId" : "5eac9dfe8e8ecfd3cdc41aa0",
"unlocked" : true
}
]
},
},
// ...
And a second document in a separate table that is as follows:
// user
{
"_id" : ObjectId("5e987f8e4b88382a98c84042"),
"username" : "KuluGary",
"campaigns" : [
"5eac9dfe8e8ecfd3cdc41aa0",
"5eac9e458e8ecfd3cdc41ac1",
"5eac9e978e8ecfd3cdc41adb",
"5eac9eae8e8ecfd3cdc41ae3"
]
}
What I want to do is make a query in which I obtain all the NPCs that are a part of a campaign the user is part of, and are unlocked. The second part is fairly easy, just thought of once I retrieve the NPCs to filter those with unclocked false, but I'm having a hard time visualizing the query since I'm fairly unfamiliar with mongoDBs syntax and usage.
Any help would be greatly appreciated.
I understand you want to "join" a user with all relevant NPC's?
A simple aggregation with $lookup would work:
db.userCollection.aggregate([
{
$match: {
// match relevant users with whatever condition you want
}
},
{
$lookup: {
from: "npc_collection",
let: {campaigns: "$campaigns"},
pipeline: [
{
$match: {
$expr: {
$gt: [
{
$size: {
$filter: {
input: "$flavor.campaign",
as: "campaign",
cond: {
$and: [
{$setIsSubset: ["$flavor.campaign.campaignId", "$$campaigns"]},
{$eq: ["$$campaign.unlocked", true]}
]
}
}
}
},
0
]
}
}
}
],
as: "relevant_npcs"
}
}
])
Note that due to the need of an NPC to be active in a specific campaign and not just a unlocked in any we require the use of $filter.
I recommend that if you only want to lookup on one user you split this into 2 calls as i feel using $elemMatch would give better performance:
let campaigns = await db.userCollection.distinct("campaigns", {_id: userId})
let results = await db.npcCollection.find({"flavor.campaign": {$elemMatch: { campaignId: {$in: campaigns}, unlocked: true}}})
Related
user data includes :
"_id" : 4
"username" : "smith"
likedVideos :[
"videoId" : 10
"title" : "something"
"speaker" : "something"
"description" : "something"
]
i have a collection with userId and a array of liked videos lists.liked videos(Array) includes videoId and video details. so i need to check that is the user is already liked this video or not. so how i match userId and videoId from the collection without using unwind ?
i tried :
const data = await userModel.findOne({ _id : userId,likedVideos: { $elemMatch: { videoId : videoId } } })
but it returns all the data of that user.
const alreadyLiked = await userModel.aggregate([
{
$match: {
'_id' : userId,
'likedVideos.videoId' : videoId,
},
},
]);
this also not working as expected.
I need a perfect solution to match a element inside array without using unwind (My boss said that unwind is a costly operation it will effect the app performance). can you please help me to solve this.
Both of your queries are valid.
It will return all the users that match your query. You are matching a specific user and a movie. So if a user is returned, it means that the user has already liked the video.
Nevertheless $elemMatch is useful when you have multiple conditions to apply to an object. So it's better to use your second solution with
{
"_id": userId,
"likedVideos.videoId": videoId
}
If you want to keep only a given element in likedVideos, you can use $filter in an aggregate.
For example
db.collection.aggregate([
{
$match: {
"_id": 1
}
},
{
$project: {
list: {
$filter: {
input: "$likedVideos",
as: "item",
cond: {
$eq: [
"$$item.videoId",
1
]
}
}
}
}
}
])
it will only return the movies in likedVideoswith id=1
try it here
The best way to filter elements in a subarray is by using an Aggregation with $match and $project.
Example:
[{
$match: {
_id: 'userId',
likedVideos.videoId: 'videoId'
}
}, {
$project: {
'likedVideos': {
$filter: {
input: '$likedVideos',
as: 'item',
cond:
{$eq: ["$$item.videoId","videoId"]}
}
}
}
}]
I have a large documents like that and want to trim blabla: from all documents.
[{
"_id" : 1,
"videoHistory" : [
"blabla:FSFS",
"blabla:CZXC",
"ADSK",
"DAOW"
]
},
{
"_id" : 2,
"videoHistory" : [
"blabla:POQW",
"blabla:QWEE",
"VCXV",
"FSGG"
]
},
{
"_id" : 3,
"videoHistory" : [
"blabla:FSSS",
"AVCK",
"DAOC"
]
}
]
What I have did?
db.collection.aggregate([
{$match: {
$and: [
{'videoHistory.1': {$exists: true}},
{videoHistory: { '$elemMatch': {'$regex': 'blabla:'} }},
]}},
{ "$set": {
"videoHistory": {
"$map": {
"input": "$videoHistory",
"as": "vid",
"in": { "$ltrim": { input: "$$vid", chars: "blabla:" } }
}
}
}},
{ $project: {"videoHistory": 1}},
])
When I run the code, the result as expected, but it doesn't apply changes to documents, So my question how can i apply this to documents?
I'm using MongoDB V4.2
this aggregation just provides the projected result to somewhere for example to the client side or to the shell, but doesn't update the original documents. Try $merge. Based on the doc, you should use the MongoDB 4.4 to output to the same collection that is being aggregated.
hope you are doing well and Happy New Year to everybody. Hope it will end up better than 2020 X). I am facing an issue here that I can not find a solution so far. Here is a data MongoDB model example :
`{ "_id" : ObjectId("5fe715bf43c3ca0009503f1d"),
"firstname" : "Nicolas ",
"lastname" : "Mazzoleni",
"email" : "nicolas.mazzoleni#orange.fr",
"items" : [
{
"item" : ObjectId("5f6c5422eeaa1364b0d7b267"),
"createdAt" : "2020-12-26T10:51:43.685Z"
},
{
"item" : ObjectId("5f6c5422eeaa1364b0d7b270"),
"createdAt" : "2021-01-04T09:21:46.260Z"
}
],
"createdAt" : ISODate("2020-12-26T10:51:43.686Z"),
"updatedAt" : ISODate("2021-01-04T09:21:46.260Z"),
"__v" : 0
}`
Let's assume that I have a lot of rows like this one above, with different 'items.createdAt' dates.
At the end, the goal is to do a query that would find an item where the ObjectId is equal to what I am looking for, and sort by the matching 'items.createdAt' date. Here is the query I use at the moment that is not sorting by the matching nested object :
`const items = await repo
.find({ 'items.item': ObjectId("5f6c5422eeaa1364b0d7b270") })
.sort({ 'items.createdAt': -1 })
.limit(5)`
PS : I also tried to use aggregations that ended up with this query
`const items = await repo.aggregate([
{ $unwind: '$items' },
{ $match: { 'items.item': ObjectId("5f6c5422eeaa1364b0d7b270") } },
{ $sort: { 'items.createdAt': -1 } },
{ $limit: 5 },
])`
This one is working like a charm, but it does not use indexes, and I am searching into millions of records, which makes this query way to long to execute.
Thank you for all your help ! Nicolas
Unfortunately, MongoDB can't sort nested array by simple Query. You need to use aggregate for that.
db.collection.aggregate([
{
"$match": {
"items.item": ObjectId("5f6c5422eeaa1364b0d7b270")
}
},
{
"$unwind": "$items"
},
{
"$sort": {
"items.createdAt": -1
}
},
{ "$limit" : 5 },
{
"$group": {
"items": {
"$push": "$items"
},
"_id": 1
}
])
After many many tries, I can't have a nice conditional aggregation of my collections.
I use two collections :
races which have a collection of reviews.
I need to obtain for my second pipeline only the reviews published.
I don't want to use a $project.
Is it possible to use only the $match ?
When I use localField, foreignField, it works perfect, but I need to filter only the published reviews.
I struggled so much on this, I don't understand why the let don't give me the foreignKey.
I tried : _id, $reviews, etc..
My $lookup looks like this :
{
$lookup: {
from: "reviews",
as: "reviews",
let: { reviewsId: "$_id" },
pipeline: [
{
$match: {
$expr: {
$and: [
// If I comment the next line, it give all the reviews to all the races
{ $eq: ["$_id", "$$reviewsId"] },
{ $eq: ["$is_published", true] }
]
}
}
}
]
// localField: "reviews",
// foreignField: "_id"
}
},
Example of a race :
{
"description":"Nice race",
"attendees":[
],
"reviews":[
{
"$oid":"5c363ddcfdab6f1d822d7761"
},
{
"$oid":"5cbc835926fa61bd4349a02a"
}
],
...
}
Example of a review :
{
"_id":"5c3630ac5d00d1dc26273dab",
"user_id":"5be89576a38d2b260bfc1bfe",
"user_pseudo":"gracias",
"is_published":true,
"likes":[],
"title":"Best race",
"__v":10,
...
}
I will become crazy soon :'(...
How to accomplish that ?
Your problem is this line:
{ $eq: ["$is_published", true] }
You are using this document _id field to match the reviews one.
The correct version looks like this:
(
[
{
"$unwind" : "$reviews"
},
{
"$lookup" : {
"from" : "reviews",
"as" : "reviews",
"let" : {
"reviewsId" : "$reviews"
},
"pipeline" : [
{
"$match" : {
"$expr" : {
"$and" : [
{
"$eq" : [
"$_id",
"$$reviewsId"
]
},
{ $eq: ["$is_published", true] }
]
}
}
}
]
}
}
],
);
and now if your want to restore the old structure add:
{
$group: {
_id: "$_id",
reviews: {$push: "$reviews"},
}
}
First you have to take correct field to get the data from the referenced collection i.e. reviews. And second you need to use $in aggregation operator as your reviews field is an array of ObjectIds.
db.getCollection('races').aggregate([
{ "$lookup": {
"from": "reviews",
"let": { "reviews": "$reviews" },
"pipeline": [
{ "$match": {
"$expr": { "$in": [ "$_id", "$$reviews" ] },
"is_published": true
}}
],
"as": "reviews"
}}
])
I have some documents with the following structure:
{
"_id": "53ad76d70ddd13e015c0aed1",
"action": "login",
"actor": {
"name": "John",
"id": 21337037
}
}
How can I make a query in Node.js that will return the number of the unique actors that have done a specific action. For example if I have a activity stream log, that shows all the actions done by the actors, and a actorscan make a specific action multiple times, how can I get the number of all the unique actors that have done the "login" action. The actors are identified by actor.id
db.collection.distinct()
db.collection.distinct("actor.id", { action: "login"})
will return all unique occiriences and then you can get count of a result set.
PS
do not forget about db.collection.ensureIndex({action: 1})
You can use aggregation framework for this:
db.coll.aggregate([
/* Filter only actions you're looking for */
{ $match : { action : "login" }},
/* finally group the documents by actors to calculate the num. of actions */
{ $group : { _id : "$actor", numActions: { $sum : 1 }}}
]);
This query will group the documents by the entire actor sub-document and calculate the number of actions by using $sum. The $match operator will filter only documents with specific action.
However, that query will work only if your actor sub-documents are the same. You said that you're identifying your actors by id field. So if, for some reason, actor sub-documents are not exactly the same, you will have problems with your results.
Consider these these three documents:
{
...
"actor": {
"name": "John",
"id": 21337037
}
},
{
...
"actor": {
"name": "john",
"id": 21337037
}
},
{
...
"actor": {
"surname" : "Nash",
"name": "John",
"id": 21337037
}
}
They will be grouped in three different groups, even though the id field is the same.
To overcome this problem, you will need to group by actor.id.
db.coll.aggregate([
/* Filter only actions you're looking for */
{ $match : { action : "login" }},
/* finally group the documents to calculate the num. of actions */
{ $group : { _id : "$actor.id", numActions: { $sum : 1 }}}
]);
This query will correctly group your documents by looking only at the actor.id field.
Edit
You didn't specify what driver you were using so I wrote the examples for MongoDB shell.
Aggregation with Node.js driver is very similar but with one difference: Node.js is async The results of the aggregation are returned in the callback. You can check the Node.js aggregation documentation for more examples:
So the aggregation command in Node.js will look like this:
var MongoClient = require('mongodb').MongoClient;
MongoClient.connect('mongodb://127.0.0.1:27017/test', function(err, db) {
if(err) throw err;
var collection = db.collection('auditlogs');
collection.aggregate([
{ $match : { action : "login" }},
{ $group : { _id : "$actor.id", numActions: { $sum : 1 }}} ],
function(err, docs) {
if (err) console.error(err);
console.log(docs);
// do something with results
}
);
});
For these test documents:
{
"_id" : ObjectId("53b162ea698171cc1677fab8"),
"action" : "login",
"actor" : {
"name" : "John",
"id" : 21337037
}
},
{
"_id" : ObjectId("53b162ee698171cc1677fab9"),
"action" : "login",
"actor" : {
"name" : "john",
"id" : 21337037
}
},
{
"_id" : ObjectId("53b162f7698171cc1677faba"),
"action" : "login",
"actor" : {
"name" : "john",
"surname" : "nash",
"id" : 21337037
}
},
{
"_id" : ObjectId("53b16319698171cc1677fabb"),
"action" : "login",
"actor" : {
"name" : "foo",
"id" : 10000
}
}
It will return the following result:
[ { _id: 10000, numActions: 1 },
{ _id: 21337037, numActions: 3 } ]
The aggregation framework is your answer:
db.actors.aggregate([
// If you really need to filter
{ "$match": { "action": "login" } },
// Then group
{ "$group": {
"_id": {
"action": "$action",
"actor": "$actor"
},
"count": { "$sum": 1 }
}}
])
Your "actor" combination is "unique", so all you need to do it have the common "grouping keys" under the _id value for the $group pipeline stage and count those "distinct" combinations with $sum.