I have a large documents like that and want to trim blabla: from all documents.
[{
"_id" : 1,
"videoHistory" : [
"blabla:FSFS",
"blabla:CZXC",
"ADSK",
"DAOW"
]
},
{
"_id" : 2,
"videoHistory" : [
"blabla:POQW",
"blabla:QWEE",
"VCXV",
"FSGG"
]
},
{
"_id" : 3,
"videoHistory" : [
"blabla:FSSS",
"AVCK",
"DAOC"
]
}
]
What I have did?
db.collection.aggregate([
{$match: {
$and: [
{'videoHistory.1': {$exists: true}},
{videoHistory: { '$elemMatch': {'$regex': 'blabla:'} }},
]}},
{ "$set": {
"videoHistory": {
"$map": {
"input": "$videoHistory",
"as": "vid",
"in": { "$ltrim": { input: "$$vid", chars: "blabla:" } }
}
}
}},
{ $project: {"videoHistory": 1}},
])
When I run the code, the result as expected, but it doesn't apply changes to documents, So my question how can i apply this to documents?
I'm using MongoDB V4.2
this aggregation just provides the projected result to somewhere for example to the client side or to the shell, but doesn't update the original documents. Try $merge. Based on the doc, you should use the MongoDB 4.4 to output to the same collection that is being aggregated.
Related
I am working in a mongoDB + node (with express and mongoose) API.
I have a document with more or less this structure:
// npcs
{
"_id" : ObjectId("5ea6c0f88e8ecfd3cdc39eae"),
"flavor" : {
"gender" : "...",
"description" : "...",
"imageUrl" : "...",
"class" : "...",
"campaign" : [
{
"campaignId" : "5eac9dfe8e8ecfd3cdc41aa0",
"unlocked" : true
}
]
},
},
// ...
And a second document in a separate table that is as follows:
// user
{
"_id" : ObjectId("5e987f8e4b88382a98c84042"),
"username" : "KuluGary",
"campaigns" : [
"5eac9dfe8e8ecfd3cdc41aa0",
"5eac9e458e8ecfd3cdc41ac1",
"5eac9e978e8ecfd3cdc41adb",
"5eac9eae8e8ecfd3cdc41ae3"
]
}
What I want to do is make a query in which I obtain all the NPCs that are a part of a campaign the user is part of, and are unlocked. The second part is fairly easy, just thought of once I retrieve the NPCs to filter those with unclocked false, but I'm having a hard time visualizing the query since I'm fairly unfamiliar with mongoDBs syntax and usage.
Any help would be greatly appreciated.
I understand you want to "join" a user with all relevant NPC's?
A simple aggregation with $lookup would work:
db.userCollection.aggregate([
{
$match: {
// match relevant users with whatever condition you want
}
},
{
$lookup: {
from: "npc_collection",
let: {campaigns: "$campaigns"},
pipeline: [
{
$match: {
$expr: {
$gt: [
{
$size: {
$filter: {
input: "$flavor.campaign",
as: "campaign",
cond: {
$and: [
{$setIsSubset: ["$flavor.campaign.campaignId", "$$campaigns"]},
{$eq: ["$$campaign.unlocked", true]}
]
}
}
}
},
0
]
}
}
}
],
as: "relevant_npcs"
}
}
])
Note that due to the need of an NPC to be active in a specific campaign and not just a unlocked in any we require the use of $filter.
I recommend that if you only want to lookup on one user you split this into 2 calls as i feel using $elemMatch would give better performance:
let campaigns = await db.userCollection.distinct("campaigns", {_id: userId})
let results = await db.npcCollection.find({"flavor.campaign": {$elemMatch: { campaignId: {$in: campaigns}, unlocked: true}}})
I had the following array of structure in my aggregation pipeline. Tried merge objects and setUnion operators.
{
combs:[
[
{
name:"A",
c_type:"A"
},
{
type:"visual",
severity:"Normal"
}
],
[
{
name:"B",
c_type:"B"
},
{
type:"visual",
severity:"Normal"
}
]
]
}
I am expecting the following results to produce some statistics. Please help me.
{
combs:[
{
name:"A",
c_type:"A",
type:"visual",
severity:"Normal"
}
{
name:"B",
c_type:"B",
type:"visual",
severity:"Normal"
}
]
}
"Is it possible to achieve without $unwind operation?"
Well YES. As long as your structure of arrays of arrays is consistently mapped that way then you really only need a single stage in the pipeline:
db.collection.aggregate([
{ "$addFields": {
"combs": {
"$map": {
"input": "$combs",
"in": { "$mergeObjects": "$$this" }
}
}
}}
])
So really the $map operator takes place here as a much more efficient method than $unwind for processing each array element. Also since $mergeObjects is expecting "an array of objects", this is what each element of your array of arrays actually is. So simply { "$mergeObjects": "$$this" } on each outer member of the array.
Produces the output from your supplied data:
{
"_id" : ObjectId("5d8865c273375a6a4cc9e76a"),
"combs" : [
{
"name" : "A",
"c_type" : "A",
"type" : "visual",
"severity" : "Normal"
},
{
"name" : "B",
"c_type" : "B",
"type" : "visual",
"severity" : "Normal"
}
]
}
Generally you should always prefer an inline processor like $map or other array operators in preference to $unwind where applicable.
You can use this aggregation query
db.collection.aggregate([
{ $unwind: "$combs" },
{ $addFields: { combs: { $mergeObjects: "$combs" }}},
{ $group: { _id: "$_id", combs: { $push: "$combs" }} }
])
After many many tries, I can't have a nice conditional aggregation of my collections.
I use two collections :
races which have a collection of reviews.
I need to obtain for my second pipeline only the reviews published.
I don't want to use a $project.
Is it possible to use only the $match ?
When I use localField, foreignField, it works perfect, but I need to filter only the published reviews.
I struggled so much on this, I don't understand why the let don't give me the foreignKey.
I tried : _id, $reviews, etc..
My $lookup looks like this :
{
$lookup: {
from: "reviews",
as: "reviews",
let: { reviewsId: "$_id" },
pipeline: [
{
$match: {
$expr: {
$and: [
// If I comment the next line, it give all the reviews to all the races
{ $eq: ["$_id", "$$reviewsId"] },
{ $eq: ["$is_published", true] }
]
}
}
}
]
// localField: "reviews",
// foreignField: "_id"
}
},
Example of a race :
{
"description":"Nice race",
"attendees":[
],
"reviews":[
{
"$oid":"5c363ddcfdab6f1d822d7761"
},
{
"$oid":"5cbc835926fa61bd4349a02a"
}
],
...
}
Example of a review :
{
"_id":"5c3630ac5d00d1dc26273dab",
"user_id":"5be89576a38d2b260bfc1bfe",
"user_pseudo":"gracias",
"is_published":true,
"likes":[],
"title":"Best race",
"__v":10,
...
}
I will become crazy soon :'(...
How to accomplish that ?
Your problem is this line:
{ $eq: ["$is_published", true] }
You are using this document _id field to match the reviews one.
The correct version looks like this:
(
[
{
"$unwind" : "$reviews"
},
{
"$lookup" : {
"from" : "reviews",
"as" : "reviews",
"let" : {
"reviewsId" : "$reviews"
},
"pipeline" : [
{
"$match" : {
"$expr" : {
"$and" : [
{
"$eq" : [
"$_id",
"$$reviewsId"
]
},
{ $eq: ["$is_published", true] }
]
}
}
}
]
}
}
],
);
and now if your want to restore the old structure add:
{
$group: {
_id: "$_id",
reviews: {$push: "$reviews"},
}
}
First you have to take correct field to get the data from the referenced collection i.e. reviews. And second you need to use $in aggregation operator as your reviews field is an array of ObjectIds.
db.getCollection('races').aggregate([
{ "$lookup": {
"from": "reviews",
"let": { "reviews": "$reviews" },
"pipeline": [
{ "$match": {
"$expr": { "$in": [ "$_id", "$$reviews" ] },
"is_published": true
}}
],
"as": "reviews"
}}
])
I have multiple data something like this
{
"_id" : ObjectId("57189fcd72b6e0480ed7a0a9"),
"venueId" : ObjectId("56ce9ead08daba400d14edc9"),
"companyId" : ObjectId("56e7d62ecc0b8fc812b2aac5"),
"cardTypeId" : ObjectId("56cea8acd82cd11004ee67a9"),
"matchData" : [
{
"matchId" : ObjectId("57175c25561d87001e666d12"),
"matchDate" : ISODate("2016-04-08T18:30:00.000Z"),
"matchTime" : "20:00:00",
"_id" : ObjectId("57189fcd72b6e0480ed7a0ab"),
"active" : 3,
"cancelled" : 0,
"produced" : 3
},
{
"matchId" : ObjectId("57175c25561d87001e666d13"),
"matchDate" : ISODate("2016-04-09T18:30:00.000Z"),
"matchTime" : "20:00:00",
"_id" : ObjectId("57189fcd72b6e0480ed7a0aa"),
"active" : null,
"cancelled" : null,
"produced" : null
}
],
"__v" : 0
}
i m doing group by companyId and its work fine But i want to search in matchData based on matchtime and matchId For that purpose i am $unwind matchData after unwind i using my search query like this
db.getCollection('matchWiseData').aggregate([
{"$match":{
"matchData.matchId":{"$in":[ObjectId("57175c25561d87001e666d12")]}
}},
{"$unwind":"$matchData"},
{"$match":{
"matchData.matchId":{"$in":[ObjectId("57175c25561d87001e666d12")]}}
}])
its give me proper result but after applying unwind is there any way to undo it I m using unwind to just search inside subdocument or there is any other way to search inside subdocument.
Well you can of course just use $push and $first in a $group to get the document back to what it was:
db.getCollection('matchWiseData').aggregate([
{ "$match":{
"matchData.matchId":{"$in":[ObjectId("57175c25561d87001e666d12")]}
}},
{ "$unwind":"$matchData"},
{ "$match":{
"matchData.matchId":{"$in":[ObjectId("57175c25561d87001e666d12")]}
}},
{ "$group": {
"_id": "$_id",
"venueId": { "$first": "$venueId" },
"companyId": { "$first": "$companyId" },
"cardTypeId": { "$first": "$cardTypeId" },
"matchData": { "$push": "$matchData" }
}}
])
But you probably should have just used $filter with MongoDB 3.2 in the first place:
db.getCollection('matchWiseData').aggregate([
{ "$match":{
"matchData.matchId":{"$in":[ObjectId("57175c25561d87001e666d12")]}
}},
{ "$project": {
"venueId": 1,
"companyId": 1,
"cardTypeId": 1,
"matchData": {
"$filter": {
"input": "$matchData",
"as": "match",
"cond": {
"$or": [
{ "$eq": [ "$$match.matchId", ObjectId("57175c25561d87001e666d12") ] }
]
}
}
}
}}
])
And if you had at least MongoDB 2.6, you still could have used $map and $setDifference instead:
db.getCollection('matchWiseData').aggregate([
{ "$match":{
"matchData.matchId":{"$in":[ObjectId("57175c25561d87001e666d12")]}
}},
{ "$project": {
"venueId": 1,
"companyId": 1,
"cardTypeId": 1,
"matchData": {
"$setDifference": [
{ "$map": {
"input": "$matchData",
"as": "match",
"in": {
"$cond": [
{ "$or": [
{ "$eq": [ "$$match.matchId", ObjectId("57175c25561d87001e666d12") ] }
]},
"$$match",
false
]
}
}},
[false]
]
}
}}
])
That's perfectly fine when every array element already has a "unique" identifier, so the "set" operation just removes the false values from $map.
Both of those a ways to "filter" content from an array without actually using $unwind
N.B: Not sure if you really grasp that $in is used to match a "list of conditions" rather than being required to match on arrays. So generally the condition can just be:
"matchData.matchId": ObjectId("57175c25561d87001e666d12")
Where you only actually have a single value to match on. You use $in and $or when you have a "list" of conditions. Arrays themselves make no difference to the operator required.
This is Collection Structure
[{
"_id" : "....",
"name" : "aaaa",
"level_max_leaves" : [
{
level : "ObjectIdString 1",
max_leaves : 4,
}
]
},
{
"_id" : "....",
"name" : "bbbb",
"level_max_leaves" : [
{
level : "ObjectIdString 2",
max_leaves : 2,
}
]
}]
I need to find the subdocument value of level_max_leaves.level filter when its matching with given input value.
And this how I tried,
For example,
var empLevelId = 'ObjectIdString 1' ;
MyModel.aggregate(
{$unwind: "$level_max_leaves"},
{$match: {"$level_max_leaves.level": empLevelId } },
{$group: { "_id": "$level_max_leaves.level",
"total": { "$sum": "$level_max_leaves.max_leaves" }}},
function (err, res) {
console.log(res);
});
But here the $match filter is not working. I can't find out exact results of ObjectIdString 1
If I filter with name field, its working fine. like this,
{$match: {"$name": "aaaa" } },
But in subdocument level its returns 0.
{$match: {"$level_max_leaves.level": "ObjectIdString 1"} },
My expected result was,
{
"_id" : "ObjectIdString 1",
"total" : 4,
}
You have typed the $match incorrectly. Fields with $ prefixes are either for the implemented operators or for "variable" references to field content. So you just type the field name:
MyModel.aggregate(
[
{ "$match": { "level_max_leaves.level": "ObjectIdString 1" } },
{ "$unwind": "$level_max_leaves" },
{ "$match": { "level_max_leaves.level": "ObjectIdString 1" } },
{ "$group": {
"_id": "$level_max_leaves.level",
"total": { "$sum": "$level_max_leaves.max_leaves" }
}}
],
function (err, res) {
console.log(res);
}
);
Which on the sample you provide produces:
{ "_id" : "ObjectIdString 1", "total" : 4 }
It is also good practice to $match first in your pipeline. That is in fact the only time an index can be used. But not only for that, as without the initial $match statement, your aggregation pipeline would perform an $unwind operation on every document in the collection, whether it met the conditions or not.
So generally what you want to do here is
Match the documents that contain the required elements in the array
Unwind the array of the matching documents
Match the required array content excluding all others