Mongoose.aggregate(pipeline) link multiple collections using $unwind, $lookup, $group - node.js

Question

I'm new to the aggregate feature with mongodb and mongoose and have been having difficulty getting the desired results after passing data through my pipeline.
Below I have simplified the using a fictional example models
The scenario
I have 3 Models (Ship, Yatch, and Sailboat) that share the interface and extend from a base class. A 4th model, Captain, which has an array watercraftContexts that contain objects used to reference the types of watercrafts associated to each Captain.
Example Mongo Data/Schema setup
// Model Name: 'Captain', Collection Name: 'captains'
{
name: 'Jack Sparrow', // Captian name
license: 'SYS-123', // License Number
classes: ['sail', 'yatch', 'ship'], // Array of allowed operational vessel types
watercraftContexts: [
{
_id: ObjectId('xxxxxxxxxxxxxxx'), // Foreign Model ID
type: 'Sailboat', // Model Name
ref: 'sailboats'. // Collection Name
},
{
_id: ObjectId('xxxxxxxxxxxxxxx'), // Foreign Model ID
type: 'Yatch', // Model Name
ref: 'yatches'. // Collection Name
},
{
_id: ObjectId('xxxxxxxxxxxxxxx'), // Foreign Model ID
type: 'Ship', // Model Name
ref: 'ships'. // Collection Name
}
]
}
As you can see, the array of objects has been setup to use the mongoose.populate() method with the ref and _id fields and I've implemented a virtual getter watercrafts to hydrate with the populate() feature (code not posted).
A new field is created as watercrafts with an array of all objects from the 3 different associated collections when using mongoose.Model queries.
The Problem
I also need a way to aggregate against this data to produce similar results since the Model methods are not available in the aggregate pipline.
Here is the query generated from my programmatic mongo aggregate:
[ { '$match':
{ _id:
{ '$in':
[ ObjectId('5f77bc653887221a703415e1'),
ObjectId('5f77bc653887221a703415df'),
ObjectId('5f77bc653887221a703415e0'),
ObjectId('5f77bc653887221a703415e5') ] } } },
{ '$unwind': '$watercraftContexts' },
{ '$lookup':
{ from: 'ships',
localField: 'watercraftContexts._id',
foreignField: '_id',
as: 'watercrafts.ships' } },
{ '$unwind': '$watercraftContexts' },
{ '$lookup':
{ from: 'yatches',
localField: 'watercraftContexts._id',
foreignField: '_id',
as: 'watercrafts.yatches' } },
{ '$unwind': '$watercraftContexts' },
{ '$lookup':
{ from: 'sailboats',
localField: 'watercraftContexts._id',
foreignField: '_id',
as: 'watercrafts.sailboats' } },
{ '$group':
{ _id: '$_id',
watercrafts:
{ '$addToSet':
{ '$concatArrays':
[ '$watercrafts.ships',
'$watercrafts.yatches',
'$watercrafts.sailboats' ] } }
I'm constructing a mongoose aggregate like so:
const Captain = mongoose.model('Captain')
const aggregate = Captain.aggregrate()
// Dynamically create Aggregate Pipeline in another function
const captains = await Captain.find({})
const captainIds = captains.map(capt => capt._id)
// Match sub-set of documents (in actual project)
aggregate.match({ _id: { $in: captainIds } })
// Collection names to apply $lookup aggregate
const collectionNames = ['sailboats', 'yatches', 'ships']
// Unwind and Lookup for each polymorphic child class's collection
collectionNames.forEach(collection => {
// Separate watercraftContexts into individual records for lookup
aggregate.unwind('watercraftContexts')
// Inner Join collection data on record
aggregate.lookup({
from: collection,
localField: '$watercrafContexts._id',
foreignField: '_id',
// Object keyed by collection name with array of collection records
// to avoid overwrite of previous collection aggregate lookup
as: `watercrafts.${collection}`
})
})
// Re-group the records by Captain Object Id
const aggregateAssociationPaths = collectionNames.map(collection =>
// Mongo Path to each collection $lookup
`$watercrafts.${collection}`
)
// Re-assemble $unwind and $group by Captain's ObjectId
aggregate.group({
_id: '$_id',
$addToSet: {
//
$concatArrays: aggregateAssociationPaths
}
})
/*** !!! !!! !!! !!! !!! !!! !!! !!! !!! !!! !!! !!! !!! !!! ***
* *
* WHAT DO I DO NEXT TO GET ALL THE CAPTAIN DATA WITH THE AGGREGATED `watercrafts`
* *
*** !!! !!! !!! !!! !!! !!! !!! !!! !!! !!! !!! !!! !!! !!! ***/
// Execute Aggregation
const captiansWithWatercraftsAssociations = await aggregate
My data is up to this point looks like this and the group isn't working with mongoose:
[ { _id: 5f77bc653887221a703415df,
watercrafts:
[ { _id: 5f77bc653887221a703415d3,
class: 'sail',
name: 'Gone with the Wind',
__v: 0 } ] },
{ _id: 5f77bc653887221a703415e0,
watercrafts:
[ { _id: 5f77bc653887221a703415d4,
class: 'yatch',
name: 'Liquid Gold',
__v: 0 } ] },
{ _id: 5f77bc653887221a703415e1,
watercrafts:
[ { _id: 5f77bc653887221a703415d5,
class: 'ship',
name: 'Jenny',
__v: 0 } ] },
{ _id: 5f77bc653887221a703415e5,
watercrafts:
[ { _id: 5f77bc653887221a703415dd,
class: 'yatch',
name: 'Audrey',
__v: 0 } ] },
{ _id: 5f77bc653887221a703415e5,
watercrafts:
[ { _id: 5f77bc653887221a703415dc,
class: 'sail',
name: 'Swell Shredder',
__v: 0 } ] },
{ _id: 5f77bc653887221a703415e5,
watercrafts:
[ { _id: 5f77bc653887221a703415de,
class: 'ship',
name: 'Jenny IV',
__v: 0 } ] } ]
Thanks for the support

Answer 1

This was a tricky one for someone new to MongoDb's aggregate. I'll break down my answer into steps to demonstrate to other attempting to aggregate an array with referencing multiple collections.
Step 1 - $match to filter on collection
The $match is accepts the same queries as db.collection.find({}) and returns an array of matching results in the case below, I select 4 specific records here
{ '$match':
{ _id:
{ '$in':
[
ObjectId('5f7bdb3eea134b5a5c976285'),
ObjectId('5f7bdb3eea134b5a5c976283'),
ObjectId('5f7bdb3eea134b5a5c976284'),
ObjectId('5f7bdb3eea134b5a5c976289')
]
}
}
}
$match Result
[
{ _id: ObjectId('5f7be0b37e2bdf5b19e4724d'),
name: 'CAPTAIN_SAIL',
classes: [ 'sail' ],
license: 'WC-1',
watercraftContexts:
[ { _id: ObjectId('5f7be0b37e2bdf5b19e47241'),
watercraftType: 'Sailboat',
ref: 'sailboats' } ],
__v: 0 },
{ _id: ObjectId('5f7be0b37e2bdf5b19e4724e'),
name: 'CAPTAIN_YATCH',
classes: [ 'yatch' ],
license: 'WC-2',
watercraftContexts:
[ { _id: ObjectId('5f7be0b37e2bdf5b19e47242'),
watercraftType: 'Yatch',
ref: 'yatches' } ],
__v: 0 },
{ _id: ObjectId('5f7be0b37e2bdf5b19e4724f'),
name: 'CAPTAIN_SHIP',
classes: [ 'ship' ],
license: 'WC-3',
watercraftContexts:
[ { _id: ObjectId('5f7be0b37e2bdf5b19e47243'),
watercraftType: 'Ship',
ref: 'ships' } ],
__v: 0 },
{ _id: ObjectId('5f7be0b37e2bdf5b19e47253'),
name: 'CAPTAIN_SAIL_YATCH_SHIP',
classes: [ 'sail', 'yatch', 'ship' ],
license: 'WC-7',
watercraftContexts:
[ { _id: ObjectId('5f7be0b37e2bdf5b19e4724a'),
watercraftType: 'Sailboat',
ref: 'sailboats' },
{ _id: ObjectId('5f7be0b37e2bdf5b19e4724b'),
watercraftType: 'Yatch',
ref: 'yatches' },
{ _id: ObjectId('5f7be0b37e2bdf5b19e4724c'),
watercraftType: 'Ship',
ref: 'ships' } ],
__v: 0 }
]
Step 2 - $unwind so we can iterate with $loopup
In this result set there is an array of objects with { _id: <ObjectId>, watercraftType: <ModelName> } to loop over the array and join each of these objects with there respective collection record, we have to break up the array into individual independent records. The $unwind feature will create a new data set for the next aggregate stage
{ '$unwind': '$watercraftContexts' },
$unwind Result
As you can see $unwind now creates a record with a single watercraftContext we are now set to use the $lookup
[ { _id: ObjectId('5f7be2231da37c5b5915bf9b'),
name: 'CAPTAIN_SAIL',
classes: [ 'sail' ],
license: 'WC-1',
watercraftContexts:
{ _id: ObjectId('5f7be2231da37c5b5915bf8f'),
watercraftType: 'Sailboat',
ref: 'sailboats' },
__v: 0 },
{ _id: ObjectId('5f7be2231da37c5b5915bf9c'),
name: 'CAPTAIN_YATCH',
classes: [ 'yatch' ],
license: 'WC-2',
watercraftContexts:
{ _id: ObjectId('5f7be2231da37c5b5915bf90'),
watercraftType: 'Yatch',
ref: 'yatches' },
__v: 0 },
{ _id: ObjectId('5f7be2231da37c5b5915bf9d'),
name: 'CAPTAIN_SHIP',
classes: [ 'ship' ],
license: 'WC-3',
watercraftContexts:
{ _id: ObjectId('5f7be2231da37c5b5915bf91'),
watercraftType: 'Ship',
ref: 'ships' },
__v: 0 },
{ _id: ObjectId('5f7be2231da37c5b5915bfa1'),
name: 'CAPTAIN_SAIL_YATCH_SHIP',
classes: [ 'sail', 'yatch', 'ship' ],
license: 'WC-7',
watercraftContexts:
{ _id: ObjectId('5f7be2231da37c5b5915bf98'),
watercraftType: 'Sailboat',
ref: 'sailboats' },
__v: 0 },
{ _id: ObjectId('5f7be2231da37c5b5915bfa1'),
name: 'CAPTAIN_SAIL_YATCH_SHIP',
classes: [ 'sail', 'yatch', 'ship' ],
license: 'WC-7',
watercraftContexts:
{ _id: ObjectId('5f7be2231da37c5b5915bf99'),
watercraftType: 'Yatch',
ref: 'yatches' },
__v: 0 },
{ _id: ObjectId('5f7be2231da37c5b5915bfa1'),
name: 'CAPTAIN_SAIL_YATCH_SHIP',
classes: [ 'sail', 'yatch', 'ship' ],
license: 'WC-7',
watercraftContexts:
{ _id: ObjectId('5f7be2231da37c5b5915bf9a'),
watercraftType: 'Ship',
ref: 'ships' },
__v: 0 } ]
Step 4 $lookup - Joins each record from the foreign collection
It is important to note that we must $unwind before calling $lookup for each different collection we need to join. Since we want to join multiple collections, we need to store the result in an objected keyed by the collection for later aggregation.
// Only performs $lookup on 'ships' collection
{ '$lookup':
{ from: 'ships', // Collection Name - Note: repeat for each collection
localField: 'watercraftContexts._id', // The field with id to link
foreignField: '_id', // The field on the foreign collection to match
as: 'watercrafts.ships' // The path where to store the lookup result
}
}
Step 5 - Repeat the $unwind and $lookup for the other joins
Repeat the above to steps for the additional joins, and key by the collection name. I have combined the aggregate stages to demonstrate the repetition.
{ '$unwind': '$watercraftContexts' },
{ '$lookup':
{ from: 'yatches',
localField: 'watercraftContexts._id',
foreignField: '_id',
as: 'watercrafts.yatches' } },
{ '$unwind': '$watercraftContexts' },
{ '$lookup':
{ from: 'sailboats',
localField: 'watercraftContexts._id',
foreignField: '_id',
as: 'watercrafts.sailboats' } }
Step 4 & 5 Results
If you look carefully, you notice that one of the Captain records exists 3 times with a different watercraftType. $lookup will only return records matching specific collection name. This is why why store them in an Object keyed by collectionName
[
{ _id: ObjectId('5f7be7145320a65b942bb450'),
name: 'CAPTAIN_SAIL',
classes: [ 'sail' ],
license: 'WC-1',
watercraftContexts:
{ _id: ObjectId('5f7be7145320a65b942bb444'),
watercraftType: 'Sailboat',
ref: 'sailboats' },
__v: 0,
watercrafts:
{ ships: [],
yatches: [],
sailboats:
[ { _id: ObjectId('5f7be7145320a65b942bb444'),
class: 'sail',
name: 'Gone with the Wind',
__v: 0 } ] } },
{ _id: ObjectId('5f7be7145320a65b942bb451'),
name: 'CAPTAIN_YATCH',
classes: [ 'yatch' ],
license: 'WC-2',
watercraftContexts:
{ _id: ObjectId('5f7be7145320a65b942bb445'),
watercraftType: 'Yatch',
ref: 'yatches' },
__v: 0,
watercrafts:
{ ships: [],
yatches:
[ { _id: ObjectId('5f7be7145320a65b942bb445'),
class: 'yatch',
name: 'Liquid Gold',
__v: 0 } ],
sailboats: [] } },
{ _id: ObjectId('5f7be7145320a65b942bb452'),
name: 'CAPTAIN_SHIP',
classes: [ 'ship' ],
license: 'WC-3',
watercraftContexts:
{ _id: ObjectId('5f7be7145320a65b942bb446'),
watercraftType: 'Ship',
ref: 'ships' },
__v: 0,
watercrafts:
{ ships:
[ { _id: ObjectId('5f7be7145320a65b942bb446'),
class: 'ship',
name: 'Jenny',
__v: 0 } ],
yatches: [],
sailboats: [] } },
{ _id: ObjectId('5f7be7145320a65b942bb456'),
name: 'CAPTAIN_SAIL_YATCH_SHIP',
classes: [ 'sail', 'yatch', 'ship' ],
license: 'WC-7',
watercraftContexts:
{ _id: ObjectId('5f7be7145320a65b942bb44d'),
watercraftType: 'Sailboat',
ref: 'sailboats' },
__v: 0,
watercrafts:
{ ships: [],
yatches: [],
sailboats:
[ { _id: ObjectId('5f7be7145320a65b942bb44d'),
class: 'sail',
name: 'Swell Shredder',
__v: 0 } ] } },
{ _id: ObjectId('5f7be7145320a65b942bb456'),
name: 'CAPTAIN_SAIL_YATCH_SHIP',
classes: [ 'sail', 'yatch', 'ship' ],
license: 'WC-7',
watercraftContexts:
{ _id: ObjectId('5f7be7145320a65b942bb44e'),
watercraftType: 'Yatch',
ref: 'yatches' },
__v: 0,
watercrafts:
{ ships: [],
yatches:
[ { _id: ObjectId('5f7be7145320a65b942bb44e'),
class: 'yatch',
name: 'Audrey',
__v: 0 } ],
sailboats: [] } },
{ _id: ObjectId('5f7be7145320a65b942bb456'),
name: 'CAPTAIN_SAIL_YATCH_SHIP',
classes: [ 'sail', 'yatch', 'ship' ],
license: 'WC-7',
watercraftContexts:
{ _id: ObjectId('5f7be7145320a65b942bb44f'),
watercraftType: 'Ship',
ref: 'ships' },
__v: 0,
watercrafts:
{ ships:
[ { _id: ObjectId('5f7be7145320a65b942bb44f'),
class: 'ship',
name: 'Jenny IV',
__v: 0 } ],
yatches: [],
sailboats: [] } } ]
Step 6 $project - Use project to flatten the Object Map of joins
We can use project to select all the existing data and flatten the Object Map of join results into a single Array.
{ '$project':
// keys with the value 'true' will be included
{ name: true,
license: true,
classes: true,
_id: true,
watercraftContexts: true,
__v: true,
watercrafts: // Re-assigns value of watercrafts
{ '$setUnion': // Accepts an array of arrays to flatten
[
'$watercrafts.ships',
'$watercrafts.yatches',
'$watercrafts.sailboats'
]
}
}
}
$project Result
The results of the above $project will replace the watercrafts object with an flatten array of watercrafts, but it is important to note that there are still duplicate records of Captain where matching many different lookups. We will re-piece them together in the next step.
[ { _id: ObjectId('5f7bea8d79dfe25bf3cb9695'),
name: 'CAPTAIN_SAIL',
classes: [ 'sail' ],
license: 'WC-1',
watercraftContexts:
{ _id: ObjectId('5f7bea8d79dfe25bf3cb9689'),
watercraftType: 'Sailboat',
ref: 'sailboats' },
__v: 0,
watercrafts:
[ { _id: ObjectId('5f7bea8d79dfe25bf3cb9689'),
class: 'sail',
name: 'Gone with the Wind',
__v: 0 } ] },
{ _id: ObjectId('5f7bea8d79dfe25bf3cb9696'),
name: 'CAPTAIN_YATCH',
classes: [ 'yatch' ],
license: 'WC-2',
watercraftContexts:
{ _id: ObjectId('5f7bea8d79dfe25bf3cb968a'),
watercraftType: 'Yatch',
ref: 'yatches' },
__v: 0,
watercrafts:
[ { _id: ObjectId('5f7bea8d79dfe25bf3cb968a'),
class: 'yatch',
name: 'Liquid Gold',
__v: 0 } ] },
{ _id: ObjectId('5f7bea8d79dfe25bf3cb9697'),
name: 'CAPTAIN_SHIP',
classes: [ 'ship' ],
license: 'WC-3',
watercraftContexts:
{ _id: ObjectId('5f7bea8d79dfe25bf3cb968b'),
watercraftType: 'Ship',
ref: 'ships' },
__v: 0,
watercrafts:
[ { _id: ObjectId('5f7bea8d79dfe25bf3cb968b'),
class: 'ship',
name: 'Jenny',
__v: 0 } ] },
{ _id: ObjectId('5f7bea8d79dfe25bf3cb969b'),
name: 'CAPTAIN_SAIL_YATCH_SHIP',
classes: [ 'sail', 'yatch', 'ship' ],
license: 'WC-7',
watercraftContexts:
{ _id: ObjectId('5f7bea8d79dfe25bf3cb9692'),
watercraftType: 'Sailboat',
ref: 'sailboats' },
__v: 0,
watercrafts:
[ { _id: ObjectId('5f7bea8d79dfe25bf3cb9692'),
class: 'sail',
name: 'Swell Shredder',
__v: 0 } ] },
{ _id: ObjectId('5f7bea8d79dfe25bf3cb969b'),
name: 'CAPTAIN_SAIL_YATCH_SHIP',
classes: [ 'sail', 'yatch', 'ship' ],
license: 'WC-7',
watercraftContexts:
{ _id: ObjectId('5f7bea8d79dfe25bf3cb9693'),
watercraftType: 'Yatch',
ref: 'yatches' },
__v: 0,
watercrafts:
[ { _id: ObjectId('5f7bea8d79dfe25bf3cb9693'),
class: 'yatch',
name: 'Audrey',
__v: 0 } ] },
{ _id: ObjectId('5f7bea8d79dfe25bf3cb969b'),
name: 'CAPTAIN_SAIL_YATCH_SHIP',
classes: [ 'sail', 'yatch', 'ship' ],
license: 'WC-7',
watercraftContexts:
{ _id: ObjectId('5f7bea8d79dfe25bf3cb9694'),
watercraftType: 'Ship',
ref: 'ships' },
__v: 0,
watercrafts:
[ { _id: ObjectId('5f7bea8d79dfe25bf3cb9694'),
class: 'ship',
name: 'Jenny IV',
__v: 0 } ] } ]
Step 7 $unwind and $group
We $unwind so that we can now group all watercrafts belonging to the same Captain. We also have to use $mergeObjects to temporarily store the additional data from the Captain collection under a new temporary variable to prepare for the final stages.
{ '$unwind': '$watercrafts' },
{ '$group':
{ _id: '$_id',
data:
{ '$mergeObjects':
{ name: '$name',
license: '$license',
classes: '$classes',
watercraftContexts: '$watercraftContexts',
__v: '$__v' } },
watercrafts: { '$push': '$watercrafts' } } }
$unwind and $group Result
Now we're really getting somewhere. We have reduced our transformation to our initial 4 Captains and flattened our joins into a single array.
[ { _id: ObjectId('5f7bed5e271dd95c306c25a4'),
data:
{ name: 'CAPTAIN_SHIP',
license: 'WC-3',
classes: [ 'ship' ],
watercraftContexts:
{ _id: ObjectId('5f7bed5e271dd95c306c2598'),
watercraftType: 'Ship',
ref: 'ships' },
__v: 0 },
watercrafts:
[ { _id: ObjectId('5f7bed5e271dd95c306c2598'),
class: 'ship',
name: 'Jenny',
__v: 0 } ] },
{ _id: ObjectId('5f7bed5e271dd95c306c25a8'),
data:
{ name: 'CAPTAIN_SAIL_YATCH_SHIP',
license: 'WC-7',
classes: [ 'sail', 'yatch', 'ship' ],
watercraftContexts:
{ _id: ObjectId('5f7bed5e271dd95c306c25a1'),
watercraftType: 'Ship',
ref: 'ships' },
__v: 0 },
watercrafts:
[ { _id: ObjectId('5f7bed5e271dd95c306c259f'),
class: 'sail',
name: 'Swell Shredder',
__v: 0 },
{ _id: ObjectId('5f7bed5e271dd95c306c25a0'),
class: 'yatch',
name: 'Audrey',
__v: 0 },
{ _id: ObjectId('5f7bed5e271dd95c306c25a1'),
class: 'ship',
name: 'Jenny IV',
__v: 0 } ] },
{ _id: ObjectId('5f7bed5e271dd95c306c25a2'),
data:
{ name: 'CAPTAIN_SAIL',
license: 'WC-1',
classes: [ 'sail' ],
watercraftContexts:
{ _id: Object('5f7bed5e271dd95c306c2596'),
watercraftType: 'Sailboat',
ref: 'sailboats' },
__v: 0 },
watercrafts:
[ { _id: ObjectId('5f7bed5e271dd95c306c2596'),
class: 'sail',
name: 'Gone with the Wind',
__v: 0 } ] },
{ _id: ObjectId('5f7bed5e271dd95c306c25a3'),
data:
{ name: 'CAPTAIN_YATCH',
license: 'WC-2',
classes: [ 'yatch' ],
watercraftContexts:
{ _id: ObjectId('5f7bed5e271dd95c306c2597'),
watercraftType: 'Yatch',
ref: 'yatches' },
__v: 0 },
watercrafts:
[ { _id: ObjectId('5f7bed5e271dd95c306c2597'),
class: 'yatch',
name: 'Liquid Gold',
__v: 0 } ] } ]
Step 8 $replaceRoot and $project
All we have left is to merge our data into the root of each record and remove the temporary variable data
// Merges 'data' into the root of each record
{ '$replaceRoot': { newRoot: { '$mergeObjects': [ '$data', '$$ROOT' ] } } },
// Use $project to remove data (include only the fields we want)
{ '$project':
{ name: true,
license: true,
classes: true,
_id: true,
watercraftContexts: true,
__v: true,
watercrafts: true }
}
$replaceRoot & $project Result
Now we have the result we set out for...A Captain with an array of mixed associated types watercrafts
[
{ name: 'CAPTAIN_SAIL_YATCH_SHIP',
license: 'WC-7',
classes: [ 'sail', 'yatch', 'ship' ],
watercraftContexts:
{ _id: ObjectId('5f7bf3b3680b375ca1755ea6'),
watercraftType: 'Ship',
ref: 'ships' },
__v: 0,
_id: ObjectId('5f7bf3b3680b375ca1755ead'),
watercrafts:
[ { _id: ObjectId('5f7bf3b3680b375ca1755ea4'),
class: 'sail',
name: 'Swell Shredder',
__v: 0 },
{ _id: ObjectId('5f7bf3b3680b375ca1755ea5'),
class: 'yatch',
name: 'Audrey',
__v: 0 },
{ _id: ObjectId('5f7bf3b3680b375ca1755ea6'),
class: 'ship',
name: 'Jenny IV',
__v: 0 } ] },
{ name: 'CAPTAIN_SAIL',
license: 'WC-1',
classes: [ 'sail' ],
watercraftContexts:
{ _id: ObjectId('5f7bf3b3680b375ca1755e9b'),
watercraftType: 'Sailboat',
ref: 'sailboats' },
__v: 0,
_id: ObjectId('5f7bf3b3680b375ca1755ea7'),
watercrafts:
[ { _id: ObjectId('5f7bf3b3680b375ca1755e9b'),
class: 'sail',
name: 'Gone with the Wind',
__v: 0 } ] },
{ name: 'CAPTAIN_YATCH',
license: 'WC-2',
classes: [ 'yatch' ],
watercraftContexts:
{ _id: ObjectId('5f7bf3b3680b375ca1755e9c'),
watercraftType: 'Yatch',
ref: 'yatches' },
__v: 0,
_id: ObjectId('5f7bf3b3680b375ca1755ea8'),
watercrafts:
[ { _id: ObjectId('5f7bf3b3680b375ca1755e9c'),
class: 'yatch',
name: 'Liquid Gold',
__v: 0 } ] },
{ name: 'CAPTAIN_SHIP',
license: 'WC-3',
classes: [ 'ship' ],
watercraftContexts:
{ _id: ObjectId('5f7bf3b3680b375ca1755e9d'),
watercraftType: 'Ship',
ref: 'ships' },
__v: 0,
_id: ObjectId('5f7bf3b3680b375ca1755ea9'),
watercrafts:
[ { _id: ObjectId('5f7bf3b3680b375ca1755e9d'),
class: 'ship',
name: 'Jenny',
__v: 0 } ] } ]
And there you have it...only took 2 days to figure this out. I hope it saves you some time if you're attempting a similar aggregate association. Happy coding!
Final Pipeline
[
{ '$match':
{ _id:
{ '$in':
[ ObjectId('5f7bf3b3680b375ca1755ea9'),
ObjectId('5f7bf3b3680b375ca1755ea7'),
ObjectId('5f7bf3b3680b375ca1755ea8'),
ObjectId('5f7bf3b3680b375ca1755ead')
]
}
}
},
{ '$unwind': '$watercraftContexts' },
{ '$lookup':
{ from: 'ships',
localField: 'watercraftContexts._id',
foreignField: '_id',
as: 'watercrafts.ships' } },
{ '$unwind': '$watercraftContexts' },
{ '$lookup':
{ from: 'yatches',
localField: 'watercraftContexts._id',
foreignField: '_id',
as: 'watercrafts.yatches' } },
{ '$unwind': '$watercraftContexts' },
{ '$lookup':
{ from: 'sailboats',
localField: 'watercraftContexts._id',
foreignField: '_id',
as: 'watercrafts.sailboats' } },
{ '$project':
{ name: true,
license: true,
classes: true,
_id: true,
watercraftContexts: true,
__v: true,
watercrafts:
{ '$setUnion':
[ '$watercrafts.ships',
'$watercrafts.yatches',
'$watercrafts.sailboats' ] } } },
{ '$unwind': '$watercrafts' },
{ '$group':
{ _id: '$_id',
data:
{ '$mergeObjects':
{ name: '$name',
license: '$license',
classes: '$classes',
watercraftContexts: '$watercraftContexts',
__v: '$__v' } },
watercrafts: { '$push': '$watercrafts' } } },
{ '$replaceRoot': { newRoot: { '$mergeObjects': [ '$data', '$$ROOT' ] } } },
{ '$project':
{ name: true,
license: true,
classes: true,
_id: true,
watercraftContexts: true,
__v: true,
watercrafts: true } }
]