Mongoose nested (2 level) find - node.js

I'm trying to use CASL for authorization check of nested items.
It uses mongoose for query data and check access.
My domain is that:
A "User" could has more "Vehicles"
A "Document" must have a Vehicle
Schema:
vehicle { users: [ {type: objectId, ref: 'user'} ] }
document { vehicle: {type: objectId, ref: 'vehicle' }}
To find the vehicle "by user" I do:
db.getCollection('vehicle').find(
{ users: {$in: [ ObjectId("5ae1a957d67500018efa2c9d") ]} }
)
That works.
In the documents collection, the data has records such as this:
{
"_id": ObjectId("5aeaad1277e8a6009842564d"),
"vehicle": ObjectId("5aea338b82d8170096b52ce9"),
"company": "Allianz",
"price": 500,
"date_start": ISODate("2018-05-02T22:00:00.000Z"),
"date_end": ISODate("2019-05-02T22:00:00.000Z"),
"createdAt": ISODate("2018-05-03T06:32:50.590Z"),
"updatedAt": ISODate("2018-05-03T06:32:50.590Z"),
"__v": 0
}
To find the document "by user" I do:
db.getCollection('document').find(
{ "vehicle.users": {$in: [ ObjectId("5ae1a957d67500018efa2c9d") ]} }
)
It doesn't work. Is possibile to do that in one single "find" query?

You can't do it in a simple MongoDB find() query, because the data about vehicle users exists in the vehicle collection, not the documents collection.
However, it is possible with an aggregation pipeline using the $lookup operator to link the data in two different collections. The aggregation would be something like this:
db.document.aggregate([
{$lookup: {
"from": "vehicle",
"localField": "vehicle",
"foreignField": "_id",
"as": "vehicleDetails",
}},
{$match: {"vehicleDetails.users" : ObjectId("5ae1a957d67500018efa2c9d")}}
])
You will probably need to add more stages to reshape the data the way you need it, but the key is to use $lookup to link the data from the two collections, then use $match to filter the set of results.

In order for this query to work you need to store users ids array in vehicle document. Neither Mongo nor CASL doesn't manage external references automatically.
Alternative solutions:
So, I see few ways:
Retrieve ids of all vehicles when you define rules. This works good in case if amount of vehicles not big (<= 1000)
const vehicleIds = await getVehicleIds(user)
can(['read', 'update'], 'document', { vehicle: { $in: vehicleIds } })
Denormalize your scheme. For example, add additional user_id field to vehicle document
Think whether you can embed document as subdocument to vechicle, something like this:
vehicle {
documents: [Document],
users: [ {type: objectId, ref: 'user'} ]
}
Just don't define rule per documents and enforce them in routes (REST or GraphQL doesn't matter).
app.get('/vehicle/:id/documents', async (req, res) => {
const vehicle = await Vehicle.findById(req.params.id)
req.ability.throwUnlessCan('read', vehicle)
const documents = Document.find({ vehicle: vehicle.id })
res.send({ documents })
})

Related

How to get categories and sub-categories in single API response

I have two collections Categories and Subcategories inside a Categories collection. I have an array that is storing the ids of subcategories that are stored in Subcategories collection. Below is my document structure:
Categories collection
{
id:65,
title:"Automotive",
active:true,
subcategories:[35,28,30]
}
subcategories collection
{
id:35,
title:"Automotive technology",
category_id:65,
active:true
},
{
id:28,
title:"Automotive coatings",
category_id:65,
active:true
},
{
id:30,
title:"Machinery and equipments",
category_id:65,
active:true
}
As seen in above collection 3 documents from subcategories collection have been associated with the category document. I want to fetch data in below format on single API hit.
API response should be in below format:
{
data:{
category:{
id:65,
title:"Automotive",
subcategories:[{
id:35,
name:"Automotive technology",
},
{
id:28,
name:"Automotive coatings",
},
{
id:30,
name:"Machinery and equipments",
}]
},
category:{
id:66,
title:"Food",
subcategories:[{
id:23,
name:"Drinks",
},
{
id:2,
name:"Additives",
}]
},
},
messsage:"Success",
code:200
}
As of now I am able to get data in 2 api hits that is like first getting all the categories
const category = await db.categories.find({});
Then on click of some particular category fetching all the sub categories based on the category id.
const subCategories = await db.SubCategories.find({category_id:id});
How can I get the above data in desired format in single API hit?
You need something like this, also, if you use mongoose, you can use .populate()
To format data you can use $project stage in aggregation pipeline or projection in .find()
If you want to use Mongoose with populate:
CategorySchema:
const CategorySchema= new mongoose.Schema({
...
subCategories: [{ type: mongoose.Schema.Types.ObjectId, ref: 'SubCategory' }],
...
});
need _id column on reference table
ref content is must be equal to model name like
module.exports = mongoose.model('SubCategory', SubCategorySchema);
Controller:
const categories = await Category.find({})
.populate({
path: 'subCategories'
})
path content is must be equal to column name
If you want to write with Mongo query:
db.getCollection("categories").aggregate([
{
$lookup:
{
from: 'subCategories',
localField: 'subCategories',
foreignField: 'id',
as: 'subCategories'
}
}
])
You get empty object because you are not using "_id" field for join schemas. If you want to use "id" field u need to use virtuals on Mongoose:
https://mongoosejs.com/docs/tutorials/virtuals.html#populate

MongoDB: Dynamic Counts

I have two collections. A 'users' collection and an 'events' collection. There is a primary key on the events collection which indicates which user the event belongs to.
I would like to count how many events a user has matching a certain condition.
Currently, I am performing this like:
db.users.find({ usersMatchingACondition }).forEach(user => {
const eventCount = db.events.find({
title: 'An event title that I want to find',
userId: user._id
}).count();
print(`This user has ${eventCount} events`);
});
Ideally what I would like returned is an array or object with the UserID and how many events that user has.
With 10,000 users - this is obviously producing 10,000 queries and I think it could be made a lot more efficient!
I presume this is easy with some kind of aggregate query - but I'm not familiar with the syntax and am struggling to wrap my head around it.
Any help would be greatly appreciated!
You need $lookup to get the data from events matched by user_id. Then you can use $filter to apply your event-level condition and to get a count you can use $size operator
db.users.aggregate([
{
$match: { //users matching condition }
},
{
$lookup:
{
from: 'events',
localField: '_id', //your "primary key"
foreignField: 'user_id',
as: 'user_events'
}
},
{
$addFields: {
user_events: {
$filter: {
input: "$user_events",
cond: {
$eq: [
'$$this.title', 'An event title that I want to find'
]
}
}
}
}
},
{
$project: {
_id: 1,
// other fields you want to retrieve: 1,
totalEvents: { $size: "$user_events" }
}
}
])
There isn't much optimization that can be done without aggregate but since you specifically said that
First, instead of
const eventCount = db.events.find({
title: 'An event title that I want to find',
userId: user._id
}).count();
Do
const eventCount = db.events.count({
title: 'An event title that I want to find',
userId: user._id
});
This will greatly speed up your queries because the find query actually fetches the documents first and then does the counting.
For returning an array you can just initialize an array at the start and push {userid: id, count: eventCount} objects to it.

MongoDB - How to find a doc by Id and find the docs that matches with an array of ids

I have the following problem, i have many documents following this structure in my collection
{
"_id": ObjectId("5c6ec80df2f9d02d08ce6b3c"),
"id": "0",
"name": "first",
"details": "these are the details of each doc",
"relatedDocsIds": ["1","2"]
}
The "id" parameter is not repeated in the collection.
I want to get the complete documents with the ids that have relatedDocsIds.
I give the "id" of the doc from whom i want to get the complete related Doc from the array of ids.
So in this example, i want to get the related documents of the document with "0", the id of his related documents are stored in the array "relatedDocsIds".
How would be that query?
Im in a nodejs backend and i need that the response be the json containing the documents of that query.
you should use mongodb aggregate for that, in conjuction with the $unwind and $lookup operator.
$unwind Deconstructs an array field from the input documents to output a document for each element.
$lookup Performs a left outer join to an unsharded collection in the same database to filter in documents from the “joined” collection for processing.
here's the final query:
db.yourCollection.aggregate([
{
$unwind: "$relatedDocsIds"
},
{
$lookup:{
from: "yourCollection",
localField: "relatedDocsIds",
foreignField: "id",
as: "relatedDocs"
}
}
])
In mongoose you can do it like this:
Model.aggregate([
{
$unwind: "$relatedDocsIds"
},
{
$lookup:{
from: "yourCollection",
localField: "relatedDocsIds",
foreignField: "id",
as: "relatedDocs"
}
}
]).exec((err, docs) => {
if (err) throw err;
console.log(docs);
});
I would recommend going against this approach, because mongoose has a populate method, which will allow you to do this exact thing and their benchmarks suggest it's faster than the mongodb aggregate.
The easiest way I think it's using MongoDB Query Arrays operators
MongoDB Query Arrays
MongoDB $in operator
Example:
getCollection('collectionName').find({
relatedDocsIds: {
$in: ['0']
}
})

MongoDB sort by property in other document

In order to expand the JSON-API capabilities of my node.js application, I'm trying to sort a query based on relationships (AKA other documents), although I don't want to return them.
According to the JSON-API documentation:
a sort field of author.name could be used to request that the primary data be sorted based upon the name attribute of the author relationship.
E.g. db.collection('books').find({}) returns:
[
{
type: "book",
id: "2349",
attributes: {
title: "My Sweet Book"
},
relationships: {
author: {
data: {
type: "authors",
id: "9"
}
}
}
},
{} // etc ...
]
db.collection('authors').find({id: "9"}) returns:
[
{
type: "author",
id: "9",
attributes: {
name: "Hank Moody"
}
}
]
Now I need some way to do something similar to e.g.:
db.collection('books').find({}).sort({"author.name": -1})
I think I need to convert the query to an aggregation so I can use the $lookup operator, but I'm not sure how to use localField and foreignField.
db.collection('books').aggregate([
{$match: {}},
{$lookup: {from: "authors", localField: "attributes.author.data.id", foreignField: "id", as: "temp.author"}},
{$sort: {"$books.temp.author.name": -1}},
{$project: {temp: false}},
])
Notes
This will be a global function for fetching JSON-API data.
This means we don't know wether a sort key is an attribute or a relationship.
Most servers run LTS versions and have MongoDB 3.2
You can try below aggregation.
$lookup to join to authors collection followed by $unwind to flatten the book_author array for applying $sort on name field and $project with exclusion to remove book_author field ( only works starting Mongo 3.4 version ). For lower versions you have to include all the other fields you want to keep and excluding book_author field in the $project stage.
db.collection('books').aggregate([{
$lookup: {
from: "authors",
localField: "relationships.author.data.id",
foreignField: "id",
as: "book_author"
}
}, {
$unwind: "$book_author"
}, {
$sort: {
"book_author.attributes.name": -1
}
}, {
$project: {
"book_author": 0
}
}])

Mongoose join operation

I have 3 schema's like below:
User
var UserSchema = new Schema({
name: String
});
Actor
var ActorSchema = new Schema({
name: String
});
Rating
var RatingSchema = new Schema({
actor: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Actor'
},
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Actor'
},
userRating: Number
});
I want to send all actors info to the front end like [actor1, actor2 ...].
Each actor contain actor details and 'userRating' which is given by the user who is currently logged in.
A user can give ratings to multiple actors and an actor can receive ratings from multiple users. These will be stored in Ratings table.
I wrote something like this
Actor
.find({}) // get all actors and populate userRating into each actor
.populate({
path: 'userRating',
model: 'Rating',
match: { actor: {$eq: req.actor}, user: {$eq: req.user}},
select: 'userRating'
})
.exec(function(error, actors){
if(error)
res.status(501).json({error: error});
else
res.json(actors);
});
I got only actors in the result. actor object doesn't contain 'userRating'. can someone correct my query
It depends on what you are actually sending as input for the query parameters here. Also the main thing that you need to understand is that this is not a "JOIN", but in fact separate queries being issued by the mongoose software layer, so there are distinct differences in handling.
In the basic case where the "values" being supplied as parameters are actually the ObjectId values of the references, then you actually just want these directly in the main "query" rather than arguments to the .populate() action ( which is actually where the "additional queries" are happening ).
Furthermore your "relations/references" are in the Rating model, so that is where your query is issued instead:
Rating.find({
"actor": req.actor,
"user": req.user
}).populate("actor user").exec(function(err,ratings) {
// Matched ratings by actor and user supplied
})
If your parameters are instead the "name" data of each object, then since that information is not present in the Rating model until populated the only way mongoose can do this is to retrieve "all" of the Rating objects, then do the "population" with the "match" criteria, and finally filter out any results where the population was null due to un-matched items:
Rating.find().populate([
{ "path": "actor", "match": { "name": req.actor } },
{ "path": "user", "match": { "name": req.user } }
]).exec(function(err,ratings) {
// Now filter out the null results
ratings = ratings.filter(function(rating) {
return ( rating.actor != null && rating.user != null )
});
// Then work with filtered data
})
Of course that is highly inefficient since this is a "client" side operation and you are pulling in all of the Rating content "first". So what you really mean to do in this case is to actually do the "three" query operations yourself, and by getting the ObjectId values from both User and Actor models in order to apply the match to the Rating model instead:
async.parallel(
{
"user": function(callback) {
User.findOne({ "name": req.user },callback)
},
"actor": function(callback) {
Actor.findOne({ "name": req.actor },callback)
}
},
function(err,data) {
// Use returned _id values in query
Rating.find({
"actor": data.actor._id,
"user": data.user._id
}).populate("actor user").exec(err,ratings) {
// populated Rating results
});
}
)
Then the queries resolve the "only" ObjectId values you actually require and the final query on Rating only retrieves those results that actually match the conditions, rather than everything and doing a "post filter" operation.
As a final approach, if you have MongoDB 3.2 available, then you could alternately use the $lookup operation instead to perform the "JOINS" on the "server" instead:
Rating.aggregate(
[
{ "$lookup": {
"from": "users",
"localField": "user",
"foreignField": "_id",
"as": "user"
}},
{ "$unwind": "$user" },
{ "$match": { "user.name": req.user } },
{ "$lookup": {
"from": "actors",
"localField": "actor",
"foreignField": "_id",
"as": "actor"
}},
{ "$unwind": "actor" },
{ "$match": { "actor.name": req.actor } }
],
function(err,ratings) {
// populated on the server in one request
}
)
From the "client" point of view, this is just "one" request and response as opposed to what .populate() does. But it really is not more than a "server" side rendition of the "client" logic presented before.
So if looking up by values of "name", you should instead do the "three" query approach for optimal performance, since the aggregation version is still really working with a lot more data than it needs to.
Of course the "best" perspective is to simply use the ObjectId values to begin with.
Of course the main thing here is that information like "userRating" belongs to the Rating model, and that is therefore where you provide the "query" in all cases in order to retrieve that data. These are not "JOIN" operations like in SQL, so the "server" is not looking at the combined results then selecting the fields.
As a bit of self education turn on "debugging" to see how mongoose is actually issuing statements to the server. Then you will see how .populate() is actually applied:
mongoose.set("debug",true)

Resources