Is it possible to query on associated fields properties in sails. For example, let's consider these two model:
Product:
attributes: {
name:{
type:'string'
},
code:{
type:'string',
required:true,
unique:true
},
inventory:{
collection:'inventory',
via:'product'
}
}
Inventory
attributes: {
product:{
model:'product',
required:true
},
quantity:{
type:'float'
}
}
Now is there any way to get Inventory records that have Products of certain codes. One way is to get products of those specific codes and then get Inventory records having those products. But is it possible to do a single find query to get desired result? I have tried following, but this does not filter products on code.
Inventory
.find()
.populate('product',{where:{code:{'contains':'something'}}})
.exec(function (err, inventories) {
//do something
})
In fact with your request you take all Inventory and populate only product with code:{'contains':'something'} other inventory will have empty products.
You can make your association two way and do this :
Product
.find().where({code:{'contains':'something'}})
.populate('inventory')
.exec(function (err, products) {
//do something
//Build an array of inventory with async on all products
})
As I know there is no other way to do this actually :/
No, you can`t. Best option you have - it is NOT use sails ever!
Related
Hello I am new to nodejs and mongodb.
I have 3 models:
"user" with fields "name phone"
"Shop" with fields "name, address"
"Member" with fields "shop user status". (shop and user hold the "id" of respective collections).
Now when I create "shops" api to fetch all shop, then I need to add extra field "isShopJoined" which is not part of the model. This extra field will true if user who see that shop is joined it otherwise it will be false.
The problem happens when I share my model with frontend developers like Android/iOS and others, They will not aware of that extra field until they see the API response.
So is it ok if I add extra field in shops listing which is not part of the model? Or do I need to add that extra field in model?
Important note
All the code below has NOT been tested (yet, I'll do it when I can setup a minimal environment) and should be adapted to your project. Keep in mind that I'm no expert when it comes to aggregation with MongoDB, let alone with Mongoose, the code is only here to grasp the general idea and algorithm.
If I understood correctly, you don't have to do anything since the info is stored in the Member collection. But it forces the front-end to do an extra-request (or many extra-requests) to have both the list of Shops and to check (one by one) if the current logged user is a Member of the shop.
Keep in mind that the front-end in general is driven by the data (and so, the API/back-end), not the contrary. The front-end will have to adapt to what you give it.
If you're happy with what you have, you can just keep it that way and it will work, but that might not be very effective.
Assuming this:
import mongoose from "mongoose";
const MemberSchema = new mongoose.Schema({
shopId: {
type: ObjectId,
ref: 'ShopSchema',
required: true
},
userId: {
type: ObjectId,
ref: 'UserSchema',
required: true
},
status: {
type: String,
required: true
}
});
const ShopSchema = new mongoose.Schema({
name: {
type: String,
required: true
},
address: {
//your address model
}
});
const UserSchema = new mongoose.Schema({
name: {
type: String,
required: true
},
phone: {
type: String,
required: true,
},
// Add something like this
shopsJoined: {
type: Array,
default: [],
required: true
}
});
You could tackle this problem via 2 ways:
MongoDB Aggregates
When retrieving (back-end side) the list of shops, if you know the user that made the request, instead of simply returning the list of Shops, you could return an aggregate of Shops and Members resulting in an hybrid document containing both the info of Shops and Models. That way, the front-end have all the info it needs with one back-end request.
Important note
The following code might not work as-is and you'll have to adapt it, I currently have nothing to test it against. Keep in mind I'm not very familiar with aggregates, let alone with Mongoose, but you'll get the general idea by looking the code and comments.
const aggregateShops = async (req, res, next) => {
try {
// $lookup will merge the "Model" and "Shop" documents into one
// $match will return only the results matching the condition
const aggreg = await Model.aggregate({$lookup: {
from: 'members', //the name of the mongodb collection
localField: '_id', //the "Shop" field to match with foreign collection
foreignField: 'shopId', //the "Member" field to match with local collection
as: 'memberInfo' //the field name in which to store the "Member" fields;
}, {
$match: {memberInfo: {userId: myUserId}}
}});
// the result should be an array of object looking like this:
/*{
_id: SHOP_OBJECT_ID,
name: SHOP_NAME,
address: SHOP_ADDRESS,
memberInfo: {
shopId: SHOP_OBJECT_ID,
userId: USER_OBJECT_ID,
status: STATUS_JOINED_OR_NOT
}
}*/
// send back the aggregated result to front-end
} catch (e) {
return next(e);
}
}
Drop the Members collection and store the info elsewhere
Instinctively, I would've gone this way. The idea is to either store an array field shopsJoined in the User model, or a membersJoined array field in the Shops model. That way, the info is retrieved no matter what, since you still have to retrieve the Shops and you already have your User.
// Your PATCH route should look like this
const patchUser = async (req, res, next) => {
try {
// How you chose to proceed here is up to you
// I tend to facilitate front-end work, so get them to send you (via req.body) the shopId to join OR "un-join"
// They should already know what shops are joined or not as they have the User
// For example, req.body.shopId = "+ID" if it's a join, or req.body.shopId = "-ID" if it's an un-join
if (req.body.shopId.startsWith("+")) {
await User.findOneAndUpdate(
{ _id: my_user_id },
{ $push: { shopsJoined: req.body.shopId } }
);
} else if (req.body.shopId.startsWith("-")) {
await User.findOneAndUpdate(
{ _id: my_user_id },
{ $pull: { shopsJoined: req.body.shopId } }
);
} else {
// not formatted correctly, return error
}
// return OK here depending on the framework you use
} catch (e) {
return next(e);
}
};
Of course, the above code is for the User model, but you can do the same thing for the Shop model.
Useful links:
MongoDB aggregation pipelines
Mongoose aggregates
MongoDB $push operator
MongoDB $pull operator
Yes you have to add the field to the model because adding it to the response will be only be a temporary display of the key but what if you need that in the future or in some list filters, so its good to add it to the model.
If you are thinking that front-end will have to be informed so just go it, and also you can set some default values to the "isShopJoined" key let it be flase for the time.
I'm stuck with some complex query joins in mongoose (using nodejs)
I have the model User
{ active_payment: {type: mongoose.Schema.Types.ObjectId, ref: 'Payment'}}
And the model payment:
{status : Number, creation_date: Date}
I want to find all the users with the a paument of a status = 2.
I've tried:
User.find({'active_payment.status': 2,}).populate('active_payment')
But its not working. Is there any way to do it without having to sort all the users by a for loop?
I would also like to sort users by the creation_date of the payment.
You can not reach status in User.find() directly, Please try this one
User
.find({})
.populate({
path: 'active_payment',
match: { status: 2},
})
.exec(function(err, users) {
users = users.filter(function(user) {
return user.active_payment; // filter the user by active_payment field
});
});
With match to filter the status is 2 populate documents, then filter user after populate though active_payment.
I have two models - User and Tag. Part of User model
module.exports = {
attributes: {
tags: {
collection: "tag",
via: "users"
}
}};
Tag model looks like
module.exports = {
attributes: {
name: {
type: 'string',
required: true
},
users: {
collection: "user",
via: "tags"
},
}};
And I need to sort tags by "users.length" (each tag can has different number of users). I tried this
Tag.find().populate('users').sort('users.length DESC').exec(function(err, tags){});
And also I tried something like this
Tag.native(function (err, collection) {
if (err) return res.serverError(err);
collection.aggregate(
[
{ $sort : { users : -1 } }
], function (err, results) {
});
});
But I had a fail. Documents are not sorted in right order. Can you help me to sort documents as I want please? I'm a new in Sails and in Mongo too. And I have no idea how I can do this.
P.S.
I use sails-mongo adapter version 0.11.2
Unfortunately, there's no way of doing this as of now.
I would suggest maintaining a count field in the Tag model itself and updating it each time a User adds or removes a Tag. Then run a normal query like you did with count as the sort key. Doing this also allows you to query faster if you just need Tag information (and not the users) since you can skip the populate() call which internally references another collection.
I have the following schemas which are used to represent a many-to-many relationship :
var CategorySchema = new Schema({
title: {type: String},
});
mongoose.model('Category', CategorySchema);
var ProductSchema = new Schema({
title: {type: String},
categories: [
{
type: Schema.ObjectId,
ref: 'Category'
}
]
});
mongoose.model('Product', ProductSchema );
When I query the Categories or the Products I want to be able to get in the result all the linked documents.
Populating the categories when querying the Product is straightforward:
Product.find().populate('categories').exec(...)
But how to do this from the Category side? I know I can add an array of ObjectId ref to the Product documents in the CategorySchema. But I'd like to avoid bi-directional referencing (I don't want to maintain it, and have a risk of inconsistency).
EDIT: here is the solution I implemented
/**
* List all Categories
*/
exports.all = function (req, res) {
//Function needed in order to send the http response only once all
//the categories' product has been retrieved and added to the returned JSON document.
function sendResponse(categories) {
res.json(categories);
}
AppCategory.list(function (err, categories) {
if (err) {
errors.serverError();
} else {
_.forEach(categories, function (category, index) {
category.products = [];
Product.byCategory(category._id, function (err, products) {
category.products= category.products.concat(products);
if (index === categories.length - 1) {
sendResponse(categories);
}
});
});
}
});
};
ProductSchema.statics = {
byCategory: function (categoryId, callback) {
this.find({'categories': categoryId})
.sort('-title')
.exec(callback);
}
};
You probably don't want to do that. :-) I would guess a product can be in some reasonably-small number of categories, but a category might have many thousands of products. In that case, trying to do Category.populate('products') is not going to work from an efficiency standpoint. You'll use lots of memory, not be able to do pagination in a straightforward way, load duplicate product data into memory when a product belongs to several categories, etc. Better to load the products in a category by querying directly against the products collection. You can filter by category easily enough a la Product.find({'categories._id': $in: arrayOfCategoryIds}}).
I'm trying to hide certain fields on my GET output for my REST server. I have 2 schema's, both have a field to embed related data from eachother into the GET, so getting /people would return a list of locations they work at and getting a list of locations returns who works there. Doing that, however, will add a person.locations.employees field and will then list out the employees again, which obviously I don't want. So how do I remove that field from the output before displaying it? Thanks all, let me know if you need any more information.
/********************
/ GET :endpoint
********************/
app.get('/:endpoint', function (req, res) {
var endpoint = req.params.endpoint;
// Select model based on endpoint, otherwise throw err
if( endpoint == 'people' ){
model = PeopleModel.find().populate('locations');
} else if( endpoint == 'locations' ){
model = LocationsModel.find().populate('employees');
} else {
return res.send(404, { erorr: "That resource doesn't exist" });
}
// Display the results
return model.exec(function (err, obj) {
if (!err) {
return res.send(obj);
} else {
return res.send(err);
}
});
});
Here is my GET logic. So I've been trying to use the query functions in mongoose after the populate function to try and filter out those references. Here are my two schema's.
peopleSchema.js
return new Schema({
first_name: String,
last_name: String,
address: {},
image: String,
job_title: String,
created_at: { type: Date, default: Date.now },
active_until: { type: Date, default: null },
hourly_wage: Number,
locations: [{ type: Schema.ObjectId, ref: 'Locations' }],
employee_number: Number
}, { collection: 'people' });
locationsSchema.js
return new Schema({
title: String,
address: {},
current_manager: String, // Inherit person details
alternate_contact: String, // Inherit person details
hours: {},
employees: [{ type: Schema.ObjectId, ref: 'People' }], // mixin employees that work at this location
created_at: { type: Date, default: Date.now },
active_until: { type: Date, default: null }
}, { collection: 'locations' });
You should specify the fields you want to fetch by using the select() method. You can do so by doing something like:
if( endpoint == 'people' ){
model = PeopleModel.find().select('locations').populate('locations');
} else if( endpoint == 'locations' ){
model = LocationsModel.find().select('employees').populate('employees');
} // ...
You can select more fields by separating them with spaces, for example:
PeopleModel.find().select('first_name last_name locations') ...
Select is the right answer but it also may help to specify it in your schema so that you maintain consistency in your API and I've found it helps me to not remember to do it everywhere I perform a query on the object.
You can set certain fields in your schema to never return by using the select: true|false attribute on the schema field.
More details can be found here: http://mongoosejs.com/docs/api.html#schematype_SchemaType-select
SOLUTION!
Because this was so hard for me to find i'm going to leave this here for anybody else. In order to "deselect" a populated item, just prefix the field with "-" in your select. Example:
PeopleModel.find().populate({path: 'locations', select: '-employees'});
And now locations.employee's will be hidden.
If you remember from you SQL days, SELECT does a restriction on the table(s) being queried. Restrict is one of the primitive operations from the relational model and continues to be a useful feature as the relational model has evolved. blah blah blah.
In mongoose, the Query.select() method allows you to perform this operation with some extra features. Particularly, not only can you specify what attributes (columns) to return, but you can also specify what attributes you want to exclude.
So here's the example:
function getPeople(req,res, next) {
var query = PeopleModel.find().populate({path: 'locations', select: '-employees'});
query.exec(function(err, people) {
// error handling stuff
// process and return response stuff
});
}
function getLocations(req,res, next) {
var query = LocationModel.find().populate({path: 'employees', select: '-locations'});
query.exec(function(err, people) {
// error handling stuff
// processing and returning response stuff
});
}
app.get('people', getPeople);
app.get('locations', getLocations);
Directly from the Mongoose Docs:
Go to http://mongoosejs.com/docs/populate.html and search for "Query conditions and other options"
Query conditions and other options
What if we wanted to populate our fans array based on their age,
select just their names, and return at most, any 5 of them?
Story
.find(...)
.populate({
path: 'fans',
match: { age: { $gte: 21 }},
select: 'name -_id',
options: { limit: 5 }
})
.exec()
I just wanted to remark, for the simplicity of the endpoint you may be able to get away with this way to define the endpoints. However, in general this kind of dispacher pattern is not necessary and may pose problems later in development when developing with Express.