I'm trying to hide certain fields on my GET output for my REST server. I have 2 schema's, both have a field to embed related data from eachother into the GET, so getting /people would return a list of locations they work at and getting a list of locations returns who works there. Doing that, however, will add a person.locations.employees field and will then list out the employees again, which obviously I don't want. So how do I remove that field from the output before displaying it? Thanks all, let me know if you need any more information.
/********************
/ GET :endpoint
********************/
app.get('/:endpoint', function (req, res) {
var endpoint = req.params.endpoint;
// Select model based on endpoint, otherwise throw err
if( endpoint == 'people' ){
model = PeopleModel.find().populate('locations');
} else if( endpoint == 'locations' ){
model = LocationsModel.find().populate('employees');
} else {
return res.send(404, { erorr: "That resource doesn't exist" });
}
// Display the results
return model.exec(function (err, obj) {
if (!err) {
return res.send(obj);
} else {
return res.send(err);
}
});
});
Here is my GET logic. So I've been trying to use the query functions in mongoose after the populate function to try and filter out those references. Here are my two schema's.
peopleSchema.js
return new Schema({
first_name: String,
last_name: String,
address: {},
image: String,
job_title: String,
created_at: { type: Date, default: Date.now },
active_until: { type: Date, default: null },
hourly_wage: Number,
locations: [{ type: Schema.ObjectId, ref: 'Locations' }],
employee_number: Number
}, { collection: 'people' });
locationsSchema.js
return new Schema({
title: String,
address: {},
current_manager: String, // Inherit person details
alternate_contact: String, // Inherit person details
hours: {},
employees: [{ type: Schema.ObjectId, ref: 'People' }], // mixin employees that work at this location
created_at: { type: Date, default: Date.now },
active_until: { type: Date, default: null }
}, { collection: 'locations' });
You should specify the fields you want to fetch by using the select() method. You can do so by doing something like:
if( endpoint == 'people' ){
model = PeopleModel.find().select('locations').populate('locations');
} else if( endpoint == 'locations' ){
model = LocationsModel.find().select('employees').populate('employees');
} // ...
You can select more fields by separating them with spaces, for example:
PeopleModel.find().select('first_name last_name locations') ...
Select is the right answer but it also may help to specify it in your schema so that you maintain consistency in your API and I've found it helps me to not remember to do it everywhere I perform a query on the object.
You can set certain fields in your schema to never return by using the select: true|false attribute on the schema field.
More details can be found here: http://mongoosejs.com/docs/api.html#schematype_SchemaType-select
SOLUTION!
Because this was so hard for me to find i'm going to leave this here for anybody else. In order to "deselect" a populated item, just prefix the field with "-" in your select. Example:
PeopleModel.find().populate({path: 'locations', select: '-employees'});
And now locations.employee's will be hidden.
If you remember from you SQL days, SELECT does a restriction on the table(s) being queried. Restrict is one of the primitive operations from the relational model and continues to be a useful feature as the relational model has evolved. blah blah blah.
In mongoose, the Query.select() method allows you to perform this operation with some extra features. Particularly, not only can you specify what attributes (columns) to return, but you can also specify what attributes you want to exclude.
So here's the example:
function getPeople(req,res, next) {
var query = PeopleModel.find().populate({path: 'locations', select: '-employees'});
query.exec(function(err, people) {
// error handling stuff
// process and return response stuff
});
}
function getLocations(req,res, next) {
var query = LocationModel.find().populate({path: 'employees', select: '-locations'});
query.exec(function(err, people) {
// error handling stuff
// processing and returning response stuff
});
}
app.get('people', getPeople);
app.get('locations', getLocations);
Directly from the Mongoose Docs:
Go to http://mongoosejs.com/docs/populate.html and search for "Query conditions and other options"
Query conditions and other options
What if we wanted to populate our fans array based on their age,
select just their names, and return at most, any 5 of them?
Story
.find(...)
.populate({
path: 'fans',
match: { age: { $gte: 21 }},
select: 'name -_id',
options: { limit: 5 }
})
.exec()
I just wanted to remark, for the simplicity of the endpoint you may be able to get away with this way to define the endpoints. However, in general this kind of dispacher pattern is not necessary and may pose problems later in development when developing with Express.
Related
Hello I am new to nodejs and mongodb.
I have 3 models:
"user" with fields "name phone"
"Shop" with fields "name, address"
"Member" with fields "shop user status". (shop and user hold the "id" of respective collections).
Now when I create "shops" api to fetch all shop, then I need to add extra field "isShopJoined" which is not part of the model. This extra field will true if user who see that shop is joined it otherwise it will be false.
The problem happens when I share my model with frontend developers like Android/iOS and others, They will not aware of that extra field until they see the API response.
So is it ok if I add extra field in shops listing which is not part of the model? Or do I need to add that extra field in model?
Important note
All the code below has NOT been tested (yet, I'll do it when I can setup a minimal environment) and should be adapted to your project. Keep in mind that I'm no expert when it comes to aggregation with MongoDB, let alone with Mongoose, the code is only here to grasp the general idea and algorithm.
If I understood correctly, you don't have to do anything since the info is stored in the Member collection. But it forces the front-end to do an extra-request (or many extra-requests) to have both the list of Shops and to check (one by one) if the current logged user is a Member of the shop.
Keep in mind that the front-end in general is driven by the data (and so, the API/back-end), not the contrary. The front-end will have to adapt to what you give it.
If you're happy with what you have, you can just keep it that way and it will work, but that might not be very effective.
Assuming this:
import mongoose from "mongoose";
const MemberSchema = new mongoose.Schema({
shopId: {
type: ObjectId,
ref: 'ShopSchema',
required: true
},
userId: {
type: ObjectId,
ref: 'UserSchema',
required: true
},
status: {
type: String,
required: true
}
});
const ShopSchema = new mongoose.Schema({
name: {
type: String,
required: true
},
address: {
//your address model
}
});
const UserSchema = new mongoose.Schema({
name: {
type: String,
required: true
},
phone: {
type: String,
required: true,
},
// Add something like this
shopsJoined: {
type: Array,
default: [],
required: true
}
});
You could tackle this problem via 2 ways:
MongoDB Aggregates
When retrieving (back-end side) the list of shops, if you know the user that made the request, instead of simply returning the list of Shops, you could return an aggregate of Shops and Members resulting in an hybrid document containing both the info of Shops and Models. That way, the front-end have all the info it needs with one back-end request.
Important note
The following code might not work as-is and you'll have to adapt it, I currently have nothing to test it against. Keep in mind I'm not very familiar with aggregates, let alone with Mongoose, but you'll get the general idea by looking the code and comments.
const aggregateShops = async (req, res, next) => {
try {
// $lookup will merge the "Model" and "Shop" documents into one
// $match will return only the results matching the condition
const aggreg = await Model.aggregate({$lookup: {
from: 'members', //the name of the mongodb collection
localField: '_id', //the "Shop" field to match with foreign collection
foreignField: 'shopId', //the "Member" field to match with local collection
as: 'memberInfo' //the field name in which to store the "Member" fields;
}, {
$match: {memberInfo: {userId: myUserId}}
}});
// the result should be an array of object looking like this:
/*{
_id: SHOP_OBJECT_ID,
name: SHOP_NAME,
address: SHOP_ADDRESS,
memberInfo: {
shopId: SHOP_OBJECT_ID,
userId: USER_OBJECT_ID,
status: STATUS_JOINED_OR_NOT
}
}*/
// send back the aggregated result to front-end
} catch (e) {
return next(e);
}
}
Drop the Members collection and store the info elsewhere
Instinctively, I would've gone this way. The idea is to either store an array field shopsJoined in the User model, or a membersJoined array field in the Shops model. That way, the info is retrieved no matter what, since you still have to retrieve the Shops and you already have your User.
// Your PATCH route should look like this
const patchUser = async (req, res, next) => {
try {
// How you chose to proceed here is up to you
// I tend to facilitate front-end work, so get them to send you (via req.body) the shopId to join OR "un-join"
// They should already know what shops are joined or not as they have the User
// For example, req.body.shopId = "+ID" if it's a join, or req.body.shopId = "-ID" if it's an un-join
if (req.body.shopId.startsWith("+")) {
await User.findOneAndUpdate(
{ _id: my_user_id },
{ $push: { shopsJoined: req.body.shopId } }
);
} else if (req.body.shopId.startsWith("-")) {
await User.findOneAndUpdate(
{ _id: my_user_id },
{ $pull: { shopsJoined: req.body.shopId } }
);
} else {
// not formatted correctly, return error
}
// return OK here depending on the framework you use
} catch (e) {
return next(e);
}
};
Of course, the above code is for the User model, but you can do the same thing for the Shop model.
Useful links:
MongoDB aggregation pipelines
Mongoose aggregates
MongoDB $push operator
MongoDB $pull operator
Yes you have to add the field to the model because adding it to the response will be only be a temporary display of the key but what if you need that in the future or in some list filters, so its good to add it to the model.
If you are thinking that front-end will have to be informed so just go it, and also you can set some default values to the "isShopJoined" key let it be flase for the time.
I have an answers schema with over 2M entries.
Schema is
{
user: {
type: Schema.Types.ObjectId,
ref: 'User',
},
q: {
type: Schema.Types.ObjectId,
ref: 'Question',
},
answer: {
type: String,
required: [true, 'Answer required'],
}
}
On the submitAnswers route, I'm checking with findOne based on user and q to see if an answer exists, if it exists then update else create
let check = await Answer.findOne({user: u_id, q: q_id})
if(check) {
check.answer = new_answer
await check.save()
return res.json({message: "Answer saved"})
} else {
let result = //Aggregate query
if(result) {
throw Error();
}
await Answer.create({user: u_id, q:q_id, answer: new_answer})
return res.json({message: "Answer saved"})
}
I currently have the {q: 1, user: 1} index on the Schema for faster searches. Still, there are a lot of repeated data in the database with the same user and q values.
Can't use upsert as that would result in calling that aggregate query for check everytime and it's a heavy query
Am I missing something? OR the index creation takes time? It is a heavy user server.
Facing this only on production server with concurrent users
What you likely have there is a race condition. If 2 people submit the same u_id and q_id at that same time, both will find that there isn't one already, and skip down to the else block where they both create a new document.
I know that the title might be a bit vague, but I will elaborate here.
basically what I am trying to achieve is the following:
I have a collection with documents that have the following scheme:
bookId: <uuid>
genre: <string>
isTaken: true
historyIndex: each time something happens i increment this
returnedAt: not required but will be present on documents with historyIndex
takenAt: not required but will be present on documents with historyIndex
there are documents with no historyIndex field because they are representing the book itself without the action that were done to the book.
what i want to do is this:
I want to query the books by their unique uuid and then use the documents with historyIndex and add them to the main documents as in an array as called bookEvents
so the final results will be
bookId:
bookEvents: [] --> an array with all the entries that contain history index
basically everytime the status of the book changes, i am inserting an event with the date it was taken on and the date it was returned on
What would be the best way of achieving such thing ?
Should I query once and iterate in my code ?
Should I query twice (once where the fields exist and once where they don't) ?
Thank you in advance for all the people that will try to help!
You can use the plugin or events to achieve this.
var CounterSchema = new mongoose.Schema({
_id: {type: String, required: true},
seq: { type: Number, default: 0 }
});
var counter = mongoose.model('counter', CounterSchema);
var entitySchema = mongoose.Schema({
sort: {type: String}
});
entitySchema.pre('save', function(next) {
var doc = this;
counter.findByIdAndUpdateAsync({_id: 'entityId'}, {$inc: { seq: 1} }, {new: true, upsert: true}).then(function(count) {
console.log("...count: "+JSON.stringify(count));
doc.sort = count.seq;
next();
})
.catch(function(error) {
console.error("counter error-> : "+error);
throw error;
});
});
refer https://stackoverflow.com/a/40137723/8201020
I have these schemas:
var Store = mongoose.model('Store', new Schema({
name: String
}));
var Client = mongoose.model('Cllient', new Schema({
name: String,
store: { type: Schema.ObjectId, ref: 'Store' }
}));
var Order = mongoose.model('Order', new Schema({
number: String,
client: { type: Schema.ObjectId, ref: 'Client' }
}));
I'm trying to code the Url handler of the API that returns the order details, which looks like this:
app.get('/api/store/:storeId/order/:orderId', function (...));
I'm passing the store id in the Url to quickly check if the logged user has permissions on the store. If not, it returns a 403 status. That said, I think this storeId and the orderId are enough data to get the order, so I'm trying to do a query on a nested document, but it just doesn't work.
Order.findOne(
{ 'client.store': req.params.storeId, _id: req.params.orderId },
function (err, order) { ... });
But the order object is null;
Even when I perform a find, it returns an empty array:
Order.find(
{ 'client.store': req.params.storeId },
function (err, results) { ... });
I know that I could as well pass the cliendId to the Url and check first if the client belongs to the store, and then retrieve the order from the client, but I think the client part is redundant, don't you think? I should be able to get the order in a secure way by using only these two fields.
What am I doing wrong here?
Ok, I found it. The secret was in the match option of populate. The final code looks like this:
Order
.findOne({ _id: req.params.orderId })
.populate({ path: 'client', match: { store: req.params.storeId } })
.exec(function (err, order) { ... });
I've got an stock application where I want to set some details about the stock and then insert all the items of the stock. I want to insert the stock details and the items in two different collection so then I can filter the items. I'm using the MEAN Stack where I've modified the crud module to accept some extra fields and also made the UI for filling the items array.This what I have so far:
scope.stockItems = [];
$scope.createStockItem = function () {
$scope.stockItems.push(
{
brand: $scope.brand,
style: $scope.style,
amount: $scope.amount
}
);
$scope.brand = false;
$scope.style = false;
$scope.amount = '';
};
// Create new Stock
$scope.create = function() {
// Create new Stock object
var stock = new Stocks ({
name: this.name,
details: this.details,
stockDate: this.stockDate
});
// Redirect after save
stock.$save(function(response) {
$location.path('stocks/' + response._id);
// Clear form fields
$scope.name = '';
}, function(errorResponse) {
$scope.error = errorResponse.data.message;
});
};
The stock model:
var StockSchema = new Schema({
name: {
type: String,
default: '',
required: 'Please fill Stock name',
trim: true
},
details: {
type: String,
default: '',
required: 'Please fill Stock details'
},
stockDate: Date
created: {
type: Date,
default: Date.now
},
user: {
type: Schema.ObjectId,
ref: 'User'
}
});
and the method in the server controller:
exports.create = function(req, res) {
var stock = new Stock(req.body);
stock.user = req.user;
stock.save(function(err) {
if (err) {
return res.status(400).send({
message: errorHandler.getErrorMessage(err)
});
} else {
res.jsonp(stock);
}
});
};
How can I send into the request and save the stockItems also?
By saying 'simultaneously' I think you are requiring transaction feature, which is really an RDBMS thing, and is not supported by MongoDB. If your application strongly relies on such features, I'm afraid MongoDB is not the right choice for you.
So back to your question, I don't understand why you have to store stock and stock item in 2 different collections. Store them in one collection would be a better choice. You can refer to the Data Model Design of MongoDB Manual for more information. If it's just to filter all the stock items, aggregation framework is designed for such purpose. As well as Map/Reduce. Here aggregation framework suits better for your issue. You would have something like:
db.stock.aggregate([
{$match: {...}}, // same as find criteria. to narrow down data range
{$unwind: "$items"}, // unwind items.
... // probably some other $match to filter the items
]);