I'm developing a RESTful API with Node.js, Mongoose and Koa and I'm a bit stuck on what are the best practices when it comes to schemas and input validation.
Currently I have both a Mongoose and Joi schema for each resource. The Mongoose schema only includes the basic info about the specific resource. Example:
const UserSchema = new mongoose.Schema({
email: {
type: String,
lowercase: true,
},
firstName: String,
lastName: String,
phone: String,
city: String,
state: String,
country: String,
});
The Joi schema includes details about each property of the object:
{
email: Joi.string().email().required(),
firstName: Joi.string().min(2).max(50).required(),
lastName: Joi.string().min(2).max(50).required(),
phone: Joi.string().min(2).max(50).required(),
city: Joi.string().min(2).max(50).required(),
state: Joi.string().min(2).max(50).required(),
country: Joi.string().min(2).max(50).required(),
}
The Mongoose schema is used to create new instances of the given resource at endpoint handler level when writing to the database.
router.post('/', validate, routeHandler(async (ctx) => {
const userObj = new User(ctx.request.body);
const user = await userObj.save();
ctx.send(201, {
success: true,
user,
});
}));
The Joi schema is used in validation middleware to validate user input. I have 3 different Joi schemas for each resource, because the allowed input varies depending on the request method (POST, PUT, PATCH).
async function validate(ctx, next) {
const user = ctx.request.body;
const { method } = ctx.request;
const schema = schemas[method];
const { error } = Joi.validate(user, schema);
if (error) {
ctx.send(400, {
success: false,
error: 'Bad request',
message: error.details[0].message,
});
} else {
await next();
}
}
I am wondering if my current approach of using multiple Joi schemas on top of Mongoose is optimal, considering Mongoose also has built-int validation. If not, what would be some good practices to follow?
Thanks!
It is a common practice to implement a validation service even if you have mongoose schema. As you stated yourself it will return an validation error before any login is executed on the data. so, it will definitely save some time in that case.
Moreover, you get better validation control with joi. But, it highly depends upon your requirement also because it will increase the extra code you have to write which can be avoided without making much difference to the end result.
IMO, I don't think there's a definite answer to this question. Like what #omer said in the comment section above, Mongoose is powerful enough to stand its own ground.
But if your code's logic/operations after receiving input is pretty heavy and expensive, it won't hurt adding an extra protection in the API layer to prevent your heavy code from running.
Edit: I just found this good answer by a respectable person.
Related
Hello I am new to nodejs and mongodb.
I have 3 models:
"user" with fields "name phone"
"Shop" with fields "name, address"
"Member" with fields "shop user status". (shop and user hold the "id" of respective collections).
Now when I create "shops" api to fetch all shop, then I need to add extra field "isShopJoined" which is not part of the model. This extra field will true if user who see that shop is joined it otherwise it will be false.
The problem happens when I share my model with frontend developers like Android/iOS and others, They will not aware of that extra field until they see the API response.
So is it ok if I add extra field in shops listing which is not part of the model? Or do I need to add that extra field in model?
Important note
All the code below has NOT been tested (yet, I'll do it when I can setup a minimal environment) and should be adapted to your project. Keep in mind that I'm no expert when it comes to aggregation with MongoDB, let alone with Mongoose, the code is only here to grasp the general idea and algorithm.
If I understood correctly, you don't have to do anything since the info is stored in the Member collection. But it forces the front-end to do an extra-request (or many extra-requests) to have both the list of Shops and to check (one by one) if the current logged user is a Member of the shop.
Keep in mind that the front-end in general is driven by the data (and so, the API/back-end), not the contrary. The front-end will have to adapt to what you give it.
If you're happy with what you have, you can just keep it that way and it will work, but that might not be very effective.
Assuming this:
import mongoose from "mongoose";
const MemberSchema = new mongoose.Schema({
shopId: {
type: ObjectId,
ref: 'ShopSchema',
required: true
},
userId: {
type: ObjectId,
ref: 'UserSchema',
required: true
},
status: {
type: String,
required: true
}
});
const ShopSchema = new mongoose.Schema({
name: {
type: String,
required: true
},
address: {
//your address model
}
});
const UserSchema = new mongoose.Schema({
name: {
type: String,
required: true
},
phone: {
type: String,
required: true,
},
// Add something like this
shopsJoined: {
type: Array,
default: [],
required: true
}
});
You could tackle this problem via 2 ways:
MongoDB Aggregates
When retrieving (back-end side) the list of shops, if you know the user that made the request, instead of simply returning the list of Shops, you could return an aggregate of Shops and Members resulting in an hybrid document containing both the info of Shops and Models. That way, the front-end have all the info it needs with one back-end request.
Important note
The following code might not work as-is and you'll have to adapt it, I currently have nothing to test it against. Keep in mind I'm not very familiar with aggregates, let alone with Mongoose, but you'll get the general idea by looking the code and comments.
const aggregateShops = async (req, res, next) => {
try {
// $lookup will merge the "Model" and "Shop" documents into one
// $match will return only the results matching the condition
const aggreg = await Model.aggregate({$lookup: {
from: 'members', //the name of the mongodb collection
localField: '_id', //the "Shop" field to match with foreign collection
foreignField: 'shopId', //the "Member" field to match with local collection
as: 'memberInfo' //the field name in which to store the "Member" fields;
}, {
$match: {memberInfo: {userId: myUserId}}
}});
// the result should be an array of object looking like this:
/*{
_id: SHOP_OBJECT_ID,
name: SHOP_NAME,
address: SHOP_ADDRESS,
memberInfo: {
shopId: SHOP_OBJECT_ID,
userId: USER_OBJECT_ID,
status: STATUS_JOINED_OR_NOT
}
}*/
// send back the aggregated result to front-end
} catch (e) {
return next(e);
}
}
Drop the Members collection and store the info elsewhere
Instinctively, I would've gone this way. The idea is to either store an array field shopsJoined in the User model, or a membersJoined array field in the Shops model. That way, the info is retrieved no matter what, since you still have to retrieve the Shops and you already have your User.
// Your PATCH route should look like this
const patchUser = async (req, res, next) => {
try {
// How you chose to proceed here is up to you
// I tend to facilitate front-end work, so get them to send you (via req.body) the shopId to join OR "un-join"
// They should already know what shops are joined or not as they have the User
// For example, req.body.shopId = "+ID" if it's a join, or req.body.shopId = "-ID" if it's an un-join
if (req.body.shopId.startsWith("+")) {
await User.findOneAndUpdate(
{ _id: my_user_id },
{ $push: { shopsJoined: req.body.shopId } }
);
} else if (req.body.shopId.startsWith("-")) {
await User.findOneAndUpdate(
{ _id: my_user_id },
{ $pull: { shopsJoined: req.body.shopId } }
);
} else {
// not formatted correctly, return error
}
// return OK here depending on the framework you use
} catch (e) {
return next(e);
}
};
Of course, the above code is for the User model, but you can do the same thing for the Shop model.
Useful links:
MongoDB aggregation pipelines
Mongoose aggregates
MongoDB $push operator
MongoDB $pull operator
Yes you have to add the field to the model because adding it to the response will be only be a temporary display of the key but what if you need that in the future or in some list filters, so its good to add it to the model.
If you are thinking that front-end will have to be informed so just go it, and also you can set some default values to the "isShopJoined" key let it be flase for the time.
thanks in advance for your attention on this pretty basic question! I'm currently working on a FreeCodeCamp project involving Node, MongoDB and Mongoose (here's my Glitch project), and I'm very confused as to what is going on when I call .save() on a Mongoose model. You can see the full context on Glitch, but here is the specific part that's not behaving as I'd expect it to:
const { User } = require("./models/user");
...
app.post("/api/exercise/new-user", (req, res) => {
var newUser = new User({ name: req.body.username });
newUser.save((error) => console.error(error));
res.json({username: newUser.name, _id: newUser._id})
});
Here's models/user.js:
const mongoose = require("mongoose");
exports.User = mongoose.model(
"User",
new mongoose.Schema({
name: { type: String, required: true }
})
);
I'm seeing the JSON response in my browser with the information I'd expect, but I'm not seeing a new document in my database on MongoDB. What am I missing here? As far as I know when I call .save() on newUser the document should be showing up in MongoDB. In previous projects I did just this and it was working, and I can't figure out what's different in this situation.
And a broader question that I have that I didn't feel like was explained in the FCC curriculum is: at what point is a collection created in MongoDB by Mongoose? Is it when I create a new model? Like at this point in my code:
const mongoose = require("mongoose");
exports.User = mongoose.model(
"User",
new mongoose.Schema({
name: { type: String, required: true }
})
);
Or does it happen when I go to save an instance of the model? When is MongoDB told to create the collection? Thanks very much in advance!
Well, I've done it again, the answer was very simple. I saw that I was getting an error from the mongoose.connect() call, because it was having trouble with the URI starting with mongodb+srv:// instead of just mongodb://. I then saw that because I had forked the start project from FCC, it had some dated versions of Mongoose and MongoDB, just updating those two to the most recent versions (3.5.7 of MongoDB and 5.9.12 of Mongoose) solved the problem, and I'm now able to save the documents correctly!
I am still kind of new to Mongoose in general. I am building my blogging app which has backend based on Node and MongoDB, I am using Angular for frontend.
I am creating my Restful API which is supposed to allow user click on a post and update it. However, I don't know for sure whether I am doing it the right way here.
This is the schema for my post:
const mongoose = require('mongoose');
// Schema Is ONly bluePrint
var postSchema = mongoose.Schema({
title: {type: String, required: true },
content: {type: String, required: true},
}, {timestamps: true});
module.exports = mongoose.model("Post", postSchema);
In my angular service, I have this function to help me to send the http request to my backend server, the id for this function comes from backend mongoDB, title and content is from the form on the page
updatePost(id: string, title: string, content: string) {
console.log('start posts.service->updatePost()');
const post: Post = {
id: id,
title: title,
content: content
};
this._http.put(`http://localhost:3000/api/posts/${id}`, post)
.subscribe(res => console.log(res));
}
It appears to me that there are at least couple of ways of approaching this for creating my API
Method 1 ( works but highly doubt if this is good practice):
here I am passing the id retrieved from mongoDB back to server via my service.ts file to avoid the 'modifying immutable field _id' error
app.put("/api/posts/:id", (req,res)=>{
console.log('update api called:', req.params.id);
const post = new Post({
id: req.body.id,
title: req.body.title,
content: req.body.content
});
Post.updateOne({_id: req.params.id}, post).then( result=> {
console.log(result);
res.json({message:"Update successful!"});
});
});
Method 2 I consider this is more robust than method 1 but still I don't think its good practice:
app.put("/api/posts/:id", (req, res)=> {
Post.findOne(
{_id:req.params.id},(err,post)=>{
if(err){
console.log('Post Not found!');
res.json({message:"Error",error:err});
}else{
console.log('Found post:',post);
post.title=req.body.title;
post.content=req.body.content;
post.save((err,p)=>{
if(err){
console.log('Save from update failed!');
res.json({message:"Error",error:err});
}else{
res.json({message:"update success",data:p});
}
})
}
}
);
});
I am open to all opinions in the hope that I can learn something from guru of Mongoose and Restful : )
Justification to Choose findOneAndUpdate() in this scenario in simple words are as follow:
You can use findOneAndUpdate() as it updates document based on the
filter and sort criteria.
While working with mongoose mostly we prefer to use this function as compare to update() as it has a an option {new:
true} and with the help of that we can get updated data.
As your purpose here is to updating a single document so you can use findOneAndUpdate(). On the other hand update() should be
used in case of bulk modification.
As update() Always returns on of document modified it won't return updated documents and while working with such a scenario like
your we always returns updated document data in response so we
should use findOneAndUpdate() here
I'm a (relatively inexperienced) Java EE developer who is looking to learn node.js. I'm working with the express framework, mongodb, and the mongoose framework. I've been working on building a simple blog site (just for practice) with an mvc like architecture. It would have would have 4 mongodb collections: post, image, user, comment. The basic Schemas are as follows:
postSchema = mongoose.Schema({
id: Number,
dateCreated, {type: Date, default: Date.now}
title: String,
content: String
});
var Post = mongoose.model('Post', postSchema);
imageSchema = mongoose.Schema({
id: Number,
postId: Number,
path: String
});
var Image = mongoose.model('Image', imageSchema);
userSchema = mongoose.Schema({
id: Number,
username: String,
password: String,
email: String
});
var User = mongoose.model('User', userSchema);
commentSchema = mongoose.Schema({
id: Number,
postId: Number,
userId: Number,
dateCreated: {type: Date, default: Date.now},
content: String
});
var Comment = mongoose.model('Comment', commentSchema);
I want to be able to show a post, an image, comments, and user info all on one page. My issue is that I can't quite figure out how retrieve and send all this data in an asynchronous way. This seems to be what most of the examples I have found do (not necessarily all in one file):
app.get('/', function(res, req) {
Post.findOne(function(err, post) {
if (err) return res.send(500);
res.render('index', post);
});
});
This wouldn't work for me because I would info from the image, comment, and user collections as well. Is there an asynchronous way to do this? If not is there a way to reconfigure what I have so that it could be asynchronous? (I'm trying to get a feel for asynchronous programming.)
Thanks in advance!
Simples way to do this as-is would be to use promises and perform simultaneous async operations:
Post.findOne(id).then(post => {
let postId = post.id;
return Promise.all([
Comments.find({postId}),
Images.find({postId}),
// Not sure what query you need here
User.find(),
post,
]);
}).then(data => {
let [comments, images, users, post] = data;
res.render('index', {comments, images, users, post});
});
In your index template you would have an object with the four properties.
You can perform simultaneous async operations without promises, but I'll leave that for someone else to talk about. I would prefer to work with promises.
Mongoose will also allow you to use other schema definitions as data types as in:
commentSchema = mongoose.Schema({
id: Number,
postId: Post,
userId: User,
dateCreated: {type: Date, default: Date.now},
content: String
});
In this case you can use .populate after some queries in order to perform another query under the hood -- e.g. get the post data for a comment.
Since you are using MongoDB -- a NoSQL Database -- I would look into denormalizing the data and keeping it flat. Firebase, which stores data in a similar structure has a great article on how to store and use denormalized data
How are you handling form validation with Express and Mongoose? Are you using custom methods, some plugin, or the default errors array?
While I could possibly see using the default errors array for some very simple validation, that approach seems to blow up in the scenario of having nested models.
I personally use node-validator for checking if all the input fields from the user is correct before even presenting it to Mongoose.
Node-validator is also nice for creating a list of all errors that then can be presented to the user.
Mongoose has validation middleware. You can define validation functions for schema items individually. Nested items can be validated too. Furthermore you can define asyn validations. For more information check out the mongoose page.
var mongoose = require('mongoose'),
schema = mongoose.Schema,
accountSchema = new schema({
accountID: { type: Number, validate: [
function(v){
return (v !== null);
}, 'accountID must be entered!'
]}
}),
personSchema = new schema({
name: { type: String, validate: [
function(v){
return v.length < 20;
}, 'name must be max 20 characters!']
},
age: Number,
account: [accountSchema]
}),
connection = mongoose.createConnection('mongodb://127.0.0.1/test');
personModel = connection.model('person', personSchema),
accountModel = connection.model('account', accountSchema);
...
var person = new personModel({
name: req.body.person.name,
age: req.body.person.age,
account: new accountModel({ accountID: req.body.person.account })
});
person.save(function(err){
if(err) {
console.log(err);
req.flash('error', err);
res.render('view');
}
...
});
I personaly use express-form middleware to do validation; it also has filter capabilities. It's based on node-validator but has additional bonuses for express. It adds a property to the request object indicating if it's valid and returns an array of errors.
I would use this if you're using express.