I am developing a MEAN stack application but i stuck at validation(user-inputs). I know the client-side validations are done in Angular-JS but i am confused where to validate on server side. Either to use mongooose for validations or don't do any validation on mongooose(just store) do validations only in server(Node/express using some modules) or validate on both server-level and database-level....????
What should i do pls help me choose which practice is best.....????
Thanks.
As a MEAN stack developer there are various ways to validate the form...
1.) AngularJs form validations
2.) Mongoose Validation
3.) Backend Validation ( Express validator ot nodeValidator )
==> Validation in mongoose
Ex.
var userSchema = new Schema({
phone: {
type: String,
required: [true, 'User phone number required']
},
name: String,
subDoc: [
{
newName:String,
data:String
}
]
});
Above code shows simple integration of mongoose validator.
Validation in mongoose schema can create problems when writing the document
and worse situation can be generated when there are nested field and i am sure there will be. There are chances of modification of schema in such situation you to manage the validation, so much trouble i have faced..
So its a better IDEA to go with node validator or express validator which are super simple to use and provide lots of different type of validation options..
TRY
node-validator,
Express validator
js.com/package/node-validator
I personally suggest Express validator. which is simple and provide most of thing you need.
Banking just on mongoose validation is not enough. Your routes must be secured to not let invalid data pass through to mongoose validation.
The best place for this validation is middleware.
router.get('/', validateParameters, controller.index);
The function takes req, res and next. Call next if req parameters are valid, else respond with 400 status code. This way, only valid call reaches the end point.
function validateParameters(req, res, next){
// if req.params/req.body/req.query is valid
// return next();
// else return res.status(400).end;
}
As to actually how to validate, you can check express-validator package, which gives you lots of options to validate req body, query and params. One good option is creating a schema for each route and just passing that schema to validateParameters function and validating it with one of req.check(schema) (to check all), req.checkQuery(schema), req.checkBody(schema) orreq.checkParams(schema), depending on where your parameters are.
You may also want to take a look at mongoose-validatorjs.
Related
I recently used nestjs, But I realized its overcomplicated, I mean look the following code:
#post('/products')
getAllProducts(#Body('title') title, #Body('price') price, #Body('description') description) { }
It makes function parameters much dirty, Also there could be more decorators above function like #Header, #Params, etc.. Which reduces readability in my opinion.
Same code in nodejs
const { title: title, price: price, description: description } = req.body
nodejs is much more readable...
Then I researched why developers use nestjs, Reason was Modularity. Why we don't implement this on our own...
See below:
See my directory sutructure
In app.js I just kicked the app:
const express = require('express');
const app = express();
// express config
require('./startup/config')(app);
// handling routes
require('./startup/routes')(app);
// db setup
require('./startup/db')(app);
In startup folder I did the basic work like mongoose configuration and connection to db etc..
However, In startup/routes, I just kicked the module as follow:
const shopModule = require('../shop/shop.module');
module.exports = app => {
app.use('/', shopModule);
};
In shop module, I just kicked the routes as follow:
const router = require('express').Router();
const productsRouter = require('./products/index');
const cartRouter = require('./cart/index');
// Products
router.use('/products', productsRouter)
// Cart
router.use('/cart', cartRouter)
module.exports = router;
Now in cart/index.js, I handled the routes related to cart and same for products as follow (I will just show cart):
const router = require('express').Router();
const { getCart } = require('./cart.controller');
router.get('/', getCart);
module.exports = router;
In controller, basically we will do validation stuff etc or extracting data.. Then controller will kick service for database work..
const { userCart } = require('./cart.service');
exports.getCart = (req, res, next) => {
const cart = userCart();
return res.status(200).json(cart);
};
And finally in cart service:
exports.userCart = _ => {
// ... go to database and fetch cart
return [{ prodId: 123, quantity: 2 }];
};
And cart.model.js is responsible for DB schema,
I know the question was too long, but I wanted to explain my question.
I am not saying nestjs should not be used, I am just saying, what about the following structure as it follows the same pattern as angular or nestjs, Right?
With your first point about making the code more readable, why not do something like
#Post('/products')
getAllProducts(#Body() body: any) {}
instead of calling for each part of the body individually, then you can deconstruct the body as you showed with
const {title: title, price: price, description: description} = body;
No need to specify each part of the body that you need as a new parameter, just grab the object itself. The same also goes for #Header(), #Param(), and #Query().
As for how you are setting up your express app, you are completely correct that you can do that; however, if you are working on an open source project, or collaborating with other developers there is nothing that says they have to follow the same format and it could eventually lead to a messy code base. Nest enforces these patterns, similar to how Angular does. Sure, it is still possible to write terrible code, but with an opinionated architecture it does make it more difficult.
NestJS also treats Typescript as a first class citizen, which in my opinion helps get rid of a lot of development problems as well. Plus you get some really cool packages to play with like class-validator and class-transformer to help with validation and transformation of your requests via pipes. You can write an Express server in Typescript, but it isn't necessary, and you can write a NestJS server in JavaScript, but I think you lose a lot of good functionality if you do.
The last point that I don't think has been touched too much it that Nest handles a lot of black boxing for your code through it's modules. If you define a service for a module, it is only available for that module. If you need it in another module you have options, but it helps cut down on the cross-contamination of code and keeps a separations of concerns ideology in mind.
In my opinion, the fact the NestJS gives us specified files for things like authentication (guards), validation and transformation (pipes and interceptors), and error handling (exception filters) makes it easier for anyone to pick up a NestJS server and have a quick understanding of how the request will flow through the server, even if by just looking at the file structure. Personally I know if I see an AuthModule or a guards folder I'm going to be dealing with authentication to some extent.
To answer your final question: there is nothing wrong with how you are showing the express example, especially as you are using Inversion of Control by passing the app into the router to make it work that way, you can absolutely write an Express server that way. Nest just makes you do it that way. (I mean, you could write an entire server using just AppController, AppService and AppModule, but that's really an anti-pattern)
In the end, definitely use what you're comfortable with, but there is a reason Nest has become so popular recently.
it's dirty because you write it in the wrong way
why not use it like this
#Get('/')
getAllProducts(#Body() product: Product) {}
and then destructure it
const {title, price, description} = product
In my node application, I have the following express route.
router.post('/register', asyncMiddleware(async (req, res) => {
const { error } = validateUser(req.body);
if (error)
return res.status(400).send(error.details[0].message);
const { token, user } = await userService.createNewUser(req.body);
res.header('x-authorization-token', token).send(_.pick(user, ['_id', 'name', 'email']));
}));
Is it a good practice to handle validation results like this? Or should I put this in to a middlerware and use it? If so, how can I put this in to a middlerware and use it?
Is it a good practice to handle validation results like this?
The answer is that It Depends.
If it is a small project or there are only a few APIs or you need validation for only this route then probably you can get around with validation like this.
But, in bigger projects, you should definitely have a separate validation middleware.
I personally like to use express-validation along with Joi for validation.
Joi is great for creating flexible and powerful schemas to validate the incoming request. and express-validation takes care of the appropriate status and body.
If you want to see an example of this, checkout my express-api-structure.
I am reusing a Joi schema in multiple places in my code and I would like to find a clean way to test that my endpoints are assigned the validation schema. This would be helpful since I could verify the schema behaved the way I expect without having to repeat the same series of tests everywhere the schema is used.
If I have a hapi server route:
server.route({
method: POST,
path: 'myUrl',
config: {
validate: {
payload: validation.myJoiValidation,
}
}
})
how would I test that the validation.myJoiValidation object has been assigned to the config.validate.payload element?
I dug down into the hapi request object and found that what I am looking for is located in the request.route.settings.validate.payload._inner.children object, but I really don't want to have to rely on that for what I am trying to do.
If you have a server running in the context of your test, you can get the validation schema being used with:
const schema = server.match('post', 'myUrl').settings.validate.payload;
Schemas can't be directly compared (as with Hoek.deepEqual), but they can be compared by using joi.describe, so:
expect(joi.describe(schema)).to.equal(joi.describe(validation.myValidation));
Or, if you are using mocha/chai I think this would be:
expect(joi.describe(schema)).to.deep.equal(joi.describe(validation.myValidation));
In your unit tests make a request with request or similar package with a payload that doesn't pass the validation. Ensure the response code is a 400.
Example test from a project of mine. It tests a regex Join validation on this route. This uses a small promise wrapper around request called yarp.
I'm writing a website which implements a usermanagement system and I wonder what best practices regarding form processing I have to consider.
Especially performance, security, SEO and user experience are important to me. When I was working on it I came across a couple questions and I didn't find an complete node/express code snippet where I could figure out all of my below questions.
Use case: Someone is going to update the birthday of his profile. Right now I am doing a POST request to the same URL to process the form on that page and the POST request will respond with a 302 redirect to the same URL.
General questions about form processing:
Should I do a POST request + 302 redirect for form processing or rather something else like an AJAX request?
How should I handle invalid FORM requests (for example invalid login, or email address is already in use during signup)?
Express specific questions about form processing:
I assume before inserting anything into my DB I need to sanitize and validate all form fields on the server side. How would you do that?
I read some things about CSRF but I have never implemented a CSRF protection. I'd be happy to see that in the code snippet too
Do I need to take care of any other possible vulnerabilities when processing forms with Express?
Example HTML/Pug:
form#profile(method='POST', action='/settings/profile')
input#profile-real-name.validate(type='text', name='profileRealName', value=profile.name)
label(for='profile-real-name') Name
textarea#profile-bio.materialize-textarea(placeholder='Tell a little about yourself', name='profileBio')
| #{profile.bio}
label(for='profile-bio') About
input#profile-url.validate(type='url', name='profileUrl', value=profile.bio)
label(for='profile-url') URL
input#profile-location.validate(type='text', name='profileLocation', value=profile.location)
label(for='profile-location') Location
.form-action-buttons.right-align
a.btn.grey(href='' onclick='resetForm()') Reset
button.btn.waves-effect.waves-light(type='submit')
Example Route Handlers:
router.get('/settings/profile', isLoggedIn, profile)
router.post('/settings/profile', isLoggedIn, updateProfile)
function profile(req, res) {
res.render('user/profile', { title: 'Profile', profile: req.user.profile })
}
function updateProfile(req, res) {
var userId = req.user._id
var form = req.body
var profile = {
name: form.profileRealName,
bio: form.profileBio,
url: form.profileUrl,
location: form.profileLocation
}
// Insert into DB
}
Note: A complete code snippet which takes care of all form processing best practices adapted to the given example is highly appreciated. I'm fine with using any publicly available express middleware.
Should I do a POST request + 302 redirect for form processing or rather something else like an AJAX request?
No, best practice for a good user experience since 2004 or so (basically since gmail launched) has been form submission via AJAX and not web 1.0 full-page load form POSTs. In particular, error handling via AJAX is less likely to leave your user at a dead end browser error page and then hit issues with the back button. The AJAX in this case should send an HTTP PATCH request to be most semantically correct but POST or PUT will also get the job done.
How should I handle invalid FORM requests (for example invalid login, or email address is already in use during signup)?
Invalid user input should result in an HTTP 400 Bad Request status code response, with details about the specific error(s) in a JSON response body (the format varies per application but either a general message or field-by-field errors are common themes)
For email already in use I use the HTTP 409 Conflict status code as a more particular flavor of general bad request payload.
I assume before inserting anything into my DB I need to sanitize and validate all form fields on the server side. How would you do that?
Absolutely. There are many tools. I generally define a schema for a valid request in JSON Schema and use a library from npm to validate that such as is-my-json-valid or ajv. In particular, I recommend being as strict as possible: reject incorrect types, or coerce types if you must, remove unexpected properties, use small but reasonable string length limits and strict regular expression patterns for strings when you can, and of course make sure your DB library property prevents injection attacks.
I read some things about CSRF but I have never implemented a CSRF protection.
The OWSAP Node Goat Project CSRF Exercise is a good place to start with a vulnerable app, understand and exploit the vulnerability, then implement the fix (in this case with a straightforward integration of the express.csrf() middleware.
Do I need to take care of any other possible vulnerabilities when processing forms with Express?
Yes generally application developers must understand and actively code securely. There's a lot of material out there on this but particular care must be taken when user input gets involved in database queries, subprocess spawning, or being written back out to HTML. Solid query libraries and template engines will handle most of the work here, you just need to be aware of the mechanics and potential places malicious user input could sneak in (like image filenames, etc).
I am certainly no Express expert but I think I can answer at least #1:
You should follow the Post/Redirect/Get web development pattern in order to prevent duplicate form submissions. I've heard a 303-redirect is the proper http statuscode for redirecting form submissions.
I do process forms using the POST route and once I'm done I trigger a 302-redirect.
As of #3 I recommend looking into express-validator, which is well introduce here: https://developer.mozilla.org/en-US/docs/Learn/Server-side/Express_Nodejs/forms . It's a middleware which allows you to validate and sanitize like this:
req.checkBody('name', 'Invalid name').isAlpha();
req.checkBody('age', 'Invalid age').notEmpty().isInt();
req.sanitizeBody('name').escape();
I wasn't able to comment hence the answer even though it's not a complete answer. Just thought it might help you.
If user experience is something you're thinking about, a page redirection is a strong no. Providing a smooth flow for the people visiting your website is important to prevent drops, and since forms are already not such a pleasure to fill, easing their usage is primary. You don't want to reload their page that might have already took some time to load just to display an error message. Once the form is valid and you created the user cookie, a redirection is fine though, even if you could do things on the client app to prevent it, but that's out-of-scope.
As stated by Levent, you should checkout express-validator, which is the more established solution for this kind of purpose.
req.check('profileRealName', 'Bad name provided').notEmpty().isAlpha()
req.check('profileLocation', 'Invalid location').optional().isAlpha();
req.getValidationResult().then(function (result) {
if (result.isEmpty()) { return null }
var errors = result.array()
// [
// { param: "profileRealName", msg: "Bad name provided", value: ".." },
// { param: "profileLocation", msg: "Invalid location", value: ".." }
// ]
res.status(400).send(errors)
})
.then(function () {
// everything is fine! insert into the DB and respond..
})
From what it looks like, I can assume you are using MongoDB. Given that, I would recommend using an ODM, like Mongoose. It will allow you to define models for your schemas and put restrictions directly on it, letting the model handles these kind of redundant validations for you.
For example, a model for your user could be
var User = new Schema({
name: { type: String, required: [true, 'Name required'] },
bio: { type: String, match: /[a-z]/ },
age: { type: Number, min: 18 }, // I don't know the kind of site you run ;)
})
Using this schema on your route would be looking like
var user = new User({
name: form.profileRealName,
bio: form.profileBio,
url: form.profileUrl,
location: form.profileLocation
})
user.save(function (err) {
// and you could grab the error here if it exists, and react accordingly
});
As you can see it provides a pretty cool api, which you should read about in their docs if you want to know more.
About CRSF, you should install csurf, which has pretty good instructions and example usages on their readme.
After that you're pretty much good to go, there is not much more I can think about apart making sure you stay up to date with your critical dependencies, in case a 0-day occurs, for example the one that happened in 2015 with JWTs, but that's still kinda rare.
I am using express-validator for validation. I am using mongoose for database, it also has validation built in. I want to know which one should I use?
I also want to know if the validation in express-validator is parallel. Take this code for example:
req.checkBody('email', 'Invalid email').notEmpty().isEmail().isUnique();
req.checkBody('password', 'Invalid possword').notEmpty().len(8, 30);
req.checkBody('first_name', 'Invalid first_name').notEmpty().isAlpha();
req.checkBody('last_name', 'Invalid last_name').notEmpty().isAlpha();
req.checkBody('dateofbirth', 'Invalid dateofbirth').notEmpty.isDate();
isUnique() is a custom validation method that checks if the email has not already been registered or not, it queries to database to validate so. Though not mentioned in the code above but I also have few other post request where I need to validate multiple fields where database queries will be made in each of them.
So I wanted to know if its possible to run each of the above check method in parallel as that would make it faster and would also me more node like. I obviously will like to use a module for running these in parallel like async. I would also like to know if already these check methods are running in parallel already?
Please help me figure this out? Thanks in advance.
express-validator is meant to validate input passed by the browser/client; Mongoose's validation is meant to validate newly created documents. Both serve a different purpose, so there isn't a clean-cut answer to which one you should use; you could use both, even.
As for the order of validation: the checks will be performed in series. You could use async.parallel() to make it appear as if the checks are performed in parallel, but in reality they won't be since the checks are synchronous.
EDIT: node-validator (and therefore express-validator) is a string validator. Testing for uniqueness isn't a string operation but operates on your data model, so you shouldn't try to use node-validator for it (in fact, I don't even think you can).
Instead, I would suggest using Mongoose's unique feature to ensure that an e-mail address only occurs once in your database.
Alternatively, use a validator module that supports async operations, like async-validate.
Using both mongodb unique constraint and express-validator will cause a little bit complicated errors processing: you need to analyse errors in different places and translate it to common format for rest response.
So another approach will be creating custom express validator which will find record with such value in mongodb:
router.use(expressValidator({
customValidators: {
isUsernameAvailable(username) {
return new Promise((resolve, reject) => {
User.findOne({ username: username }, (err, user) => {
if (err) throw err;
if(user == null) {
resolve();
} else {
reject();
}
});
});
}
}
})
);
...
req.checkBody('username', 'Username is required').notEmpty();
req.checkBody('username', 'Username already in use').isUsernameAvailable();
req.asyncValidationErrors().then(() => {
...
If you need, see total example with this code on my website: https://maketips.net/tip/161/validate-username-available-using-express-validator
Conclusion: express-validator supports asynchronous validation as well so you can perform any data and string validation you need without mixing user validation and low-level db backend validation
There is no way that Express-Validator can check a email address is unique or not, on the other hand MongoDB is not responsible for checking your entered email address is in correct format.
So you must have to use both of them when they are in need because they serve completly diffrent purpose.