I am using express-validator for validation. I am using mongoose for database, it also has validation built in. I want to know which one should I use?
I also want to know if the validation in express-validator is parallel. Take this code for example:
req.checkBody('email', 'Invalid email').notEmpty().isEmail().isUnique();
req.checkBody('password', 'Invalid possword').notEmpty().len(8, 30);
req.checkBody('first_name', 'Invalid first_name').notEmpty().isAlpha();
req.checkBody('last_name', 'Invalid last_name').notEmpty().isAlpha();
req.checkBody('dateofbirth', 'Invalid dateofbirth').notEmpty.isDate();
isUnique() is a custom validation method that checks if the email has not already been registered or not, it queries to database to validate so. Though not mentioned in the code above but I also have few other post request where I need to validate multiple fields where database queries will be made in each of them.
So I wanted to know if its possible to run each of the above check method in parallel as that would make it faster and would also me more node like. I obviously will like to use a module for running these in parallel like async. I would also like to know if already these check methods are running in parallel already?
Please help me figure this out? Thanks in advance.
express-validator is meant to validate input passed by the browser/client; Mongoose's validation is meant to validate newly created documents. Both serve a different purpose, so there isn't a clean-cut answer to which one you should use; you could use both, even.
As for the order of validation: the checks will be performed in series. You could use async.parallel() to make it appear as if the checks are performed in parallel, but in reality they won't be since the checks are synchronous.
EDIT: node-validator (and therefore express-validator) is a string validator. Testing for uniqueness isn't a string operation but operates on your data model, so you shouldn't try to use node-validator for it (in fact, I don't even think you can).
Instead, I would suggest using Mongoose's unique feature to ensure that an e-mail address only occurs once in your database.
Alternatively, use a validator module that supports async operations, like async-validate.
Using both mongodb unique constraint and express-validator will cause a little bit complicated errors processing: you need to analyse errors in different places and translate it to common format for rest response.
So another approach will be creating custom express validator which will find record with such value in mongodb:
router.use(expressValidator({
customValidators: {
isUsernameAvailable(username) {
return new Promise((resolve, reject) => {
User.findOne({ username: username }, (err, user) => {
if (err) throw err;
if(user == null) {
resolve();
} else {
reject();
}
});
});
}
}
})
);
...
req.checkBody('username', 'Username is required').notEmpty();
req.checkBody('username', 'Username already in use').isUsernameAvailable();
req.asyncValidationErrors().then(() => {
...
If you need, see total example with this code on my website: https://maketips.net/tip/161/validate-username-available-using-express-validator
Conclusion: express-validator supports asynchronous validation as well so you can perform any data and string validation you need without mixing user validation and low-level db backend validation
There is no way that Express-Validator can check a email address is unique or not, on the other hand MongoDB is not responsible for checking your entered email address is in correct format.
So you must have to use both of them when they are in need because they serve completly diffrent purpose.
Related
i was told that using res.locals will decrease the performance of my application , and it's better to attach variables on the request.
in my case i want to attach variables that are accessible only on the server side , and i don't want it to be sent back to the user , and i also came across sending the variable using
next(value) , what is the best approach for my case??
i have this middleware that gets the id of the user from jwt
jwt.verify(
accessToken,
process.env.ACCESS_TOKEN_KEY,
function (err, payload) {
if (err)
return res.status(401).send({
status: "failure",
response: "access token is not valid",
});
id = payload.id;
}
);
res.locals.userId = id;
next();
then this middleware that gets the role of the user based on the id
const RoleId = await sequelize.models.User.findByPk(res.locals.userId);
if (RoleId === 1) {
res.locals.title = "Admin";
next();
} else {
res.locals.title = "Customer";
next();
}
res.locals.xxx will have no different performance from setting res.xxx or req.xxx. No difference at all. So, without a specific reference to whomever said one would be slower than the other that somehow has more context, that's not correct.
And, res.locals are not sent to the client. Template engines doing server-side rendering by convention will look in res.locals to find variables that the template may reference that were not explicitly passed to res.render(). This is very useful for having common data (like a user's name) that you want the template to use, but don't want to have to manually pass to every single res.render() call.
This is entirely under your control since you, on the server, control both the template and the server-side rendering. This allows you to insert things into the page which is then sent to the user, but nothing goes in the template that you don't put there. So, res.locals is not a security risk unless you somehow give an outside agent control over your template (which would be subject to all sorts of security issues beyond just res.locals.
Another advantage of res.locals is that its an independent namespace that is entirely reserved for your use. No variable you use there will conflict with any existing functionality of the http class or Express or whatever web server engine you're using. The res object, on the other hand, has all sorts of existing methods and properties that you have to make sure you don't overwrite. So, res.locals is safer in that regard as it is specifically reserved for your own use. You can put any named property in there without any risk of conflict.
Passing a value to next(value) is how you abort further routing and immediately invoke your error handler, passing value to the error handler. This is not how middleware communicates data to downstream routing or rendering.
Both your code examples look like proper use of res.locals to me. That is exactly what it's for.
On a completely separate topic, your first code example has a coding error in it. You need to put these:
res.locals.userId = id;
next();
Inside the jwt.verify() callback. jwt.verify() is asynchronous. You won't have the id value until that callback is called. It should be like this:
jwt.verify(accessToken, process.env.ACCESS_TOKEN_KEY, function(err, payload) {
if (err) {
return res.status(401).send({
status: "failure",
response: "access token is not valid",
});
}
res.locals.userId = payload.id;
next();
});
My web app will receive a lot of user-generated content. I want to improve basic security by validating inputs that users generate. My backend is running Node+Express.
How should I implement input validation?
With assert?
Coming from Python, I started writing assert statements:
assert(title.length > 0)
With express-validator?
Then I discovered that there are form validation libraries, such as express-validator. This seemed like the 'right' way of doing it. But compared to simple assertion statements, it also seemed actually like I would double or triple the number of lines of code I need to write.
With TypeScript?
Then I realised that perhaps I could just write some of my Node code in TypeScript, and it would do all the validation for me.
?
Which approach should I use?
The first thing you should do is sanitize the user input to prevent XSS.
There are many NPM packages for that, DOMPurify does a great job.
The most basic way to validate the input is to use if statements, for example:
app.post("/user", (req, res) => {
const { email, password } = req.body;
if (!email || !validator.isEmail(email)) {
return res.status(400).send("Invalid email");
} else if (!password || password.length < 8) {
return res.status(400).send("Invalid password");
}
// Do stuff
return res.sendStatus(200);
});
Validation libraries are pretty useful, in the example above I used the isEmail function from validator.
Note that TypeScript does not do any runtime validation, it performs type-checking during compile time and then compiles the whole TypeScript code into JavaScript, stripping away all the types.
Which means that, for example, even if a field you specified as a string in TypeScript turns out to be a number during runtime, JavaScript won't throw any error, so it's really important that you make sure that every input field is validated.
Validating each input in each request could become quite repetitive and ugly to look at, but thankfully you can implement Express middlewares that you can easily add to the requests that need them.
Hi please can anyone help me. I want user to be able to access only what they are permitted to access.
I have been looking at several Access Control List packages. I have not made a final decision.
A restaurant which would have several levels of permission.
The customer can place several orders and can see what foods he has ordered
He can also modify the order only within a specified time period e.g. before the order is being processed.
The customer can only view his own order and the stage which the order is.
A staff can only check the order than is under his menu and state how much the order would cost and how long the order would take.
Another staff would be in charge of the stores and how things goes in and goes out.
A Staff can be in charge of a department and at the same time allow input to a menu which is under another department.
I have been looking at how I can go about putting this into Express.js and mongodb
I have looked at the following
https://github.com/optimalbits/node_acl main focus
https://www.npmjs.com/package/acl
https://www.npmjs.com/package/express-acl
But I have not got the granularity and the mix which I stated above.
The permission would be based majorly on data. It has been a little confusing as to how I can go about that.
Any help will be useful
I use mongoose as my driver
As I said in my comments, this design has a bit of business logic that might make it not the best fit for regular ACL-type security controls. On the surface, it seems like the easier to figure out solution would be to just implement your business rules in your Mongoose models or Controller code, depending on your preference. That said, a key piece of doing any of this with an ACL-like approach comes down to your URL design. For example, it's tempting to make your API such that all orders are available through /api/orders and maybe a person would query their own orders via /api/orders?userId=12345. But that makes most ACL-based approaches fail. Instead you have to think about the API in terms of the hierarchy as you want it secured (regardless of if all orders are stored in the Orders Mongoose model, and persist in the orders collection).
So using your first requirement as an example
The customer can place several orders and can see what foods he has ordered
The focus here is that you are securing things by the customer 'owner' of the orders, so to secure it that way you need to setup your route that way, eg (assuming you're using the first middleware you asked about):
app.post('/api/customer/:customerId/orders', acl.middleware(), (req, res, next) => {
const order = new Order(req.body); // TODO: whitelist what info you take in here
order.customerId = req.user.id; // assuming you have a logged-in user that does this
order.save(e => {
if (e) return next(e);
return res.status(201).send(order);
});
});
To support this, you'd register your ACL info as such:
acl.allow('12345', '/api/customer/12345/orders', ['post']);
Minimally, you'd do that. You would likely provide more options such as 'get', etc. As you can guess, this means that you'll need to register permissions for individual users whenever you create them (to support the concept of 'ownership').
For your second requirement,
He can also modify the order only within a specified time period e.g. before the order is being processed.
Despite what I said before, you could arguably do this in an ACL if you really wanted to. For example, you could make the URL account for the status, like '/api/customers/12345/orders/modifiable/6789', but that becomes hard to maintain in my experience. You're better off putting that logic in the controller or the Mongoose logic. It's probably simpler to do it in the controller, unless you plan on using your Mongoose models outside of the Express app. Something like this (note, not using the ACL in this case, though you could if you wanted):
app.param('orderId', (req, res, next, id) => {
Order.findById(id, (err, order) => {
if (err) return next(err);
if (order) {
req.order = order;
return next();
}
const notFound = new Error('Order not found');
notFound.status = 404;
return next(notFound);
});
});
app.put('/api/orders/:orderId', (req, res, next) => {
if (req.order.status !== 'pending') {// or whatever your code setup is
const notProcessable = new Error('Cannot modify an order in process');
notProcessable.status = 422;
return next(notProcessable);
}
// handle the modification and save stuff
});
I am developing a MEAN stack application but i stuck at validation(user-inputs). I know the client-side validations are done in Angular-JS but i am confused where to validate on server side. Either to use mongooose for validations or don't do any validation on mongooose(just store) do validations only in server(Node/express using some modules) or validate on both server-level and database-level....????
What should i do pls help me choose which practice is best.....????
Thanks.
As a MEAN stack developer there are various ways to validate the form...
1.) AngularJs form validations
2.) Mongoose Validation
3.) Backend Validation ( Express validator ot nodeValidator )
==> Validation in mongoose
Ex.
var userSchema = new Schema({
phone: {
type: String,
required: [true, 'User phone number required']
},
name: String,
subDoc: [
{
newName:String,
data:String
}
]
});
Above code shows simple integration of mongoose validator.
Validation in mongoose schema can create problems when writing the document
and worse situation can be generated when there are nested field and i am sure there will be. There are chances of modification of schema in such situation you to manage the validation, so much trouble i have faced..
So its a better IDEA to go with node validator or express validator which are super simple to use and provide lots of different type of validation options..
TRY
node-validator,
Express validator
js.com/package/node-validator
I personally suggest Express validator. which is simple and provide most of thing you need.
Banking just on mongoose validation is not enough. Your routes must be secured to not let invalid data pass through to mongoose validation.
The best place for this validation is middleware.
router.get('/', validateParameters, controller.index);
The function takes req, res and next. Call next if req parameters are valid, else respond with 400 status code. This way, only valid call reaches the end point.
function validateParameters(req, res, next){
// if req.params/req.body/req.query is valid
// return next();
// else return res.status(400).end;
}
As to actually how to validate, you can check express-validator package, which gives you lots of options to validate req body, query and params. One good option is creating a schema for each route and just passing that schema to validateParameters function and validating it with one of req.check(schema) (to check all), req.checkQuery(schema), req.checkBody(schema) orreq.checkParams(schema), depending on where your parameters are.
You may also want to take a look at mongoose-validatorjs.
I'm writing a website which implements a usermanagement system and I wonder what best practices regarding form processing I have to consider.
Especially performance, security, SEO and user experience are important to me. When I was working on it I came across a couple questions and I didn't find an complete node/express code snippet where I could figure out all of my below questions.
Use case: Someone is going to update the birthday of his profile. Right now I am doing a POST request to the same URL to process the form on that page and the POST request will respond with a 302 redirect to the same URL.
General questions about form processing:
Should I do a POST request + 302 redirect for form processing or rather something else like an AJAX request?
How should I handle invalid FORM requests (for example invalid login, or email address is already in use during signup)?
Express specific questions about form processing:
I assume before inserting anything into my DB I need to sanitize and validate all form fields on the server side. How would you do that?
I read some things about CSRF but I have never implemented a CSRF protection. I'd be happy to see that in the code snippet too
Do I need to take care of any other possible vulnerabilities when processing forms with Express?
Example HTML/Pug:
form#profile(method='POST', action='/settings/profile')
input#profile-real-name.validate(type='text', name='profileRealName', value=profile.name)
label(for='profile-real-name') Name
textarea#profile-bio.materialize-textarea(placeholder='Tell a little about yourself', name='profileBio')
| #{profile.bio}
label(for='profile-bio') About
input#profile-url.validate(type='url', name='profileUrl', value=profile.bio)
label(for='profile-url') URL
input#profile-location.validate(type='text', name='profileLocation', value=profile.location)
label(for='profile-location') Location
.form-action-buttons.right-align
a.btn.grey(href='' onclick='resetForm()') Reset
button.btn.waves-effect.waves-light(type='submit')
Example Route Handlers:
router.get('/settings/profile', isLoggedIn, profile)
router.post('/settings/profile', isLoggedIn, updateProfile)
function profile(req, res) {
res.render('user/profile', { title: 'Profile', profile: req.user.profile })
}
function updateProfile(req, res) {
var userId = req.user._id
var form = req.body
var profile = {
name: form.profileRealName,
bio: form.profileBio,
url: form.profileUrl,
location: form.profileLocation
}
// Insert into DB
}
Note: A complete code snippet which takes care of all form processing best practices adapted to the given example is highly appreciated. I'm fine with using any publicly available express middleware.
Should I do a POST request + 302 redirect for form processing or rather something else like an AJAX request?
No, best practice for a good user experience since 2004 or so (basically since gmail launched) has been form submission via AJAX and not web 1.0 full-page load form POSTs. In particular, error handling via AJAX is less likely to leave your user at a dead end browser error page and then hit issues with the back button. The AJAX in this case should send an HTTP PATCH request to be most semantically correct but POST or PUT will also get the job done.
How should I handle invalid FORM requests (for example invalid login, or email address is already in use during signup)?
Invalid user input should result in an HTTP 400 Bad Request status code response, with details about the specific error(s) in a JSON response body (the format varies per application but either a general message or field-by-field errors are common themes)
For email already in use I use the HTTP 409 Conflict status code as a more particular flavor of general bad request payload.
I assume before inserting anything into my DB I need to sanitize and validate all form fields on the server side. How would you do that?
Absolutely. There are many tools. I generally define a schema for a valid request in JSON Schema and use a library from npm to validate that such as is-my-json-valid or ajv. In particular, I recommend being as strict as possible: reject incorrect types, or coerce types if you must, remove unexpected properties, use small but reasonable string length limits and strict regular expression patterns for strings when you can, and of course make sure your DB library property prevents injection attacks.
I read some things about CSRF but I have never implemented a CSRF protection.
The OWSAP Node Goat Project CSRF Exercise is a good place to start with a vulnerable app, understand and exploit the vulnerability, then implement the fix (in this case with a straightforward integration of the express.csrf() middleware.
Do I need to take care of any other possible vulnerabilities when processing forms with Express?
Yes generally application developers must understand and actively code securely. There's a lot of material out there on this but particular care must be taken when user input gets involved in database queries, subprocess spawning, or being written back out to HTML. Solid query libraries and template engines will handle most of the work here, you just need to be aware of the mechanics and potential places malicious user input could sneak in (like image filenames, etc).
I am certainly no Express expert but I think I can answer at least #1:
You should follow the Post/Redirect/Get web development pattern in order to prevent duplicate form submissions. I've heard a 303-redirect is the proper http statuscode for redirecting form submissions.
I do process forms using the POST route and once I'm done I trigger a 302-redirect.
As of #3 I recommend looking into express-validator, which is well introduce here: https://developer.mozilla.org/en-US/docs/Learn/Server-side/Express_Nodejs/forms . It's a middleware which allows you to validate and sanitize like this:
req.checkBody('name', 'Invalid name').isAlpha();
req.checkBody('age', 'Invalid age').notEmpty().isInt();
req.sanitizeBody('name').escape();
I wasn't able to comment hence the answer even though it's not a complete answer. Just thought it might help you.
If user experience is something you're thinking about, a page redirection is a strong no. Providing a smooth flow for the people visiting your website is important to prevent drops, and since forms are already not such a pleasure to fill, easing their usage is primary. You don't want to reload their page that might have already took some time to load just to display an error message. Once the form is valid and you created the user cookie, a redirection is fine though, even if you could do things on the client app to prevent it, but that's out-of-scope.
As stated by Levent, you should checkout express-validator, which is the more established solution for this kind of purpose.
req.check('profileRealName', 'Bad name provided').notEmpty().isAlpha()
req.check('profileLocation', 'Invalid location').optional().isAlpha();
req.getValidationResult().then(function (result) {
if (result.isEmpty()) { return null }
var errors = result.array()
// [
// { param: "profileRealName", msg: "Bad name provided", value: ".." },
// { param: "profileLocation", msg: "Invalid location", value: ".." }
// ]
res.status(400).send(errors)
})
.then(function () {
// everything is fine! insert into the DB and respond..
})
From what it looks like, I can assume you are using MongoDB. Given that, I would recommend using an ODM, like Mongoose. It will allow you to define models for your schemas and put restrictions directly on it, letting the model handles these kind of redundant validations for you.
For example, a model for your user could be
var User = new Schema({
name: { type: String, required: [true, 'Name required'] },
bio: { type: String, match: /[a-z]/ },
age: { type: Number, min: 18 }, // I don't know the kind of site you run ;)
})
Using this schema on your route would be looking like
var user = new User({
name: form.profileRealName,
bio: form.profileBio,
url: form.profileUrl,
location: form.profileLocation
})
user.save(function (err) {
// and you could grab the error here if it exists, and react accordingly
});
As you can see it provides a pretty cool api, which you should read about in their docs if you want to know more.
About CRSF, you should install csurf, which has pretty good instructions and example usages on their readme.
After that you're pretty much good to go, there is not much more I can think about apart making sure you stay up to date with your critical dependencies, in case a 0-day occurs, for example the one that happened in 2015 with JWTs, but that's still kinda rare.