Proper Handling of fetch errors for Mongoose? - node.js

This is a pure best practice question. I am pretty new to Node and Mongoose. I absolutely love the technology and have been cranking away on a project to build a JSON-backed API for an app that I'm building.
I am finding that I am continuously repeating code when I fetch objects from my database. For example:
Playlist.findById(req.params.id, function(err,playlist){
if (err)
return res.json({error: "Error fetching playlist"});
else if (!playlist)
return res.json({error: "Error finding the playlist"});
//Actual code being performed on the playlist that I'm fetching
});
The error handling at the top of the function call is annoying because I have to repeat that code for every call to the database... or so I think.
I thought about using a callback like:
var fetchCallback = function(err,objOrDoc,callback){
//Handle the error messages
callback(objOrDoc);
};
However, this approach would mess up my sequential flow since I would have to define the callback function before I performed the fetch. So, if I had a lot of database queries chained together, I would have to place the callbacks in reverse order, which is far from ideal in a clean-coding perspective.
I'm wondering if anyone has run into this issue and has any best practices for cutting down on the repetition.
I'm also using the express framework, so if there's a helpful way to handle it in express, I'd be interested to know, too.

There are a couple interesting approaches you could try here.
At the most simple, you could simply have a function that loads up an object and handles the output in an error condition.
fetchResource = function(model, req, res, callback) {
model.findById(req.params.id, function(err, resource) {
if (err)
return res.json({error: "Error fetching " + model.toString()});
else if (!playlist)
return res.json({error: "Error finding the " + model.toString()});
callback(resource);
});
};
app.on('/playlists/1', function(req, res) {
fetchResource(Playlist, req, res, function(playlist) {
// code to deal with playlist.
});
});
That's still quite a bit of duplication, so I might try to move this out into a middleware.
Route Middleware
Routes may utilize route-specific middleware by passing one or more additional callbacks (or arrays) to the method. This feature is extremely useful for restricting access, loading data used by the route etc.
Now I haven't tested this and it's a bit hand-wavey (read: pseudocode), but I think it should serve as a decent example.
// assuming a URL of '/playlist/5' or '/user/10/edit', etc.
function loadResource(model) {
return function(req, res, next) {
model.findById(req.params.id, function(err, resource) {
if (err)
return res.json({error: "Error fetching " + model.toString()});
else if (!resource)
return res.json({error: "Error finding the " + model.toString()});
req.resource = resource;
next();
});
}
}
app.get('/playlist/:id', loadResource(Playlist), function(req, res) {
var playlist = req.resource;
...
});
app.get('/user/:id', loadResource(User), function(req, res) {
var user = req.resource;
...
});
The express source contains a pretty good example of this pattern, and the middleware section in the docs (specifically under 'Route Middleware') details it further.

Related

NodeJS best practices for catching errors

I'm starting out w/ NodeJS and Express. Coming from the other popular scripting languages and C++ background, asynchronously calling DB functions is a bit foreign. I've sorted out a pattern, but I'm still curious about catching exceptions. Below is my basic pattern.
var callback = function(req, res) {
// do stuff
connection.query(queryString, function(err,result){
if (err) throw err;
// process results.
};
};
var express = require('express');
var app = express();
app.get('/', callback);
app.listen(3000,function() {
console.log('listening');
};
Generally I have a lot of endpoints and callbacks. I'm a bit lost on where I set up ta try/catch block to catch errors thrown in the callback though. I've looked around for some suggestions, but a lot of them seem to be on the web framework (if any) being used.
When you throw in an asynchronous callback, the exception just goes back to the internals of the database event handler and there is NO way for you to ever catch or handle that exception. So, basically it does no good at all. It will just cause you to abort the handling of that request and you will never send a response on that request.
Basically, you have several choices for how to handle the error. You can handle it completely right in each endpoint and send some sort of error response.
Send Response right at each point of error
app.get('/', function(req, res) {
// do stuff
connection.query(queryString, function(err,result){
if (err) return res.status(500).send(someErrorResponse);
// process results.
};
});
Forward on to centralized error handler
Or, you can forward the error on to a centralized error handler by calling next(err):
app.get('/', function(req, res, next) {
// do stuff
connection.query(queryString, function(err,result){
// if error, forward it on to our centralized error handler
if (err) return next(err);
// process results.
};
});
// centralized error handler - note how it has four parameters
app.use(function(err, req, res, next) {
// formulate an error response here
console.log(err);
res.status(500).send(someErrorMessage)
});
See Nodejs handle unsupported URLs and request types for more info on the ways to have generalized error handlers in Express.
Use promises to collect errors within each route
If you are using more involved sequences of asynchronous operations where you may have more than one async operation sequenced together, then it does get to be a pain to handle errors at every single async operation. This is where using promises with all your async operations more easily allows all the errors to percolate up to one .catch() statement at the top level of each route. You don't say what database you're using, but here's an idea what that looks like. The general idea is that you can write your code so that all promise rejections (e.g. errors) will propagate up to one central .catch() in each route handler and you can then call next(err) from that .catch(), sending the error to your centralized error handler. Here's how that looks for a recent version of Mongoose (you didn't say which database you were using) with one database operation.
app.get('/', function(req, res, next) {
// do stuff
connection.query(queryString).exec().then(function(result){
// process results.
}).catch(next);
});
// centralized error handler - note how it has four parameters
app.use(function(err, req, res, next) {
// formulate an error response here
console.log(err);
res.status(500).send(someErrorMessage)
});
And, here's what it looks like if you have more than one operation:
app.get('/', function(req, res, next) {
// do stuff
connection.query(queryString).exec().then(function(result){
// process results, then make another query
// return the promise from this second operaton so both results
// and error are chained to the first promise
return connection.query(...).exec();
}).then(function(result) {
// process chained result
}).catch(next);
});
// centralized error handler - note how it has four parameters
app.use(function(err, req, res, next) {
// formulate an error response here
console.log(err);
res.status(500).send(someErrorMessage)
});
Since ES6 built in support for promises and ES7 will add support for async/await for asynchronous operations (which is based on promises) and all significant libraries that offer asynchronous operations have added or are adding support for promises, it is clear that promises are the future of the language for managing asynchronous operations. That would be my strong recommendation.
You should never, ever throw an error like that! :) The reason is that at some point your whole node app will just stop working, because of some db query failed. This should be handled instead of just die.
And because this is a route handler - handles specific url that the user is getting (for example /), there should be some output. You can always show a page with status 500 and a nice design, if there was such an error that you cannot handle or you might have your internal state messed up.
So basically just act as nothing happened - return respones of any kind, or even render a page, but provide information that something went wrong.
Also, a common scenario is something like what Alon Oz presented. All routes in express are actually a middleware functions, that are called one after another. If the route does not match the requested one, the function just skips and the next one is called. You can manually control that. The actual pattern of the router is this:
app.get('/', function(req, res, next) {
// you can have the request
// you can send response like res.send('hello')
// OR you can skip this function using NEXT
});
The actual signature of next is next(err). So if you call it without any arguments, it will just skip to the next middleware. If you call it with an argument, it will skip all regular functions and go to the last ones in the stack, or more specifically the ones that handle errors. They are like the regular ones, but taking four arguments instead of three:
app.use(function (err, req, res, next) { });
It's very important to understand that this function will be called if you call next with an argument. Throwing an error won't do any good! Of course if none of your routes match the specific criteria (url) the final one will in the call will be called, so you can still handle the "not found" error.
This is a common scenario that you will use:
// development error handler, will print stacktrace
if (app.get('env') === 'development') {
app.use(function(err, req, res, next) {
debug('ERROR [ip: %s]:: dev env -> ', req.ip, err); // I'm using debug library - very helpful
res.status(err.status || 500);
res.render('deverr', { // I render custom template with the whole stack beautifully displayed
errMessage: err.message,
error: err
});
});
}
// production error handler, no stacktraces leaked to user
app.use(function(err, req, res, next) {
res.status(err.status || 500);
res.render('pages/error', { // custom error page with nice design and a message
errMessage: err.message,
error: {}
});
});
Hope that helps! :)
Since you are using express, it has its own way to handle exceptions,
defined like this:
function clientErrorHandler (err, req, res, next) {
if (req.xhr) {
res.status(500).send({ error: 'Something failed!' })
} else {
next(err)
}
}
app.use(clientErrorHandler)
For more info:
https://expressjs.com/en/guide/error-handling.html
There are most commonly three major types of errors that we need to take into account.
Promise failures (Any failures that come up during async/await or result of a promise in then/catch)
In order to handle promise failures, as suggested in the strong loop document or node js 2018 best practices, it's important to have a common function that can handle it.
// app.js file
app.get('/:id', async (req,res,next) => {
if(!req.params.id) {
return res.status(412).send('enter a valid user id');
}
try {
const results = await UserDAL(id);
} catch(e) {
next(e);
}
}
// common error middleware defined in middleware/error.js
module.exports = function (err,req,res,next) {
logger.error(`${err.status || 500} - ${err.message} - ${req.originalUrl} - ${req.method} - ${req.ip}`);
return res.status(500).send('something failed.');
};
Unhandled Rejections
process.on('unhandledRejection', e => {
// do something
});
Unhandled exceptions
process.on('uncaughtException', e => {
// do something
});
If you see a lot of try/ catch blocks in your express methods you can abstract that to a separate async function like below:
module.exports = function asyncMiddleWare(handler) {
return async (req,res,next) => {
try {
await handler(req,res)
} catch(e) {
next(e);
}
}
};

Effective way to get data that's needed on all pages

I'm using nodejs and express and I have a navigation menu that is built using data that is in mongodb currently I'm just making a call to the database to get a list of companies and passing that back inside each route. There doesn't seem to be a way to store this information in localstorage client side. So I"m wondering what is the most effective way to handle this situation. Sample of my code
admin.get('/', function(res, resp){
mongodb.connect(url, function(err, db){
var collection = db.collection('companies')
collection.find({}).toArray(function(err, companies){
res.render('adminview', {companies:companies})//once the page is rendered I would like to save the company list to localstorage.
})
})
})
admin.get('/:company', function(res, resp){
/* repeating code from above because I need this list */
mongodb.connect(url, function(err, db){
var collection = db.collection('companies')
collection.find({}).toArray(function(err, companies){
/* more code to do stuff to render the company page */
res.render('companyadminview', {companies:companies, company:company})
}) })
I could be going about this the wrong way I'm new to web development this feels wrong to me but can't figure out a different way.
So, first off you should be able to store it in localstorage or sessionstorage just fine, unless you're targeting browsers that don't support it.
That said, I think it's best not to, as the fact that you're storing it in the DB implies that it changes with enough frequency that you will get buggy clientside behavior if you cache it there for too long.
Instead, I'd just setup a middleware and attach it to the locals object on a per request basis, unless you want to do some kind of cache on the server:
app.use(function(req, res, next) {
mongodb.connect(url, function(err, db){
if (err) return next(err);
var collection = db.collection('companies')
collection.find({}).toArray(function(err, companies){
if (err) return next(err);
res.locals.companies = companies;
next();
});
});
});

Sending different POST response in NodeJS

I would like to get some help with the following problem. I'm writing my bsc thesis, and this small part of code would be responsible for registering a user. (I'm new at nodejs actually). I'm using express and mongoose for this too.
I would like to process the request data, and check for some errors, first I would like to check if all fields exist, secondly if someone already registered with this e-mail address.
Based on the errors (or on success), I would like to send different responses. If a field is missing, then a 400 Bad request, if a user exists, then 409 Conflict, and 200 OK, if everything is ok. But I would only like to do the callback if there are no errors, but I'm kinda stuck here... I get the error Can't set headers after they are sent, which is obvious actually, because JS continues processing the code even if a response is set.
app.post('/register', function (req, res) {
var user = new User(req.body);
checkErrors(req, res, user, registerUser);
});
var registerUser = function(req, res, user){
user.save(function(err, user){
if (err) return console.log(err);
});
res.sendStatus(200);
};
var checkErrors = function(req, res, user, callback){
var properties = [ 'firstName', 'lastName', 'email', 'password', 'dateOfBirth' ];
for(var i = 0; i < properties.length; i++){
if(!req.body.hasOwnProperty(properties[i])){
res.status(400).send('field ' + properties[i] + ' not found');
}
}
var criteria = {
email: req.body.email
};
User.find(criteria).exec(function(err, user){
if(user.length > 0){
res.status(409).send('user already exists');
}
});
callback(req, res, user);
};
I think the problem is in the for loop in checkErrors. Since you call res.status(400).send() within the loop, you can end up calling it multiple times, which will trigger an error after the first call since a response will already have been sent back to the client.
Inside the loop, you can instead add missing fields to an array, then check the length of the array to see if you should respond with a 400 or continue. That way, you will only call res.status(400).send() one time.
For example:
...
var missingFields = [];
for(var i = 0; i < properties.length; i++){
if(!req.body.hasOwnProperty(properties[i])){
missingFields.push(properties[i]);
}
}
if(missingFields.length > 0) {
return res.status(400).send({"missingFields" : missingFields});
}
...
In general, I advise that you put return in front of each res.send() call, to ensure that no others are accidentally called later on.
An example of this is:
User.find(criteria).exec(function(err, user){
// We put return here in case you later add conditionals that are not
// mutually exclusive, since execution will continue past the
// res.status() call without return
if(user.length > 0){
return res.status(409).send('user already exists');
}
// Note that we also put this function call within the block of the
// User.find() callback, since it should not execute until
// User.find() completes and we can check for existing users.
return callback(req, res, user);
});
You probably noticed that I moved callback(req, res, user). If we leave callback(req, res, user) outside the body of the User.find() callback, it is possible that it will be executed before User.find() is completed. This is one of the gotchas of asynchronous programming with Node.js. Callback functions signal when a task is completed, so execution can be done "out of order" in relation to your source code if you don't wrap operations that you want to be sequential within callbacks.
On a side note, in the function registerUser, if user.save fails then the client will never know, since the function sends a 200 status code for any request. This happens for the same reason I mentioned above: because res.sendStatus(200) is not wrapped inside the user.save callback function, it may run before the save operation has completed. If an error occurs during a save, you should tell the client, probably with a 500 status code. For example:
var registerUser = function(req, res, user){
user.save(function(err, user){
if (err) {
console.error(err);
return res.status(500).send(err);
}
return res.sendStatus(201);
});
};
Your call to registerUser() is defined after the route and would be undefined since it's not a hoisted function.
Your use of scope in the closures isn't correct. For your specific error, it's because you're running res.send() in a loop when it's only supposed to be called once per request (hence already sent headers a.k.a. response already sent). You should be returning from the function directly after calling res.send() as well.

MEAN Stack - what's the purpose of returning next() inside a router.param exec function?

I'm learning MEAN stack with an awesome tutorial online. In it, there is a section about preloading objects by id using express's param().
router.param('post', function(req, res, next, id) {
var query = Post.findById(id);
query.exec(function (err, post){
if (err) { return next(err); }
if (!post) { return next(new Error('can\'t find post')); }
req.post = post;
return next();
});
});
My question are:
Where can I find the API documentation on the exec() function? Is it a Mongoose function? How can I tell if it's a Mongoose function or JS or express function?
What's the purpose of final return next() here? Is it necessary in the exec() function to have return next()? What would happen if that line is absent? I have read that next() is for the next middleware, but in other functions from the tutorial, like below, there is no final next() or return next(). What's the difference between next() and return next() anyways?
router.post('/posts', function(req, res, next) {
var post = new Post(req.body);
post.save(function(err, post){
if(err){ return next(err); }
res.json(post);
});
});
Thanks.
First of all please read document of every library you used, if you read just document of mongoose you figure out what is the exec() !
Ok now we can focus on the point
Q1: The exec() is mongoose function. if you being stuck in this situation which you can't figure out what's going on in code, please check this out.
this web app can help you when you need some info about functions (methods) , properties or ... (work perfectly in offline times).
Q2: Please read this question (By the way i think your last question is duplicated)
MEAN is using express.
next() tells the express server to go to invoke the next middleware.
http://expressjs.com/guide/using-middleware.html
The exec is not expressjs by the way.

Response headers of previous request affecting current request

The following code is the user-facing part of a new node app we are building:
var loadInvoice = function(req, res, next) {
Invoice.findById(req.params.invoiceId, function (err, invoice) {
if (err) {
res.send(404, 'Page not found');
} else {
req.invoice = invoice;
next();
}
});
};
app.namespace('/invoices/:invoiceId', loadInvoice, function () {
app.get('', function(req, res){
var templateVals = {
//some template data
};
res.render('paymentselection', templateVals);
});
app.post('', function(req, res){
var data = {
// some data for the apiCall
};
someAPI.someRequest(data, function(err, data) {
console.log(res.status());
res.redirect(data.url);
});
});
});
The first method returns a confirmation page where the user presses a button to post to the same url, which triggers a redirect to an external website.
This all works exactly once. Every second request will crash the app with the message Cant set headers after they are sent. After carefull inspection of the code I could find no reason for this to happen so I added the console.log line which indeed confirms the location header has been set. But it is set to the value i got from someAPI on the previous request not the current one.
This makes absolutely no sense to me. I do not store this value anywhere nor do I do caching or persistence of this data in any way.
Does anybody know what could be causing this?
I use express, express-namespace, mogoose and swig
I found out the problem was being caused bij the 'Restler' libaray used within 'someAPI'. I have no idea how this is possible but swapping it out with something else fixed the problem.

Resources