I'm quite new to NodeJS, and I came across a construction that I can't wrap my head around. Consider the following code:
router.get("/", function(req, res, next) {
let db = new sqlite3.Database(dbname, sqlite3.OPEN_READONLY, (err) => {
if (err) next(err);
});
res.render("views/page");
}
If an exception is raised by Database (for instance because dbname does not exist), then the callback will pass it to the Express error handler (next). But the code does not stop there, and it will continue to execute the next line and attempt to render something, which is problematic if the error handler also sends headers. If I add a return statement within the callback, it will simply terminate Database but not the rest of the function.
My question: is there anyway to prevent the rest of the code from being executed if an exception is raised?
Please read the sqlite3 npm documentation. Ideally, you should not be opening a database connection on every request. Instead, you should open the database connection when the server is started, and share a reference to the db variable to be used in the API routes (gets/posts).
Unless you just want to make it work, I believe the below should do it.
router.get("/", function(req, res, next) {
let db = new sqlite3.Database(dbname, sqlite3.OPEN_READONLY, (err) => {
if (err) return next(err);
res.render("views/page");
});
}
EDIT: Javascript does not allow for async/await when instantiating objects, so in this case I believe would be better to just initialize the database connection outside the router method, like below, and use the error event to handle errors
const db = new sqlite3.Database(dbname, sqlite3.OPEN_READONLY);
db.on('error', function(err) { /*handle error here*/ });
router.get("/", function(req, res, next) {
res.render("views/page");
});
}
Related
I'm starting out w/ NodeJS and Express. Coming from the other popular scripting languages and C++ background, asynchronously calling DB functions is a bit foreign. I've sorted out a pattern, but I'm still curious about catching exceptions. Below is my basic pattern.
var callback = function(req, res) {
// do stuff
connection.query(queryString, function(err,result){
if (err) throw err;
// process results.
};
};
var express = require('express');
var app = express();
app.get('/', callback);
app.listen(3000,function() {
console.log('listening');
};
Generally I have a lot of endpoints and callbacks. I'm a bit lost on where I set up ta try/catch block to catch errors thrown in the callback though. I've looked around for some suggestions, but a lot of them seem to be on the web framework (if any) being used.
When you throw in an asynchronous callback, the exception just goes back to the internals of the database event handler and there is NO way for you to ever catch or handle that exception. So, basically it does no good at all. It will just cause you to abort the handling of that request and you will never send a response on that request.
Basically, you have several choices for how to handle the error. You can handle it completely right in each endpoint and send some sort of error response.
Send Response right at each point of error
app.get('/', function(req, res) {
// do stuff
connection.query(queryString, function(err,result){
if (err) return res.status(500).send(someErrorResponse);
// process results.
};
});
Forward on to centralized error handler
Or, you can forward the error on to a centralized error handler by calling next(err):
app.get('/', function(req, res, next) {
// do stuff
connection.query(queryString, function(err,result){
// if error, forward it on to our centralized error handler
if (err) return next(err);
// process results.
};
});
// centralized error handler - note how it has four parameters
app.use(function(err, req, res, next) {
// formulate an error response here
console.log(err);
res.status(500).send(someErrorMessage)
});
See Nodejs handle unsupported URLs and request types for more info on the ways to have generalized error handlers in Express.
Use promises to collect errors within each route
If you are using more involved sequences of asynchronous operations where you may have more than one async operation sequenced together, then it does get to be a pain to handle errors at every single async operation. This is where using promises with all your async operations more easily allows all the errors to percolate up to one .catch() statement at the top level of each route. You don't say what database you're using, but here's an idea what that looks like. The general idea is that you can write your code so that all promise rejections (e.g. errors) will propagate up to one central .catch() in each route handler and you can then call next(err) from that .catch(), sending the error to your centralized error handler. Here's how that looks for a recent version of Mongoose (you didn't say which database you were using) with one database operation.
app.get('/', function(req, res, next) {
// do stuff
connection.query(queryString).exec().then(function(result){
// process results.
}).catch(next);
});
// centralized error handler - note how it has four parameters
app.use(function(err, req, res, next) {
// formulate an error response here
console.log(err);
res.status(500).send(someErrorMessage)
});
And, here's what it looks like if you have more than one operation:
app.get('/', function(req, res, next) {
// do stuff
connection.query(queryString).exec().then(function(result){
// process results, then make another query
// return the promise from this second operaton so both results
// and error are chained to the first promise
return connection.query(...).exec();
}).then(function(result) {
// process chained result
}).catch(next);
});
// centralized error handler - note how it has four parameters
app.use(function(err, req, res, next) {
// formulate an error response here
console.log(err);
res.status(500).send(someErrorMessage)
});
Since ES6 built in support for promises and ES7 will add support for async/await for asynchronous operations (which is based on promises) and all significant libraries that offer asynchronous operations have added or are adding support for promises, it is clear that promises are the future of the language for managing asynchronous operations. That would be my strong recommendation.
You should never, ever throw an error like that! :) The reason is that at some point your whole node app will just stop working, because of some db query failed. This should be handled instead of just die.
And because this is a route handler - handles specific url that the user is getting (for example /), there should be some output. You can always show a page with status 500 and a nice design, if there was such an error that you cannot handle or you might have your internal state messed up.
So basically just act as nothing happened - return respones of any kind, or even render a page, but provide information that something went wrong.
Also, a common scenario is something like what Alon Oz presented. All routes in express are actually a middleware functions, that are called one after another. If the route does not match the requested one, the function just skips and the next one is called. You can manually control that. The actual pattern of the router is this:
app.get('/', function(req, res, next) {
// you can have the request
// you can send response like res.send('hello')
// OR you can skip this function using NEXT
});
The actual signature of next is next(err). So if you call it without any arguments, it will just skip to the next middleware. If you call it with an argument, it will skip all regular functions and go to the last ones in the stack, or more specifically the ones that handle errors. They are like the regular ones, but taking four arguments instead of three:
app.use(function (err, req, res, next) { });
It's very important to understand that this function will be called if you call next with an argument. Throwing an error won't do any good! Of course if none of your routes match the specific criteria (url) the final one will in the call will be called, so you can still handle the "not found" error.
This is a common scenario that you will use:
// development error handler, will print stacktrace
if (app.get('env') === 'development') {
app.use(function(err, req, res, next) {
debug('ERROR [ip: %s]:: dev env -> ', req.ip, err); // I'm using debug library - very helpful
res.status(err.status || 500);
res.render('deverr', { // I render custom template with the whole stack beautifully displayed
errMessage: err.message,
error: err
});
});
}
// production error handler, no stacktraces leaked to user
app.use(function(err, req, res, next) {
res.status(err.status || 500);
res.render('pages/error', { // custom error page with nice design and a message
errMessage: err.message,
error: {}
});
});
Hope that helps! :)
Since you are using express, it has its own way to handle exceptions,
defined like this:
function clientErrorHandler (err, req, res, next) {
if (req.xhr) {
res.status(500).send({ error: 'Something failed!' })
} else {
next(err)
}
}
app.use(clientErrorHandler)
For more info:
https://expressjs.com/en/guide/error-handling.html
There are most commonly three major types of errors that we need to take into account.
Promise failures (Any failures that come up during async/await or result of a promise in then/catch)
In order to handle promise failures, as suggested in the strong loop document or node js 2018 best practices, it's important to have a common function that can handle it.
// app.js file
app.get('/:id', async (req,res,next) => {
if(!req.params.id) {
return res.status(412).send('enter a valid user id');
}
try {
const results = await UserDAL(id);
} catch(e) {
next(e);
}
}
// common error middleware defined in middleware/error.js
module.exports = function (err,req,res,next) {
logger.error(`${err.status || 500} - ${err.message} - ${req.originalUrl} - ${req.method} - ${req.ip}`);
return res.status(500).send('something failed.');
};
Unhandled Rejections
process.on('unhandledRejection', e => {
// do something
});
Unhandled exceptions
process.on('uncaughtException', e => {
// do something
});
If you see a lot of try/ catch blocks in your express methods you can abstract that to a separate async function like below:
module.exports = function asyncMiddleWare(handler) {
return async (req,res,next) => {
try {
await handler(req,res)
} catch(e) {
next(e);
}
}
};
I currently have a POST route defined in an Express Node.js application as so:
var locationService = require("../app/modules/locationservice.js");
app.post('/createstop', isLoggedIn, function(req, res) {
locationService.createStop(res, req.body);
});
(for this question, please assume the routing in & db works.. my record is created on form submission, it's the response I am struggling with)
In the locationservice.js class I then currently have
var models = require('../models');
exports.createStop = function(res, formData) {
models.location.build({ name: formData.name })
.save()
.then(function(locationObj) {
res.json({ dbResult : locationObj });
});
};
So as you can see, my route invokes the exported function CreateStop which uses the Sequelize persistent layer to insert a record asynchronously, after which I can stick the result on the response in the promised then()
So at the moment this only works by passing the response object into the locationservice.js method and then setting res.json in the then() there. This is sub-optimal to me with regards to my service classes, and doesn't feel right either.
What I would like to be able to do is "treat" my createStop method as a promise/with a callback so I can just return the new location object (or an error) and deal with it in the calling method - as future uses of this method might have a response context/parameter to pass in/be populated.
Therefore in the route I would do something more like:
var locationService = require("../app/modules/locationservice.js");
app.post('/createstop', isLoggedIn, function(req, res) {
locationService.createStop(req.body)
.then(dataBack) {
res.json(dataBack);
};
});
Which means, I could call createStop from else where in the future and react to the response in that promise handler. But this is currently beyond me. I have done my due diligence research, but some individual expert input on my specific case would be most appreciated.
Your locationservice.js could look like that
exports.createShop = function(data){
// here I have used create instead of build -> save
return models.location.create(data).then(function(location){
// here you return instance of saved location
return location;
});
}
And then your post() method should be like below
app.post('/createstop', isLoggedIn, function(req, res){
locationService.createShop(req.body).then(function(location){
// here you access the location created and saved in createShop function
res.json(location);
}).catch(function(error){
// handle the error
});
});
Wrap your createStop function with a promise like so:
exports.createStop = function(res, formData) {
return new Promise(function(resolve, reject) {
models.location.build({ name: formData.name })
.save()
.then(function(locationObj) {
resolve({ dbResult : locationObj });
});
//in case of error, call reject();
});
};
This will allow you to use the .then after the createStop within your router.
I'm trying to add Bugsnag to my Node Restify service. We have a ton of routes already and such so I'm trying not to add Bugsnag calls all over our code base and I'm also trying to do something global so there's never a mistake where a dev forgets to add the error reporting.
Conceptually I want after any res.send() to get the status code. If the statusCode is >=400 i want to notify Bugsnag by calling Bugsnag.notify. I already check for errors everywhere so no errors ever show up to the clients (browsers, phones, etc) but they do get sent, for example, res.send(401, { message: 'You dont have permission to do that' }) which I'd like to be able to hook into and pass who tried to do that, what route, etc. Problem is I can't get the after event to fire at all:
server.on('after', function (req, res, route, error) {
console.log('AFTER')
});
I think I misunderstand what after is for. It's at the top of my index.js before any routes or other middleware (server.use) is defined.
My general code structure looks something like:
server.post('/foo', function (req, res, next) {
FooPolicy.create(req, function (err) {
if (err) return res.send(err.code, err.data);
FooController.create(req.params, function (response) {
res.send(response.code, response.data)
next();
});
});
});
FooPolicy == checking permissions
FooController == actually creating the model/data
The issue is that the after event is currently treated like any other handler. That means that if you don't call next in every code path, the after event will never be emitted.
In the meantime, adding a next call will cause your after event handler to fire.
if (err) {
res.send(err.code, err.data);
next();
return;
}
This is something that is a bit unclear to me (I'm just getting started with Node and Mongo), and it really concerns me because of server performance and strain (which I guess is another question, but I'll get to that at the end of the post).
So, assuming I'm writing an API with Node.js and Restify, where each API endpoint corresponds to a function, should I:
a) open the db connection and store it in a global var, and then just use that in every function?
Example:
// requires and so on leave me with a db var, assume {auto_reconnect: true}
function openDB() {
db.open(function(err, db) {
// skip err handling and so on
return db;
}
}
var myOpenDB = openDB(); // use myOpenDB in every other function I have
b) open the db connection and then just put everything in one giant closure?
Example:
// same as above
db.open(function(err, db) {
// do everything else here, for example:
server.get('/api/dosomething', function doSomething(req, res, next) { // (server is an instance of a Restify server)
// use the db object here and so on
});
}
c) open and close the db each time it is needed?
Example:
// again, same as above
server.get('/api/something', function doSomething(req, res, next) {
db.open(function(err, db) {
// do something
db.close();
});
});
server.post('/api/somethingelse', function doSomethingElse(req, res, next) {
db.open(function(err, db) {
// do something else
db.close();
});
});
This last one is what I would do out of intuition, but at the same time I don't feel entirely comfortable doing this. Doesn't it put too much strain on the Mongo server? Especially when (and I hope I do get to that) it gets hundreds — if not thousands — of calls like this?
Thank you in advance.
I like MongoJS a lot. It lets you use Mongo in a very similar way to the default command line and it's just a wrapper over the official Mongo driver. You only open the DB once and specify which collections you'll be using. You can even omit the collections if you run Node with --harmony-proxies.
var db = require('mongojs').connect('mydb', ['posts']);
server.get('/posts', function (req, res) {
db.posts.find(function (err, posts) {
res.send(JSON.stringify(posts));
});
});
Option A is not a great idea since there is no guarantee that the DB will be finished opening before an HTTP request is handled (granted this is very unlikely)
Option C is also not ideal since it needlessly opens and closes the DB connection
The way that I like to handle this is using deferreds/promises. There are a bunch of different promise libraries available for Node but the basic idea is to do something like this:
var promise = new Promise();
db.open(function(err, db) {
// handle err
promise.resolve(db);
});
server.get('/api/something', function doSomething(req, res, next) {
promise.then(function(db)
// do something
});
});
I believe Mongoose handles connections in a way that vaguely resembles this.
This is a pure best practice question. I am pretty new to Node and Mongoose. I absolutely love the technology and have been cranking away on a project to build a JSON-backed API for an app that I'm building.
I am finding that I am continuously repeating code when I fetch objects from my database. For example:
Playlist.findById(req.params.id, function(err,playlist){
if (err)
return res.json({error: "Error fetching playlist"});
else if (!playlist)
return res.json({error: "Error finding the playlist"});
//Actual code being performed on the playlist that I'm fetching
});
The error handling at the top of the function call is annoying because I have to repeat that code for every call to the database... or so I think.
I thought about using a callback like:
var fetchCallback = function(err,objOrDoc,callback){
//Handle the error messages
callback(objOrDoc);
};
However, this approach would mess up my sequential flow since I would have to define the callback function before I performed the fetch. So, if I had a lot of database queries chained together, I would have to place the callbacks in reverse order, which is far from ideal in a clean-coding perspective.
I'm wondering if anyone has run into this issue and has any best practices for cutting down on the repetition.
I'm also using the express framework, so if there's a helpful way to handle it in express, I'd be interested to know, too.
There are a couple interesting approaches you could try here.
At the most simple, you could simply have a function that loads up an object and handles the output in an error condition.
fetchResource = function(model, req, res, callback) {
model.findById(req.params.id, function(err, resource) {
if (err)
return res.json({error: "Error fetching " + model.toString()});
else if (!playlist)
return res.json({error: "Error finding the " + model.toString()});
callback(resource);
});
};
app.on('/playlists/1', function(req, res) {
fetchResource(Playlist, req, res, function(playlist) {
// code to deal with playlist.
});
});
That's still quite a bit of duplication, so I might try to move this out into a middleware.
Route Middleware
Routes may utilize route-specific middleware by passing one or more additional callbacks (or arrays) to the method. This feature is extremely useful for restricting access, loading data used by the route etc.
Now I haven't tested this and it's a bit hand-wavey (read: pseudocode), but I think it should serve as a decent example.
// assuming a URL of '/playlist/5' or '/user/10/edit', etc.
function loadResource(model) {
return function(req, res, next) {
model.findById(req.params.id, function(err, resource) {
if (err)
return res.json({error: "Error fetching " + model.toString()});
else if (!resource)
return res.json({error: "Error finding the " + model.toString()});
req.resource = resource;
next();
});
}
}
app.get('/playlist/:id', loadResource(Playlist), function(req, res) {
var playlist = req.resource;
...
});
app.get('/user/:id', loadResource(User), function(req, res) {
var user = req.resource;
...
});
The express source contains a pretty good example of this pattern, and the middleware section in the docs (specifically under 'Route Middleware') details it further.