let's take this snippet for example
const express = require('express')
const bodyParser = require('body-parser')
const app = express()
app.use(bodyParser.urlencoded({ extended: false }))
app.get(...)
# all my routes #
app.use(function (err, req, res, next) {
console.error(err.stack)
res.status(500).send('Something broke!')
})
app.listen(9000)
....
if body-parser throws an error?
Maybe the client closes the connection or the request payload is too large
the client will receive an error with status code 500 from the default express error handler but it’s probably a error that should return 4xx response .
I thought of putting a middleware right after it and calling res.send with the error and status code 400.
maybe wrap all my 3rd party middle ware with try-catch handler
Or adding in my error handler middleware a huge switch case
And over time adding error thrown from middleware third party middleware and change there status code to 400
All the solutions not really bolt proof and need maintenance over time
What us the best practice any suggestions?
The body-parser module should call next(err) when it encounters an error and you can then intercept that by providing an Express error handler such as:
app.use(function(err, req, res, next) {
console.error(err.stack)
res.status(500).send('Put your content here')
});
When you define this special middleware (with four arguments), then Express will keep track of that separately as an error handler that should be called when/if some request handler calls next(err) or if a synchronous exception is caught by Express.
Express has a default error handler if you do not provide one which you can read about here. But, if you supply one, then yours will take over and you can do whatever you want from the error handler. You can even just log the error and take the user to an error page - you don't have to even send an http error status if it's a web page request. If it's an API, you should probably be using http error statuses.
what is the best practice?
Best practice is to include your own Express error handler where you will be able to intercept all errors that are sent on to next(err).
I thought of putting a middleware right after it and calling res.send with the error and status code 400 or maybe wrap all my 3rd party middle ware with try-catch handler
Your third party middleware needs to behave properly in the Express eco-system. That means that it can throw synchronously and Express will catch it automatically and send to the Express error handler. The code should not throw asynchronously (as is true for all types of code) because nobody can catch those exceptions (unless they are in promise-structured code). You shouldn't have to wrap middleware yourself.
Internal errors should generally not be 4xx errors, but rather 5xx errors. Whether or not you surface an internal error back to the end user depends entirely upon the context. Usually, the user will be shown some sort of general purpose error page. Or, if it's an API, the API will have a scheme for communicating errors and typically the API will return actual http status codes that appropriately map to the type of problem.
after further investigation, this is the best solution suited for me
simply adding another middleware after body parse
https://www.npmjs.com/package/express-body-parser-error-handler
Related
Is it valid to call next() in route handler?
In my app I have a middleware at the end of the stack to handle all the unidentified requests.
What I want to do is to send an error page as response when the invalid title is encountered.
The snippet in question is the following:
app.get('/:title', (req, res, next) => {
const title = req.params.title;
if (title.isValid()) {
res.render('post', { title });
} else {
next();
}
});
This is the middleware to handle the unidentified requests:
app.use((req, res, next) => {
res.status(404).render('error');
next();
});
FYI: the snippets above do work. I am just not sure if this is the right way to implement this sort of behavior.
You have basically three choices when you want to send an error response.
You can just send the error response right where the error occurs and not call next().
You can call next(err) where you create an appropriate Error object and let Express' error handling process that error and send the response (which could also be your own custom error handler that the error would flow to).
You can call next() and rely on hitting your default 404 error handler (as you show in your question). This will only work if there are no other routes that might also try to handle this request.
There are logical reasons for any of these methods, depending upon your app architecture plan and just how you want to do things. All can work.
Is it valid to call next() in route handler?
Yes, if you want to continue routing to other route handlers or you know there are no more route handlers that will match and you want it to flow to the 404 error handler.
You can use next() equally well from either middleware or a route handler. It behaves no different in either. The main difference between app.use() and app.get() is just in how it matches the route, not in what happens once the middleware/route handler executes on a matching route. They work the same in that regard.
FYI, app.get() is JUST a more specific version of app.use().
app.get() only matches the GET http verb whereas app.use() matches all http verbs.
app.get() requires a full path match, app.use() will match on a partial path match. For example the path /category/mountain will match app.use("/category", ...), but not app.get("/category", ...).
See What is the difference between a route handler and middleware function in ExpressJS? for some further explanation.
Once they've matched are are executing the code in the handler, they work identically with next() or res.send(), etc...
So, it's perfectly fine to use next() in a route handler if your desired architecture wants to do it that way. You do have to make sure you're always sending exactly one http response for an incoming request. Never failing to send a response and never sending more than one response. So, that becomes a requirement of however you choose to do things. For example, you would never call next() AND send a response in that same handler because that would likely lead to sending multiple responses.
There are even cases where you might have two matching routes for the same path and if the first one decides that it shouldn't handle the request (perhaps because of some state or some query parameter or something like that), it can call next() to allow the second one to get a chance to process that request.
Environment: Node.js, Express
The pattern below will forward all synchronous and asynchronous Error objects to my error handling middleware.
app.get('/', wrapAsync(async function(req, res, next) {
// synchronous error
throw new Error('oops');
// asynchronous error
//next( new Error('oops I did it again');
res.send('hello world');
}));
function wrapAsync(fn) {
return function(req, res, next) {
fn(req, res, next).catch(next);
};
}
app.use( function(error, req, res, next) {
res.send(error);
});
However what if an unexpected error occurs in part of my code where I didn't set up an Error object? Will node.js or Express detect that an error occurred in my route, create an Error object and forward it to my middleware through the wrapAsync wrapper function? This is hard for me to wrap my mind around because I'm not sure how to test for something unexpected.
Is there a pattern that ensures that all possible errors that occur in a route are forwarded to the error handling middleware without crashing the server?
However what if an unexpected error occurs in part of my code where I didn't set up an Error object?
If something throws besides your own code (a programming error or unexpected exception), than whatever threw will have created its own exception. It is convention, but not entirely required that you throw Error objects. Custom code could throw a string or its own object if it wanted to, though that is not the common convention.
An interpreter-generated exception (such as a TypeError) will always throw some type of Error object.
Will node.js or Express detect that an error occurred in my route, create an Error object and forward it to my middleware through the wrapAsync wrapper function?
That's not really the right way to think of it. It's not Express or node.js doing it. It's whatever code caused or threw the exception in the first place (either manually throwing an exception or the interpreter ran into a error that leads to an exception. That's an exception and where they come from. Because you have wrapped things in an async function, you are likely to see that exception (and it's associated Error object) in your .catch() handler.
There are however situations where you still won't see the exception. If some asynchronous code inside your wrapper uses plain callbacks (not promises) and throws an exception inside that plain asynchronous callback, then your wrapper won't catch that exception (nothing will). That's why all asynchronous code in this architecture should be using promisified asynchronous functions only because it enables the automatic error propagation that you are relying on.
Is there a pattern that ensures that all possible errors that occur in a route are forwarded to the error handling middleware without crashing the server?
No. Not if a function uses plain, non-promisified asynchronous callbacks. As described above, in that circumstance the errors will not propagate up to your wrapper.
FYI, see Express middleware cannot trap errors thrown by async/await, but why? for a scheme for building in rejected promise detection into Express. There's also the Express cousin, koa that does this more automatically as part of its architecture.
Is there a way to add middleware to the end of an express app or router chain that gets called to track whether or not the res / response was sent or not?
I mean, regardless of if:
A response is sent (string, JSON, etc.)
A static served file.
No file found in the static folder.
A catch-all callback was reached.
An error middleware was reached.
Example
For instance, if I wanted to log everything...
whether a response was successful or not, ie: it served a file via a express.static( ... ) middleware, some data fetched from a DB, or a custom middleware, or again... if it failed / threw an error...,
is there a way to invoke a callback at the very end?
So far from what I can understand, it seems like, by design, if a static file gets served successfully (via express.static), it doesn't call next(), so the chain stops there.
And for any custom-made middlewares using res.send(), you normally wouldn't want to call next() afterwards since it could cause some undesirable side-effects (errors with headers getting resent).
For error-handlers, that's easier since all unsuccessful responses can be caught here.
But how can it output both successful / unsuccessful responses? Could this be something that should be done without middlewares?
The solution I went with ended up being slightly different from this one by #idbehold, but in a nutshell, at the very top of the express app middleware chain, I had to hook a callback to the res Response object's finish event which gets triggered for most (all?) HTTP status-codes I needed to track a successfully served request.
app.use( ( req, res, next ) => {
res.on( 'finish', () => {
var codeStr = String( res.statusCode );
codeStr = codeStr[res.statusCode < 400 ? 'green' : 'red'];
var output = [req.method.green, req.fullUrl().green, codeStr];
trace( output.join( ' ' ) );
} );
next();
});
I can now get things like:
EDIT
Alright! So provided you also have an error-handler at the "end" of your middleware chain that serves something with an error 404 code, that will trigger the finish event on the res object.
Example of such an error-handler:
app.use( ( err, req, res, next ) => {
trace( "Error!".red );
trace( err );
res.status( 404 ).send(); // Triggers 'finish' on res.
})
There's a conceptual difficulty with the asynchronous architecture of node.js and Express for doing this. I'll describe the general problem and then discuss a few possible work-arounds.
First, each Express handler can be asynchronous. Thus, it gets called and returns pretty much immediately and nobody outside of that world knows whether it is still waiting for some asynchronous operation to finish before eventually sending its response or if it just failed to do anything. You literally can't tell from the outside world.
Second, you can monitor a given request to see if it either calls an error handler or if it sends a response. There is no way to monitor a request handler to see if it just failed to send anything because of the reason above - you have no way of knowing if its still waiting for some asynchronous thing to finish.
So, here's the best I could recommend:
Hook res.end() to see when it gets called. This is an indication that the response is now done (whether error or success). You can see an example of doing that in the express-afterware module that Medet linked in an above comment. The general idea is that you'd have your own middleware somewhere very early in the chain that overrides res.end() so you can see when its called. That early middleware would just install the override and call next() to continue the handler chain. Then, when the response is finished, your override would see that res.end() got called. This should work for all cases where the response is sent.
Then, you still need to handle cases where no response is sent (which is probably due to faulty code since all requests should get a response eventually). The only way I know of to do that is to implement some sort of timeout for a request. You can either use a built-in mechanism server.setTimeout() or you can implement your own inside your middleware (same middleware as describe in step 1). Then, after some timeout that you specify, if no response has yet been sent, you would take over and send some error response.
Install your own error middlewares early in the chain that will see and log all errors. Note that res.end() will still be called so the behavior in step 1 will still be triggered even for errors (error responses still call res.end()).
You can trigger a piece of code at the end of a request by using the finish event of the response object. The finish event is emitted when the response has been sent to the client and all the data has been flushed to the network.
app.use(function(req, res, next) {
res.on('finish', function() {
console.log('Request finished');
});
next();
});
I am overall clueless about how and why you set up a node.js app, and how any of the app.use functions work - the tutorials on it don't explain the why of anything.
Anyway, I have socket.io, res.locals and index.js set up like so in the app.js root file.
const sockets = require('./models/socket')(io)
app.use(function (req, res, next) {
res.locals.user_id = req.session.user_id;
next();
});
const routes = require('./routes/index');
app.use('/', routes);
I'd like to be able to access res.locals in the socket.js model, like I can in index.js found in the routes folder.
I can't guess how to go about doing this. If anybody is able to explain how and why I can or can't that would be a bonus. Thanks!
Welcome to Expressjs, there are a few fundamentals you should probably research before going any further, they'll help solve some of your confusion. I'll give a brief explanation of them but I suggest you do further research. I'll then answer your actual question at the end.
Middleware and app.use
Expressjs is built upon an idea that everything is just "middleware". Middleware is a function which runs as part of a request chain. A request chain is essentially a single client request, which then goes through a chain of a number of middleware functions until it either reaches the end of the chain, exits early by returning a response to the client, or errors.
Express middleware is a function which takes the following three arguments.
req (request) - Representing the request made by a client to your
server.
res (response) - Representing the response you will return to
the client.
next - A way of telling express that your current
middleware function is done, and it should now call the next piece of
middleware. This can either be called "empty" as next(); or with an
error next(new Error());. If it is called empty, it will trigger
the next piece of middleware, if it is called with an error then it
will call the first piece of error middleware. If next is not called at the
end of a piece of middleware, then the request is deemed finished and the
response object is sent to the user.
app.use is a way of setting middleware, this means it will run for every request (unless next() is either not called by the previous piece of middleware for some reason, or it's called with an error). This middleware will run for any HTTP request type (GET, POST, PUT, DELETE, etc).
app.use can take multiple arguments, the important ones for beginners to learn are: app.use(func) and app.use(path, func). The former sets "global" middleware which runs no matter what endpoint (url path) the client requests, the latter (with a specific path) is run only if that specific path is hit. I.e. app.use('/hello', (req, res, next) => { res.send('world'); }); will return "world" when the endpoint "/hello" is hit, but not if the client requests "/hi". Where as app.use((req, res, next) => { res.send('world'); }); would return "world" when you hit any endpoint.
There are more complex things you can do with this, but that's the basics of attaching middleware to your application. The order they are attached to the application, is the order in which they will run.
One more thing, this will blow your mind, an express application made with the standard const app = express() can also be used as middleware. This means you can create several express applications, and then mount them using app.use to a single express application. This is pretty advanced, but does allow you to do some really great things with Express.
Why can you not access res.locals in socket.io? (The real question)
Within your middleware handler, you are setting up a res.locals.use_id property. This only lives with that individual request, you can pass it around as long as the request is alive by passing it into other functions, but outside of that request it doesn't exist. res is literally the response object that tells Express how to respond to the clients request, you can set properties of it during the request but once that HTTP request has ended it's gone.
Socket.io is a way of handling web socket requests, not standard HTTP requests. Thus, in a standard express HTTP request you will not be able to hand off the connection to anything with socket.io, because the connection is a single short lived HTTP request. Likewise, you won't be able to do the same the other way.
If you wish to find the users id in a socket.io request, you'll have to do this within the socket.io request itself.
Right now, you're entering a piece of middleware for an Express.js request, you are then calling next() which runs the next piece of express middleware, at no point does it cross over into Socket.io realms. This is often confused by tutorials because Socket.io can handle requests across the same port as Express is listening on, but the two are not crossed over. So you will need to write separate middleware for both Express.js requests chains, and socket.io request chains. There are ways of writing this code once and then writing an adapter to use it across both platforms, but that's not what you've tried to do here.
I would suggest you look at doing just nodejs and express for a time before taking on socket.io as well, otherwise you're trying to learn a whole heap of technologies all at once is quite a lot to try and take on board all at once.
Using the express-generator it spits out some error handling code like this:
app.use('/', routes);
app.use('/users', users);
// catch 404 and forward to error handler
app.use(function(req, res, next) {
var err = new Error('Not Found');
err.status = 404;
next(err);
});
// error handlers
// development error handler
// will print stacktrace
if (app.get('env') === 'development') {
app.use(function(err, req, res, next) {
res.status(err.status || 500);
res.render('error', {
message: err.message,
error: err
});
});
}
In this example, if (for whatever reason) my routes are broken or the route isn't found or whatever, the code falls back to throw a 404 in the second code block. How come the 3rd code block (the 500 handler) doesn't execute immediately after the 2nd code block (the 404 handler) begins executing?
I thought that the way node.js callbacks work is that the function begins executing and continues to be executed in the background and then the next callback begins executing at the same time. But apparently I am a bit confused by how the synchronous callbacks work. Does the above code somehow know to "wait" until the 404 handler code is done executing before running the 500 error handler?
All of the app.use() statements run when your app is initialized. They each set up a "middleware" handler. They don't actually run the handlers at that time, they just register them into the Express middleware stack. If no handler before them handles a page, then these two last app.use() middleware handlers will get a shot at the request in order and the second one only gets to see the request if the first passes the request on to more handlers.
The 404 handler will set the status to 404 and will then call the next() handler in the middleware stack. That will end up being your last app.use() statement which will see if a status has already been set and if not, will set it to 500, but if it was previously set to 404, it will leave it at that. It will then apply a default rendering for a missing page that shows the status in the page.
This is a means of having a single place where the default rendering is applied, but multiple places that could set errors.
None of this really has anything to do with asynchronous behavior. The next request handler in the list is started only when next() is called by an earlier request handler. There is no "waiting". You can think of the 404 request handler using the last app.use() statement like a synchronous function call when it calls next() it is just saying please execute the next request handler in the chain right now (which it happens to know is the one that provides default rendering for the error status code).
It might be helpful to review how app.use() works in Express.
Each call to app.use() adds a request handler to a list. When a given http request comes in, Express starts with the first request handler in the list and checks to see if the parameters of that first request handler in the list match the current request (e.g. does the path match or any other parameters set in the app.use() statement). If it matches, then it calls that request handler. If that request handler does not call next() to let the next request handler in the list have a chance at the request, then all processing is done and Express assumes that the first request handler has completely handled the request. If this first request handler has not completely handled the request (say it was just checking a cookie value in the header and wants processing to continue to other handlers), then it will call next(). This tells express to look at the next app.use() handler in the list and see if it is a match for this request.
As long as no request handler matches the current request or each one that does keeps calling next() to keep the chain going, Express will keep marching down the list looking for some request handler to handle the request and generate a server response. In your specific example, the second to the last request in the chain is a 404 handler. It assumes that if Express got this far down the chain, then no handler has yet handled this request so it must be a request for a page that this server is not designed to handle. Thus, it sets the status to 404. And, then because the default rendering for an error page is in the very last request handler, it calls next() in order to trigger that last default page rendering with the error in it.
Only one thread in the interpreter is running your code. I/O operations are performed concurrently so that JS execution can continue without blocking on I/O. It's called asynchronous because the timing and sequence of callback execution is not under your direct control. Two JavaScript functions do not execute simultaneously.
Your code above will run fully without either of the callback functions executing. After your code runs the http module will listen for client requests (typically, you didn't show this above). The callbacks will execute as needed in response to those client requests. They are not running all the time in separate threads and waiting for data. app.use just registers the functions within the express middleware stack. When requests come in that match the routes (or not) that you specify, the applicable callbacks are called in order. That is why you must call next within your middleware; if you don't, processing of that request object stops (this design is called continuation passing style).
The exact order in which these functions are executed is not known to you, and it's not important. Only the relative order matters, i.e. which of two functions will be called first. Usually the code structure will guarantee this (i.e. supplying a callback function to an I/O call). This means the interpreter is able to process the result of each I/O activity immediately without you having to worry about thread management, etc.