I have a simple module that authenticates. If the user is not authenticated I use this function to move him forward :
function forbidden() {
return next({ status: 403 });
}
I can't seem to find what's the next function that picks this up in case of 403. Is there a way I can see all the middleware stack ?
Thanks
next is always a synthetic function that connect creates to know when your middleware is done and it's time to proceed either down the regular middleware stack (if no error is passed to next) or the error handling middleware stack (when an error is passed to next as in your example). So it's always going to be the same function, but you can explore with node-inspector if you want to peek at the state of the connect middleware stack for educational purposes. Even then since most functions added to the connect middleware are coded as anonymous functions, the middleware stack Array is just going to look like [Function, Function, Function] and not be very illuminating.
Related
I'm learning to create web applications using Express.js.
In the process, we tried to implement a feature to prevent users from accessing certain pages when they are not logged in.
// teamController.redirectView : Redirect the screen according to res.locals.redirect
app.get('/user/:id/team/member', teamController.showMember, teamController.redirectView)
// The following is a middleware function written in another file(teamController.js)
showMember: (req, res, next) => {
// I want to set the redirect to the '/login' and skip the following process when no user are logging in.
if(!res.locals.loggedIn) {
res.locals.redirect = 'login'
next()
}
// access the property that is set only when the user logs in
let userID = res.locals.currentUser.userID
// Other processes...
When I accessed the URI when the user was not logged in, I was indeed taken to the login page, but I got the following error in the console.
TypeError: Cannot read property 'userID' of undefined
at // the position of 'let userID...'
Error occured: TypeError: Cannot read property 'userID' of undefined
// Abbreviated below...
Does this mean that excuting next() does not skip the following process, like 'return' does?
Or is there some fatal error that is causing this error?
Can you please help me?
I could have avoided the error by enclosing all subsequent processes in 'else', but if there is a better way, I would appreciate it if you could tell me that too.
Does this mean that executing next() does not skip the following process, like 'return' does?
Exactly. Calling next() alone doesn't stop further execution.
This is actually intentional because it allows for additional logic without making the client wait longer (especially with multiple middle-wares).
However to solve your problem, you can combine your next() and a return:
if(!res.locals.loggedIn) {
res.locals.redirect = 'login';
return next();
}
This way you don't have to warp all the code below in an else.
It's a common practice as well, as described here.
You must use return next() or the code keeps executing after you call next()
I have a basic misunderstanding regarding asynchronic calls using express and middlewares and i would really appreciate some help understanding it.
Suppose we have this code:
var express = require('express')
var cookieParser = require('cookie-parser')
var app = require('express')
var router = express.Router()
app.use(function timeLog (req, res, next) {
req.requestTime = Date.now()
next()
})
app .use(express.json());
app .use(express.urlencoded());
app.use(cookieParser())
router.post('/hello', function (req, res) {
//write async to file
res.send('bye')
})
Now, when the client calls this endpoint "hello":
are the middlewares which defined in App-level called asynchronicaly? I understood that they do(because they are called "Callbacks" in the documentation... so basically before reaching the router: parsing cookies, parsing json to req.body and adding req.requestTime will run asynchronicaly and then will be routed to '/hello' end point.
after routing, is the callback will run asynchronicaly? if yes, then how in this case the request is not left hanging? i see that the response is being terminated inside a body of a callback... how this make any sense? :(
would somebody please explain to me this flow?
I will try to explain how I understood "async calls" through this code above: lets say alot of users trying to get this end point. all these calls added to the call stack, then because of these callbacks are async, then they are moved to event Queue/table and will be handled after the call stack will be "empty". if this is the case, how the first user will ever get a response? the requestTime done async, parsing the json done async and when reaching the router, the callback is done async.... so when the first user will ever get a repsone if all these async calls located inside the event Queue/table and they will be handled only after the callstack is empty? what am i missing here?
Thanks.
The middleware doesn't appear to be asynchronous of its own. In other words, as you said in your comment on another answer, it is not forcing each layer in the expressjs "stack" of middleware/handlers into a separate frame in the JavaScript event queue.
If you trace the next() function in a .use(), there are a couple of setImmediates fairly early on to handle "exit router" or "no more layers," but then you get into a while loop on a stack of handlers. This is happening at around this point in the code.
So if all your middleware was similar to this section, all middleware etc would happen in the same frame within the event queue:
app.use(function(req, res, next){
console.log('synchronous layer');
next();
});
Whereas a step like this next one would put the next into a separate frame in the event queue, and potentially allow the process to handle other frames that may be queued up.
app.use(function(req, res, next){
setImmediate(()=> {
console.log('setImmediate puts this next() into a separate frame in the event queue');
next();
});
});
I can't imagine this would normally be a problem. Most things that would happen in middleware that might take some time (a database call etc) are very likely all going to be happening asynchronously (in a way that puts the next into a new frame in the event queue). But it is something worth considering when you're adding middleware...
All those middleware are using the continuation passing style. So basically they COULD run asynchonous. But they don't have to. It depends on, whether those middlewares are doing some IO. You could take a look into the code to check how the functions behave exactly, but at least, this does not matter. Just keep in mind, that they COULD run asynchronous.
In express and connect, is it bad to use "next" in middleware if I do not need it? Are there any possible negative outcomes? Assume there is no middleware which will be called after this middleware, and therefore the next will not call anything. I know it is bad for modularity, as if you want to add a callback for another middleware it may be accidentally triggered by the next in this middleware. However, in this case next is bad for modularity anyway, as middleware often interact in unexpected ways.
As an example of an unneeded next, consider the sample MEAN.JS stack, constructed by the guys who originally came up with the stack's name. It seems to have some next callbacks which do not ever get called. Many are in the users controller, including the signin function:
exports.signin = function(req, res, next) {
passport.authenticate('local', function(err, user, info) {
if (err || !user) {
res.status(400).send(info);
} else {
// Remove sensitive data before login
user.password = undefined;
user.salt = undefined;
req.login(user, function(err) {
if (err) {
res.status(400).send(err);
} else {
res.json(user);
}
});
}
})(req, res, next);
};
This function has a next callback defined. This next callback is then used by the passport.authenticate() custom middleware function as a parameter. However, this parameter is never used in the function itself. I have tried taking out the next definition from the function definition, as well as the custom passport middleware, and the route seems to still work. However, perhaps passport uses it in its authenticate() function, and leaving it out did not cause any trouble here but it may cause trouble in some cases.
I was recently looking at passport's tutorials on http://passportjs.org, and I came across a function in the section on custom callbacks on the authenticate page that looks almost exactly like the signin function in MEAN.JS. One difference was that it actually had some next callbacks (for error handling), so the next parameter was actually useful. Is it possible that the MEAN.JS app took a lot of code from passportjs.org's guide and changed it over time, but left in some vestigial remnants that do not do anything but were causing no harm? Or does the next parameter actually do something in passport.authenticate() that is not immediately obvious? Regardless of why this happened, does an extra next parameter in connect middleware cause any bad side effects if it is not used?
When writing middleware, the next parameter is optional. It's purpose is so that the next middleware in the chain will be called. If you want the current middleware to be the last one called for a given request, not executing the next parameter will accomplish that. This is fine for code that you write for yourself, but it's typically better to always execute the next parameter in middleware that may be used elsewhere because you don't know what else they could be adding.
For example, maybe you wanted to add some kind of logging that happens after a request is completed. If your middleware that runs before the logging middleware doesn't execute next, it won't be logged.
http://expressjs.com/api.html#middleware
Not executing next will simply not start the next middleware. There are no other side effects of not executing it other than those caused by not moving to the next middleware (for example, if the response hasn't ended yet, not calling next will result in a timeout.)
I'm having trouble understanding how to handle something that it seems like would be a pretty basic aspect of express. If I have some code that throws an exception in an async callback, there is no way I can catch that exception because the try/catch block is no longer in scope by the time the callback is running. In these scenarios the browser will hang until it eventually give up stating that the server is unresponsive. This is a very bad user experience. I would much rather be able to immediately return a 500 error to the client. The default express error handler apparently does not handle this situation. Here is some sample code:
var express = require("express");
var app = express();
app.use(app.router);
//express error handler (never called)
app.use(function(err, req, res, next) {
console.log(err);
res.send(500);
});
app.get("/test", function(req, res, next) {
require("fs").readFile("/some/file", function(err, data) {
a.b(); //blow up
});
});
app.listen(8888);
In the above code, the line a.b() throws a "ReferenceError: a is not defined" exception. The defined error handler is never called. Notice that the err object returned by fs.readFile() is null in this case because the file was correctly read. The bug is the code inside the async handler.
I have read this post about using node's uncaughtExpception even, but the documentation says not use that method. Even if I did use it, how would I then send the 500 response back to the user? The express response object is no longer around for me to use.
So how do you handle this scenario?
OK, I'm just going to post an entirely different answer since my first answer has some valuable information but is off topic in hindsight.
The short answer: the correct thing to do is what already happens: your program should print a stack trace and exit with an error.
The underlying thinking:
So I think you need to think about errors in different categories. My first answer deals with data-related errors that a well-written program can and should handle cleanly. What you are describing is a CRASH. If you read the node.js documentation you linked to, it is correct. The only useful thing your program can do at this point is exit with a stack trace and allow a process supervisor to restart it and attain an understood state. Once your program has crashed, it is essentially unrecoverable due to the extremely wide range of errors that could be the root cause of an exception getting to the top of the stack. In your specific example, this error is just going to continue happening every time until the source code bug is fixed and the application is redeployed. If you are worried that untested and buggy code is going to get into your application, adding more untested and buggy error handling code isn't really addressing the right problem.
But in short, no, there's no way to get a reference to the HTTP request object that caused this exception so AFAIK you cannot change the way this is perceived by the end user in the browser, outside of handling this at an intermediate reverse proxy layer where you could configure a crude timeout and send a more friendly error page (which of course would be useless for any request that isn't for a full HTML document).
The Bible of Error Handling in Node
Error Handling in Node.js by Dave Pacheco is the definitive work on this topic in my opinion. It is comprehensive, extensive, and thorough. I recommend reading and re-reading periodically.
To address #asparagino's comments, if an unhandled exception is easily reproducible or happening with high frequency, it's not an edge case, it's a bug. The correct thing to do is improve your code to not generate uncaught exceptions in face of this situation. Actually handle the condition, thus converting a programmer error into an operational error, and your program can continue without a restart and without an uncaught exception.
You should use express's error handling middleware via app.use(error, req, res, next). Express maintains a separate middleware stack that it uses when the normal middleware stack throws an uncaught exception. (Side note that express just looks at the callback's arity (number of expected arguments) to categorize it as regular middleware or error handling middleware, which is a bit magical, so just keep in mind you must declare the arguments as above for express to understand this is an error handling middleware).
And based on your question and comments, just understand that exceptions aren't all that useful in node.js because every async call gets a new stack, which is why callbacks are used everywhere and the 1st argument is an error universally. Your try/catch block in the example is only going to catch an exception thrown directly by findById (like if id were undefined, as it is in your snippet), but once an actual call to the database is made, that's it, the call stack is over and no further exceptions can happen until a totally different call stack starts when node invokes the async IO callback.
Thanks for the answer, but this only works if I put the try/catch inside the async callback and have the catch do next(exp). I'd like to avoid having separate try/catch blocks in every single async callback.
Nope, that's not true. You don't have to manually call next(exp). Express will catch the error and trigger the error handling middleware for you (that's how express does it's developer-friendly exception report pages in dev mode anyway). And async libraries don't throw exceptions even under "normal" error conditions. They pass an error to the callback, so in general you don't have to bother with try/catch that much in node. Just never ignore an error parameter passed to a callback function, and you're fine.
You don't see this boilerplate in node:
someDb.query(someCriteria, function (error, result) {
try {
//some code to deal with result
} catch (exception) {
callback(exception);
}
});
You do see this though:
someDb.query(someCriteria, function (error, result) {
if (error) {
callback(error);
return;
}
//some code to deal with result
});
Node handles IO differently, which means the call stack works differently, which means exceptions work differently, which means error handling works differently. You can write a stable node/express app that handles errors without crashing without writing a single try/catch. (express has one that handles uncaught errors that bubble all the way to the top). It's not a functional limitation, it's just a consquence of async IO that means you have to write your code differently and handle errors with callbacks instead of exceptions. Thinking of it as a "limitation" as opposed to the "way it is" is putting a negative connotation on something that is really just a technical reality. There are clean and robust patterns for exception handling in both a synchronous and asynchronous paradigm.
I'm new to javascript, and jumped right into node.js. I've read a lot of theory, and began well with the practical side (I'm writing an API for a mobile app), but I have one basic problem, which has lead me to middleware. I've successfully implemented a middleware function, but I would like to know if the use I'm giving the idea of middleware is OK, and also resolve the original problem which brought me to middleware. My question is two-fold, it's as follows:
1) From what I could gather, the idea of using middleware is repeating a process before actually processing the request. I've used it for token verification, as follows:
Only one of my urls doesn't receive a token parameter, so
app.js
app.get('/settings', auth.validateToken, auth.settings);
auth.js
function validateToken(req, res, next){ //code };
In validateToken, my code checks the token, then calls next() if everything is OK, or modifies res as json to return a specific error code.
My questions regarding this are: a) Is this a correct use of middleware? b) is there a [correct] way of passing a value onto the next function? Instead of calling next only if everything is OK, is there a [correct] way of calling next either way, and knowing from inside the next function (whichever it is), if the middleware was succesful or not? If there is, would this be a proper use of middleware? This precise point brings me to my original problem, and part two of this question, which is encapsulating functions:
THIS PART WAS FIXED, SEE MY SECOND COMMENT.
2) I discovered middleware trying to simply encapsulate validateToken, and be able to call it from inside the functions that the get handlers point to, for example auth.settings.
I'm used to common, sequential programming, and not in javascript, and haven't for the life of me been able to understand how to do this, taking into account the event-based nature of node.js.
What I want to do right now is write a function which simply verifies the user and password. I have it perfectly written inside a particular handler, but was about to copy-paste it to another one, so I stopped. I want to do things the right way from scratch, and understand node.js. One of the specific problems I've been having, is that the error code I have to return when user and password don't match are different depending on the parent function, so I would need this function to be able to tell the callback function "hey, the password and user don't match", so from the parent function I can respond with the correct message.
I think what I actually want is to write an asynchronous function I can call from inside another one.
I hope I've been clear, I've been trying to solve this on my own, but I can't quite finish wrapping my head around what my actual problem is, I'm guessing it's due to my recent introduction to node.js and JS.
Thanks in advance! Jennifer.
1) There is res.locals object (http://expressjs.com/api.html#res.locals) designed to store data local to the request and to pass them from one middleware to another. After request is processed this object is disposed of. If you want to store data within the session you can use req.session.
2) If I understand your question, you want a function asynchronously passing the response to the caller. You can do it in the same way most node's functions are designed.
You define a function in this way:
function doSomething(parameters, callback) {
// ... do something
// if (errorConddition()) err = errorCode();
if (callback) callback(err, result)
}
And the caller instead of using the return value of the function passes callback to this function:
function caller(req, res, next) {
//...
doSomething(params, function(err, result) {
if (! err && result) {
// do something with the result
next();
} else {
// do something else
next();
// or even res.redirect('/error');
}
});
}
If you find yourself writing similar callback functions you should define them as function and just pass the function as parameter:
//...
doSomething(param, processIt);
function processIt(err, result) {
// ...
}
What keeps you confused, probably, is that you don't treat functions as values yet, which is a very specific to JavaScript (not counting for languages that are little used).
In validateToken, my code checks the token, then calls next() if everything is OK, or modifies res as json to return a specific error code.
a) Is this a correct use of middleware?
b) is there a [correct] way of passing a value onto the next function?
Yes that is the correct way of using middleware, although depending on the response message type and specifications you could use the built in error handling of connect. That is in this example generate a 401 status code by calling next({status:401,stack:'Unauthorized'});
The middleware system is designed to handle the request by going through a series of functions until one function replies to the request. This is why the next function only takes one argument which is error
-> if an error object is passed to the next function then it will be used to create a response and no further middleware will be processed. The manner in which error response is created is as follows
// default to 500
if (res.statusCode < 400) res.statusCode = 500;
debug('default %s', res.statusCode);
// respect err.status
if (err.status) res.statusCode = err.status;
// production gets a basic error message
var msg = 'production' == env
? http.STATUS_CODES[res.statusCode]
: err.stack || err.toString();
-> to pass values down the middleware stack modifying the request object is the best method. This ensures that all processing is bound to that specific request and since the request object goes through every middleware function it is a good way to pass information down the stack.