I have to test for server errors (Express) in acceptance tests that can't (or shouldn't) be sent with response, for example
Error: Can't set headers after they are sent.
Catching an error with error handler and responding with 5XX code would provide valuable feedback here, but the problem is that the headers have been sent already.
This kind of bugs may be noncritical and hard to spot, and usually they are figured out from the logs.
The spec is
it('should send 200', function (done) {
request(app).get('/').expect(200, done);
});
And tested app is
app.get('/', function (req, res, next) {
res.sendStatus(200);
next();
});
app.use(function (req, res) {
res.sendStatus(200);
});
What is the most appropriate way to communicate between Express app instance and request testing library (i.e. Supertest) in similar cases?
The question is not restricted to Supertest. If there are packages that can solve the problem that Supertest can't, they may be considered as well.
Try to just set the status code without send it and avoid send it twice throwing the error use res.status().
As the express documentation say it
Sets the HTTP status for the response. It is a chainable alias of
Node’s response.statusCode.
IMHO if you want to detect it in a end-to-end (E2E) testing tool like supertest (or Selenium) you have to handle the express error and send a correct output (500 status error, some message...) to permit detect it.
Or use instead a unit test, to test that the controller function doesn't throw any error using chai or a native assertion.
I answered a similar question here. The "can't set headers after they have been set" error should be raised by express as an unhandled exception. So, you should be able to get access to it via the unhandledException event that is raised by the process.
However, this is much more tricky because of the timing. Your test case's expect function and done function will be queued for processing on the event loop on the tick right after the first res.statusCode call. Unfortunately, the next call for res.statusCode can happen at an indeterminate amount of time afterwards. For example, what if the second route handler called a really slow webservice or db and then called res.statusCode.
With that in mind, your options are pretty hard. The brute force way is to wait in your test code for a determinate amount of time and then check. It's effective but slow and non-deterministic, which will cause your test to be flaky.
Another option is to check any instrumentation code that you might have in express. If you have code in express that keeps metrics of the number of in process calls for the various route handlers you could expose these metrics to your test code. Then one of your conditions for finishing your test is that all metrics for in process route calls are 0. The second option would allow you to write deterministic tests and be much faster because you could poll the metrics.
The final option would be to handle this test case through unit tests. This is probably the best solution because it would be deterministic and wouldn't require any sort of polling. However, the downside is that you need to know that both of your functions are called in order which leads you down a path of trying to recreate in your test code the logic that express uses to call route handlers.
I did this using HAPI instead of Express, but I solved the same problem. I used an external library to make the call (like request-promise) and it worked. Catch the error in the request-promise response.
Related
I’m always coding backend api’s and I don’t really get how express does its bidding with my code. I know what the request and response objects offer, I just don’t understand how they come to be.
This simplified code for instance:
exports.getBlurts = function() {
return function(req, res) {
// build query…
qry.exec(function(err, results) {
res.json(results);
}
});
}
}
Then I’d call in one of my routes:
app.get('/getblurts/, middleware.requireUser, routes.api.blurtapi.getBlurts());
I get that the function is called upon the route request. It’s very abstract to me though and I don’t understand the when, where, or how as it pertains to the req\res params being injected.
For instance. I use a CMS that modifies the request object by adding a user property, which is then available globally on all requests made whether ajax or otherwise, making it easy at all times to determine if a user is logged in.
Are the req and res objects just pre-cooked by express but allow freedom for them to be modified to your needs? When are they actually 'built'
At its heart express is actually using node's default http-module and passing the express-application as a callback to the http.createServer-function. The request and response objects are populated at that point, i.e. from node itself for every incoming connection. See the nodeJS documentation for more details regarding node's http-module and what req/res are.
You might want to check out express' source code which shows how the express application is passed as a callback to http.createServer.
https://github.com/expressjs/express/blob/master/lib/request.js and https://github.com/expressjs/express/blob/master/lib/response.js show how node's request/response are extended by express specific functions.
Is there a way to add middleware to the end of an express app or router chain that gets called to track whether or not the res / response was sent or not?
I mean, regardless of if:
A response is sent (string, JSON, etc.)
A static served file.
No file found in the static folder.
A catch-all callback was reached.
An error middleware was reached.
Example
For instance, if I wanted to log everything...
whether a response was successful or not, ie: it served a file via a express.static( ... ) middleware, some data fetched from a DB, or a custom middleware, or again... if it failed / threw an error...,
is there a way to invoke a callback at the very end?
So far from what I can understand, it seems like, by design, if a static file gets served successfully (via express.static), it doesn't call next(), so the chain stops there.
And for any custom-made middlewares using res.send(), you normally wouldn't want to call next() afterwards since it could cause some undesirable side-effects (errors with headers getting resent).
For error-handlers, that's easier since all unsuccessful responses can be caught here.
But how can it output both successful / unsuccessful responses? Could this be something that should be done without middlewares?
The solution I went with ended up being slightly different from this one by #idbehold, but in a nutshell, at the very top of the express app middleware chain, I had to hook a callback to the res Response object's finish event which gets triggered for most (all?) HTTP status-codes I needed to track a successfully served request.
app.use( ( req, res, next ) => {
res.on( 'finish', () => {
var codeStr = String( res.statusCode );
codeStr = codeStr[res.statusCode < 400 ? 'green' : 'red'];
var output = [req.method.green, req.fullUrl().green, codeStr];
trace( output.join( ' ' ) );
} );
next();
});
I can now get things like:
EDIT
Alright! So provided you also have an error-handler at the "end" of your middleware chain that serves something with an error 404 code, that will trigger the finish event on the res object.
Example of such an error-handler:
app.use( ( err, req, res, next ) => {
trace( "Error!".red );
trace( err );
res.status( 404 ).send(); // Triggers 'finish' on res.
})
There's a conceptual difficulty with the asynchronous architecture of node.js and Express for doing this. I'll describe the general problem and then discuss a few possible work-arounds.
First, each Express handler can be asynchronous. Thus, it gets called and returns pretty much immediately and nobody outside of that world knows whether it is still waiting for some asynchronous operation to finish before eventually sending its response or if it just failed to do anything. You literally can't tell from the outside world.
Second, you can monitor a given request to see if it either calls an error handler or if it sends a response. There is no way to monitor a request handler to see if it just failed to send anything because of the reason above - you have no way of knowing if its still waiting for some asynchronous thing to finish.
So, here's the best I could recommend:
Hook res.end() to see when it gets called. This is an indication that the response is now done (whether error or success). You can see an example of doing that in the express-afterware module that Medet linked in an above comment. The general idea is that you'd have your own middleware somewhere very early in the chain that overrides res.end() so you can see when its called. That early middleware would just install the override and call next() to continue the handler chain. Then, when the response is finished, your override would see that res.end() got called. This should work for all cases where the response is sent.
Then, you still need to handle cases where no response is sent (which is probably due to faulty code since all requests should get a response eventually). The only way I know of to do that is to implement some sort of timeout for a request. You can either use a built-in mechanism server.setTimeout() or you can implement your own inside your middleware (same middleware as describe in step 1). Then, after some timeout that you specify, if no response has yet been sent, you would take over and send some error response.
Install your own error middlewares early in the chain that will see and log all errors. Note that res.end() will still be called so the behavior in step 1 will still be triggered even for errors (error responses still call res.end()).
You can trigger a piece of code at the end of a request by using the finish event of the response object. The finish event is emitted when the response has been sent to the client and all the data has been flushed to the network.
app.use(function(req, res, next) {
res.on('finish', function() {
console.log('Request finished');
});
next();
});
I am overall clueless about how and why you set up a node.js app, and how any of the app.use functions work - the tutorials on it don't explain the why of anything.
Anyway, I have socket.io, res.locals and index.js set up like so in the app.js root file.
const sockets = require('./models/socket')(io)
app.use(function (req, res, next) {
res.locals.user_id = req.session.user_id;
next();
});
const routes = require('./routes/index');
app.use('/', routes);
I'd like to be able to access res.locals in the socket.js model, like I can in index.js found in the routes folder.
I can't guess how to go about doing this. If anybody is able to explain how and why I can or can't that would be a bonus. Thanks!
Welcome to Expressjs, there are a few fundamentals you should probably research before going any further, they'll help solve some of your confusion. I'll give a brief explanation of them but I suggest you do further research. I'll then answer your actual question at the end.
Middleware and app.use
Expressjs is built upon an idea that everything is just "middleware". Middleware is a function which runs as part of a request chain. A request chain is essentially a single client request, which then goes through a chain of a number of middleware functions until it either reaches the end of the chain, exits early by returning a response to the client, or errors.
Express middleware is a function which takes the following three arguments.
req (request) - Representing the request made by a client to your
server.
res (response) - Representing the response you will return to
the client.
next - A way of telling express that your current
middleware function is done, and it should now call the next piece of
middleware. This can either be called "empty" as next(); or with an
error next(new Error());. If it is called empty, it will trigger
the next piece of middleware, if it is called with an error then it
will call the first piece of error middleware. If next is not called at the
end of a piece of middleware, then the request is deemed finished and the
response object is sent to the user.
app.use is a way of setting middleware, this means it will run for every request (unless next() is either not called by the previous piece of middleware for some reason, or it's called with an error). This middleware will run for any HTTP request type (GET, POST, PUT, DELETE, etc).
app.use can take multiple arguments, the important ones for beginners to learn are: app.use(func) and app.use(path, func). The former sets "global" middleware which runs no matter what endpoint (url path) the client requests, the latter (with a specific path) is run only if that specific path is hit. I.e. app.use('/hello', (req, res, next) => { res.send('world'); }); will return "world" when the endpoint "/hello" is hit, but not if the client requests "/hi". Where as app.use((req, res, next) => { res.send('world'); }); would return "world" when you hit any endpoint.
There are more complex things you can do with this, but that's the basics of attaching middleware to your application. The order they are attached to the application, is the order in which they will run.
One more thing, this will blow your mind, an express application made with the standard const app = express() can also be used as middleware. This means you can create several express applications, and then mount them using app.use to a single express application. This is pretty advanced, but does allow you to do some really great things with Express.
Why can you not access res.locals in socket.io? (The real question)
Within your middleware handler, you are setting up a res.locals.use_id property. This only lives with that individual request, you can pass it around as long as the request is alive by passing it into other functions, but outside of that request it doesn't exist. res is literally the response object that tells Express how to respond to the clients request, you can set properties of it during the request but once that HTTP request has ended it's gone.
Socket.io is a way of handling web socket requests, not standard HTTP requests. Thus, in a standard express HTTP request you will not be able to hand off the connection to anything with socket.io, because the connection is a single short lived HTTP request. Likewise, you won't be able to do the same the other way.
If you wish to find the users id in a socket.io request, you'll have to do this within the socket.io request itself.
Right now, you're entering a piece of middleware for an Express.js request, you are then calling next() which runs the next piece of express middleware, at no point does it cross over into Socket.io realms. This is often confused by tutorials because Socket.io can handle requests across the same port as Express is listening on, but the two are not crossed over. So you will need to write separate middleware for both Express.js requests chains, and socket.io request chains. There are ways of writing this code once and then writing an adapter to use it across both platforms, but that's not what you've tried to do here.
I would suggest you look at doing just nodejs and express for a time before taking on socket.io as well, otherwise you're trying to learn a whole heap of technologies all at once is quite a lot to try and take on board all at once.
I want to have a system which will either send me a text message or email based on severity of error on respective occurrence. I have done some research on the subject but I haven't been able to simplify matters.
All in all my end goal is to get notification whenever something is going wrong with my production server. In case of serious issues which are not caught at all I have used process's uncaughtException and for specific routes which are called via separate API calls I can simply use try catch blocks.
My first issue is I cannot handle errors thrown from different routes directly via my main server file.
Second is I'll have to write same kinda' code in every route for exception handling. If something could just listen to all the errors from main server file then I can do things easily.
You can use an error handling middleware in Express:
app.use(function (err, req, res, next) {
console.error(err.stack);
// do whatever you need to send a notification
res.status(500).send('Internal error');
});
See the documentation on the error handling in Express:
https://expressjs.com/en/guide/error-handling.html
Everything you ask about is explained very well there.
I'm creating a module, which is an express middleware. My question is, when it encounters some kind of error (for example an acl module that sees that the user has no access right to this content), what is the best practice, calling next() with an error, so the app using the module can handle it the way it chooses, or can I send a response and not even call next?
Yes, you pass a error into next function to allow user catch error in callback and do what he needs.
More you can read in this articles:
Who provides the next() function in Express middleware?
https://blog.safaribooksonline.com/2014/03/10/express-js-middleware-demystified/