Behavior of expressjs next middleware when the response has gone - node.js

I am using express (v:4.17.1) and here is my code snippet:
//upload.array is for multer npm file uplaoding....
//LIMIT is how many files you can upload via multer ....
app.post('/processFiles', upload.array('myFile', LIMIT), (req, res, next) => {
if (!validEnv) {
res.setHeader('X-Correlation-ID', corrId);
res.status(400).json({'error':'invalid env'});
return next(); //A
}
//checking how many file hvae been uplaoded
if (totalFiles > LIMIT) {
res.status(400).json({'error':'Too many files uploaded'});
return next(); //B
}
//Some xml parsing code and later it inserts into db
}
Few questions on the way I have added 'return next();' in the routes. Please note that I have not added any explicit middleware, just relying on what express is providing:
If I keep [A] or [B] as it is, then it gives a proper error message to the browser if the conditions get TRUE and returns the response.
But if I comment the line //A or //B, it gives the error response back to browser but also prints few error logs as mentioned below:
Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client
I don't know why the above line is appearing when I comment the //A or //B (but wont complain if I keep as it is without commenting them) as the response was already out to the browser and the express will not execute any piece of code after the response has gone.
Please correct my understanding ?
Also, Is this the correct way to handle errors as I have coded? If No, what could be the proper way, please advice? Is there any way by which we can see the complete middleware execution in sequence until the last one via any debug flags

There are a several ExpressJS things to know that apply here:
next() tells Express to continue routing to other requests. Only call next() if you want other routes to continue to be able to process this request and send a response. Or the corallary, don't call next() if you've already sent a response.
You can only send one response for a given request. Once, you've called res.send() or .res.json() or any other way of sending a response, you should be completely done with your request processing and no other code should try to send a response.
res.send() and next() are not Javascript flow control. If you want to stop further processing of the request in your function, you need to code that with traditional Javascript flow control such as return, if/else, etc... to prevent the rest of your code from continuing to run or process the request.
To that end, your code should look like this:
//upload.array is for multer npm file uplaoding....
//LIMIT is how many files you can upload via multer ....
app.post('/processFiles', upload.array('myFile', LIMIT), (req, res, next) => {
if (!validEnv) {
res.setHeader('X-Correlation-ID', corrId);
res.status(400).json({'error':'invalid env'});
return;
}
//checking how many file hvae been uplaoded
if (totalFiles > LIMIT) {
res.status(400).json({'error':'Too many files uploaded'});
return;
}
// more code here that sends some other response
}
But if I comment the line //A or //B, it gives the error response back to browser but also prints few error logs as mentioned below: Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client
This error occurs when you attempt to send more than one response for the same request (e.g. call res.send(...) more than once. Your code should not allow that. Once you send a response, your code should use normal Javascript flow control (such as return or if/else) to avoid executing any more code that might send a response. Typically after you send a response, you are done processing the request and would just return from the request handler.

This is happening because of the //A. See, what is going on here in your code :
If you comment line //B it reaches the next if (totalFiles > LIMIT) which is sending another response but above the return next() is called. That is why it is generating error that once they are sent in if(!validEnv) how can Headers be sent again!

You have two if conditions that mean both might get executed, in both the conditions you are setting the headers and ending the request. Once the request is ended the headers cannot be set. So if you comment [A] which has the return statement and if the second condition is also satisfied, it will set the headers again after sending the request. so it has nothing to do with the next() function or express since there is no next middleware in the stack.
Also the error handling is looks fine. however you can use the node --inspect flag while starting the script and set breakpoint in your middleware to debug entire node code.

Related

Good practice to handle useless response in express js/nodejs?

Am using express js on node for my api service ! In which am using sequelize for query handling purposes !
So in some usecase like creating record, or updating record its simply returning "1" or sometimes nothing !
In this case , am just using
res.sendStatus(200);
or sometimes
res.send("success");
Is there any better way or this is the correct way to handle ? Or should in need .end() in order to end the process ??
which is a good way to handle these kind of useless responses which we dont need to send back ?
This is where Status 204 comes in to play: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#2xx_success
It states: everything is OK (like in 200), but there is simple no need to send a body.
Using express, it's just as simple as: res.sendStatus(204)
Usually a json is send back to the client with different information in it that lets the client-side know that the operation that they request through the api is successfull or failed. for e.g. here, you can define a standard json response like this
//in case of success of the request
{
requestStatus: 1,
...other data if you want to like new Id of the created data/updated data or info about it if you want or depending upon use case
}
or
// in case of failure of the request
{
requestStatus: 0,
...other data if you want to like reason for failure, error message or depending upon your use case
}
Just add this line of code, should be fine:
//For all bad endpoints
app.use('*', (req, res) => {
res.status(404).json({ msg: 'Not a good endpoint, why are you here? Go play FIFA.'})
})
If you want you can generate an error HTML file, but since it's back-end, JSON format is strongly suggested. You can also add success: false for more clarity.

Why does my express router crash only when i send a JSON and not when i send a text ?

statusRouter.route('/')
.all(function(req,res,next){
res.writeHead(200, {'Content-Type': 'application/json'});
next();
})
.get(function(req, res, next) {
res.json({
name : "xyz"
});
});
This crashes with - Header cant be set after it has been sent.
But the catch is , this works :
statusRouter.route('/')
.all(function(req,res,next){
res.writeHead(200, {'Content-Type': 'text/plain'});
next();
})
.get(function(req, res, next) {
res.end("xyz");
});
NOTE : If I remove the writeHead function in the first case where I am sending JSON it starts working as well. Why does it not work when i do a writeHead on it ? This thing is driving me crazy can anyone explain why this happens ?
P.S I'm working with express-generated app with my own router.
Both res.writeHead() and res.end() are not implemented by Express, but by the Node.js http module.
Its documentation states, for res.end():
If data is specified, it is equivalent to calling response.write(data, encoding) followed by response.end(callback)
So res.end("xyz") is short for:
res.write("xyz");
res.end();
For res.write() the documentation states:
If this method is called and response.writeHead() has not been called, it will switch to implicit header mode and flush the implicit headers.
So res.end("xyz") is actually short for:
if (! res.headersSent) {
res.writeHead(...);
}
res.write("xyz");
res.end();
This means that it's perfectly okay to issue res.writeHead() in your own code, before using res.end(). Internally, the http module will know that you already flushed the headers, so it won't do it again (therefore preventing the error you're getting). However, you can't set different headers, or change existing ones, once writeHead() has been called.
Now, res.json() is another matter: this isn't part of the http module, but of Express itself. Because it's used to send JSON responses, it will set the content type header to application/json (so you don't have to).
But this will only work if the headers haven't yet been sent already: you can't set headers when they have already been sent out. That's why you're getting the error.
If you want to set particular headers in Express, use res.set().
Express evaluates its routes in the order they are added. This means that the route handler passed to .all() will be executed first. In this handler, a response header is written and then next() tells Express to continue iterating its list of routes for another possible matching route.
It then matches the .get() route handler which tries to send a JSON response. However, .json() implicitly sends a response header as well as writes the value passed as JSON to the response and ends the response. This is why you are seeing an error about writeHead() being called more than once.

Error: Can't set headers after they are sent only on page refresh

I have this problem only when I try refresh the page and I can not solve it, I tried everything but still happens the same. It began to happen when I add socket.io at the project. The project run in several servers which are connected one each other throught sockets.
TEST CASES: When I render the page, at the first time everything goes well but, if I refresh the same page, I get this error:
ERROR: "Error: Can't set headers after they are sent. at ServerResponse.OutgoingMessage.setHeader (_http_outgoing.js:344:11)"
ATTENTION: when get in IF() and send "return res.end('The Activation Code is INVALID!');" it DOESN'T HAPPEND! I refresh it and refresh it and everything goes well. My problem is in the RENDER.
MY CODE BELOW:
activationUser = function(req,res,next){
var data = {
activationCode : req.params.activationCode,
now : new Date().valueOf(),
ip : req.connection.remoteAddress,
fId : frontalId
}
socketCore.emit('activationUser', data);
socketCore.on(frontalId + 'activationUserResp', function(data){
if(data.msg == "CHECKED!"){
next();
}else{
return res.end(data.msg);
}
});
}
router.get('/activationUser/:activationCode',activationUser,function(req,res){
var data = {
activationCode : req.params.activationCode,
fId : frontalId
}
socketCore.emit('step2', data);
socketCore.on(frontalId + 'step2Resp', function(data){
if(data.msg == 'err'){
return res.end('The Activation Code is INVALID!');
}else{
return res.render('registro2', {title: 'title | '+ data.name + ' ' + data.lastname, user:data});
}
});
});
Thank you!
The particular error you are getting happens when you try to send anything on the res object after the complete response has already been sent. This often occurs because of errors in asynchronous logic. In your particular case, it apepars to be because you are assigning a new event handler with socketCore.on() every single time the router is hit. Those event handlers will accumulate and after the first time the route is hit, they will execute multiple times triggering the sending of multiple responses on the same response object, thus trigger that error.
The main ways to fix your particular problem are:
Use .once() instead of .on() so the event handler automatically removes itself after being triggered.
Manually remove the .on() event handler after you get the response.
Move the event handler outside of the route so it's only ever installed once.
In your particular case, since socketCore is a shared object available to all requests, it appears that you also have a race condition. If multiple users trigger the '/activationUser/:activationCode' route in the same general time frame, then you will register two event handlers with socketCore.on() (one for each route that is hit) and you will do two socketCore.emit('step2', data);. But, you have no way of associating which response belongs with which request and the two responses could easily get mixed up - going to the wrong request.
This highlights how socket.io connections are not request/response. They are message/answer, but unless you manually code a correspondence between a specific message request and a specific answer, there is no way to correlate which goes with which. So, without assigning some particular responseID that lets you know which response belongs to which message, you can't use a socket.io connection like this in a multi-user environment. It will just cause race conditions. It's actually simpler to use an HTTP request/response for this type of data fetching because each response goes only with the request that made it in the HTTP architecture.
You can change your architecture for making the socketCore request, but you will have to manually assign an ID to each request and make sure the server is sending back that ID with the response that belongs to that request. Then, you can write a few lines of code on the receiving side of things that will make sure the right response gets fed to the code with the matching request.

nodeJS prevent timeout on res.download

I have a POST call to my nodeJS server that searches for some data on the mongo database and returns a CSV file with the requested data. The problem is that the data search and processing exceeds the 2 minute nodeJS default timeout.
On a diferent scenario y used:
res.writeHeader(200,'application/json');
res.write('starting fetch .... ');
to keep alive the request and prevent the timeout from the client by sending some res.write(''); from time to time.
Now Im using res.download() to download the resulting csv file, so not just sending JSON as response.
Tried to use this solution like this:
res.writeHeader(200,'application/json');
res.write('starting fetch .... ');
res.download()
But i get the "headers already sent" error.
Any idea on how to prevent the timeout until the data is processed and the file download is done ?
Thanks in advance
A few small things first of all, i think it's supposed to be res.writeHead() instead of res.writeHeader(). Secondly once you send the headers you can not send them again. You are getting the error because res.download() uses res.sendFile() which re-sends the headers. You can't use res.download() or res.sendFile() or res.send() because all of these three methods implicitly set the headers and send as part of the response. You'll need to use a combination of res.writeHead() ,res.write() and res.end(). I am just GET so i can test it from the browser, you can use post obviously.
router.get('/test', function (req,res,next) {
res.writeHead(200, {'Content-Type': 'application/json',
'Content-Disposition': 'attachment',
'filename':'package.json'
});
fs.readFile("package.json",function (err,data) {
if (err) {
throw err;
}else{
setTimeout(function () {
res.write(data);
res.end()
},200000);
}
});
function keepBusy(){
if(!res.finished){
res.writeProcessing();
setTimeout(keepBusy,1000)
}
}
setTimeout(keepBusy,1000);
});
This works fine for me. res.writeProcessing() sends the 102 status code. It is recommended for requests that last greater than 20 second but this blog post says that
102 has been left out more recent revisions of HTTP specifications
but it works fine for me in most browsers. If you have some control over the client as well there can be a much better and reliable solution.

Why can't we do multiple response.send in Express.js?

3 years ago I could do multiple res.send in express.js.
even write a setTimeout to show up a live output.
response.send('<script class="jsbin" src="http://code.jquery.com/jquery-1.7.1.min.js"></script>');
response.send('<html><body><input id="text_box" /><button>submit</button></body></html>');
var initJs = function() {
$('.button').click(function() {
$.post('/input', { input: $('#text_box').val() }, function() { alert('has send');});
});
}
response.send('<script>' + initJs + '</script>');
Now it will throw:
Error: Can't set headers after they are sent
I know nodejs and express have updated. Why can't do that now? Any other idea?
Found the solution but res.write is not in api reference http://expressjs.com/4x/api.html
Maybe you need: response.write
response.write("foo");
response.write("bar");
//...
response.end()
res.send implicitly calls res.write followed by res.end. If you call res.send multiple times, it will work the first time. However, since the first res.send call ends the response, you cannot add anything to the response.
response.send sends an entire HTTP response to the client, including headers and content, which is why you are unable to call it multiple times. In fact, it even ends the response, so there is no need to call response.end explicitly when using response.send.
It appears to me that you are attempting to use send like a buffer: writing to it with the intention to flush later. This is not how the method works, however; you need to build up your response in code and then make a single send call.
Unfortunately, I cannot speak to why or when this change was made, but I know that it has been like this at least since Express 3.
res.write immediately sends bytes to the client
I just wanted to make this point about res.write clearer.
It does not build up the reply and wait for res.end(). It just sends right away.
This means that the first time you call it, it will send the HTTP reply headers including the status in order to have a meaningful response. So if you want to set a status or custom header, you have to do it before that first call, much like with send().
Note that write() is not what you usually want to do in a simple web application. The browser getting the reply little by little increases the complexity of things, so you will only want to do it it if it is really needed.
Use res.locals to build the reply across middleware
This was my original use case, and res.locals fits well. I can just store data in an Array there, and then on the very last middleware join them up and do a final send to send everything at once, something like:
async (err, req, res, next) => {
res.locals.msg = ['Custom handler']
next(err)
},
async (err, req, res, next) => {
res.locals.msg.push('Custom handler 2')
res.status(500).send(res.locals.msg.join('\n'))
}

Resources