I have a strange edge case of HTTP and HTTP-like handling where I have to support some broken clients. Specifically, a client will make an HTTP PUT request, and I need to send a 200 OK after the request headers are received, before the client sends its request body. Normally, I do this:
app.put('/*', function (req, res, next) {
res.socket.write('HTTP/1.0 200 OK\r\n\r\n');
/* Pipe socket and handle data here */
}
However, with newer versions of Node.js (v6.2.2 as of this writing), when a client connects with a proper HTTP/1.1 request and Expect: 100-continue in its request headers, the HTTP server will (correctly) send an HTTP/1.1 100 Continue message, immediately before I get a chance to send my own response status line. As a result, the client would get two status lines:
HTTP/1.1 100 Continue
HTTP/1.0 200 OK
This isn't a huge problem. I just need to detect when the Node.js HTTP server took care of the status line, so I don't send a duplicate. The trick is, I don't think this API is properly exposed. I prodded around and came up with this method:
app.put('/*', function (req, res, next) {
if (!res._sent100) {
res.socket.write('HTTP/1.0 200 OK\r\n\r\n');
}
/* Pipe socket and handle data here */
}
My Question: Is there a better way to detect when the built-in HTTP server has sent 100 Continue than by using res._sent100?
There is an HTTP server checkContinue event. However, if I handle that event then the normal events don't fire. I'm using this in conjunction with Express, and I need the normal request events so that the normal middleware stack will run. If I could handle checkContinue, write my own property to res, and then pass it back to the normal stack for handling, then all would be well. But, I don't see a documented way to do this.
This seems to work:
server.on('checkContinue', (req, res) => {
res.mySent100 = true;
res.writeContinue();
server.emit('request', req, res);
});
(where server is the HTTPServer instance that is serving your Express app)
Related
I am doing this essentially:
app.get('/:id.:ext', (req, res) => {
const remote = bucket.file(`images/${req.params.id}.${req.params.ext}`)
// if (isProd)
// res.set('Cache-Control', 'public, max-age=604800')
remote.createReadStream({ validation: false })
.on('error', error => {
console.log(error)
res.send(`data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7`)
})
.pipe(res)
.on('end', () => {
res.set('Cache-Control', 'public, max-age=86400')
})
})
When there is an error, I just return an empty image. But in this case, I don't want to set the cache headers because I don't want the blank image to be cached.
So I try setting the cache headers after the response is sent, but they don't go through.
If I set the cache headers before piping the response, I cache the error in case of error.
Wondering how to get around this.
You can't do what you're trying to do the way you're trying to do it. The sequence of an http response is to send the http headers, then start sending the respond body. So if you're piping a response body (like you are), you HAVE to send the headers first. You can't start sending the response body, then changing your mind about the headers. They've already been sent.
And, you can't send headers after the http library has already starting sending the respond body. Starting to send the response body writes out the currently stored state of the http headers that go with this response and then starts writing the response.
As best I know, the only way to deal with an error that occurs in the middle of sending the http response body is to close the http connection prematurely. The client will see the socket close without seeing the end of the http response and will understand that it got a terminated, unfinished response. You don't get a chance to send another response at that point. Error handling for that case will need to be client-side in order to decide what to do.
Another option is to prefetch all the data you want to send to the client BEFORE you send anything. This allows you the most chances to determine if anything is going to cause an error before you've started sending the http response and then you can craft the entire response to match your error condition. You obviously can't use something like .pipe(res) if you're going to do that. Instead, you've have to load the entire response into memory (or at least a chunk of the response if you're going to send it in chunks) and only when that has been successfully pre-flighted and loaded and ready to go do you start sending the response.
Also, another way to avoid getting an error image cached is to do a 301 (temporary redirect) to the error image URL rather than return it as the response to the original request. Then, when the browser loads that redirected URL and gets the image, it will not cache it as the original URL.
Is there a way to add middleware to the end of an express app or router chain that gets called to track whether or not the res / response was sent or not?
I mean, regardless of if:
A response is sent (string, JSON, etc.)
A static served file.
No file found in the static folder.
A catch-all callback was reached.
An error middleware was reached.
Example
For instance, if I wanted to log everything...
whether a response was successful or not, ie: it served a file via a express.static( ... ) middleware, some data fetched from a DB, or a custom middleware, or again... if it failed / threw an error...,
is there a way to invoke a callback at the very end?
So far from what I can understand, it seems like, by design, if a static file gets served successfully (via express.static), it doesn't call next(), so the chain stops there.
And for any custom-made middlewares using res.send(), you normally wouldn't want to call next() afterwards since it could cause some undesirable side-effects (errors with headers getting resent).
For error-handlers, that's easier since all unsuccessful responses can be caught here.
But how can it output both successful / unsuccessful responses? Could this be something that should be done without middlewares?
The solution I went with ended up being slightly different from this one by #idbehold, but in a nutshell, at the very top of the express app middleware chain, I had to hook a callback to the res Response object's finish event which gets triggered for most (all?) HTTP status-codes I needed to track a successfully served request.
app.use( ( req, res, next ) => {
res.on( 'finish', () => {
var codeStr = String( res.statusCode );
codeStr = codeStr[res.statusCode < 400 ? 'green' : 'red'];
var output = [req.method.green, req.fullUrl().green, codeStr];
trace( output.join( ' ' ) );
} );
next();
});
I can now get things like:
EDIT
Alright! So provided you also have an error-handler at the "end" of your middleware chain that serves something with an error 404 code, that will trigger the finish event on the res object.
Example of such an error-handler:
app.use( ( err, req, res, next ) => {
trace( "Error!".red );
trace( err );
res.status( 404 ).send(); // Triggers 'finish' on res.
})
There's a conceptual difficulty with the asynchronous architecture of node.js and Express for doing this. I'll describe the general problem and then discuss a few possible work-arounds.
First, each Express handler can be asynchronous. Thus, it gets called and returns pretty much immediately and nobody outside of that world knows whether it is still waiting for some asynchronous operation to finish before eventually sending its response or if it just failed to do anything. You literally can't tell from the outside world.
Second, you can monitor a given request to see if it either calls an error handler or if it sends a response. There is no way to monitor a request handler to see if it just failed to send anything because of the reason above - you have no way of knowing if its still waiting for some asynchronous thing to finish.
So, here's the best I could recommend:
Hook res.end() to see when it gets called. This is an indication that the response is now done (whether error or success). You can see an example of doing that in the express-afterware module that Medet linked in an above comment. The general idea is that you'd have your own middleware somewhere very early in the chain that overrides res.end() so you can see when its called. That early middleware would just install the override and call next() to continue the handler chain. Then, when the response is finished, your override would see that res.end() got called. This should work for all cases where the response is sent.
Then, you still need to handle cases where no response is sent (which is probably due to faulty code since all requests should get a response eventually). The only way I know of to do that is to implement some sort of timeout for a request. You can either use a built-in mechanism server.setTimeout() or you can implement your own inside your middleware (same middleware as describe in step 1). Then, after some timeout that you specify, if no response has yet been sent, you would take over and send some error response.
Install your own error middlewares early in the chain that will see and log all errors. Note that res.end() will still be called so the behavior in step 1 will still be triggered even for errors (error responses still call res.end()).
You can trigger a piece of code at the end of a request by using the finish event of the response object. The finish event is emitted when the response has been sent to the client and all the data has been flushed to the network.
app.use(function(req, res, next) {
res.on('finish', function() {
console.log('Request finished');
});
next();
});
What is the difference between the request in this line of code:
http.createServer(function(request,response){. . .}
and request in
http.request()
Are both requests done to the server?
I am new to node.js and I am sorry if I sound dumb!
How does http.request() work?
In http.request() we fetch data from another site but in order to fetch data from another site we first need to go to our site and then make a request? Explain it with a simple real-life example!
http.request() makes a request to another HTTP server. Suppose for some reason I wanted to go download Stack Overflow's home page...
http.request('https://stackoverflow.com/', (res) => {
// ...
});
http.createServer()... it creates an HTTP server. That is, it binds your application to a socket to listen on. When a new connection is made from somewhere or something else, it handles the underlying HTTP protocol of that request and asks your application to deal with it by way of a callback. From the Node.js documentation:
http.createServer((req, res) => {
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.end('okay');
});
These two methods have absolutely nothing to do with each other. http.request() is for making a request to an HTTP server. http.createServer() is for creating your own HTTP server. Don't get confused by the callbacks.
Based on the source code of nodejs (extract below), createServer is just a helper method to instantiate a Server.
Extract from line 1674 of http.js.
exports.Server = Server;
exports.createServer = function(requestListener) {
return new Server(requestListener);
};
The http.request() API is for when you want your server code to act as a client and request content from another site and has GET, POST, PUT, DELETE methods.
For a special use case with an express 4 server (forcing a client, that doesn't respond to 503 status code repeat a request) I need to not respond to a client request at all.
But Express sends a 502 Bad Gateway after apr. 2 minutes, when I just omit to send a result. How can I achieve this?
I tried to catch the timeout, but that doesn't help:
app.use((req, res, next) => {
if (!req.timedout) {
next();
return;
}
console.log('Timeout!')
});
Tell me if I didn't understand the question correctly...
You can use res.end();. That will end the request.
What is the difference between response.status() and response.sendStatus() in Express.
I notice that one is used generally for post, get, and other middleware, while the later is used in delete requests. Why is this?
status() sets a HTTP status on the response (as a Javascript object on the server side).
sendStatus() sets the status and sends it to the client.
The usage doesn't depend on the HTTP method of the request. In most cases you would use sendStatus anyway since it's unlikely that the status changes once the request is processed (especially since status code is the first line in a raw HTTP response).
Read more in the docs:
https://expressjs.com/en/4x/api.html#res.sendStatus
res.sendStatus is shorthand for implementing res.send and res.status
the link shared by #freakish explains it all.
res.sendStatus(200); // equivalent to res.status(200).send('OK')
res.sendStatus(403); // equivalent to res.status(403).send('Forbidden')
res.sendStatus(404); // equivalent to res.status(404).send('Not Found')
res.sendStatus(500); // equivalent to res.status(500).send('Internal Server Error')
As freakish already pointed out, sendStatus sets and sends the status.
This means that if you want to both set a status and send a body you have to use status. For example, you want to set an error status, and send a body with a JSON that explains why the error occured, you first have to set the status (using status), and then send the JSON (using send). If you had already set the status with sendStatus, it is no longer possible to send the JSON, because you already used a form of sent.
sendStatus(code) is a function that duplicates the standard status message corresponding to the provided status code into the body:
app.get('/a', (req, res) => {
res.sendStatus(500)
})
HTTP/1.1 500 Internal Server Error <--- this status message (aka "reason phrase")
<response headers>
Internal Server Error <--- gets duplicated here in body
See here for its implementation. There is no obvious need to do that, because the status message (the text after "500 ") will be there regardless. The above is NOT a normal HTTP idiom. Who knows why it was added into the express response. Moreover, when you send a non-empty body along with a status to a browser, it will render that body rather than a standard stylish page:
res.status(500).send() is sufficient to respond in a standard way:
app.get('/b', (req, res) => {
res.status(500).send()
})
HTTP/1.1 500 Internal Server Error <--- status message
<response headers>
<--- empty body
This is what everyone expects if there is nothing to say beyond status code & message.
no it's not for special routers like delete
The diffrence is res.status() only sets header and when you send response (res.send()) it will send to user
but res.sendStatus() sends the response at the moment without response body