I'm using Express to to serve some webpages. I have a font file (woff) that I'm downloading using node's https API. I then pass the response headers along to the response I'm returning to the client. The issue I have is that express seems to be modifying the response headers, specifically the content-type and content-length headers.
I'm calling res.set() and passing the headers from the server side request
Here is some code:
app.get('/*', (req, res, next) => {
https.get(URI, (serversideRes) => {
serversideRes.on('end', () => {
res.set(serversideRes.headers);
console.log(res.getHeaders()['content-type']); //font/x-woff
console.log(res.getHeaders()['content-length']); //27756
res.send(data);
console.log(res.getHeaders()['content-type']); //font/x-woff; charset=utf-8
console.log(res.getHeaders()['content-length']); //49574
}
}
}
In the browser console I'm getting
Failed to decode downloaded font
OTS parsing error: incorrect file size in WOFF header
From the Express API
This method performs many useful tasks for simple non-streaming responses: For example, it automatically assigns the Content-Length HTTP response header field (unless previously defined) and provides automatic HEAD and HTTP cache freshness support.
In this case, the headers clearly seem to be previously defined.
Related
I am using building a node express HTTP web service that calls a web service that returns a JPEG file. The web service works to proxy the JPEG file but I need to add some content headers to the returned GET request that I respond via pipe. Any suggestions?
app.get('/rendition', function (req, res){
let uri = `https://example.com/thumbnail2x`;
axios.get(uri).then(function(response){
// how do I add a header here?
response.data.pipe(res);
});
})
Нello, please take a look at this simple file server.js:
require('http').createServer((req, res) => {
res.end('hiii');
console.log('Response headers:', res.getHeaders());
}).listen(80);
Navigating to localhost:80 in my browser hits this endpoint. This causes the response of hiii to appear in the browser, and also the headers to be logged to stdout.
The strange thing is, the headers logged to stdout disagree with the headers the browser received.
Stdout shows me an object representing 0 headers:
Response headers: [Object: null prototype] {}
Developer tools show me that in fact, 3 response headers were received:
What accounts for this difference? I understand that the 3 headers shown in chrome are very fundamental to http. Is chrome receiving 0 headers, but filling them in by default? Is node's http library filling these headers in by default? If that's the case, why aren't they exposed via res.getHeaders()? Are these headers being calculated at some lower level, as in C libraries? If so is there any means of exposing these values?
I tried the following in case there is some kind of async delay where the headers are calculated:
require('http').createServer((req, res) => {
res.end('hiii');
setTimeout(() => console.log('Response headers:', res.getHeaders()), 3000);
}).listen(80);
But nonetheless, 0 headers are sent to stdout.
Somewhere, these 3 headers are being calculated! How can I access these calculated header values??
I found them under res.socket._httpMessage._header
require('http').createServer((req, res) => {
res.setHeader('HELLO', 'WORLD')
res.end('hiii');
console.log(res.getHeaders())
console.log(res.socket._httpMessage._header)
}).listen(8000);
You can't get them using getHeaders because they are not set using the regular API but they are sent by nodejs internaly.
If you define a header yourself using setHeader then you'll be able to retrieve it using getHeaders
Response headers are not generated by the client since it needs them to be able to process the response. How can the client read the request if it doesn't know the size of the body (Content-Length).
These communication rules are defined in the HTTP RFC and the server must implement them so the client can understand the message it receives.
NodeJS calculates these information internaly (source code exampe here) and probably sends them somehow in the socket without storing them in the high level API that you are using. (another source code example)
I want to see in Node Exprees enviroment all the headers that are sent to the client
But when i see do this:
app.get("/", (req, res) => {
console.log(res.getHeaders());
});
i only see this :
At the time you're looking at the outgoing headers, those are the only ones that have been added so far. The rest will be added by the code that sends the actual response or by other middleware.
If you want to see all the headers that were eventually added to the response before it was sent, you can monitor the finish event and THEN look at the headers:
app.use(function(req, res, next) {
res.on('finish', () => {
console.log(`request url = ${req.originalUrl}`);
console.log(res.getHeaders());
});
next();
});
This will sometimes not include the date, content-type or content-length headers unless they are specifically set by the sending code. This is because if these are not set by the sending code, then the HTTP library adds these headers at a lower level as it is sending the headers and thus res.getHeaders() does not retrieve them and never knows about them.
Edit: I overlooked your first screenshot... Are you using any middleware? It looks like you're using the CORS middleware, at least - which is why you are showing more headers than the defaults..
It looks like Node/Express sends a Content-Length header when it can..
The Date header is mandatory per the HTTP spec, but it looks like you can change that behavior in Node/Express - I'm guessing by default Node/Express sets that value to true.
I did test setting res.sendDate = false and the date header was not sent, so it looks like that header is set by default for you, most likely as the last step in the response?
With res.sendDate = false;
Without setting res.sendDate (aka the default):
All in all, I'm assuming the headers you don't see when you console.log(res.getHeaders()) are set by Node/Express by default..
I wasn't able to find anything in the docs about default response headers (outside of the Date header), but it's possible I overlooked something. The Express docs don't have anything on it as they just use the built in http module from Node.
I'm using Express and Node to build a web, where the client uses RESTful API to get Json response to show a list of objects.
After creating a new object, the app should request a new json response of the updated object list. Chrome works fine - new json response returned with status code 200 OK. However, things are not good in IE and Edge - it seems the browser just fetch the json response from cache (with status code 304), instead of making a new request. If I manually clear the browser cache data things will be fine.
I tried this solution: using a middleware to set max-age of cache-control res.header to be 0
function headerSet(req, res, next) {
res.header('Cache-Control', 'public, max-age=0');
return next();
}
And in the response header I can see accordant settings taking effect, however, IE and Edge still refuse to make a new request - I'm still getting the unupdated json response with 304.
What possibly have I done wrong?
Instead of setting max age try with the following.
res.header('Cache-Control', 'private, no-cache, no-store, must-revalidate');
I have a Node http-proxy server doing some response body rewriting that basically does this:
Client GET localhost:8000/api/items
Node Proxy send localhost:8000 -> to example.com/api
Server responds with json [{ id: 1234, url: http://example.com/api/items/1234 }]
Node proxy rewrites json to [{ id: 1234, url: http://localhost:8000/api/items/1234 }]
Node proxy calculates the new content-length header, sets it, and returns the response to the client
This was working fine until the backend server enabled compression. So now, by default, responses were being gzipped. I worked around this by setting this in my proxy:
req.headers['accept-encoding'] = 'deflate';
So after that, responses weren't being gzipped, I could parse them and rewrite the body as necessary before. However, this stopped working with IE. I think the problem is that the response still has a transfer-encoding=chunked header, so IE expects a chunked response. Because that transfer-encoding header is present, there is no content-length header, even though I'm explicitly setting it (those two headers being mutually exclusive). I've tried everything I can think of to remove the transfer-encoding header and get the content-length header in instead, but nothing is working. I've tried all of these:
// In the context of my middleware response.writeHead function
res.setHeader('transfer-encoding', null);
res.setHeader('transfer-encoding', '');
res.removeHeader('transfer-encoding');
res.setHeader('content-length', modifiedBuffer.length); // this line alone worked before
res.originalWriteHead.call(res, statusCode, { 'Content-Length', modifiedBuffer.length });
// In the context of my middleware response.write function res.write(data, encoding)
// Here, encoding parameter is undefined
// According to docs, encoding defaults to utf8, could be 'chunked'
res.oldWrite.call(res, modifiedBuffer, 'utf8');
res.oldWrite.call(res, modifiedBuffer, '');
res.oldWrite.call(res, modifiedBuffer, null);
// tried all three previous the same for res.end
Basically, no matter what I do, the response is not chunked, but has the transfer-encoding header set, and not the content-length. Firefox, safari, chrome all seem to handle this fine, but IE fails with the error XMLHttpRequest: Network Error 0x800c0007, No data is available for the requested resource.. This is (from what I can tell) because it's waiting for chunks (because of the transfer-encoding header), but gets the end of the response, and doesn't have a content-length to read it.
Does anyone know how I can solve this? Am I doing something wrong in trying to remove the transfer-encoding header in favor of content-length?
I figured this out myself:
I was effectively using two middleware components (my own, described in the question), and express.js compression. My middleware was decompressing the response, but compression ensures that the response is always written with transfer-encoding=chunked and content-length is removed.
Removing the express.js compression module solved this for me.