How can I gunzip POST request data in express? - node.js

Question

I am trying to build a server that can accept gzipped POST data with express. I think I could just write my own middleware to pipe the request stream to a zlib.createGunzip() stream. The question is, how can I achieve that, afterwards, the express.bodyParser() middleware is still able to parse my gunzipped POST data?
I tried to replace the original request stream methods by the ones of the zlib stream, but that just made the bodyParser return a "Bad Request" Error:
var express = require('express');
var app = express();
function gUnzip(req, res, next) {
var newReq;
if (req.headers['content-encoding'] === 'gzip') {
console.log("received gzipped body");
newReq = req.pipe(zlib.createGunzip());
Object.getOwnPropertyNames(newReq).forEach(function (p) {
req[p] = newReq[p];
});
}
next();
}
app.use(gUnzip);
app.use(express.bodyParser());
app.listen(8080);
Is there a way to make this work without rewriting the bodyParser() middleware within my own middleware?
EDIT:
This is the same question: Unzip POST body with node + express. But in the answer he just does in his own middleware what the express.bodyParser() should do, which is what I want to avoid. I am looking for a way to simply unzip the request data from the stream and then pass it to the bodyParser(), which expects a stream itself, as can be seen at http://www.senchalabs.org/connect/json.html.

Answer 1

compressed request bodies are generally not used because you can't negotiate content encodings between the client and server easily (there's another stackoverflow question about that i believe). most servers don't support compressed request bodies, and the only time you really need it is for APIs where the client will send large bodies.
body-parser, specifically raw-body, does not support it because the use-case is so minimal, though i've though about adding it. for now, you'll have to create your body-parser. fortunately, that's easy since you can just fork body-parser and leverage raw-body. the main code you'll add around https://github.com/expressjs/body-parser/blob/master/index.js#L80:
var zlib = require('zlib')
var stream
switch (req.headers['content-encoding'] || 'identity') {
case 'gzip':
stream = req.pipe(zlib.createGunzip())
break
case 'deflate':
stream = req.pipe(zlib.createInflate())
break
case 'identity':
break
default:
var err = new Error('encoding not supported')
err.status = 415
next(err)
return
}
getBody(stream || req, {
limit: '1mb',
// only check content-length if body is not encoded
length: !stream && req.headers['content-length'],
encoding: 'utf8'
}, function (err, buf) {
})

Answer 2

Have you tried using the built in compress middleware. It's documented in the expressjs reference documentation
app.use(express.compress());
Maybe you can find something useful here instead: Unzip POST body with node + express