I am trying to modify the response with the help of a proxy created using node-http-proxy.
However I am not able to access the response headers. I want to access the response headers since I would like to modify javascript files and send the modified javascript files to the client.
This is my code:
var httpProxy = require('http-proxy');
var url = require('url');
var i = 0;
httpProxy.createServer(function(req, res, next) {
var oldwriteHead = res.writeHead;
res.writeHead = function(code, headers) {
oldwriteHead.call(res, code, headers);
console.log(headers); //this is undefined
};
next();
}, function(req, res, proxy) {
var urlObj = url.parse(req.url);
req.headers.host = urlObj.host;
req.url = urlObj.path;
proxy.proxyRequest(req, res, {
host: urlObj.host,
port: 80,
enable: {xforward: true}
});
}).listen(9000, function() {
console.log("Waiting for requests...");
});
writeHead() doesn't necessarily have to be called with an array of headers, write() can also send headers if necessary.
If you want to access headers (or set them), you can use this:
res.writeHead = function() {
// To set:
this.setHeader('your-header', 'your-header-value');
// To read:
console.log('Content-type:', this.getHeader('content-type'));
// Call the original method !!! see text
oldwriteHead.apply(this, arguments);
};
I'm using apply() to pass all the arguments to the old method, because writeHead() can actually have 3 arguments, while your code only assumed there were two.
Related
I am building a cross system admin app, which will be used as an admin tool for multiple backend systems. The app is built on top of Mean.js.
I have setup a /proxy route using "express-http-proxy" to send all sub-routes to their respective backend system endpoints. However, I need to have each request authenticated within my admin app and then decorated with the targeted backendSystem credentials before the "express-http-proxy" can continue. Here's an example of my /proxy route...
app.use('/proxy', users.requiresLogin, expressHttpProxy(config.backendSystem.host, {
forwardPath: function (req) {
return '/1.0' + require('url').parse(req.url).path;
},
decorateRequest: function (req) {
req.headers['content-type'] = 'application/json';
req.headers['backend-system-id'] = config.backendSystem.id;
req.headers['backend-system-key'] = config.backendSystem.key;
return req;
}
}));
NOTE:
Currently the backendSystem credentials are stored based on the environment my admin app is ran in. However, in the future the backendSystem credentials will be specified by the user, and this /proxy route will differently than what is currently shown.
THE ISSUE:
Proxy routes that require data within the request body don't work.
e.g. POST /comments {"user": user_id, "text": "rabble rabble rabble"}
WHAT I'VE FOUND:
bodyParser.json() and "express-https-proxy" don't play nice. I've confirmed this by removing bodyParser.json() from express.js.
However, this isn't a full solution since almost all of my other routes need bodyParser.json, e.g. /auth/signin.
Does anyone have a clean way that I can make a route exception for my /proxy route so that bodyParser.json won't be called for it?
As far as I understand, the root of problem is so:
if you were reading a POST request by pure node, you should be using a code like this
if (req.method == 'POST') {
console.log("POST");
var body = '';
req.on('data', function (data) {
body += data;
console.log("Partial body: " + body);
});
req.on('end', function () {
console.log("Body: " + body);
});
res.writeHead(200, {'Content-Type': 'text/html'});
res.end('post received');
}
in other words, you need to use the req.on('data') & req.on('end') events.
but the problem is,that you can use this code only once. after the 'end' is called, the request is consumed.
so then you use bodyParser , it consumes the request, and the proxy have nothing to do with it.
actually, in my opinion, the proxy wait for the 'data' event to appear , but it will newer happen, so the code halts.
The solution:
you need to 're-enable' the events. I used this code and it works for me.
var express = require('express');
var bodyParser = require('body-parser');
var http = require('http');
//call for proxy package
var devRest = require('dev-rest-proxy');
//init express (as default)
var users = require('./routes/users');
var app = express();
app.use(bodyParser.json());
//set the proxy listening port
app.set('port', 8080);
//process the POST request
app.post('/users/*', function(req, res) {
//just print the body. do some logic with it
console.log("req.body: ",req.body);
//remove listeners set by bodyParser
req.removeAllListeners('data');
req.removeAllListeners('end');
//add new listeners for the proxy to use
process.nextTick(function () {
if(req.body) {
req.emit('data', JSON.stringify(req.body));
}
req.emit('end');
});
//forward the request to another server
devRest.proxy(req,res, 'localhost', 3000);
});
//start the proxy server
http.createServer(app).listen(app.get('port'), function(){
console.log("Express server listening on port " + app.get('port'));
});
module.exports = app;
the solution found on schumacher-m post (github of nodejitsu)
I was able to resolve my issue by adding a regex that excluded my /proxy route to where bodyParser.json was being added within express.js. I found that from this answer
While this approach doesn't scale well, it solved my immediate issue.
I get it works by converting the data into query string using 3rd party query-string as follows:
proxyReqBodyDecorator: function(bodyContent, srcReq) {
return (queryString.stringify(bodyContent));
}
Have tried JSON.stringify but not working, need the data in the following format
array_field=val1&array_field=val2&array_field=val3......
To modify the request body, do this with the latest express-http-proxy v1.6.2:
const express = require('express');
const proxy = require('express-http-proxy');
const bodyParser = require('body-parser');
const conf = {
proxyHost: 'some.example.net:9200',
proxyOptions: {
proxyReqBodyDecorator: modifyRequestBody,
preserveHostHdr: true,
parseReqBody: true
},
port: 8073
};
var app = express();
app.use('/proxy', proxy(conf.proxyHost, conf.proxyOptions));
function modifyRequestBody(body, srcReq) {
if(srcReq.method.match(/^(GET|POST)$/i)) {
try {
// convert buffer to string, then to object
var str = Buffer.from(body).toString('utf-8');
var reqBody = JSON.parse(str);
if(someCondition)) {
reqBody.addStuff = 'whatever';
body = reqBody; // return modified body as object
}
} catch(error) {
console.log('- error: ' + JSON.stringify(error));
}
}
return body; // return original buffer, or modified object
}
app.listen(conf.port, function () {
log('app listening on port ' + conf.port);
});
You can fill the proxyReq.bodyContent inside the decorateRequest method with the JSON-ed data from originalReq.body to be correctly POST'ed:
app.use('/proxy', users.requiresLogin, expressHttpProxy(config.backendSystem.host, {
...
...
decorateRequest: function (proxyReq, originalReq) {
...
...
if (originalReq.body) {
proxyReq.bodyContent = JSON.stringify(originalReq.body);
}
return proxyReq;
}
...
...
}));
I am currently making an application in Node.js, with the Express web server plug-in. I want to count the total data sent by the web server.
To do this, I need to get the 'Content-Length' field of an outgoing HTTP header. However, I need to do this right after the data is added.
If I need to alter core Express scripts, can somebody tell me which file this is contained in?
If you just want to count it, you can use a middleware for that:
var totalBytes = 0;
app.use(function(req, res, next) {
res.on('finish', function() {
totalBytes += Number(res.get('content-length') || 0);
});
next();
});
You have to include very early in the middleware stack, before any other middleware whose contents you want to count.
Also, this doesn't count any streamed data, for which no Content-Length header is set.
You could add middleware to monkey-patch the response methods. It’s ugly but better than altering core Express files.
This calculates total body bytes for standard and streamed responses. Place this before any other app.use() directives.
app.use(function(req, res, next) {
res.totalLength = 0;
var realResWrite = res.write;
res.write = function(chunk, encoding) {
res.totalLength += chunk.length;
realResWrite.call(res, chunk, encoding);
};
var realResEnd = res.end;
res.end = function(data, encoding) {
if (data) { res.totalLength += data.length; }
console.log('*** body bytes sent:', res.totalLength);
realResEnd.call(res, data, encoding);
};
next();
});
If you want to count it on each request then you can try below code!!!
var totalByte=0;
app.all('*', function(req,res, next){
res.on('finish', function() {
totalByte = parseInt(res.getHeader('Content-Length'), 10);
if(isNaN(totalByte))
totalByte = 0;
});
next();
});
Please remember totalByte is a global variable here and it added value on each request
I'm currently using NodeJS/Express as a simple domain router running on my VPS on port 80. My routes.coffee looks something like this:
request = require("request")
module.exports = (app) ->
#404, 503, error
app.get "/404", (req, res, next) ->
res.send "404. Sowway. :("
app.get "/error", (req, res, next) ->
res.send "STOP SENDING ERRORS! It ain't cool, yo."
#Domain Redirects
app.all '/*', (req, res, next) ->
hostname = req.headers.host.split(":")[0]
#Website1.com
if hostname == 'website1.com'
res.status 301
res.redirect 'http://facebook.com/website1'
#Example2.com
else if hostname == 'example2.com'
pathToGo = (req.url).replace('www.','').replace('http://example2.com','')
request('http://localhost:8020'+pathToGo).pipe(res)
#Other
else
res.redirect '/404'
As you can see in Example2.com, I'm attempting to reverse proxy to another node instance on a different port. Overall it works perfectly, except for one issue. If the route on the other node instance changes (Redirects from example2.com/page1.html to example2.com/post5), the URL in the address bar doesn't change. Would anyone happen to have a nice workaround for this? Or maybe a better way to reverse proxy? Thanks!
In order to redirect the client, you should set the http-status-code to 3xx and send a location header.
I'm not familiar with request module but I believe it follows redirects by default.
On the other hand, you're piping the proxy-request's response to client's response object, discarding the headers and the status code. That's why the clients don't get redirected.
Here is a simple reverse HTTP proxy using the built-in HTTP client. It's written in javascript but you can easily translate it to coffeescript and use request module if you want.
var http = require('http');
var url = require('url');
var server = http.createServer(function (req, res) {
parsedUrl = url.parse(req.url);
var headersCopy = {};
// create a copy of request headers
for (attr in req.headers) {
if (!req.headers.hasOwnProperty(attr)) continue;
headersCopy[attr] = req.headers[attr];
}
// set request host header
if (headersCopy.host) headersCopy.host = 'localhost:8020';
var options = {
host: 'localhost:8020',
method: req.method,
path: parsedUrl.path,
headers: headersCopy
};
var clientRequest = http.request(options);
clientRequest.on('response', function (clientResponse) {
res.statusCode = clientResponse.statusCode;
for (header in clientResponse.headers) {
if (!clientResponse.headers.hasOwnProperty(header)) continue;
res.setHeader(header, clientResponse.headers[header]);
}
clientResponse.pipe(res);
});
req.pipe(clientRequest);
});
server.listen(80);
// drop root privileges
server.on('listening', function () {
process.setgid && process.setgid('nobody');
process.setuid && process.setuid('nobody');
});
I'm developing a sails.js (node.js framework based on express) aplication, which is going great but ]I can't solve this detail...
I need to send POST requests cross domain from internet explorer 8 and 9. For that I'm forced to use xDomainRequest object, wich doesn't allow to set a Content type header.
So, when the request gets to the server the content type is "text/plain", which doesn't fire the bodyParser express middleware, so my req.body is an empty object and I can't see the payload I'm sending from the client.
For this I've tried two things with no luck:
First I wanted to set a header to the request in my first middleware, which is of course not available:
req.set("Content-type", "Apli.....
Then, I created a middleware that listens to req.on('data'), like this:
var data = "";
req.on('data', function(chunk){
data += chunk;
});
req.on('end', function(){
req.rawBody = data;
next();
});
But the data event never fires!
Does anyone know how can I access my raw payload, so I can parse it myself.
Thanks!
with newest version of Sails, using express is deprecated.
I needed to use a specific parser to get raw data from Stripe API.
Here is my code, maybe it will help somebody :
bodyParser: function(req, res, next) {
var skipper = require('skipper')();
var rawParser = require("body-parser").raw({type: "*/*"});
// Create and return the middleware function
return function(req, res, next) {
sails.log.debug(req.headers);
if (req.headers && req.headers['stripe-signature']) {
sails.log.info('request using raw parser middleware');
return rawParser(req, res, next);
}
// Otherwise use Skipper to parse the body
sails.log.info('request using skipper middleware');
return skipper(req, res, next);
};
},
I think in this case you're going to have to implement your own body parser, which you can set as sails.config.express.bodyParser or create a config/express.js file like:
var express = require('express');
module.exports.express = {
bodyParser: function(options) {
// Get default body parser from Express
var defaultBodyParser = express.bodyParser(options);
// Get function for consumung raw body, yum.
var getBody = require('raw-body');
return function (req, res, next) {
// If there's no content type, or it's text/plain, parse text
if (!req.headers['content-type'] ||
req.headers['content-type'].match('text/plain')) {
// flag as parsed
req._body = true;
// parse
getBody(req, {
limit: 100000, // something reasonable here
expected: req.headers['content-length']
}, function (err, buf) {
if (err) return next(err);
// Make string from buffer
buf = buf.toString('utf8').trim();
// Set body
req.body = buf.length ? {content: buf} : {}
// Continue
next();
});
}
// Otherwise try the default parsers
else return defaultBodyParser(req, res, next);
};
}
You'll have to npm install express and npm install raw-body. Note that this example uses the default Express body parser as a fallback, not the default Sails body parser which isn't exposed anywhere (and is mostly the same as Express anyway, sans the JSON retry).
maxime's answer is mostly correct. minor modification needed as follows:
bodyParser: function(req, res, next) {
var skipper = require('skipper')();
var rawParser = require("body-parser").raw({type: "*/*"});
// Create and return the middleware function
sails.log.debug(req.headers);
if (req.headers && req.headers['stripe-signature']) {
sails.log.info('request using raw parser middleware');
return rawParser(req, res, next);
}
// Otherwise use Skipper to parse the body
sails.log.info('request using skipper middleware');
return skipper(req, res, next);
},
I'm using node-http-proxy and want to watch for a particular response
header and rewrite it if necessary. Anyone here have suggestions on to
do this?
My proxy server sits in front of a couple different node servers as
well as a java webapp. The java app is setting a cookie, but the
cookie has a path that is relative the the webapp's context. I need
the cookie to be secure and have a path to root without modifying the Java
application.
In other words, the following header is returned:
set-cookie: MYSPECIALCOOKIE=679b6291-d1cc-47be; Path=/app; HttpOnly
And I'd like to rewrite the Path value to:
set-cookie: MYSPECIALCOOKIE=679b6291-d1cc-47be; Path=/; HttpOnly; Secure
I'm not clear how I would do this using node-http-proxy. Suggestions?
Is there middleware to help with this?
You can achieve this by overloading the writeHead function of the response object. For example, this code will set the 'foo' response header to the value 'bar'. I've indicated where you can add your own logic to change the header values.
JavaScript is not my primary language, so there may be a more idiomatic way to overload the writeHead method.
httpProxy = require('http-proxy');
httpProxy.createServer(function (req, res, proxy) {
res.oldWriteHead = res.writeHead;
res.writeHead = function(statusCode, headers) {
/* add logic to change headers here */
var contentType = res.getHeader('content-type');
res.setHeader('content-type', 'text/plain');
// old way: might not work now
// as headers param is not always provided
// https://github.com/nodejitsu/node-http-proxy/pull/260/files
// headers['foo'] = 'bar';
res.oldWriteHead(statusCode, headers);
}
proxy.proxyRequest(req, res, {
host: 'localhost',
port: 3000
});
}).listen(8000);
Just listen to proxyRes event and put your logic.
proxy.on('proxyRes', (proxyRes, req, res) => {
// modifying headers goes here
});
See https://www.npmjs.com/package/http-proxy#listening-for-proxy-events
I didn't test this code, but it should allow you to edit your header before sending the request. Let me know if it works.
var httpProxy = require('http-proxy');
var server = httpProxy.createServer(function (req, res, proxy) {
var buffer = httpProxy.buffer(req);
req.headers['x-host'] = process.env.PROXY_URI;
proxy.proxyRequest(req, res, {
host: '127.0.0.1',
port: 9000,
});
});