I am proxying my api via following setup in my express config
// Proxy api calls
app.use('/api', function (req, res) {
let url = config.API_HOST + req.url
req.pipe(request(url)).pipe(res)
})
config.API_HOST in here resolves to my api url and req.url is some endpoint i.e. /users I tried following documentation on npm for request and set up my headers like so
// Proxy api calls
app.use('/api', function (req, res) {
let options = {
url: config.API_HOST + req.url,
options: { 'mycustomheader': 'test' }
}
req.pipe(request(options)).pipe(res)
})
But I am not able to see my custom headers in chrome dev tools under Network.
Was able to achieve it this way
app.use('/api', function (req, res) {
let url = config.API_HOST + req.ur
req.headers['someHeader'] = 'someValue'
req.pipe(request(url)).pipe(res)
})
Related
I have the following code snippet made with express js
import serverless from 'serverless-http'
import express from 'express';
const app = express();
app.use(express.urlencoded({ extended: true }));
app.use(express.json());
app.get('/api/info', (req, res) => {
res.send({ application: 'sample-app', version: '1.0' });
});
app.get('/jump', (req, res) => {
res.redirect('/api/info');
});
app.get('/explicit-jump', (req, res) => {
res.redirect('/dev/api/info');
});
app.post('/api/v1/getback', (req, res) => {
res.send({ ...req.body });
});
export default serverless(app)
If I deploy that code with serverless probably I will get an endpoint like https://my-api.some-region.amazonaws.com/dev/
Now if I try to reach the endpoint that redirect without the '/dev' path (/jump), I will get forbidden because is trying to reach https://my-api.some-region.amazonaws.com/api/info.
The one that set the path explicitly (/explicit-jump) works fine.
Fixing this single case is easy but I'm in the context of using an external app boilerplate (shopify express app) that has an incredible amount of redirects, really a high number.
I tried using a middleware that rewrites the urls when redirects:
app.use((req, res, next) => {
const redirector = res.redirect
res.redirect = function (url) {
console.log('CHANGING: ' + url);
url = url.replace('/api', '/dev/api')
console.log('TO: ' + url);
redirector.call(this, url)
}
next()
})
But I have the feeling that this is a brute force idea and actually it worked only in some occasions, somehow there are still redirects that goes to the base url ignoring the '/dev' path.
Is there a way I could fix this in a reasonable way so all redirects use the host in where the function is running?
For several reasons, I have a server that has to forward requests to another server. The response should be the response of the final server. I also need to add an extra header onto the request but remove this header again from the response before returning. As such, redirect isn't going to cut it.
I'm currently doing it manually copying the headers & body as required but I would like to know if there's a simple generic way to do it?
A proxy would work for this. Assuming #koa/router or something simliar and the http-proxy module (there are also wrapper modules for Koa that may work:
const proxy = httpProxy.createProxyServer({
target: 'https://some-other-server.com',
// other options, see https://www.npmjs.com/package/http-proxy
})
proxy.on('proxyReq', (proxyReq, req, res, options) => {
proxyReq.setHeader('x-foo', 'bar')
})
proxy.on('proxyRes', (proxyRes, req, res) => {
proxyRes.removeHeader('x-foo')
})
router.get('/foo', async (ctx) => {
// ctx.req and ctx.res are the Node req and res, not Koa objects
proxy.web(ctx.req, ctx.res, {
// other options, see docs
})
})
You could also lift the proxy out of a route if you happen to be starting your Koa server with http.createServer rather than app.listen:
// where app = new Koa()
const handler = app.callback()
http.createServer((req, res) => {
if (req.url === '/foo') {
return proxy.web(req, res, options)
}
return handler(req, res)
})
To avoid same-domain AJAX issues, I want my node.js web server to forward all requests from URL /api/BLABLA to another server, for example other_domain.com:3000/BLABLA, and return to user the same thing that this remote server returned, transparently.
All other URLs (beside /api/*) are to be served directly, no proxying.
How do I achieve this with node.js + express.js? Can you give a simple code example?
(both the web server and the remote 3000 server are under my control, both running node.js with express.js)
So far I found this https://github.com/http-party/node-http-proxy , but reading the documentation there didn't make me any wiser. I ended up with
var proxy = new httpProxy.RoutingProxy();
app.all("/api/*", function(req, res) {
console.log("old request url " + req.url)
req.url = '/' + req.url.split('/').slice(2).join('/'); // remove the '/api' part
console.log("new request url " + req.url)
proxy.proxyRequest(req, res, {
host: "other_domain.com",
port: 3000
});
});
but nothing is returned to the original web server (or to the end user), so no luck.
request has been deprecated as of February 2020, I'll leave the answer below for historical reasons, but please consider moving to an alternative listed in this issue.
Archive
I did something similar but I used request instead:
var request = require('request');
app.get('/', function(req,res) {
//modify the url in any way you want
var newurl = 'http://google.com/';
request(newurl).pipe(res);
});
I found a shorter and very straightforward solution which works seamlessly, and with authentication as well, using express-http-proxy:
const url = require('url');
const proxy = require('express-http-proxy');
// New hostname+path as specified by question:
const apiProxy = proxy('other_domain.com:3000/BLABLA', {
proxyReqPathResolver: req => url.parse(req.baseUrl).path
});
And then simply:
app.use('/api/*', apiProxy);
Note: as mentioned by #MaxPRafferty, use req.originalUrl in place of baseUrl to preserve the querystring:
forwardPath: req => url.parse(req.baseUrl).path
Update: As mentioned by Andrew (thank you!), there's a ready-made solution using the same principle:
npm i --save http-proxy-middleware
And then:
const proxy = require('http-proxy-middleware')
var apiProxy = proxy('/api', {target: 'http://www.example.org/api'});
app.use(apiProxy)
Documentation: http-proxy-middleware on Github
You want to use http.request to create a similar request to the remote API and return its response.
Something like this:
const http = require('http');
// or use import http from 'http';
/* your app config here */
app.post('/api/BLABLA', (oreq, ores) => {
const options = {
// host to forward to
host: 'www.google.com',
// port to forward to
port: 80,
// path to forward to
path: '/api/BLABLA',
// request method
method: 'POST',
// headers to send
headers: oreq.headers,
};
const creq = http
.request(options, pres => {
// set encoding
pres.setEncoding('utf8');
// set http status code based on proxied response
ores.writeHead(pres.statusCode);
// wait for data
pres.on('data', chunk => {
ores.write(chunk);
});
pres.on('close', () => {
// closed, let's end client request as well
ores.end();
});
pres.on('end', () => {
// finished, let's finish client request as well
ores.end();
});
})
.on('error', e => {
// we got an error
console.log(e.message);
try {
// attempt to set error message and http status
ores.writeHead(500);
ores.write(e.message);
} catch (e) {
// ignore
}
ores.end();
});
creq.end();
});
Notice: I haven't really tried the above, so it might contain parse errors hopefully this will give you a hint as to how to get it to work.
To extend trigoman's answer (full credits to him) to work with POST (could also make work with PUT etc):
app.use('/api', function(req, res) {
var url = 'YOUR_API_BASE_URL'+ req.url;
var r = null;
if(req.method === 'POST') {
r = request.post({uri: url, json: req.body});
} else {
r = request(url);
}
req.pipe(r).pipe(res);
});
I used the following setup to direct everything on /rest to my backend server (on port 8080), and all other requests to the frontend server (a webpack server on port 3001). It supports all HTTP-methods, doesn't lose any request meta-info and supports websockets (which I need for hot reloading)
var express = require('express');
var app = express();
var httpProxy = require('http-proxy');
var apiProxy = httpProxy.createProxyServer();
var backend = 'http://localhost:8080',
frontend = 'http://localhost:3001';
app.all("/rest/*", function(req, res) {
apiProxy.web(req, res, {target: backend});
});
app.all("/*", function(req, res) {
apiProxy.web(req, res, {target: frontend});
});
var server = require('http').createServer(app);
server.on('upgrade', function (req, socket, head) {
apiProxy.ws(req, socket, head, {target: frontend});
});
server.listen(3000);
First install express and http-proxy-middleware
npm install express http-proxy-middleware --save
Then in your server.js
const express = require('express');
const proxy = require('http-proxy-middleware');
const app = express();
app.use(express.static('client'));
// Add middleware for http proxying
const apiProxy = proxy('/api', { target: 'http://localhost:8080' });
app.use('/api', apiProxy);
// Render your site
const renderIndex = (req, res) => {
res.sendFile(path.resolve(__dirname, 'client/index.html'));
}
app.get('/*', renderIndex);
app.listen(3000, () => {
console.log('Listening on: http://localhost:3000');
});
In this example we serve the site on port 3000, but when a request end with /api we redirect it to localhost:8080.
http://localhost:3000/api/login redirect to http://localhost:8080/api/login
Ok, here's a ready-to-copy-paste answer using the require('request') npm module and an environment variable *instead of an hardcoded proxy):
coffeescript
app.use (req, res, next) ->
r = false
method = req.method.toLowerCase().replace(/delete/, 'del')
switch method
when 'get', 'post', 'del', 'put'
r = request[method](
uri: process.env.PROXY_URL + req.url
json: req.body)
else
return res.send('invalid method')
req.pipe(r).pipe res
javascript:
app.use(function(req, res, next) {
var method, r;
method = req.method.toLowerCase().replace(/delete/,"del");
switch (method) {
case "get":
case "post":
case "del":
case "put":
r = request[method]({
uri: process.env.PROXY_URL + req.url,
json: req.body
});
break;
default:
return res.send("invalid method");
}
return req.pipe(r).pipe(res);
});
I found a shorter solution that does exactly what I want https://github.com/http-party/node-http-proxy
After installing http-proxy
npm install http-proxy --save
Use it like below in your server/index/app.js
var proxyServer = require('http-route-proxy');
app.use('/api/BLABLA/', proxyServer.connect({
to: 'other_domain.com:3000/BLABLA',
https: true,
route: ['/']
}));
I really have spent days looking everywhere to avoid this issue, tried plenty of solutions and none of them worked but this one.
Hope it is going to help someone else too :)
I don't have have an express sample, but one with plain http-proxy package. A very strip down version of the proxy I used for my blog.
In short, all nodejs http proxy packages work at the http protocol level, not tcp(socket) level. This is also true for express and all express middleware. None of them can do transparent proxy, nor NAT, which means keeping incoming traffic source IP in the packet sent to backend web server.
However, web server can pickup original IP from http x-forwarded headers and add it into the log.
The xfwd: true in proxyOption enable x-forward header feature for http-proxy.
const url = require('url');
const proxy = require('http-proxy');
proxyConfig = {
httpPort: 8888,
proxyOptions: {
target: {
host: 'example.com',
port: 80
},
xfwd: true // <--- This is what you are looking for.
}
};
function startProxy() {
proxy
.createServer(proxyConfig.proxyOptions)
.listen(proxyConfig.httpPort, '0.0.0.0');
}
startProxy();
Reference for X-Forwarded Header: https://en.wikipedia.org/wiki/X-Forwarded-For
Full version of my proxy: https://github.com/J-Siu/ghost-https-nodejs-proxy
I think you should use cors npm
const app = express();
const cors = require('cors');
var corsOptions = {
origin: 'http://localhost:3000',
optionsSuccessStatus: 200 // some legacy browsers (IE11, various SmartTVs) choke on 204
}
app.use(cors(corsOptions));
https://www.npmjs.com/package/cors
I have the following code. And a web api which returns string array,
const express = require('express');
const proxy = require('express-http-proxy');
var app = express();
app.use('/proxy', proxy('http://localhost:56660/api/values'));
app.listen(3000);
When I tried to do localhost:3000/proxy I do not get a response,
But when I use app.use('/proxy', proxy('www.google.com')); , it redirects to google web site.
Please suggest me a best approach/solution:
I want to create a proxy server which gets url from browser (Application), modify the url, call the new url and send the response back to browser(Application).
You can get the URL to be proxied as a query parameter, modify it and then pass that URL to proxy, like this (use instead of app.use('/proxy', proxy('http://localhost:56660/api/values'));):
app.get('/proxy', (req, res, next) => {
const modifiedURL = modifyURL(req.query.url)
return proxy(modifiedURL)(req, res, next)
})
You can call you server with an URL like this (GET method):
https://my.server.com/proxy?url=https://urltobeproxied.com
UPDATE:
I think this would work according to your needs:
app.use('/proxy', (req, res, next) => {
const requestedUrl = `${req.protocol}://${req.get('Host')}${req.url}`
const modifiedURL = modifyURL(requestedUrl)
proxy(modifiedURL)(req, res, next)
})
UPDATE2:
app.use('/proxy', proxy('http://localhost:56660/api/values', {
proxyReqPathResolver: function(req) {
const requestedUrl = `${req.protocol}://${req.get('Host')}${req.url}`
const modifiedURL = modifyURL(requestedUrl)
return require('url').parse(modifiedURL).path;
}
}))
UPDATE3:
An example of proxy modifying the response (extracted from the package docs);
app.use('/proxy', proxy('http://localhost:56660/api/values', {
userResDecorator: function(proxyRes, proxyResData, userReq, userRes) {
data = JSON.parse(proxyResData.toString('utf8'));
data.newProperty = 'exciting data';
return JSON.stringify(data);
}
}))
For routing, I'd like my middleware to pass the request the routes defined in a /html folder to server HTML(ejs), and if header Content-Type is application/json, use the routes defined in the /api folder.
But I don't want to have to define that in every route.
So I'm not looking for middleware that defines some req.api property that I can check on in every route
app.get('/', function(req, res) {
if(req.api_call) {
// serve api
} else {
// serve html
}
});
But I'd like something like this:
// HTML folder
app.get('/', function(req, res) {
res.send('hi');
});
// API folder
app.get('/', function(req, res) {
res.json({message: 'hi'});
});
Is this possible and if so, how can I do this?
I'd like it to work something like this:
app.use(checkApiCall, apiRouter);
app.use(checkHTMLCall, htmlRouter);
You can insert as the first middleware in the Express chain, a middleware handler that checks the request type and then modifies the req.url into a pseudo URL by adding a prefix path to it. This modification will then force that request to go to only a specific router (a router set up to handle that specific URL prefix). I've verified this works in Express with the following code:
var express = require('express');
var app = express();
app.listen(80);
var routerAPI = express.Router();
var routerHTML = express.Router();
app.use(function(req, res, next) {
// check for some condition related to incoming request type and
// decide how to modify the URL into a pseudo-URL that your routers
// will handle
if (checkAPICall(req)) {
req.url = "/api" + req.url;
} else if (checkHTMLCall(req)) {
req.url = "/html" + req.url;
}
next();
});
app.use("/api", routerAPI);
app.use("/html", routerHTML);
// this router gets hit if checkAPICall() added `/api` to the front
// of the path
routerAPI.get("/", function(req, res) {
res.json({status: "ok"});
});
// this router gets hit if checkHTMLCall() added `/api` to the front
// of the path
routerHTML.get("/", function(req, res) {
res.end("status ok");
});
Note: I did not fill in the code for checkAPICall() or checkHTMLCall() because you were not completely specific about how you wanted those to work. I mocked them up in my own test server to see that the concept works. I assume you can provide the appropriate code for those functions or substitute your own if statement.
Prior Answer
I just verified that you can change req.url in Express middleware so if you have some middleware that modifies the req.url, it will then affect the routing of that request.
// middleware that modifies req.url into a pseudo-URL based on
// the incoming request type so express routing for the pseudo-URLs
// can be used to distinguish requests made to the same path
// but with a different request type
app.use(function(req, res, next) {
// check for some condition related to incoming request type and
// decide how to modify the URL into a pseudo-URL that your routers
// will handle
if (checkAPICall(req)) {
req.url = "/api" + req.url;
} else if (checkHTMLCall(req)) {
req.url = "/html" + req.url;
}
next();
});
// this will get requests sent to "/" with our request type that checkAPICall() looks for
app.get("/api/", function(req, res) {
res.json({status: "ok"});
});
// this will get requests sent to "/" with our request type that checkHTMLCall() looks for
app.get("/html/", function(req, res) {
res.json({status: "ok"});
});
Older Answer
I was able to successfully put a request callback in front of express like this and see that it was succesfully modifying the incoming URL to then affect express routing like this:
var express = require('express');
var app = express();
var http = require('http');
var server = http.createServer(function(req, res) {
// test modifying the URL before Express sees it
// this could be extended to examine the request type and modify the URL accordingly
req.url = "/api" + req.url;
return app.apply(this, arguments);
});
server.listen(80);
app.get("/api/", function(req, res) {
res.json({status: "ok"});
});
app.get("/html/", function(req, res) {
res.end("status ok");
});
This example (which I tested) just hardwires adding "/api" onto the front of the URL, but you could check the incoming request type yourself and then make the URL modification as appropriate. I have not yet explored whether this could be done entirely within Express.
In this example, when I requested "/", I was given the JSON.
To throw my hat in the ring, I wanted easily readable routes without having .json suffixes everywhere.
router.get("/foo", HTML_ACCEPTED, (req, res) => res.send("<html><h1>baz</h1><p>qux</p></html>"))
router.get("/foo", JSON_ACCEPTED, (req, res) => res.json({foo: "bar"}))
Here's how those middlewares work.
function HTML_ACCEPTED (req, res, next) { return req.accepts("html") ? next() : next("route") }
function JSON_ACCEPTED (req, res, next) { return req.accepts("json") ? next() : next("route") }
Personally I think this is quite readable (and therefore maintainable).
$ curl localhost:5000/foo --header "Accept: text/html"
<html><h1>baz</h1><p>qux</p></html>
$ curl localhost:5000/foo --header "Accept: application/json"
{"foo":"bar"}
Notes:
I recommend putting the HTML routes before the JSON routes because some browsers will accept HTML or JSON, so they'll get whichever route is listed first. I'd expect API users to be capable of understanding and setting the Accept header, but I wouldn't expect that of browser users, so browsers get preference.
The last paragraph in ExpressJS Guide talks about next('route'). In short, next() skips to the next middleware in the same route while next('route') bails out of this route and tries the next one.
Here's the reference on req.accepts.