I'm running a NodeJS application and am visiting it at localhost/?product=test. I want to retreive 'test' (or any URL-related information for that matter).
Google shows me 2 options, but I can't get either to work:
Use HttpRequest
const http = require('http');
const url = require('url');
http.createServer(function (req, res) {
const queryObject = url.parse(req.url,true).query;
console.log(queryObject);
res.writeHead(200, {'Content-Type': 'text/html'});
res.end('Feel free to add query parameters to the end of the url');
}).listen(3000);
I'm trying to use this code, or variations on it, but the issue is that I cannot create a server to listen to because the address is already in use. I'm not too familiar with NodeJS, but what I don't understand is how to use a HttpRequest properly to listen to the running application and get its URL.
Use Express
Using express doesn't give an error, but all code snippets that -should- work return undefined for me when retrieving the URL.
What is a simple way to retrieve the URL the user used to visit my NodeJS app, so I can retrieve the parameters?
If you want to access localhost without filling in a port like here localhost/?product=test then you should listen to port 80 and not 3000. Otherwise your request should look like this: localhost:3000/?product=test
You should also create a route method for products. In express it will look like this:
const express = require('express')
const app = express()
const port = 80
app.get('/', (req, res) => res.send('home'))
app.get('/product/:productId', function (req, res) {
res.send(req.params)
})
app.listen(port, () => console.log(`Listening on port ${port}!`))
If you run this example your request url should be http://localhost/product/test and response will look like
{
"productId": "test"
}
I have a simple node js server like this:
var express = require('express');
var app = express();
var httpProxy = require('http-proxy');
var apiProxy = httpProxy.createProxyServer();
var serverOne = 'http://<address>:<port>/sap/opu/odata/sap/Z_ATTENDANCE_SRV/';
app.use(express.static('webapp'));
app.use(express.static('./'));
app.all("/attendance/*", function(req, res) {
console.log('redirecting to Server1: ' + serverOne);
apiProxy.web(req, res, {target: serverOne});
});
app.listen(3000);
So address localhost:3000/attendance should redirect me to http://<address>:<port>/sap/opu/odata/sap/Z_ATTENDANCE_SRV/ but it is not, I am getting 404.
I was able to make it work when I set the proxy path as "/*" instead of "/attendance/*", but when I wanted to access entity set "AttendanceSet" via localhost:3000/AttendanceSet it also gave me 404. Do I need to create proxy for all my paths? Shouldn't the /* do that?
When I check initialisation of oDataModel in SAPUI5 app I can see such a request (In this case I have set "/*" for the proxy):
Request URL: http://localhost:3000/$metadata?sap-language=EN
Request Method: GET
Status Code: 200 OK (from disk cache)
Remote Address: [::1]:3000
Referrer Policy: no-referrer-when-downgrade
By this logic, I should be able to access entity set AttendanceSet, but I guess I am missing something.
Thanks.
For urls like localhost:3000/AttendanceSet or localhost:3000/Attendance you have to remove the / you have in you endpoint like this.
app.all("/attendance*", function(req, res) {
console.log('redirecting to Server1: ' + serverOne);
apiProxy.web(req, res, {target: serverOne});
});
Looks like problem was with cache. I deleted browser cache and also, when running node js server I used addition -c-1 to do no caching.
To avoid same-domain AJAX issues, I want my node.js web server to forward all requests from URL /api/BLABLA to another server, for example other_domain.com:3000/BLABLA, and return to user the same thing that this remote server returned, transparently.
All other URLs (beside /api/*) are to be served directly, no proxying.
How do I achieve this with node.js + express.js? Can you give a simple code example?
(both the web server and the remote 3000 server are under my control, both running node.js with express.js)
So far I found this https://github.com/http-party/node-http-proxy , but reading the documentation there didn't make me any wiser. I ended up with
var proxy = new httpProxy.RoutingProxy();
app.all("/api/*", function(req, res) {
console.log("old request url " + req.url)
req.url = '/' + req.url.split('/').slice(2).join('/'); // remove the '/api' part
console.log("new request url " + req.url)
proxy.proxyRequest(req, res, {
host: "other_domain.com",
port: 3000
});
});
but nothing is returned to the original web server (or to the end user), so no luck.
request has been deprecated as of February 2020, I'll leave the answer below for historical reasons, but please consider moving to an alternative listed in this issue.
Archive
I did something similar but I used request instead:
var request = require('request');
app.get('/', function(req,res) {
//modify the url in any way you want
var newurl = 'http://google.com/';
request(newurl).pipe(res);
});
I found a shorter and very straightforward solution which works seamlessly, and with authentication as well, using express-http-proxy:
const url = require('url');
const proxy = require('express-http-proxy');
// New hostname+path as specified by question:
const apiProxy = proxy('other_domain.com:3000/BLABLA', {
proxyReqPathResolver: req => url.parse(req.baseUrl).path
});
And then simply:
app.use('/api/*', apiProxy);
Note: as mentioned by #MaxPRafferty, use req.originalUrl in place of baseUrl to preserve the querystring:
forwardPath: req => url.parse(req.baseUrl).path
Update: As mentioned by Andrew (thank you!), there's a ready-made solution using the same principle:
npm i --save http-proxy-middleware
And then:
const proxy = require('http-proxy-middleware')
var apiProxy = proxy('/api', {target: 'http://www.example.org/api'});
app.use(apiProxy)
Documentation: http-proxy-middleware on Github
You want to use http.request to create a similar request to the remote API and return its response.
Something like this:
const http = require('http');
// or use import http from 'http';
/* your app config here */
app.post('/api/BLABLA', (oreq, ores) => {
const options = {
// host to forward to
host: 'www.google.com',
// port to forward to
port: 80,
// path to forward to
path: '/api/BLABLA',
// request method
method: 'POST',
// headers to send
headers: oreq.headers,
};
const creq = http
.request(options, pres => {
// set encoding
pres.setEncoding('utf8');
// set http status code based on proxied response
ores.writeHead(pres.statusCode);
// wait for data
pres.on('data', chunk => {
ores.write(chunk);
});
pres.on('close', () => {
// closed, let's end client request as well
ores.end();
});
pres.on('end', () => {
// finished, let's finish client request as well
ores.end();
});
})
.on('error', e => {
// we got an error
console.log(e.message);
try {
// attempt to set error message and http status
ores.writeHead(500);
ores.write(e.message);
} catch (e) {
// ignore
}
ores.end();
});
creq.end();
});
Notice: I haven't really tried the above, so it might contain parse errors hopefully this will give you a hint as to how to get it to work.
To extend trigoman's answer (full credits to him) to work with POST (could also make work with PUT etc):
app.use('/api', function(req, res) {
var url = 'YOUR_API_BASE_URL'+ req.url;
var r = null;
if(req.method === 'POST') {
r = request.post({uri: url, json: req.body});
} else {
r = request(url);
}
req.pipe(r).pipe(res);
});
I used the following setup to direct everything on /rest to my backend server (on port 8080), and all other requests to the frontend server (a webpack server on port 3001). It supports all HTTP-methods, doesn't lose any request meta-info and supports websockets (which I need for hot reloading)
var express = require('express');
var app = express();
var httpProxy = require('http-proxy');
var apiProxy = httpProxy.createProxyServer();
var backend = 'http://localhost:8080',
frontend = 'http://localhost:3001';
app.all("/rest/*", function(req, res) {
apiProxy.web(req, res, {target: backend});
});
app.all("/*", function(req, res) {
apiProxy.web(req, res, {target: frontend});
});
var server = require('http').createServer(app);
server.on('upgrade', function (req, socket, head) {
apiProxy.ws(req, socket, head, {target: frontend});
});
server.listen(3000);
First install express and http-proxy-middleware
npm install express http-proxy-middleware --save
Then in your server.js
const express = require('express');
const proxy = require('http-proxy-middleware');
const app = express();
app.use(express.static('client'));
// Add middleware for http proxying
const apiProxy = proxy('/api', { target: 'http://localhost:8080' });
app.use('/api', apiProxy);
// Render your site
const renderIndex = (req, res) => {
res.sendFile(path.resolve(__dirname, 'client/index.html'));
}
app.get('/*', renderIndex);
app.listen(3000, () => {
console.log('Listening on: http://localhost:3000');
});
In this example we serve the site on port 3000, but when a request end with /api we redirect it to localhost:8080.
http://localhost:3000/api/login redirect to http://localhost:8080/api/login
Ok, here's a ready-to-copy-paste answer using the require('request') npm module and an environment variable *instead of an hardcoded proxy):
coffeescript
app.use (req, res, next) ->
r = false
method = req.method.toLowerCase().replace(/delete/, 'del')
switch method
when 'get', 'post', 'del', 'put'
r = request[method](
uri: process.env.PROXY_URL + req.url
json: req.body)
else
return res.send('invalid method')
req.pipe(r).pipe res
javascript:
app.use(function(req, res, next) {
var method, r;
method = req.method.toLowerCase().replace(/delete/,"del");
switch (method) {
case "get":
case "post":
case "del":
case "put":
r = request[method]({
uri: process.env.PROXY_URL + req.url,
json: req.body
});
break;
default:
return res.send("invalid method");
}
return req.pipe(r).pipe(res);
});
I found a shorter solution that does exactly what I want https://github.com/http-party/node-http-proxy
After installing http-proxy
npm install http-proxy --save
Use it like below in your server/index/app.js
var proxyServer = require('http-route-proxy');
app.use('/api/BLABLA/', proxyServer.connect({
to: 'other_domain.com:3000/BLABLA',
https: true,
route: ['/']
}));
I really have spent days looking everywhere to avoid this issue, tried plenty of solutions and none of them worked but this one.
Hope it is going to help someone else too :)
I don't have have an express sample, but one with plain http-proxy package. A very strip down version of the proxy I used for my blog.
In short, all nodejs http proxy packages work at the http protocol level, not tcp(socket) level. This is also true for express and all express middleware. None of them can do transparent proxy, nor NAT, which means keeping incoming traffic source IP in the packet sent to backend web server.
However, web server can pickup original IP from http x-forwarded headers and add it into the log.
The xfwd: true in proxyOption enable x-forward header feature for http-proxy.
const url = require('url');
const proxy = require('http-proxy');
proxyConfig = {
httpPort: 8888,
proxyOptions: {
target: {
host: 'example.com',
port: 80
},
xfwd: true // <--- This is what you are looking for.
}
};
function startProxy() {
proxy
.createServer(proxyConfig.proxyOptions)
.listen(proxyConfig.httpPort, '0.0.0.0');
}
startProxy();
Reference for X-Forwarded Header: https://en.wikipedia.org/wiki/X-Forwarded-For
Full version of my proxy: https://github.com/J-Siu/ghost-https-nodejs-proxy
I think you should use cors npm
const app = express();
const cors = require('cors');
var corsOptions = {
origin: 'http://localhost:3000',
optionsSuccessStatus: 200 // some legacy browsers (IE11, various SmartTVs) choke on 204
}
app.use(cors(corsOptions));
https://www.npmjs.com/package/cors
In my scenario I need forward get request to another end point. In my machine there are two servers php and node.js server. Node.js is like a "man in the middle", PHP server must work in the same way.
Node.js server code
var express = require('express');
var fs = require('fs');
var path = require('path');
var http = require('http');
var https = require('https');
var app = express();
var HTTP_PORT = 3000;
// Create an HTTP service
http.createServer(app).listen(HTTP_PORT,function() {
console.log('Listening HTTP on port ' + HTTP_PORT);
});
//endpoint for tracking
app.get('/track', function(req, res) {
sendRequestToOtherEndPoint(req);
processRequest(req);
res.setHeader('Content-Type', 'application/json');
res.send('Req OK');
});
function processRequest(req){
console.log("request processed");
}
function sendRequestToOtherEndPoint(req){
//magic here :)
}
When this server receive a get request in port 3000, it process request information and it must forward the same requesto to another end point.
For example:
Get localhost:3000/track?param1=1¶m2=2
Server process get request
Server forward get request to localhost/final-endpoint?param1=1¶m2=2
Depending on what you're trying to do, you can create a new request to the end-point:
//endpoint for tracking
app.get('/track', function(req, res) {
req.get({url: 'http://end-point', headers: req.headers});
processRequest(req);
res.setHeader('Content-Type', 'application/json');
res.send('Req OK');
});
More info: https://github.com/request/request
There are a couple of useful libraries that one could use:
http-proxy-middleware:
const proxy = require('http-proxy-middleware')
var apiProxy = proxy('/track', {target: 'http://end-point'});
app.use(apiProxy)
axios-express-proxy
import express from 'express';
import { Proxy } from 'axios-express-proxy';
const app = express();
const port = 3000;
app.get('/track', (req, res) => Proxy('http://end-point', req, res));
In you case res.redirect might help.
app.get('/track', function(req, res) {
// process the request
// then redirect
res.redirect('/final-endpoint');
});
Then catch the redirected request in final endpont.
app.get('/final-endpoint', function(req, res) {
// proceess redirected request here.
});
See the Express docs
If your second endpoint is on a different server, (e.g. PHP) then you're going to need to either redirect the client (as in sohel's answer), or spoof a request from Node to the PHP server and then send the response back to the client. This latter option is definitely non-trivial so I would question whether it's crucial not to use a client redirect.
If you're talking about two express endpoints, then I think the simplest answer might be not to actually forward at all, but just use the endpoint callback directly instead:
app.get('/track', trackCallback);
app.get('/otherendpoint', otherendpointCallback);
function otherendpointCallback(req, res) {
// do your thing
}
function trackCallback(req, res) {
otherendpointCallback(req, res);
processRequest(req);
res.setHeader('Content-Type', 'application/json');
res.send('Req OK');
};
Depending on exactly what you want to do at the other end point, you might need to spoof some of req's fields (e.g. req.url)
I'm currently using NodeJS/Express as a simple domain router running on my VPS on port 80. My routes.coffee looks something like this:
request = require("request")
module.exports = (app) ->
#404, 503, error
app.get "/404", (req, res, next) ->
res.send "404. Sowway. :("
app.get "/error", (req, res, next) ->
res.send "STOP SENDING ERRORS! It ain't cool, yo."
#Domain Redirects
app.all '/*', (req, res, next) ->
hostname = req.headers.host.split(":")[0]
#Website1.com
if hostname == 'website1.com'
res.status 301
res.redirect 'http://facebook.com/website1'
#Example2.com
else if hostname == 'example2.com'
pathToGo = (req.url).replace('www.','').replace('http://example2.com','')
request('http://localhost:8020'+pathToGo).pipe(res)
#Other
else
res.redirect '/404'
As you can see in Example2.com, I'm attempting to reverse proxy to another node instance on a different port. Overall it works perfectly, except for one issue. If the route on the other node instance changes (Redirects from example2.com/page1.html to example2.com/post5), the URL in the address bar doesn't change. Would anyone happen to have a nice workaround for this? Or maybe a better way to reverse proxy? Thanks!
In order to redirect the client, you should set the http-status-code to 3xx and send a location header.
I'm not familiar with request module but I believe it follows redirects by default.
On the other hand, you're piping the proxy-request's response to client's response object, discarding the headers and the status code. That's why the clients don't get redirected.
Here is a simple reverse HTTP proxy using the built-in HTTP client. It's written in javascript but you can easily translate it to coffeescript and use request module if you want.
var http = require('http');
var url = require('url');
var server = http.createServer(function (req, res) {
parsedUrl = url.parse(req.url);
var headersCopy = {};
// create a copy of request headers
for (attr in req.headers) {
if (!req.headers.hasOwnProperty(attr)) continue;
headersCopy[attr] = req.headers[attr];
}
// set request host header
if (headersCopy.host) headersCopy.host = 'localhost:8020';
var options = {
host: 'localhost:8020',
method: req.method,
path: parsedUrl.path,
headers: headersCopy
};
var clientRequest = http.request(options);
clientRequest.on('response', function (clientResponse) {
res.statusCode = clientResponse.statusCode;
for (header in clientResponse.headers) {
if (!clientResponse.headers.hasOwnProperty(header)) continue;
res.setHeader(header, clientResponse.headers[header]);
}
clientResponse.pipe(res);
});
req.pipe(clientRequest);
});
server.listen(80);
// drop root privileges
server.on('listening', function () {
process.setgid && process.setgid('nobody');
process.setuid && process.setuid('nobody');
});