Why and how this simple node.js proxy works? - node.js

I have a frontend-only web application on Netlify which has to consume an API on OpenSubtitles.org. Although OpenSubtitles.org enables CORS, sometimes I got preflight errors, so I decided to use a proxy.
I had problems using Netlify's proxy feature, so I decided I will create my own proxy on Heroku, and send my requests from the frontend to there, so these will be proxied to OpenSubtitles.org from a server.
I came up with the following based on the code I found here:
const express = require('express');
const request = require('request');
express()
.use('/', function(req, res) {
req.pipe(
request({
url: 'http://rest.opensubtitles.org/search' + req.url,
headers: {
'User-Agent': 'TemporaryUserAgent'
}
})
).pipe(res);
})
.listen(process.env.PORT || 8000);
I thought I deploy this, try it out, then I will enable CORS on it after that. However I've just realized it is working perfectly without doing anything else. How is it possible? Why can I call this from a frontend-only app on a different domain without explicitly enabling CORS?
Also, what if the server crashes, how to handle the errors there?

CORS is working because the url you're requesting responds with the header Access-Control-Allow-Origin set with a value of *. Since you're piping that response and its headers back to the original res object, it will enable CORS as if it was coming from your local proxy.
Below is a more straightforward example of how to proxy a request to another site and return its response intact using node streams.
const express = require('express')
const request = require('request')
const port = process.env.PORT || 1337
let server = express()
const proxyMiddleware = (req, res, next) => {
let url = `https://www.google.com/${req.url}`
let proxyRequest = request(url)
// Pass request to proxied request url
req.pipe(proxyRequest)
// Respond to the original request with the response from proxyRequest
proxyRequest.pipe(res)
}
server.use(proxyMiddleware)
server.listen(port, () => console.log(`Listening on ${port}`))

Related

Node Express Cors issue

I cant figure why the cors express middleware wont work. cors, express, and ejs are all saved in package.json. The app works fine if I add corsanywhere proxy on the front end but id like to work around this on the server side. any help much appreciated I've been stuck on this.
the api is in the get View/index path
the error is:
Access to fetch at 'https://api.darksky.net/forecast/' from origin 'http://localhost:3000' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
const express = require('express');
const app = express();
const ejs = require('ejs');
const cors = require('cors');
const PORT = process.env.PORT || 3000;
// app.use((req, res, next) => {
// res.header('Access-Control-Allow-Origin', '*')
// res.header('Access-Control-Allow-Headers', 'Origin', 'X-Requested-With')
// next();
// });
app.use(cors());
app.use(express.static(__dirname + '/Public'));
app.set('view engine', 'ejs');
app.get('/', cors(), (req, res) => {
res.render(__dirname + '/Views/index')
});
app.listen(PORT, () => {
console.log(`server is listening on ${PORT}`)
});
client side:
it works with the ${proxy} in there but id like to get rid of that
if(navigator.geolocation){
navigator.geolocation.getCurrentPosition(position => {
long = position.coords.longitude;
lat = position.coords.latitude;
var proxy = 'https://cors-anywhere.herokuapp.com/'
var api = `${proxy}https://api.darksky.net/forecast/042750f3abefefdfe2c9d43cf33ce576/${lat},${long}`;
fetch(api)
.then(response => {
return response.json();
})
.then(data => {
let {temperature, summary, icon,} = data.currently;
temperatureDegree.textContent = Math.floor(temperature);
temperatureDescription.textContent = summary;
locationTimezone.textContent = data.timezone;
setIcons(icon, document.querySelector('.icon'
w
``````
So, if you're trying to access some other service https://api.darksky.net/forecast/ (that you don't control) from your web page, then there is nothing you can do to make CORs work for that. It's up to the api.darksky.net server to decide if CORs is allowed or not. You can't change that.
You could make a request from your web page to your server to ask it to get some data from api.darksky.net for you and then return it back to your webpage (working as a simple proxy). Your server is not subject to any CORs limitations when accessing api.darksky.net. Only browsers are limited by CORs.
And, as you've found, you can also use a proxy service that enables CORs and fetches data for you.
Let's suppose you want to proxy the parts of the darksky API, you could do something simple like this:
const express = require('express');
const app = express();
const request = require('request');
const apiRouter = express.Router();
// maps /api/forecast/whatever to http://api.darksky.net/forecast/developerKey/whatever
// and pipes the response back
const apiKey = "yourAPIKeyHere";
apiRouter.get("/*", (req, res, next) => {
// parse out action and params
// from an incoming URL of /api/forecast/42.3601,-71.0589
// the /api will be the root of the router (so not in the URL here)
// "forecast" will be the action
// "42.3601,-71.0589" will be the params
let parts = req.path.slice(1).split("/"); // split into path segments, skipping leading /
let action = parts[0]; // take first path segment as the action
let params = parts.slice(1).join("/"); // take everything else for params
request({
uri: `https://api.darksky.net/${action}/${apiKey}/${params}`,
method: "get"
}).pipe(res);
});
app.use("/api", apiRouter);
app.listen(80);
Now, when you send this server, this request:
/api/forecast/42.3601,-71.0589
it will request:
https://api.darksky.net/forecast/yourAPIKeyHere/42.3601,-71.0589
and pipe the result back to the caller. I ran this test app and it worked for me. While I didn't see anything other than forecast URLs in the darksky.net API, it would work for anything of the format /api/someAction/someParams.
Note, you probably do NOT want to enable CORS on your server because you don't want other people's web pages to be able to use your proxy. And, since you're just sending requests to your own server now, you don't need CORS to be able to do that.

Allowing CORS in NodeJS

I defined my Express js app:
const express = require('express')
var history = require('connect-history-api-fallback');
const path = require('path')
const http = require('http')
const socketio = require('socket.io')
require('./db/mongoose')
const userRouter = require('./routers/user')
const publicDirPath = path.join(__dirname, '../public')
const app = express()
app.use(express.static(publicDirPath))
app.use(history({
index: '../public/index.html'
}))
app.use(express.static(publicDirPath))
app.use(express.json())
app.use(userRouter)
const server = http.createServer(app)
const io = socketio(server)
io.on('connection', socket => {
console.log('New WebSocket connection')
socket.on('join', () => {
socket.emit('message', 'Welcome to the app')
})
})
module.exports = server
Then I use it my index.js file:
const app = require('./app')
const port = process.env.PORT
app.listen(port, () => {
console.log(`Server is up on port ${port}`)
})
When I run the app and send requests from the same port - everything works just fine. But when I try to send requests from different localhost port, ie. 8080, I'm getting cross origin error.
I tried to install cors package and use it as follows:
const cors = require('cors')
app.options('*', cors())
app.use(cors());
And got the same result.
I tried to pass configuration to cors:
app.use(cors({
origin: 'http://localhost:8080'
}));
And still got the same result.
What am I doing wrong and how can I make it work?
When your frontend app tries to make the request to the express server,
The express server is blocking that request because the source of the request (i.e. frontend server) is unknown to the express server
As the request that you are trying to make it out of the domain of the express server. this is the reason where you have to tell the server please accept the request from this origin a.k.a IP Address
and you can achieve the via cors
Let's take a minute to explain what "origin" is in simple words.
When a user browses to a website, he uses an address.
Normally that address is a domain, but when we run our tests we mostly work with local IPs.
For CORS, this doesn't matter. Once you enable Allow-Origins, the server looks at the address the user used to reach the website, and defines it as the "origin" of the request.
Obviously, on a deeper level, everything comes down to IP addresses. But when it comes to CORS, it's more of a high-level security method. It's helps preventing Cross-Site Request-Forgeries for example.
In conclusion, in order for CORS to work,
make sure you allowed the address which the user is using to access the HTTP service.
So, if you're serving a VUE app that's working on http://localhost:8080, and from it calling to an HTTP service on http://localhost:5555, you'll need to do this on the said HTTP service:
app.use(cors({
origin: 'http://localhost:8080'
}));

NodeJS Request handler

I have some issues with a server that does not support IPv6 requests from Apple Application Review. So they reject my update.
And i'm thinking of making a request handler as a middle server, with nodejs.
So my app will send the requests in my new server, which server will send the request to the old server, take the response json back, and serve it back as well in my app.
So lets say the old webserver request was the following
https://www.example.com/example/api/index.php?action=categories&subaction=getproducts&category_id=100304&limit=0,30
But the request parameters are not always the same!
It may vary but the main URL is always the same
https://www.example.com/example/api/index.php?
The question is how to get the request params dynamically, make a request to the old webserver and return the response to the request of the new webserver?
You just need a very simple proxy like this;
const express = require('express')
const request = require('request')
const app = express()
const BASE_URL = 'http://www.google.com' // change accordingly
app.use('/', (req, res) => {
request({
url: BASE_URL + req.originalUrl
}).pipe(res)
})
app.listen(8900, () => console.log('Listening...'))
req.originalUrl will allow to concatenate the path + the query string to your base url

Mixed content error for http express route

I'm running a local node js server on my machine and I provide it with a ngrok.io domain.
If I run it local in the web browser with localhost:3000 or with http://1234.ngrok.io everything is ok. If I call it with httpS://1234.ngrok.io I get the error
Mixed Content: The page at 'https://1234.ngrok.io/' was loaded over HTTPS, but requested an insecure XMLHttpRequest endpoint 'http://localhost:3000/calculation'. This request has been blocked; the content must be served over HTTPS.
caused because the route is not http. How can I fix this?
This is the app.js on my node.js
const express = require('express');
const app = express();
const https = require('https');
const http = require('http');
const fs = require('fs');
app.use('/', express.static(__dirname + '/../Client'));
var options = {
key: fs.readFileSync('./https_certificate/client-key.pem'),
cert: fs.readFileSync('./https_certificate/client-cert.pem')
};
app.listen(server_connection.port, () => {
console.log('Server started on port ' + server_connection.port);
//do stuff
});
//load the index.html from the server
app.get('/', function(req, res) {
res.sendFile(path.join(__dirname + '/../Client/index.html'));
});
This is the other file with the route who causes the error:
const express = require('express');
const router = express();
router.post('/calculation', function(req, res) {
//do stuff
}
I already tried to configure the server for https how you can see on the options object in the first file but this resolved in a 502 Bad Gateway.
I also would like to use just https. Is it possible to redirect to https if someone uses http?
It seems that you are sending an ajax request using http to the server, make sure that all your ajax calls are over https to get ride from this error.

Cross-domain POST request in Node.JS with preflight?

I have just started with Node.
I am trying to get cross-domain form data from an HTML form to parse in a Node.js server. I have been able to do this with simple POST data, not with POST requests that require preflight.
I am running the Node code on cloud9 app servers. I am also using the Cors module to handle the requests. This module works well with simple requests (test here to see a simple request work), however with requests that require preflight I get this result from the Chrome inspector console.
XMLHttpRequest cannot load https://nms-motaheri-1.c9.io:8080/mail.
The request was redirected to 'https://c9.io:8080/api/nc/auth?.....SHORTENED',
which is disallowed for cross-origin requests that require preflight.
Here is my server.js code:
// Define dependencies
var express = require('express')
, cors = require('cors')
, app = express()
, parse_post = require("parse-post");
// Core module config
var corsOptions = {
origin: '*',
preflightContinue: true // <- I am assuming this is correct
};
app.use(cors(corsOptions));
// Respond to option request with HTTP 200
// ?? Why is this not answering my OPTION requests sufficiently ??
app.options('*',function(req,res){
res.send(200);
});
// Give a hello world response to all GET requests
app.get('/',function(req,res){
res.sendFile(__dirname + '/index.html');
});
// Handle all POST requests to /mail
app.post('/mail', parse_post(function(req, res) {
console.log(req.body);
res.json({msg: 'This is CORS-enabled for all origins!'});
})
);
// Listen on default Cloud9 port which is 8080 in this case
app.listen(process.env.PORT, function(){
console.log('CORS-enabled web server listening on port ' + process.env.PORT);
});
Why is this happening and how can I satisfactorily answer the OPTION request for my POST with pre-flight?
Here is the post request and response in Chrome dev tools:
Turns out that part of the problem was that the cloud9 server was set to private making these requests all redirect.
After making the server public, the redirections stopped. However, I received an error that the Node.js server did not have any Access-Control-Allow-Origin headers to allow requests from my cross origin domain. I noticed that "simple" with-out preflight requests would go through. So instead of trying to understand why it was not accepting my allow-all-origin-configuration on the Node.js side I decided to serialized the POST data to get rid of the preflight requirement and changed the data type in my angular request to plain text.
To get rid of preflight, first get rid of any POST header configuration (cache, etc), make sure your request Content-Type is plain text and make sure your actual content is plain text too. So if it is in JSON serialize it in jQuery before sending it with POST.
This is what my new Angular Post request code looked like:
sendEmail: function(email) {
var config = {
headers: {
'Content-Type': 'text/plain'
}
};
var POSTDATA= JSON.stringify(POSTDATAJSON);
return $http.post(POSTURL, POSTDATA, config)
}
And in Node.js this, I am using the cors Node.js module:
app.post('/mail', parse_post(function(req, res) {
var postReq = JSON.parse(Object.keys(req.body));
}));

Resources