Issue SSL handshake with own https proxy on NodeJS - node.js

I try to work on my own HTTPS Proxy and I can't create my https server, during the connexion initialization, I have this error message "tlsClientError Error: 101057795:error:1407609B:SSL routines:SSL23_GET_CLIENT_HELLO:https proxy request:openssl\ssl\s23_srvr.c:400".
Below the node JS source code:
private runHttpsServer() {
const instance = this;
const cert = fs.readFileSync("./certificate/server.crt", "utf8");
const key = fs.readFileSync("./certificate/key.pem", "utf8");
const options : https.ServerOptions = {
cert: cert,
key: key
};
const httpsServer = https.createServer(options, (req, res)=>{
console.log("Request ...");
instance.handleRequest.call(instance, req, res);
});
httpsServer.on("tlsClientError", (err : Error, tlsSocket : TLSSocket)=>{
console.log("tlsClientError", err.stack);
});
httpsServer.listen(7001);
}
When I browse to "https://localhost:7001/" directly without my proxy, I have no SSL handshake error.
When I browse to other website throughout my proxy, I have this SSL Handshake error.
Someone have already encountered this issue ?
Some can help me ?

It looks like you have the wrong understanding of how proxying HTTPS works. It is not that the client will make a TLS connection to the proxy as you assume but instead the client will send a plain HTTP request with the method CONNECT to the proxy in order to make the proxy establish a tunnel to the target host. Then the client will upgrade this tunnel to TLS and thus get an end-to-end protection between client and final server.
On the wire it will look something like this (the exact message might slightly vary - see RFC 2817 for details):
Client ------------------------> Proxy
- create TCP connection
- send to proxy:
> CONNECT google.com:443 HTTP/1.0\r\n
> \r\n
Proxy -----------------------------> Server
- create TCP connection
Client <------------------------ Proxy
- send to client
< HTTP/1.0 200 Connection established\r\n
< \r\n
Client <------------------------(Proxy)-----------------------------> Server
use tunnel from client to server through proxy to
- make the end-to-end TLS handshake
- transfer the HTTP messages within the TLS connection
... SSL routines:SSL23_GET_CLIENT_HELLO:https proxy request: ...
This error message says essentially that your proxy expected the start of the TLS handshake (ClientHello) but instead got a https proxy request, i.e. CONNECT .... You need to fix your proxy to properly handle https proxy requests which work as I've described.

I also had the same problem. As mr. Steffen Ullrich answered we (me and you) have wrong understanding how proxying HTTPS work. Firstly youd need listen event CONNECTION in http then doing tls handshake. In code it look likes so:
http
.createServer((req, res) => requestHandler(req, res, false))
.on('connect', (req, cltSocket, head) => {
const srvUrl = url.parse(`http://${req.url}`);
const srvSocket = net.connect(
srvUrl.port,
srvUrl.hostname,
() => {
cltSocket.write('HTTP/1.1 200 Connection Established\r\n'
+ 'Proxy-agent: Node.js-Proxy\r\n'
+ '\r\n');
srvSocket.write(head);
srvSocket.pipe(cltSocket).on('error', (e) => console.log('srvSocket', e));
cltSocket.pipe(srvSocket).on('error', (e) => console.log('cltSocket', e));
},
);
})
.listen(9000, () => console.log('HTTP Server started listening on port 9000'));
you can learn about it in official site of Node.js
Full code of my https proxy is available here.

Related

Websockets token authentication using middleware and express in node.js

I use node.js, express and express-ws that is based on ws
Express-ws allows to create express-like endpoints for websockets.
I am looking for a solution to authenticate users in websocket connections, based on a token. Since my ws server is based on an HTTP one
const wsHttpServer = http.createServer();
wsHttpServer.listen(5001);
const expressWs = require('express-ws')(app , wsHttpServer);
and since the ws connection is based on an HTTP one that gets upgraded to a ws, WHY I cannot pass a token in my ws that the express route checks, like any other one? My logic is, send the token, check it, if it is ok, proceed to upgrade to a ws connection. So, I can reuse the token-middleware solution that I have in my HTTP connections.
In node
My ws server
const wsHttpServer = http.createServer();
wsHttpServer.listen(5001);
const expressWs = require('express-ws')(app , wsHttpServer);
//set the route
app.use('/ws', require('./routes/wsroute'));
In that route, I would like to use the token.validate() middleware -that in HTTP connections, checks the Authorization header
router.ws('/user/:name/:id', token.validate(), (ws, req) => {
console.log('ws route data : ',vessel, req.params.name, req.params.id);
});
In my client
const socket = new WebSocket('ws://localhost',{
path: '/user/Nick/25/',
port: 5001, // default is 80
protocol : "echo-protocol", // websocket protocol name (default is none)
protocolVersion: 13, // websocket protocol version, default is 13
keepAlive: 60,
headers:{ some:'header', 'ultimate-question':42 } // websocket headers to be used e.g. for auth (default is none)
});
this errors Failed to construct 'WebSocket': The subprotocol '[object Object]' is invalid
I also tried
const socket = new WebSocket('ws://localhost:5001/user/Nick/25', ["Authorization", localStorage.getItem('quad_token')]);
I dont get any errors, but I dont know how to get the Authorization "header" in node
I could
just send const socket = new WebSocket(currentUrl); with some data and include a valid token in that data. But to check it, I have to allow the connection first. I dont want that, I would like to use a middleware solution that automatically checks a token and allows or not to continue.
Questions
Please help me understand:
1 Is it possible to use a token-based, middleware-based solution in ws?
2 How to set a header with a token in a ws connection?
3 How to get that token in node?
1) In my experience there is no available express.js middleware and the solution i found requires to listen to the upgrade event on your http server and blocking access to your socket connection before it reaches ws routes.
2) Your browser will not allow setting additional headers during websocket connection on the client side. It will send though the cookies so you can make use of express-session to authorize on your server first the user, a cookie will be set on the browser and that cookie will be sent over during the websocket connection.
3) You can do like in this answer Intercept (and potentially deny) web socket upgrade request Copying the code here from there for your own perusal.
**wsHttpServer**.on('upgrade', function (req, socket, head) {
var validationResult = validateCookie(req.headers.cookie);
if (validationResult) {
//...
} else {
socket.write('HTTP/1.1 401 Web Socket Protocol Handshake\r\n' +
'Upgrade: WebSocket\r\n' +
'Connection: Upgrade\r\n' +
'\r\n');
socket.close();
socket.destroy();
return;
}
//...
});
As outlined here, it seems that it is not possible for a standard browser websocket client to handle a http error response to an upgrade request. Thus what I ended up using was something like this:
HTTPserver.on('upgrade' (req, sock, head) => {
if (req.url === wsRoute) {
webSocketServer.handleUpgrade(req, sock, head, ws => {
const authenticated = validateToken(req.headers.cookie) // your authentication method
if (!authenticated) {
ws.close(1008, 'Unauthorized') // 1008: policy violation
return
}
webSocketServer.emit('connection', ws, req)
})
} else {
sock.destroy()
}
}
This way we accept the connection first before closing it with an appropriate code and reason, and the websocket client is able to process this close event as required.
On your client side, you should pass an array of strings instead of object, but you must set a header for your HTTP response with a key and value:
key : headeSec-WebSocket-Protocol
value : corresponding protocol used in front.

Socket.IO, SSL Problems With cloudflare

I'm having a socket.io app that basically receives signals from a frontend in order to kill and start a new ffmpeg process (based on .spawn()).
Everything works like expected, but often I get a 525 error from cloudflare. The error message is: Cloudflare is unable to establish an SSL connection to the origin server.
It works like 9 out of 10 times.I noticed that more of these errors pop up whenever a kill + spawn is done. Could it be the case that something block the event loop and because of this blocks all incoming requests and cloudflare logs these as a handshake failed error?
Contacting cloudflare support gives me back this info (this is the request they do to my server):
Time id host message upstream
2017-08-16T09:14:24.000Z 38f34880faf04433 xxxxxx.com:2096 peer closed connection in SSL handshake while SSL handshaking to upstream https://xxx.xxx.xxx.xxx:2096/socket.io/?EIO=3&transport=polling&t=LtgKens
I'm debugging for some time now, but can't seem to find a solutions myself.
This is how I initialize my socketIO server.
/**
* Start the socket server
*/
var startSocketIO = function() {
var ssl_options = {
key: fs.readFileSync(sslConfig.keyFile, 'utf8'),
cert: fs.readFileSync(sslConfig.certificateFile, 'utf8')
};
self.app = require('https').createServer(ssl_options, express);
self.io = require('socket.io')(self.app);
self.io.set('transports', ['websocket', 'polling']);
self.app.listen(2096, function() {
console.log('Socket.IO Started on port 2096');
});
};
This is the listener code on the server side
this.io.on('connection', function (socket) {
console.log('new connection');
/**
* Connection to the room
*/
socket.on('changeVideo', function (data) {
//Send to start.js and start.js will kill the ffmpeg process and
start a new one
socket.emit('changeVideo');
});
});
Another thing that I observer while debugging (I only got this a few times):
The text new connection displayed on the server and the connected client emits the changevideo event but nothing happens on the server side instead the client just
keeps reconnecting.
This is a simplified version of the nodejs code. If you have more questions, just let me know.
Thanks!

net::ERR_INSECURE_RESPONSE Angular 2 + Node.js + Socket.io

I am developing a web app with Angular 2, Node.js and socket.io. It is working perfectly fine with http, however I wanted to implement https protocol and I hit the wall.
I am getting net::ERR_INSECURE_RESPONSE on any http call from angular to my web api and also when trying to make a socket connection.
Here is what I did:
1) Generated my private key and csr.
2) Requested SSL certificate from Comodo using that csr.
3) Received 2 files from Comodo:
- domain.ca-bundle
- domain.crt
4) I copied those 2 files and my private key to a folder: /etc/apache2/ssl/
5) I modified default-ssl.conf in etc/apache2/sites-available to point to these files:
SSLCertificateFile /etc/apache2/ssl/domain.crt
SSLCertificateKeyFile /etc/apache2/ssl/mypriv.key
SSLCertificateChainFile /etc/apache2/ssl/domain.ca-bundle
plus added:
ServerName domain.io
ServerAlias www.domain.io
6) Enabled https and restarted apache. Verified using SSL analyzer (https://sslanalyzer.comodoca.com/) if my cert is properly set up. It confirmed that connection to my site is now secure, so I assume this part was done properly and now I needed to fix the app and the server to handle https.
7) This is how socket connection is created in angular:
this.socket = io.connect('https://server_ip' + ':3000', {
rejectUnauthorized: false });
This is gonna give me net::ERR_INSECURE_RESPONSE later on
8) Angular app is also making http requests to the server like this:
var headers = new Headers();
headers.append('Content-Type', 'application/json');
let options = new RequestOptions({ headers: headers });
return this.http.post('https://server_ip' + ':3000/register',
JSON.stringify(userModel), options)
.map(res => res.json())
This is gonna give me net::ERR_INSECURE_RESPONSE as well (in response to OPTIONS call)
9) Now on the server side, this is the relevant piece of code:
var https = require('https');
var options = {
ca: splitca('/etc/apache2/ssl/domain.ca-bundle'),
key: fs.readFileSync('/etc/apache2/ssl/mypriv.key'),
cert: fs.readFileSync('/etc/apache2/ssl/domain.crt')
};
var server = https.createServer(options, app);
let io = require('socket.io')(https);
server.listen(3000, () => {
logger.debug('server started on port');
})
Server starts up fine, but any call from angular results in net::ERR_INSECURE_RESPONSE response (tried chrome, mozilla and IE). Both server and angular app are running on the same IP.
Any idea what I might be doing wrong here? Thanks!

Non-http TCP connection on Cloudfoundry

I'm a nooby mobile developer trying to take advantage of cloudfoundry's service to run my server to handle some chats and character movements.
I'm using Noobhub to achieve this (TCP connection between server and client using Node.js and Corona SDK's TCP connection API)
So basically I'm trying a non-http TCP connection between Cloudfoundry(Node.js) and my machine(lua).
Link to Noobhub(There is a github repo with server AND client side implementation.
I am doing
Client
...
socket.connect("myappname.cloudfoundry.com", 45234)
...
(45234 is from server's process.env.VCAP_APP_PORT value I retrieved from console output I got through "vmc logs myappname" after running the application.)
Server
...
server.listen(process.env.VCAP_APP_PORT)
When I try to connect, it just times out.
On my local machine, doing
Client
...
socket.connect("localhost",8989)
Server
...
server.listen(8989)
works as expected. It's just on cloudfoundry that it doesn't work.
I tried a bunch of other ways of doing this such as setting the client's port connection to 80 and a bunch of others. I saw a few resources but none of them solved it.
I usually blow at asking questions so if you need more information, please ask me!
P.S.
Before you throw this link at me with an angry face D:< , here's a question that shows a similar problem that another person posted.
cannot connect to TCP server on CloudFoundry (localhost node.js works fine)
From here, I can see that this guy was trying to do a similar thing I was doing.
Does the selected answer mean that I MUST use host header (i.e. use http protocol) to connect? Does that also mean cloudfoundry will not support a "TRUE" TCP socket much like heroku or app fog?
Actually, process.env.VCAP_APP_PORT environment variable provides you the port, to which your HTTP traffic is redirected by the Cloud Foundry L7 router (nginx) based on the your apps route (e.g. nodejsapp.vcap.me:80 is redirected to the process.env.VCAP_APP_PORT port on the virtual machine), so you definitely should not use it for the TCP connection. This port should be used to listen HTTP traffic. That is why you example do work locally and do not work on Cloud Foundry.
The approach that worked for me is to listen to the port provided by CF with an HTTP server and then attach Websocket server (websocket.io in my example below) to it. I've created sample echo server that works both locally and in the CF. The content of my Node.js file named example.js is
var host = process.env.VCAP_APP_HOST || "localhost";
var port = process.env.VCAP_APP_PORT || 1245;
var webServerApp = require("http").createServer(webServerHandler);
var websocket = require("websocket.io");
var http = webServerApp.listen(port, host);
var webSocketServer = websocket.attach(http);
function webServerHandler (req, res) {
res.writeHead(200);
res.end("Node.js websockets.");
}
console.log("Web server running at " + host + ":" + port);
//Web Socket part
webSocketServer.on("connection", function (socket) {
console.log("Connection established.");
socket.send("Hi from webSocketServer on connect");
socket.on("message", function (message) {
console.log("Message to echo: " + message);
//Echo back
socket.send(message);
});
socket.on("error", function(error){
console.log("Error: " + error);
});
socket.on("close", function () { console.log("Connection closed."); });
});
The dependency lib websocket.io could be installed running npm install websocket.io command in the same directory. Also there is a manifest.yml file which describes CF deploy arguments:
---
applications:
- name: websocket
command: node example.js
memory: 128M
instances: 1
host: websocket
domain: vcap.me
path: .
So, running cf push from this directory deployed app to my local CFv2 instance (set up with the help of cf_nise_installer)
To test this echo websocket server, I used simple index.html file, which connects to server and sends messages (everything is logged into the console):
<!DOCTYPE html>
<head>
<script>
var socket = null;
var pingData = 1;
var prefix = "ws://";
function connect(){
socket = new WebSocket(prefix + document.getElementById("websocket_url").value);
socket.onopen = function() {
console.log("Connection established");
};
socket.onclose = function(event) {
if (event.wasClean) {
console.log("Connection closed clean");
} else {
console.log("Connection aborted (e.g. server process killed)");
}
console.log("Code: " + event.code + " reason: " + event.reason);
};
socket.onmessage = function(event) {
console.log("Data received: " + event.data);
};
socket.onerror = function(error) {
console.log("Error: " + error.message);
};
}
function ping(){
if( !socket || (socket.readyState != WebSocket.OPEN)){
console.log("Websocket connection not establihed");
return;
}
socket.send(pingData++);
}
</script>
</head>
<body>
ws://<input id="websocket_url">
<button onclick="connect()">connect</button>
<button onclick="ping()">ping</button>
</body>
</html>
Only thing to do left is to enter server address into the textbox of the Index page (websocket.vcap.me in my case), press Connect button and we have working Websocket connection over TCP which could be tested by sending Ping and receiving echo. That worked well in Chrome, however there were some issues with IE 10 and Firefox.
What about "TRUE" TCP socket, there is no exact info: according to the last paragraph here you cannot use any port except 80 and 443 (HTTP and HTTPS) to communicate with your app from outside of Cloud Foundry, which makes me think TCP socket cannot be implemented. However, according to this answer, you can actually use any other port... It seems that some deep investigation on this question is required...
"Cloud Foundry uses an L7 router (ngnix) between clients and apps. The router needs to parse HTTP before it can route requests to apps. This approach does not work for non-HTTP protocols like WebSockets. Folks running node.js are going to run into this issue but there are no easy fixes in the current architecture of Cloud Foundry."
- http://www.subbu.org/blog/2012/03/my-gripes-with-cloud-foundry
I decided to go with pubnub for all my messaging needs.

Is it possible to enable tcp, http and websocket all using the same port?

I am trying to enable tcp, http and websocket.io communication on the same port. I started out with the tcp server (part above //// line), it worked. Then I ran the echo server example found on websocket.io (part below //// line), it also worked. But when I try to merge them together, tcp doesn't work anymore.
SO, is it possible to enable tcp, http and websockets all using the same port? Or do I have to listen on another port for tcp connections?
var net = require('net');
var http = require('http');
var wsio = require('websocket.io');
var conn = [];
var server = net.createServer(function(client) {//'connection' listener
var info = {
remote : client.remoteAddress + ':' + client.remotePort
};
var i = conn.push(info) - 1;
console.log('[conn] ' + conn[i].remote);
client.on('end', function() {
console.log('[disc] ' + conn[i].remote);
});
client.on('data', function(msg) {
console.log('[data] ' + conn[i].remote + ' ' + msg.toString());
});
client.write('hello\r\n');
});
server.listen(8080);
///////////////////////////////////////////////////////////
var hs = http.createServer(function(req, res) {
res.writeHead(200, {
'Content-Type' : 'text/html'
});
res.end(['<script>', "var ws = new WebSocket('ws://127.0.0.1:8080');", 'ws.onmessage = function (data) { ws.send(data); };', '</script>'].join(''));
});
hs.listen(server);
var ws = wsio.attach(hs);
var i = 0, last;
ws.on('connection', function(client) {
var id = ++i, last
console.log('Client %d connected', id);
function ping() {
client.send('ping!');
if (last)
console.log('Latency for client %d: %d ', id, Date.now() - last);
last = Date.now();
};
ping();
client.on('message', ping);
});
You can have multiple different protocols handled by the same port but there are some caveats:
There must be some way for the server to detect (or negotiate) the protocol that the client wishes to speak. You can think of separate ports as the normal way of detecting the protocol the client wishes to speak.
Only one server process can be actually listening on the port. This server might only serve the purpose of detecting the type of protocol and then forwarding to multiple other servers, but each port is owned by a single server process.
You can't support multiple protocols where the server speaks first (because there is no way to detect the protocol of the client). You can support a single server-first protocol with multiple client-first protocols (by adding a short delay after accept to see if the client will send data), but that's a bit wonky.
An explicit design goal of the WebSocket protocol was to allow WebSocket and HTTP protocols to share the same server port. The initial WebSocket handshake is an HTTP compatible upgrade request.
The websockify server/bridge is an example of a server that can speak 5 different protocols on the same port: HTTP, HTTPS (encrypted HTTP), WS (WebSockets), WSS (encrypted WebSockets), and Flash policy response. The server peeks at the first character of the incoming request to determine if it is TLS encrypted (HTTPS, or WSS) or whether it begins with "<" (Flash policy request). If it is a Flash policy request, then it reads the request, responds and closes the connection. Otherwise, it reads the HTTP handshake (either encrypted or not) and the Connection and Upgrade headers determine whether it is a WebSocket request or a plain HTTP request.
Disclaimer: I made websockify
Short answer - NO, you can't have different TCP/HTTP/Websocket servers running on the same port.
Longish answer -
Both websockets and HTTP work on top of TCP. So you can think of a http server or websocket server as a custom TCP server (with some state mgmt and protocol specific encoding/decoding). It is not possible to have multiple sockets bind to the same port/protocol pair on a machine and so the first one will win and the following ones will get socket bind exceptions.
nginx allows you to run http and websocket on the same port, and it forwards to the correct appliaction:
https://medium.com/localhost-run/using-nginx-to-host-websockets-and-http-on-the-same-domain-port-d9beefbfa95d

Resources