I need to create a small authentication layer on top of a 3rd party web-socket based chat application. I have a simple API (get) that can validate api tokens from requests. What I want to do is essentially validate their token (which I know how to do) then proxy the websocket connection to the actual chat server.
I've been looking for solutions and this thread seems to give some pointers in the right direction however I can't get any of the solutions to work.
var http = require('http'),
WebSocket = require('faye-websocket'),
conf = require('./conf.json');
var server = http.createServer();
server.on('upgrade', function(request, socket, body) {
console.log('upgrade fired');
var frontend = new WebSocket(request, socket, body),
backend = new WebSocket.Client('ws://echo.websocket.org');
frontend.pipe(backend).pipe(frontend);
});
server.on('connection', function(socket) {
console.log('connection')
backend = new WebSocket.Client('ws://echo.websocket.org');
console.log(backend);
socket.pipe(backend).pipe(socket);
})
server.listen(conf.port);
console.log('Listening on '+port.conf);
The connection event is fired, however the upgrade event which is supposed to be fired on a ws connection never is.
The goal is to first authenticate a api key to an external server, then open a proxy to the chat websocket all through a request to this node server via a websocket connection. I'll most likely pass the api key as a get parameter for simplicity. I also looked at this package and attempted using it however it didn't work as well.
The issue ended up being with nginx. I forgot I was proxying requests through the reverse proxy and by default nginx does not support the ws:// connection so it drops it.
Related
I'm using Socket.IO to run a WebSocket server locally in NodeJS using the following code:
import express = require('express');
const path = require('path');
import http = require('http');
import { Socket } from 'socket.io';
const app = express();
const server = http.createServer(app);
const socketio = require('socket.io')(server);
app.get('/', (req, res) => {
res.send("Node Server is running");
});
server.listen(3000, function () {
console.log('Example app listening on port 3000!');
});
socketio.on("connection", (socket: Socket) => {
console.log(`connect ${socket.id}`);
console.log(`connect ${socket.handshake.url}`);
socket.on("disconnect", () => {
console.log(`disconnect ${socket.id}`);
});
});
Using a tool like Firecamp, I try to establish a connection on ws://localhost:3000, but to no avail. I eventually use the Socket.IO client to connect from a simple web page by running let socket = io(). It seems the only reason this works is because that call connects to the host serving the page by default, as stated here. Running console.log(socket) and looking at the output, I eventually find that the URL inside the engine field is ws://localhost:3000/socket.io/?EIO=4&transport=websocket&sid=qerg3iHm3IKMOjdNAAAA.
My question is why is the URL so complicated rather than simply ws://localhost:3000? And is there no easier way to get the URL instead of having to access it through dev tools?
A socket.io server does not accept generic webSocket connections. It only accepts socket.io connections as socket.io goes through an extra layer of preparation stuff (over http) before establishing the actual webSocket connection. It then also adds a layer on top of the regular webSocket packet format to support some of its features (such as message names).
When using a socket client to connect to a socket.io server in the default configuration, socket.io first makes a few regular http requests to the socket.io server and with those http requests it sends a few parameters. In your URL:
ws://localhost:3000/socket.io/?EIO=4&transport=websocket&sid=qerg3iHm3IKMOjdNAAAA
The path:
/socket.io/
Is the path that the socket.io server is looking for requests on as destined for the socket.io server. Since this is a unique path and not generally used by other requests, this allows you to share an http server between socket.io and other http requests. In fact, this is a common way to deploy a socket.io server (hooking into an http server that you are already using for http requests).
In fact, the path /socket.io/socket.io.js is also served by the socket.io server and that will return the client-side socket.io.js file. So, clients often use this in their HTML files:
<script src="/socket.io/socket.io.js"></script>
as a means of getting the socket.io client code. Again you see the use of the path prefix /socket.io on all socket.io related URLs.
In your original URL, you can see parameters for:
EIO=4 // engine.io protocol version
transport=websocket // desired transport once both sides agree
sid=qerg3iHm3IKMOjdNAAAA // client identifier so the server knows which client this
// is before the actual webSocket connection is established
Once both sides agree that the connection looks OK, then the client will make a webSocket connection to the server. In cases where webSocket connections are blocked (by network equipment that doesn't support them or blocks them), then socket.io will use a form of http polling where it repeatedly "polls" the server asking for any more data and it will attempt to simulate a continuous connection. The client configuration can avoid this http polling and go straight to a webSocket connection if you want, but you would give up the fallback behavior in case continuous webSocket connections are blocked.
And is there no easier way to get the URL instead of having to access it through dev tools?
Not really. This URL is not something you have to know at all. The socket.io client will construct this URL for you. You just specify http://localhost:3000 as the URL you want to connect to and the socket.io client will add the other parameters to it.
I'm using ws webSocket (can't use socket.io) and I'm a bit new to it. I know how it works, but don't fully get a couple of things.
How can I authenticate using jwt? The docs say that using the upgrade event is the correct way to go about it, but when the client connected, the code in the upgrade function didn't run.
What's the correct way to add an event type? In socket.io, one can listen using client.on('eventType', function). What's the webSocket ws equivalent?
When and why would one use paths in webSocket ws?
Update
The update event doesn't get fired on my end. Here's the code I have:
File 1
const server = require('http').createServer(app);
require('./socket/socket')(server);
File 2
module.exports = function(server) {
const WebSocket = require('ws');
const wss = new WebSocket.Server({ port: 8080 });
server.on('upgrade', function(request, socket, head) {
console.log('Upgraded');
wss.handleUpgrade(request, socket, head, function(ws) {
wss.emit('connection', ws, request);
});
});
wss.on('connection', function connection(ws) {
ws.on('message', function incoming(message) {
console.log('received: %s', message);
});
});
};
How can I authenticate using jwt? The docs say that using the upgrade event is the correct way to go about it, but when the client connected, the code in the upgrade function didn't run.
Well, there are lots of different choices. You could pass a token as a query parameter of a custom header when first making the connection. You could require that the first data sent over the webSocket is a token and disconnect if such a token does not arrive soon and first. If the client was already authenticated and there's an http cookie that indicates that, you could examine that cookie upon websocket connection.
What's the correct way to add an event type? In socket.io, one can listen using client.on('eventType', function). What's the webSocket ws equivalent?
The socket.io message types (or event types as you call them) are something that socket.io adds on top of webSocket. The webSocket protocol itself does not have such a thing. You send a data packet and that data packet arrives at the other end. All data packets are of the same webSocket "type". If you want to identify your data packets as being a certain type of message, then you have to invent a way of doing that inside your data back. For example, if your data was JSON formatted, you could add a type: "someType" property to event message and then examine that property upon receipt (this is essentially what socket.io does although it does it outside of the socket.io payload).
When and why would one use paths in webSocket ws?
You may have two completely separate parts of code that use a webSocket for different things that have nothing in common and reside in separate modules and it's just more straightforward for each to have its own webSocket connection. And, they may be used by two completely separate types of clients.
How to access the orignal URL when a webSocket client connects to your server is shown here on your server's upgrade event used with the ws library.
You will note in that example that the upgrade event occurs on the http server, not on the websocket server instance.
ws provide auth examples.
These work when a HTTP server first handle the auth requests. Then pass a
HTTP upgrade request to ws rather than ws itself listening on a port. Note
the noServer: true option they include in the setup:
const wss = new WebSocket.Server({ clientTracking: false, noServer: true });
The jwt component will be easier using koa or express in HTTP
first then doing the upgrade in this way. Otherwise you would need to
write a ws message handler to look for some token data and verify it.
The message is the event in ws. You get to write anything more specific on top of the message event, which is a big reason to use socket.io that has done all that for you (including client callback functions, which are super helpful).
Using a URI path in the ws:// or http upgrade request would usually be to connect to a separate ws context on the server side. A simple way to namespace or separate connection into discreet groups.
I see a lot of examples hooking an http server during the creation of a WS server, more or less like the following
var server = http.createServer(function(request, response) {
// process HTTP request. Since we're writing just WebSockets
// server we don't have to implement anything.
});
server.listen(1337, function() { });
// create the server
wsServer = new WebSocketServer({
httpServer: server
});
or
var httpServer = http.createServer().listen(websocketport);
/*
* Hook websockets in to http server
*/
socketServer.installHandlers(httpServer, { prefix: '/websockets' });
I dont understand the reason why. Is there any benefit from that?
What is wrong with the classic WS setup, like so
const WebSocket = require('ws')
const wss = new WebSocket.Server({ port: 8080 })
wss.on('connection', ws => {....
Why shouldn't I just use a WS server with no http server at all?
This depends entirely on the project.
Typically, most examples you will see are not pure websocket server examples and instead will assume that you are incorporating websocket functionality into a larger application stack.
If you have no need for an HTTP server, then you shouldn't attach your websocket listener to an instance of one. If you do need an HTTP server, then you are best off attaching your websocket listener in the following situations:
If your HTTP server and your WebSocket listener are on the same domain
You want to be able to manage or push to your socket connections when some HTTP requests are made.
This is most likely because usually, you would provide some REST APIs which uses the HTTP server along with your WS server.
If you are trying to use the same host and port for both of them (HTTP & WS) together, you need to sort of combine or encapsulate one of them into another so that it can accept and handle both kinds of protocol.
However, if you are only trying to use WS without HTTP, then you probably do not need the HTTP server, just the WS server will do, as you have shown in your classic WS setup.
On the other hand, if you only need the HTTP server without WS, then you do not need to implement WS, just HTTP server will do.
I made a basic chat app using node.js, express and socket.io. It's not too different from the tutorial chat app for socket.io, it simply emits events between connected clients. When I ran it on port 3001 on my server, it worked fine.
Then I made a proxy server app using node-http-proxy which listens on port 80 and redirects traffic based on the requested url to various independent node apps I have running on different ports. Pretty straightforward. But something is breaking. Whenever anyone disconnects, every single socket dis- and re-connects. This is bad for my chat app, which has connection-based events. The client consoles all show:
WebSocket connection to 'ws://[some socket info]' failed: Connection closed before receiving a handshake response
Here's what I think are the important parts of my code.
proxy-server.js
var http = require('http');
var httpProxy = require('http-proxy');
//create proxy template object with websockets enabled
var proxy = httpProxy.createProxyServer({ws: true});
//check the header on request and return the appropriate port to proxy to
function sites (req) {
//webapps get their own dedicated port
if (req == 'mychatwebsite.com') {return 'http://localhost:3001';}
else if (req == 'someothersite.com') {return 'http://localhost:3002';}
//static sites are handled by a vhost server on port 3000
else {return 'http://localhost:3000';}
}
//create node server on port 80 and proxy to ports accordingly
http.createServer(function (req, res) {
proxy.web(req, res, { target: sites(req.headers.host) });
}).listen(80);
chat-app.js
/*
...other modules
*/
var express = require("express");
var app = exports.app = express(); //I probably don't need "exports.app" anymore
var http = require("http").Server(app);
var io = require("socket.io")(http);
io.on("connection", function (socket) {
/*
...fun socket.on and io.emit stuff
*/
socket.on("disconnect", function () {
//say bye
});
});
http.listen(3001, function () {
console.log("listening on port 3001");
});
Now from what I've read on socket.io's site, I might need to use something to carry the socket traffic through my proxy server. I thought that node-http-proxy did that for me with the {ws: true} option as it states in their docs, but apparently it doesn't work like I thought it would. socket.io mentions three different things:
sticky session based on node's built in cluster module
socket.io-redis, which allows separate socket.io instances to talk to each other
socket.io-emitter, which allows socket.io to talk to non-socket.io processes
I have exactly no idea what any of this means or does. I am accidentally coding way above my skill level here, and I have no idea which of these tools will solve my problem (if any) or even what the cause of my problem really is.
Obligatory apology: I'm new to node.js, so please forgive me.
Also obligatory: I know other apps like nginx can solve a lot of my issues, but my goal is to learn and understand how to use this set of tools before I go picking up new ones. And, the less apps I use, the better.
I think your intuition about needing to "carry the socket traffic through" the proxy server is right on. To establish a websocket, the client makes an HTTP request with a special Upgrade header, signalling the server to switch protocols (RFC 6455). In node, http.Server instances emit an upgrade event when this happens and if the event is not handled, the connection is immediately closed.
You need to listen for the upgrade event on your http server and handle it:
var proxy = httpProxy.createProxyServer({ws: true})
var http = http.createServer(/* snip */).listen(80)
// handle upgrade events by proxying websockets
// something like this
http.on('upgrade', function (req, socket, head) {
proxy.ws(req, socket, head, {target:sites(req.headers.host)})
})
See the node docs on the upgrade event and the node-http-proxy docs for more.
There is a way to manage the socket.io creation at Server Side?, Currently, I couldn't found any doc, Only found in relation with the socket is created per request from a client "io.connect(server)".
The current flow work OK:
Set Socket.io (at Node.js) at SERVER:PORT
Client connect to SERVER
using io.connect(SERVER:PORT)
I wonder if it is possible ? Trying to do:
Set Socket.io (at Node.js) at SERVER:PORT
Recieved a POST (REST) - Server side
Create/Open Socket.io a server side.
At response of Post send the id?
the clien open a socke.io
Sent to client socket.id to client as
So Far, looking in deep on the code and doc, I found that socket.io support namespaces, so I used this in order to manage client connection id.
at server.js
var app = express();
var server = require('http').createServer(app),
io = require('socket.io').listen(server,{ log: false });
// Rest New Process
function generateNameSpaceWs (responce, request) {
io.of("/" + id).on('connection', handler);
response.send(id);
}
app.post("/newWS", function (res, req) {
return generateNameSpaceWs(res, req);
}
at Client.js
function makeWS(){
var ws, c = new XMLHttpRequest();
c.open("GET", url, false);
c.send();
if (c.status == 200){
id = JSON.parse(c.responseText);
ws = new io.connect("server/" + id)
}
So far you are doing right, if I understand your question correctly, you are trying to authenticate connection via POST, so that user can only connect to server via socket if server responds to ID. This is a roundabout way. Use the socket instead of POST.
Socket server has to be running already, and accepts connection via io.sockets.on('connection'), at server you can choose whether to accept it or reject it ,do socket.disconnect('unauthorized') to close connection from server.
I would you suggest you do this :
Set Socket.io (at Node.js) at SERVER:PORT
Client connect to SERVER using io.connect(SERVER:PORT)
Send what you are sending in POST over socket.
Authenticate/Process on io.sockets.on('connection', function(socket) at server.
Close socket if unathorized.
Send back ID data to client.
This doesn't seem possible -- while the official documentation for socket.io is lacking, the documentation for the net module indicates that the only way to create a socket is to initiate it server side.
However, you can still achieve the desired effect by creating an id for the socket on the server to associate with the socket. That is,
Set Socket.io (at Node.js) at SERVER:PORT
Recieved POST (REST) - Server side
Create id (Note:This could be done before step 2)
At response of Post send the id!
Client connect to SERVER
using io.connect(SERVER:PORT)
The client sends the id to the server using something like
socket.emit("set_id",id)
The server recieves the id and associates it with the socket using something like
socket.on("set_id",function(id){
socket.set("id",id)
}
Now you can reference the socket using the id that you created!
Good luck!