I am looking to implement real-time updates in my application that has a node.js backend. I want to use socket.io for this and I understand the library that needs to be implemented. However, I already have a node.js server running:
app.listen(process.env.PORT, () => {
console.log("Server is up and running")
})
The question is fairly simple: is it possible to use the same server with the same port to listen for socket.io connections, or should I run this entirely on a different server or just open another port for this? Because what I often see in the examples is that it listens to http. See below.
What I usually see (this example comes from this post)
var app = require('express')();
var http = require('http').Server(app);
var io = require('socket.io')(http);
http.listen(3000, function () {
console.log('listening on *:3000');
});
Now, if I would use the same server and port, how could I also listen for incoming socket.io requests or emit to its clients? Can I just keep listening with app.listen and will it also connect to socket.io?
In short yes, the socket connection starts as an http request on the same address and port.
The WebSocket protocol was designed to work well with the existing Web
infrastructure. As part of this design principle, the protocol
specification defines that the WebSocket connection starts its life as
an HTTP connection, guaranteeing full backwards compatibility with the
pre-WebSocket world. The protocol switch from HTTP to WebSocket is
referred to as a the WebSocket handshake.
At this point the HTTP connection breaks down and is replaced by the
WebSocket connection over the same underlying TCP/IP connection. The
WebSocket connection uses the same ports as HTTP (80) and HTTPS (443),
by default.
https://websocket.org/aboutwebsocket.html
Related
I'm using Socket.IO to run a WebSocket server locally in NodeJS using the following code:
import express = require('express');
const path = require('path');
import http = require('http');
import { Socket } from 'socket.io';
const app = express();
const server = http.createServer(app);
const socketio = require('socket.io')(server);
app.get('/', (req, res) => {
res.send("Node Server is running");
});
server.listen(3000, function () {
console.log('Example app listening on port 3000!');
});
socketio.on("connection", (socket: Socket) => {
console.log(`connect ${socket.id}`);
console.log(`connect ${socket.handshake.url}`);
socket.on("disconnect", () => {
console.log(`disconnect ${socket.id}`);
});
});
Using a tool like Firecamp, I try to establish a connection on ws://localhost:3000, but to no avail. I eventually use the Socket.IO client to connect from a simple web page by running let socket = io(). It seems the only reason this works is because that call connects to the host serving the page by default, as stated here. Running console.log(socket) and looking at the output, I eventually find that the URL inside the engine field is ws://localhost:3000/socket.io/?EIO=4&transport=websocket&sid=qerg3iHm3IKMOjdNAAAA.
My question is why is the URL so complicated rather than simply ws://localhost:3000? And is there no easier way to get the URL instead of having to access it through dev tools?
A socket.io server does not accept generic webSocket connections. It only accepts socket.io connections as socket.io goes through an extra layer of preparation stuff (over http) before establishing the actual webSocket connection. It then also adds a layer on top of the regular webSocket packet format to support some of its features (such as message names).
When using a socket client to connect to a socket.io server in the default configuration, socket.io first makes a few regular http requests to the socket.io server and with those http requests it sends a few parameters. In your URL:
ws://localhost:3000/socket.io/?EIO=4&transport=websocket&sid=qerg3iHm3IKMOjdNAAAA
The path:
/socket.io/
Is the path that the socket.io server is looking for requests on as destined for the socket.io server. Since this is a unique path and not generally used by other requests, this allows you to share an http server between socket.io and other http requests. In fact, this is a common way to deploy a socket.io server (hooking into an http server that you are already using for http requests).
In fact, the path /socket.io/socket.io.js is also served by the socket.io server and that will return the client-side socket.io.js file. So, clients often use this in their HTML files:
<script src="/socket.io/socket.io.js"></script>
as a means of getting the socket.io client code. Again you see the use of the path prefix /socket.io on all socket.io related URLs.
In your original URL, you can see parameters for:
EIO=4 // engine.io protocol version
transport=websocket // desired transport once both sides agree
sid=qerg3iHm3IKMOjdNAAAA // client identifier so the server knows which client this
// is before the actual webSocket connection is established
Once both sides agree that the connection looks OK, then the client will make a webSocket connection to the server. In cases where webSocket connections are blocked (by network equipment that doesn't support them or blocks them), then socket.io will use a form of http polling where it repeatedly "polls" the server asking for any more data and it will attempt to simulate a continuous connection. The client configuration can avoid this http polling and go straight to a webSocket connection if you want, but you would give up the fallback behavior in case continuous webSocket connections are blocked.
And is there no easier way to get the URL instead of having to access it through dev tools?
Not really. This URL is not something you have to know at all. The socket.io client will construct this URL for you. You just specify http://localhost:3000 as the URL you want to connect to and the socket.io client will add the other parameters to it.
I am trying to build a command-line chat room using Node.js and Socket.io.
This is my server-side code so far, I have tried this with both http initialisations (with express, like on the official website's tutorial, and without it):
#app = require('express')()
#http = require('http').Server(app)
http = require('http').createServer()
io = require('socket.io')(http)
io.sockets.on 'connect', (socket) ->
console.log 'a user connected'
http.listen 3000, () ->
console.log 'listening on *:3000'
I start this with nodejs server.js, the "Listening on" is showing up.
If I run lsof -i tcp:3000, the server.js process shows up.
However, when I start this client-side code:
socket = require('socket.io-client')('localhost:3000', {})
socket.on 'connect', (socket) ->
console.log "Connected"
No luck... When I run nodejs client.js, neither "connect" events, from server nor client, are fired!
My questions are :
- What am I doing wrong?
- Is it necessary to start a HTTP server to use it? Sockets are on the transport layer, right? So in theory I don't need a HTTP protocol to trade messages.
If this is a server to server connection and you're only making a socket.io connection (not also setting it up for regular HTTP connections), then this code shows the simple way for just a socket.io connection:
Listening socket.io-only server
// Load the library and initialize a server on port 3000
// This will create an underlying HTTP server, start it and bind socket.io to it
const io = require('socket.io')(3000);
// listen for incoming client connections and log connect and disconnect events
io.on('connection', function (socket) {
console.log("socket.io connect: ", socket.id);
socket.on('disconnect', function() {
console.log("socket.io disconnect: ", socket.id);
});
});
Node.js socket.io client - connects to another socket.io server
// load the client-side library
const io = require('socket.io-client');
// connect to a server and port
const socket = io('http://localhost:3000');
// listen for successful connection to the server
socket.on('connect', function() {
console.log("socket.io connection: ", socket.id);
});
This code works on my computer. I can run two separate node.js apps on the same host and they can talk to one another and both see the connect and disconnect events.
Some Explaining
The socket.io protocol is initiated by making an HTTP connection to an HTTP server. So, anytime you have a socket.io connection, there is an HTTP server listening somewhere. That HTTP connection is initially sent with some special headers that indicate to the server that this is a request to "upgrade" to the webSocket protocol and some additional security info is included.
This is pretty great reference on how a webSocket connection is initially established. It will show you step by step what happens.
Once both sides agree on the "upgrade" in protocol, then the protocol is switched to webSocket (socket.io is then an additional protocol layer on top of the base webSocket protocol, but the connection is all established at the HTTP/webSocket level). Once the upgrade is agreed upon, the exact same TCP connection that was originally the incoming HTTP connection is repurposed and becomes the webSocket/socket.io connection.
With the socket.io server-side library, you can either create the HTTP server yourself and then pass that to socket.io or you can have socket.io just create one for you. If you're only using socket.io on this server and not also sharing using http server for regular http requests, then you can go either way. The minimal code example above, just lets socket.io create the http server for you transparently and then socket.io binds to it. If you are also fielding regular web requests from the http server, then you would typically create the http server first and then pass it to socket.io so socket.io could bind to the http server you already have.
Then, keep in mind that socket.io is using the webSocket transport. It's just some additional packet structure on top of the webSocket transport. It would akin to agreeing to send JSON across an HTTP connection. HTTP is the host transport and underlying data format. Both sides then agree to format some data in JSON format and send it across HTTP. The socket.io message format sits on top of webSocket in that way.
Your Questions
Is it necessary to start a HTTP server to use it?
Yes, an HTTP server must exist somewhere because all socket.io connections start with an HTTP request to an HTTP server.
Sockets are on the transport layer, right?
The initial connection protocol stack works like this:
TCP <- HTTP protocol
Then, after the protocol upgrade:
TCP <- webSocket <- socket.io
So after the protocol upgrade from HTTP to the webSocket transport, you then have socket.io packet format sitting on top of the webSocket format sitting on top of TCP.
So in theory I don't need a HTTP protocol to trade messages.
No, that is not correct. All connections are initially established with HTTP. Once the upgrade happens to the webSocket transport, HTTP is no longer used.
I made a basic chat app using node.js, express and socket.io. It's not too different from the tutorial chat app for socket.io, it simply emits events between connected clients. When I ran it on port 3001 on my server, it worked fine.
Then I made a proxy server app using node-http-proxy which listens on port 80 and redirects traffic based on the requested url to various independent node apps I have running on different ports. Pretty straightforward. But something is breaking. Whenever anyone disconnects, every single socket dis- and re-connects. This is bad for my chat app, which has connection-based events. The client consoles all show:
WebSocket connection to 'ws://[some socket info]' failed: Connection closed before receiving a handshake response
Here's what I think are the important parts of my code.
proxy-server.js
var http = require('http');
var httpProxy = require('http-proxy');
//create proxy template object with websockets enabled
var proxy = httpProxy.createProxyServer({ws: true});
//check the header on request and return the appropriate port to proxy to
function sites (req) {
//webapps get their own dedicated port
if (req == 'mychatwebsite.com') {return 'http://localhost:3001';}
else if (req == 'someothersite.com') {return 'http://localhost:3002';}
//static sites are handled by a vhost server on port 3000
else {return 'http://localhost:3000';}
}
//create node server on port 80 and proxy to ports accordingly
http.createServer(function (req, res) {
proxy.web(req, res, { target: sites(req.headers.host) });
}).listen(80);
chat-app.js
/*
...other modules
*/
var express = require("express");
var app = exports.app = express(); //I probably don't need "exports.app" anymore
var http = require("http").Server(app);
var io = require("socket.io")(http);
io.on("connection", function (socket) {
/*
...fun socket.on and io.emit stuff
*/
socket.on("disconnect", function () {
//say bye
});
});
http.listen(3001, function () {
console.log("listening on port 3001");
});
Now from what I've read on socket.io's site, I might need to use something to carry the socket traffic through my proxy server. I thought that node-http-proxy did that for me with the {ws: true} option as it states in their docs, but apparently it doesn't work like I thought it would. socket.io mentions three different things:
sticky session based on node's built in cluster module
socket.io-redis, which allows separate socket.io instances to talk to each other
socket.io-emitter, which allows socket.io to talk to non-socket.io processes
I have exactly no idea what any of this means or does. I am accidentally coding way above my skill level here, and I have no idea which of these tools will solve my problem (if any) or even what the cause of my problem really is.
Obligatory apology: I'm new to node.js, so please forgive me.
Also obligatory: I know other apps like nginx can solve a lot of my issues, but my goal is to learn and understand how to use this set of tools before I go picking up new ones. And, the less apps I use, the better.
I think your intuition about needing to "carry the socket traffic through" the proxy server is right on. To establish a websocket, the client makes an HTTP request with a special Upgrade header, signalling the server to switch protocols (RFC 6455). In node, http.Server instances emit an upgrade event when this happens and if the event is not handled, the connection is immediately closed.
You need to listen for the upgrade event on your http server and handle it:
var proxy = httpProxy.createProxyServer({ws: true})
var http = http.createServer(/* snip */).listen(80)
// handle upgrade events by proxying websockets
// something like this
http.on('upgrade', function (req, socket, head) {
proxy.ws(req, socket, head, {target:sites(req.headers.host)})
})
See the node docs on the upgrade event and the node-http-proxy docs for more.
It's pretty easy to configure a http server (using express) and a socket server (socket.io) assigned to it:
var app = require('express')();
var http = require('http').Server(app);
var io = require('socket.io')(http);
How can I run http server and socket server in two different node.js instances?
My idea is to leverage the performance this way, releasing the http node instance from the responsibility of also sending notifications back to the clients.
In a regular Socket.IO + Express app, Socket.IO intercepts requests starting with /socket.io/.
You may set Nginx (or any other webserver that supports proxying) listening 80 port, and make it proxy to Socket.IO process if request starts with /socket.io/, and to Express process otherwise.
Edit: To set up Socket.IO in separate process you may use the following code:
var io = require('socket.io')();
io.on('connection', function(socket){
//here you can emit and listen messages
});
io.listen(3000);
I am currently experiencing with Websockets.
By reviewing some active projects/implementations like einaros/ws (and others as well) I found out that they implement the server their own. Instead of using the node net module which provides a tcp server. Is there a reason for this approach?
https://github.com/einaros/ws/blob/master/lib/WebSocketServer.js
Regards
Update:
var server = net.createServer(function(c) {
c.on('data', function(data) {
// data is a websocket fragment which has to get parsed
});
// transformToSingleUtfFragment is building a websocket valid
// byte fragment which contains hello as application payload
// and sets the right flags so the receiver knows we have a single text fragment
c.write(transformToSingleUtfFragment('hello'));
c.pipe(c);
});
server.listen(8124, function() { //'listening' listener
console.log('server bound');
});
WebSocket's a a protocol layered on top of normal HTTP.
How it works is basically that the browser sends a UPGRADE HTTP request and then makes use of the HTTP 1.1 keep alive functionality to keep the underlying TCP socket of the HTTP connection open.
The data is then send via the WebSocket Protocol (Rather large RFC behind the link), which itself is built on top of TCP.
Since the HTTP part is required, and you need to re-use the TCP connection from that one, it makes sense to go with the normal HTTP server instead of net.Server. Otherwise you'd had to implement the HTTP handling part yourself.
Implementing the WebSocket Protocol needs to be done in either case, and since any HTTP connection can be upgraded, you can, in theory, simply connect your WebSocket "server" to the normal HTTP Server on Port 80 and thus handle both normal HTTP requests and WebSockets on the same port.