I have a Node.js http server that occasionally acts as a proxy for some server side code written in a different language.
So sometimes (but not always) http requests are to be passed through to an server application via sockets. The responses coming from the application already contain the http headers.
The problem is that I would like to simply write the application response into the http response stream without worrying about writing the headers and content separately.
I could implement the entire http server using net sockets but I would like to eventually implement a node http framework as the front-end.
Using the http module, is there a way to write directly to the underlying response socket?
When a request comes in, you should be able to access the connection property of the request.
var http = require('http');
http.createServer(function (req, res) {
req.connection.write(/* your data here */);
}).listen(1337, '127.0.0.1');
console.log('Server running at http://127.0.0.1:1337/');
Note that if you do this, you are also responsible for closing the connection when done.
You could also just pipe the two streams together.
Related
I'm using Socket.IO to run a WebSocket server locally in NodeJS using the following code:
import express = require('express');
const path = require('path');
import http = require('http');
import { Socket } from 'socket.io';
const app = express();
const server = http.createServer(app);
const socketio = require('socket.io')(server);
app.get('/', (req, res) => {
res.send("Node Server is running");
});
server.listen(3000, function () {
console.log('Example app listening on port 3000!');
});
socketio.on("connection", (socket: Socket) => {
console.log(`connect ${socket.id}`);
console.log(`connect ${socket.handshake.url}`);
socket.on("disconnect", () => {
console.log(`disconnect ${socket.id}`);
});
});
Using a tool like Firecamp, I try to establish a connection on ws://localhost:3000, but to no avail. I eventually use the Socket.IO client to connect from a simple web page by running let socket = io(). It seems the only reason this works is because that call connects to the host serving the page by default, as stated here. Running console.log(socket) and looking at the output, I eventually find that the URL inside the engine field is ws://localhost:3000/socket.io/?EIO=4&transport=websocket&sid=qerg3iHm3IKMOjdNAAAA.
My question is why is the URL so complicated rather than simply ws://localhost:3000? And is there no easier way to get the URL instead of having to access it through dev tools?
A socket.io server does not accept generic webSocket connections. It only accepts socket.io connections as socket.io goes through an extra layer of preparation stuff (over http) before establishing the actual webSocket connection. It then also adds a layer on top of the regular webSocket packet format to support some of its features (such as message names).
When using a socket client to connect to a socket.io server in the default configuration, socket.io first makes a few regular http requests to the socket.io server and with those http requests it sends a few parameters. In your URL:
ws://localhost:3000/socket.io/?EIO=4&transport=websocket&sid=qerg3iHm3IKMOjdNAAAA
The path:
/socket.io/
Is the path that the socket.io server is looking for requests on as destined for the socket.io server. Since this is a unique path and not generally used by other requests, this allows you to share an http server between socket.io and other http requests. In fact, this is a common way to deploy a socket.io server (hooking into an http server that you are already using for http requests).
In fact, the path /socket.io/socket.io.js is also served by the socket.io server and that will return the client-side socket.io.js file. So, clients often use this in their HTML files:
<script src="/socket.io/socket.io.js"></script>
as a means of getting the socket.io client code. Again you see the use of the path prefix /socket.io on all socket.io related URLs.
In your original URL, you can see parameters for:
EIO=4 // engine.io protocol version
transport=websocket // desired transport once both sides agree
sid=qerg3iHm3IKMOjdNAAAA // client identifier so the server knows which client this
// is before the actual webSocket connection is established
Once both sides agree that the connection looks OK, then the client will make a webSocket connection to the server. In cases where webSocket connections are blocked (by network equipment that doesn't support them or blocks them), then socket.io will use a form of http polling where it repeatedly "polls" the server asking for any more data and it will attempt to simulate a continuous connection. The client configuration can avoid this http polling and go straight to a webSocket connection if you want, but you would give up the fallback behavior in case continuous webSocket connections are blocked.
And is there no easier way to get the URL instead of having to access it through dev tools?
Not really. This URL is not something you have to know at all. The socket.io client will construct this URL for you. You just specify http://localhost:3000 as the URL you want to connect to and the socket.io client will add the other parameters to it.
I have the following code:
express = require('express');
app = express();
http = require('http').createServer(app);
io = require('socket.io')(http);
app.use(express.static(__dirname + '/'));
http.listen(80);
I know it creates a server that clients can connect to and it works. But I don't know what exactly happens. Can you explain in detail?
Also, why things don't work when I forget about Express.js and just use this line:
io = require('socket.io').listen(80);
It appears to listen for connections. However, inside the browser when I go to http://localhost/, nothing happens. My guess is that I don't specify the directory for my app like that:
app.use(express.static(__dirname + '/'));
Is that why I need Express? To specify the directory?
At the client, I use:
socket = io('http://localhost/'); // this
socket = io(); // or this
None of them work with the single line code at the server-side.
Also, why do I need an HTTP server when Socket.IO uses the WebSocket protocol?
When your browser goes to http://localhost/, you need a web server that's going to respond back to the browser with a web page. That's what Express and the express.static() lines were doing. When you remove those, you do indeed have a server listening for webSocket connections on a specific path, but you don't have anything serving web pages. So, when the browser goes to http://localhost/, there's nothing responding back with a plain web page.
Also, why do I need an HTTP server when Socket.IO uses the WebSocket
protocol?
All socket.io connections start with an HTTP request. socket.io is based on the webSocket protocol and all webSocket connections are initiated with an HTTP request. So, to accept a socket.io connection, you need a web server that responds to an HTTP request and you then need a web server that is smart enough to recognize a request for a webSocket connection so it can "upgrade" the protocol from HTTP to webSocket.
For a well written overview of how a webSocket connection is established, see this overview on MDN.
The socket.io infrastructure then runs on top of that webSocket once it is connected.
I know it creates a server that clients can connect to and it works.
But I don't know what exactly happens. Can you explain in detail?
Here's a line-by-line explanation of your code:
express = require('express');
This loads the Express library.
app = express();
This creates an Express app object which can be used as a webServer request handler.
http = require('http').createServer(app);
This creates a web server and passes it the Express app object as the webServer request handler.
io = require('socket.io')(http);
This hooks socket.io into your web server as another request handler so it can see any incoming http requests that are actually the first stage of starting a webSocket/socket.io connection.
app.use(express.static(__dirname + '/'));
This tells Express that if any request is made for a web page that it should look in the __dirname for a file that matches the requested path. If found, it should return that path.
http.listen(80);
This starts the web server listening on port 80.
None of them work with the single line code at the server-side.
Both of those lines of code to create a socket.io connection will work when used properly. You don't say how this code is being run. If you're trying to run this code from a web page that the browser loads from http://localhost/, then I've already explained why that web page won't load if you don't start Express. If you're trying to run those lines of code from a web page loaded some other way, then you're probably having a same-origin security issue were the browser by default won't let you access a domain that is different than the one the web page came from.
You need the express http server to deliver the socket client to the browser.
Express server starts on port 80
Browser connects to express on port 80, the socket.io server component delivers socket client javascript to the browser (http://localhost:80/socket.io/socket.io.js)
Socket client (running in browser) can then connect to socket.io server
I am learning node.js, and I am not managing to find a direct answer to this question. How does node.js deal with HTTP incoming requests, if they come in virtually at the same time? Let's say that one HTTP request comes in at a given time. As a result, the value of a global variable might change. However, at virtually the same time, another request comes in. In order to service the new request, the value of that one global variable is needed, but the code for the first request is still executing. How does node react to this?
Node.js processes the request one after the other. There is only one thread.
However, if you for example query the database for some information and pass a callback, while the query is executed, node.js can process new requests. Once the database query is completed, node.js calls the callback and finishes processing the first request.
EDIT:
Simple server example:
var http = require('http');
var numresponses = 0;
http.createServer(function (request, response) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('This is response #' + (++numresponses));
}).listen(80);
this server will always print out the number of the request even if two requests happen simultaneously, node will choose one that gets processed first, and both will have different numbers.
I'd like to add a live functionality to a PHP based forum - new posts would be automatically shown to users as soon as they are created.
What I find a bit confusing is the interaction between the PHP code and NodeJS+socket.io.
How would I go about informing the NodeJS server about new posts and have the server inform the clients that are watching the thread in which the post was posted?
Edit
Tried the following code, and it seems to work, my only question is whether this is considered a good solution, as it looks kind of messy to me.
I use socket.io to listen on port 81 to clients, and the server running om port 82 is only intended to be used by the forum - when a new post is created, a PHP script sends a POST request to localhost on port 82, along with the data.
Is this ok?
var io = require('socket.io').listen(81);
io.sockets.on('connection', function(socket) {
socket.on('init', function(threadid) {
socket.join(threadid);
});
});
var forumserver = require('http').createServer(function(req, res) {
if (res.socket.remoteAddress == '127.0.0.1' && req.method == 'POST') {
req.on('data', function(chunk) {
data = JSON.parse(chunk.toString());
io.sockets.in(data.threadid).emit('new-post', data.content);
});
}
res.end();
}).listen(82);
Your solution of a HTTP server running on a special port is exactly the solution I ended up with when faced with a similar problem. The PHP app simply uses curl to POST to the Node server, which then pushes a message out to socket.io.
However, your HTTP server implementation is broken. The data event is a Stream event; Streams do not emit messages, they emit chunks of data. In other words, the request entity data may be split up and emitted in two chunks.
If the data event emitted a partial chunk of data, JSON.parse would almost assuredly throw an exception, and your Node server would crash.
You either need to manually buffer data, or (my recommendation) use a more robust framework for your HTTP server like Express:
var express = require('express'), forumserver = express();
forumserver.use(express.bodyParser()); // handles buffering and parsing of the
// request entity for you
forumserver.post('/post/:threadid', function(req, res) {
io.sockets.in(req.params.threadid).emit('new-post', req.body.content);
res.send(204); // HTTP 204 No Content (empty response)
});
forumserver.listen(82);
PHP simply needs to post to http​://localhost:82/post/1234 with an entity body containing content. (JSON, URL-encoded, or multipart-encoded entities are acceptable.) Make sure your firewall blocks port 82 on your public interface.
Regarding the PHP code / forum's interaction with Node.JS, you probably need to create an API endpoint of sorts that can listen for changes made to the forum. Depending on your forum software, you would want to hook into the process of creating a new post and perform the API callback to Node.js at this time.
Socket.io out of the box is geared towards visitors of the site being connected on the frontend via Javascript. Upon the Node server receiving notification of a new post update, it would then notify connected clients of this new post and its details, at which point it would probably add new HTML to the DOM of the page the visitor is viewing.
You may want to arrange the Socket.io part of things so that users only subscribe to specific events being emitted by them being in a specific room such as "subforum123" so that they only receive notifications of applicable posts.
I am currently experiencing with Websockets.
By reviewing some active projects/implementations like einaros/ws (and others as well) I found out that they implement the server their own. Instead of using the node net module which provides a tcp server. Is there a reason for this approach?
https://github.com/einaros/ws/blob/master/lib/WebSocketServer.js
Regards
Update:
var server = net.createServer(function(c) {
c.on('data', function(data) {
// data is a websocket fragment which has to get parsed
});
// transformToSingleUtfFragment is building a websocket valid
// byte fragment which contains hello as application payload
// and sets the right flags so the receiver knows we have a single text fragment
c.write(transformToSingleUtfFragment('hello'));
c.pipe(c);
});
server.listen(8124, function() { //'listening' listener
console.log('server bound');
});
WebSocket's a a protocol layered on top of normal HTTP.
How it works is basically that the browser sends a UPGRADE HTTP request and then makes use of the HTTP 1.1 keep alive functionality to keep the underlying TCP socket of the HTTP connection open.
The data is then send via the WebSocket Protocol (Rather large RFC behind the link), which itself is built on top of TCP.
Since the HTTP part is required, and you need to re-use the TCP connection from that one, it makes sense to go with the normal HTTP server instead of net.Server. Otherwise you'd had to implement the HTTP handling part yourself.
Implementing the WebSocket Protocol needs to be done in either case, and since any HTTP connection can be upgraded, you can, in theory, simply connect your WebSocket "server" to the normal HTTP Server on Port 80 and thus handle both normal HTTP requests and WebSockets on the same port.