I have a simple node.js /socket.io (websockets) application running #localhost. I am trying to see how many concurrent connections it can handle. Is it possible to simulate several concurrent users on localhost itself ?
This is my half baked attempt using socket.io-client:
function connectAndSend(){
socket.emit('qand',{
code :'ubuntu'
});
}
socket.on('connect', function () {
});
socket.on('q', function (data) {
console.log(data);
});
function callConnect(){
console.log('calling');
connectAndSend() ;
setTimeout(callConnect,100) ;
}
callConnect() ;
As I see it this only 'emits' a new message every 100 ms and is not simulating concurrent connections.
In your call to connect, you must tell socket.io to create a new connection for each call to connect. For example:
var socket = io.connect(server, { "force new connection": true });
Also, if you want to raise the outbound TCP connection limit (which seems to default to 5 connections per target), do something like
require('http').globalAgent.maxSockets = 1000;
before connecting.
But note that creating and closing tcp sockets at a fast rate will make TCP connections pile up in state TIME_WAIT and depending on your OS and your network settings you'll hit a limit pretty soon, meaning you'll have to wait for those old sockets to timeout before you can establish new connections.
If I recall correctly, the limit was around 16k connections (per target ip/port combo) on Windows (both Server 2008 R2 and Windows 7), and the default TIME_WAIT timeout in Windows is 4 minutes, so if you create more than 16k connections in 4 minutes on Windows, you'll probably hit that wall.
Check here:
Long connections with Node.js, how to reduce memory usage and prevent memory leak? Also related with V8 and webkit-devtools
and specifically - test procedure used by the author of question mentioned above
EDIT:
You can use following tools to check how many requests per second your server is capable of serving
ab - http://httpd.apache.org/docs/2.2/programs/ab.html
siege - http://www.joedog.org/siege-home/
Related
I am new to socket.io and Node JS and I am trying to build a scalable application with a high number of simultaneous socket connections (10,000+).
Currently, I started on a model where my server creates child process, and every child process listens a specific port with a sicket.io instance attached. Once a client connects, he is redirected on a specific port.
The big question is : Does having several socket.io instances on several ports increases the number of possible connections ?
Here is my code, just in case :
Server
var server = http.createServer(app);
server.childList = [];
for (var i = 0; i < app.portList.length; i++) {
server.childList[i] = require('child_process').fork('child.js');
}
server.listen(3443, () => {
for (var i = 0; i < app.portList.length; i++) {
server.childList[i].send({ message: 'createServer', port: app.portList[i] });;
}
});
child.js :
var app = require('./app');
var http = require('http');
var socket_io = require( "socket.io" );
process.on('message', (m) => {
if (m.message === 'createServer') {
var childServ = http.createServer(app);
childServ.listen(m.port, () => {
console.log("childServ listening on port "+m.port);
});
var io = socket_io();
io.attach( childServ );
io.sockets.on('connection', function (socket) {
console.log("A client just connected to my socket_io server on port "+m.port);
});
}
});
Feel free to release the kraken if I did something horrible there
First off, what you need to optimize depends on how busy your socket.io connections are and whether the activity is mostly asynchronous I/O operations or whether it's CPU-intensive stuff. As you may already know, node.js scales really well already for asynchronous I/O stuff, but it needs multiple processes to scale well for CPU-intensive stuff. Further, there are some situations where the garbage collector gets too busy (lots and lots of small requests being served) and you also need to go to multiple processes for that reason.
More server instances (up to at least the number of CPUs you have in the server) will give you more CPU processing power (if that's what you need). It won't necessarily increase the number of max connections you can support on a box if most of them are idle. For that, you have to custom tune your server to support lots and lots of connections.
Usually, you would NOT want N socket.io servers each listening on a different port. That puts the burden on the clients to somehow select a port and the client has to know exactly what ports to choose from (e.g. how many server instances you have).
Usually, you don't do it this way. Usually, you have N processes all listening on the same port and you use some sort of loadbalancer to distribute the load among them. This makes the server infrastructure transparent to the clients which means you can scale the servers up or down without changing the client behavior at all. In fact, you can even add more than one physical server box and increase capacity even further that way.
Here's an article from the socket.io doc on using multiple nodes with a load balancer to increase capacity: Socket.io - using multiple nodes (updated link). There's also explicit support by redis for a combination of multiple socket.io instances and redis so you can communicate with any socket.io instance regardless of process.
Does having several socket.io instances on several ports increases the number of possible connections ?
Yes, you have built a simple load-balancer which is a pretty common practice. There are several good tutorials about different ways of scaling node.js.
Horizontally scale socket.io with redis
http://goldfirestudios.com/blog/136/Horizontally-Scaling-Node.js-and-WebSockets-with-Redis
Your load balancer will speed up your code to a point because you utilize multiple threads but I read on some other thread a while ago that a rule of thumb is to start around 2-3 processes per cpu core. More than that cause more overhead then help, but that is highly dependent on situation.
I have a web app that accepts api requests from an ios app. My web app is hosted on Heroku using their free dyno which is able to process 512 mb of data per request. Because node is a single threaded application this will be a problem once we start getting higher levels of traffic from the ios end to the web server. I'm also not the richest person in the world so i'm wondering if it would be smart to create another free heroku app and use a round robin approach to balance the load received from the ios app?
I just need to be pointed into the right direction. Vertical scaling is not really an option financially.
I'm the Node.js platform owner at Heroku.
You may be doing some premature optimization. Node.js, on our smallest 1X size (512MB RAM), can handle hundreds of simultaneous connections and thousands of requests per minute.
If your iOS app is consistently maxing that out, it may be time to consider monetization!
As mentioned by Daniel it's against Heroku rules. Having said that there are probably other services that would allow you to do that.
One way to approach this problem is to use cluster module with ZeroMQ (you need to have ZeroMQ installed before using the module - see module description).
var cluster = require('cluster');
var zmq = require('zmq');
var ROUTER_SOCKET = 'tcp://127.0.0.1:5555';
var DEALER_SOCKET = 'tcp://127.0.0.1:7777';
if (cluster.isMaster) {
// this is the main process - create Router and Dealer sockets
var router = zmq.socket('router').bind(ROUTER_SOCKET);
var dealer = zmq.socket('dealer').bind(DEALER_SOCKET);
// forward messages between router and dealer
router.on('message', function() {
var frames = Array.prototype.slice.cal(arguments);
dealer.send(frames);
});
dealer.on('message', function() {
var frames = Array.prototype.slice.cal(arguments);
router.send(frames);
});
// listen for workers processes to come online
cluster.on('online', function() {
// do something with a new worker, maybe keep an array of workers
});
// fork worker processes
for (var i = 0, i < 100; i++) {
cluster.fork();
}
} else {
// worker process - connect to Dealer
let responder = zmq.socket('rep').connect(DEALER_SOCKET);
responder.on('message', function(data) {
// do something with incomming data
})
}
This is just to point you in the right direction. If you think about it you can create a script with a parameter that will tell it if it's a master or a worker process. Then on the main server run it as is, and on additional servers run it using worker flag which will force it to connect to the main dealer.
Now your main app needs to send the requests to the router, which will be later forwarded to the worker processes:
var zmq = require('zmq');
var requester = zmq.socket('req');
var ROUTER_SOCKET = 'tcp://127.0.0.1:5555';
// handle replies - for example completion status from the worker processes
requester.on('message', function(data) {
// do something with the replay
});
requester.connect(ROUTER_SOCKET);
// send requests to the router
requester.send({
// some object describing the task
});
So first off, as the other replies have pointed out, running two copies of your app to avoid Heroku's limits violates their ToS, which may not be a great idea.
There is, however, some good news. For starters (from Heroku's docs):
The dyno manager will restart your dyno and log an R15 error if the memory usage of a:
free, hobby or standard-1x dyno reaches 2.5GB, five times its quota.
As I understand it, despite the fact that your dyno has 512mb of actual RAM, it'll swap out to 5x that before it actually restarts. So you can go beyond 512mb (as long as you're willing to pay the performance penalty for swapping to disk, which can be severe).
Further to that, Heroku bills by the second and allows you to scale your dyno formation up and down as needed. This is fairly easy to do within your own app by hitting the Heroku API – I see that you've tagged this with NodeJS so you might want to check out:
Heroku's node client
the very-barebones-but-still-functional toots/node-heroku module
Both of these modules allow you to scale up and down your formation of dynos — with a simple heuristic (say, always have a spare 1X dyno running), you could add capacity while you're processing a request, and get rid of the spare capacity when api requests aren't running. Given that you're billed by the second, this can end up being very inexpensive; 1X dynos work out to something like 5¢ an hour to run. If you end up running extra dynos for even a few hours a day, it's a very, very small cost to you.
Finally: there are also 3rd party services such as Adept and Hirefire (two random examples from Google, I'm sure there are more) that allow you to automate this to some degree, but I don't have any experience with them.
You certainly could, I mean, programatically - but that would bypass Heroku's TOS:
4.4 You may not develop multiple Applications to simulate or act as a single Application or otherwise access the Heroku Services in a manner intended to avoid incurring fees.
Now, I'm not sure about this:
Because node is a single threaded application this will be a problem once we start getting higher levels of traffic from the ios end to the web server.
There are some threads discussing that, with some interesting answers:
Clustering Node JS in Heavy Traffic Production Environment
How to decide when to use Node.js?
Also, they link to this video, introducing Node.js, which talks a bit about benchmarks:
Introduction of Node JS by Ryan Dahl
I'm testing communication between two NodeJS instances over TCP, using the net module.
Since the TCP doesn't rely on messages (socket.write()), I'm wrapping each message in a string like msg "{ json: 'encoded' }"; in order to handle them individually (otherwise, I'd receive packets with a random number of concatenated messages).
I'm running two NodeJS instances (server and client) on a CentOS 6.5 VirtualBox VM with bridged network and a Core i3-based host machine. The test lies on the client emitting a request to the server and waiting for the response:
Client connects to the server.
Client outputs current timestamp (Date.now()).
Client emits n requests.
Server replies to n requests.
Client increments a counter on every response.
When finished, client outputs the current timestamp.
The code is quite simple:
Server
var net = require('net');
var server = net.createServer(function(socket) {
socket.setNoDelay(true);
socket.on('data', function(packet) {
// Split packet in messages.
var messages = packet.toString('utf-8').match(/msg "[^"]+";/gm);
for (var i in messages) {
// Get message content (msg "{ content: 'json' }";). Actually useless for the test.
//var message = messages[i].match(/"(.*)"/)[1];
// Emit response:
socket.write('msg "PONG";');
}
});
});
server.listen(9999);
Client
var net = require('net');
var WSClient = new net.Socket();
WSClient.setNoDelay(true);
WSClient.connect(9999, 'localhost', function() {
var req = 0;
var res = 0;
console.log('Start:', Date.now());
WSClient.on('data', function(packet) {
var messages = packet.toString("utf-8").match(/msg "[^"]+";/gm);
for (var i in messages) {
// Get message content (msg "{ content: 'json' }";). Actually useless for the test.
//var message = messages[i].match(/"(.*)"/)[1];
res++;
if (res === 1000) console.log('End:', Date.now());
}
});
// Emit requests:
for (req = 0; req <= 1000; req++) WSClient.write('msg "PING";');
});
My results are:
With 1 request: 9 - 24 ms
With 1000 requests: 478 - 512 ms
With 10000 requests: 5021 - 5246 ms
My pings (ICMP) to localhost are between 0.6 - 0.1 seconds. I've not intense network traffic or CPU usage (running SSH, FTP, Apache, Memcached, and Redis).
Is this normal for NodeJS and TCP or it is just my CentOS VM or my low-performance host? Should I move to another platform like Java or a native C/C++ server?
I think that a 15 ms delay (average) per request on localhost is not acceptable for my project.
Wrapping the messages in some text and searching for a Regex match isn't enough.
The net.Server and net.Socket interfaces have a raw TCP stream as an underlying data source. The data event will fire whenever the underlying TCP stream has data available.
The problem is, you don't control the TCP stack. The timing of it firing data events has nothing to do with the logic of your code. So you have no guarantee that the data event that drives your listeners has exactly one, less than one, more than one, or any number and some remainder, of messages being sent. In fact, you can pretty much guarantee that the underlying TCP stack WILL break up your data into chunks. And the listener only fires when a chunk is available. Your current code has no shared state between data events.
You only mention latency, but I expect if you check, you will also find that the count of messages received (on both ends) is not what you expect. That's because any partial messages that make it across will be lost completely. If the TCP stream sends half a message at the end of chunk 1, and the remainder in chunk 2, the split message will be totally dropped.
The easy and robust way is to use a messaging protocol like ØMQ. You will need to use it on both endpoints. It takes care of framing the TCP stream into atomic messages.
If for some reason you will connecting to or receiving traffic from external sources, they will probably use something like a length header. Then what you want to do is create a Transform stream that buffers incoming traffic, and only emits data when the amount identified in the header has arrived.
Have you done any network dump? You may be creating network congestion due to the overhead introduced by enabling 'no delay' socket property. This property will send data down to TCP stack as soon as possible and if you have very small chunks of information it will lead to many TCP packets with small payloads, thus the decreasing transmission efficiency and eventually having TCP pausing the transmission due to congestion. If u want to use 'no delay' for your sockets, try increasing your receiving socket buffer so that data is pulled from the tcp stack faster. Let us know if that helped.
Ok, so I have an app that works just fine locally. I deployed it to Azure the other day and I am regularly getting the error:
IIS Detailed Error - 503.0 - Number of active WebSocket requests has reached the maximum concurrent WebSocket requests allowed.
I don't understand why...I have read a lot of tutorials, guides, etc about socket.io (and I have been building with it for 4 months locally with no issue).
Here is my connection code.
io.sockets.on('connection', function (socket) {
var handshake = socket.handshake;
var session = socket.handshake.session;
clients.push(socket);
console.log('A socket with sessionID ' + handshake.sessionID + ' connected!');
// setup an inteval that will keep our session fresh
var intervalID = setInterval(function () {
session.reload( function () {
session.touch().save();
});
}, 60 * 1000);
socket.on('disconnect', function () {
console.log('A socket with sessionID ' + handshake.sessionID + ' disconnected!');
var i = clients.indexOf(socket);
clients.splice(i, 1);
// clear the socket interval to stop refreshing the session
clearInterval(intervalID);
});
}
The console logs when people connect and disconnect...this is working just fine.
If I reset my server my code will run for a little while. I know Azure supports 350 concurrent sockets...not sure how a single user fills that up.
I come from a .NET background so I am used to closing connections when I am done with them, but that doesn't seem to be necessary with node.js sockets.
But if I don't need to explicitly close my sockets, then why are my connections piling up?
Thanks for your help,
David
UPDATE
So, based on the answer below, I discovered that azure limits the concurrent connections pretty severely on the free plan. I updated to the standard package to get the full 350 connections.
Of note, I learned that if you use this command:
io.sockets.manager.server.connections
you will get a count of the current connections. This plainly showed me that even by myself I was using 7 (which is why the free plan died). Now I just need to figure out why...
The blog post states:
•Free: (5) concurrent connections per website instance
•Shared: (35) concurrent connections per website instance
•Standard: (350) concurrent connections per website instance
The 350 concurrent connections limits applies only to "Standard" Windows Azure Web Sites. Are you in fact using Standard?
In production, I have a game which uses connection-local variables to hold game state. However I notice that if I idle for a certain time on the connection, it disconnects and reconnects which loses the current state. During my tests on a local host, I never noticed this behavior. Is this the norm behavior for socket connections or is something else causing the connections to drop.
If it is a normal behavior how is this typically handled? Should connection values be stored globally so they can be restored should a user drop/reconnect?
Your problem is around socket timeouts. If there's no activity on a certain socket, socket.io will close it automatically.
An easy (and hackish) fix is to send a heartbeat to the connected client to create activity and stop the socket from timing out.
Server:
function sendHeartbeat(){
setTimeout(sendHeartbeat, 8000);
io.sockets.emit('ping', { beat : 1 });
}
io.sockets.on('connection', function (socket) {
socket.on('pong', function(data){
console.log("Pong received from client");
});
}
setTimeout(sendHeartbeat, 8000);
Client:
socket.on('ping', function(data){
socket.emit('pong', {beat: 1});
});
More Information:
You can get more information on configuring socket.io here.
EDIT: Mark commented that if the user does lose the connection (connection drops on his end because of internet troubles), you should be able to restore the user to his last state.
To do that, the best way would be to use a already widely used method for storing user data, cookies and sessions.
An extremely well done tutorial on how to do this located here. Although he uses express to set cookies, you can do this using anything (I do it using rails). Using this method, you can store the user data in a cookie and fetch it during the handshake. From there you can just access the data using socket.handshake.data.
What you need to do is create or identify the session per (re-) connection. You may reduce the number of reconnections per Moox's answer above but it is still not failsafe - e.g. a user loses wifi connection for a bit, etc. In other words - maintain user metadata per session and not per socket, and expect occasional disconnects and reconnects.