I have experience with a few headless browsers, but only for testing and not load testing. What's the best way to launch 500-1000 websocket clients to load test the application? Is it as simple as just looping a bunch with a setTimeout that gets longer incrementally?
I can build out the actual tests myself, I'm just curious which framework is best suited for this.
I have experience with ZombieJS and PhantomJS (along with Casper and Webspecter).
Looks like you want to do scalability/load testing on your server. How do you test client side performance with hundreds of thousands of virtual clients? anyway I assume you already tried using headless phantom clients. That's the same thing I also tried and it worked for me well I monitored CPU, network throughput and Memory usage using some utility plugins.
There is a plugin which does JMeter integration for websocket protocol. This might be helpful
https://github.com/kawasima/jmeter-websocket/
Related
Lets say I want to create a front-end where multiple users can send a request to a server that scrapes some links off a website, would puppeteer be able to process it concurrently/atleast fast enough or should I consider a different method?
Also, is there any possible way to load a page in a headless browser instance(with js enabled) on a mobile device? How could I go about coding my own headless browser in javascript if its possible?
You can always deploy your node.js instance via PM2 and spawn multiple processes concurrently to handle the incoming load. You should restrict the total processes to the total number of cores available to your box, but otherwise this would work fine.
Whether or not this could handle your load depends on your server, the workload, and the expected throughout. You'd need to do some load testing to make that determination for your system.
I have developed a basic nodejs+express server which, for one route returns status 200 and perform a couple of load tests on localhost. The thing is that I cannot pass 1000 requests/second for this simple route:
app.get('/test',function(req.res){
res.status(200).send()
}
For 1000 requests/s the server behaves alright, but after this the loadtest returns errors. How can I increase this value for nodejs, and, is this a good result? This is only a simple route without any processing so I think the server should accept much more requests. Thanks.
There are so many factors that can influence your results. This is generally something that you shouldn't do on your own dev machine if you want solid, real-life results. The reason you shouldn't use your own dev machine is because it is always running something else that's completely unrelated to your app that uses your system resources - like Google Chrome, I/O, etc.
Another factor is your hardware and how saturated your system resources currently are.
If you still want to test locally I want you to keep in mind that the results will not reflect a production environment, hence the test is useless.
You can also improve performance a bit by using Node's cluster module to make your server utilize all your machines' processor threads. Node.js is a single-threaded language.
Which would be the best option to profile a Node.js application on linux? I tried https://github.com/c4milo/node-webkit-agent and https://github.com/baryshev/look (this is based on nodetime), but they both seem pretty experimental. What surprises me the most is that the results reported by these tools are different.
The major disadvantages for look are that the heap snapshots aren't very relevant and you can't CPU profile for more than 1 minute.
With node-webkit-agent the Chrome browser is running out of memory.
I'm doing profiling while sending requests using JMeter to my web application.
Not sure if you're willing to use an online service instead of a module, but you could give http://nodefly.com/ a try, it's free and has worked quite good for me.
I wish to do a performance test on my site, simulating thousands of user and find per server capacity limit. The tool I'm using is jmeter and I have prepared a .jmx for the test scenario. But when I try to simulate 1000 of users simultaneously I start to get:
<httpSample t="0" lt="0" ts="1338538936990" s="false" lb="VerifyPassword" rc="Non HTTP response code: java.net.SocketException" rm="Non HTTP response message: Too many open files" tn="LoadConfig 1-901" dt="text" by="1375"/>
I think the error is on the client side because of the too many socket connection. If so how can I simulate the case from my local machine? Can I increase the number of open sockets on linux?
Also one thing I discover testing from a single client can give false alarm where the client is the bottleneck and the server works fine. How can I do a performance testing such that I simulate a real life scenario such that I have 10K+ users each have its own CPU/ RAM and then do a performance testing?
I have run JMeter from .NET but I think will be the same for your case.
You cannot increase the number of sockets. You should do a distributed load testing.
Luckly for you Jmeter has this ability :)
The google term you should look for is distributed JMeter testing or remote JMeter testing. If it happens that you only can use your local machine you might use virtual machines to create several JMeter distributed instances...
Check:
http://jmeter.apache.org/usermanual/remote-test.html
So the first app that people usually build with SocketIO and Node is usually a chatting app. This chatting app basically has 1 Node server that will broadcast to multiple clients. In the Node code, you would have something like.
//Psuedocode
for(client in clients){
if(client != messageSender){
user.send(message);
}
}
This is great for a low number of users, but I see a problem with this. First of all, there is a single point of failure which is the Node server. Second of all, the app will slow down as the number of clients grow. What is there to do then when we reach this bottleneck? Is there an architecture (horizontal/vertical scaling) that can be used to alleviate this problem?
For that "one day" when your chat app needs multiple, fault-tolerant node servers, and you want to use socket.io to cross communicate between the server and the client, there is a node.js module that fits the bill.
https://github.com/hookio/hook.io
It's basically an event emitting framework to cross communicate between multiple "things" -- such as multiple node servers.
It's relatively complicated to use, compared to most modules, which is understandable since this is a complex problem to solve.
That being said, you'd probably have to have a few thousand simultaneous users and lots of other problems before you begin to have problems with this.
Another thing you can do, is try to develop your application in a way so that if a connection is lost (which happens all the time anyway), eg. server goes down, client has network issues (eg. mobile user), etc, your application should be able to handle that and recover from such issues gracefully.
Since Node.js has a single event-loop thread, this single point of failure is written into its DNA. Even reloading a server after code changes require this thread to be stopped.
There are however a lot of tools available to handle such failures gracefully. You could use forever; a simple CLI tool for ensuring that a given script runs continuously. Other options include distribute and up. Distribute is a load balancing middleware for Node. Up builds on top of Distribute to offer zero downtime reloads using either a JavaScript API or command line interface:
Further reading I find you just need to use Redis Store with Socket.io to maintain connection references between two or more processes/ servers. These options have already been discussed extensively here and here.
There's also the option of using socket.io-clusterhub if you don't intend to use the Redis store.