I would like to script benchmark of my socket.io implementation.
After some research I have identified several NodeJS modules, but they have either not been updated for past years (wsbench), or are only supporting websocket protocol (wsbench, thor) or is not testing socket.io implementation but socket.io project (socket.io-benchmark).
Since socket.io project has been highly active the past year, I wonder what is the latest and greatest tool/module to use for benchmarking?
My requirements:
Easy to script and run the tests
Test reports giving good overview of test runs
Test reports should be easy to save in order to compare with later benchmarking
Just came across this in search of some benchmarking for my Socket.IO project.
I found socket.io-benchmark, however had some additional items that I wanted to work through but found one of the forks nearly there.
https://github.com/slowthinker/socket.io-benchmark
I also forked it to add a cap on messagse/second sent to give it more realistic parameters.
Hope that helps!
I would suggest Artillery: Artillery is a modern, powerful, easy-to-use, open-source load-testing toolkit: https://github.com/shoreditch-ops/artillery
Here some feature:
Mulitple protocols: Load-test HTTP, WebSocket and Socket.io applications
Scenarios: Specify scenarios to test multi-step interactions in your API or web app
Perfomance metrics: get detailed performance metrics (latency, requests per second, concurrency, throughput)
Scriptable: write custom logic in JS to do pretty much anything
High performance: generate serious load on modest hardware
Integrations: statsd support out of the box for real-time reporting (integrate with Datadog, Librato, InfluxDB etc)
Extensible: custom reporting plugins, custom protocol engines etc
and more! HTML reports, nice CLI, parameterization with CSV files
Related
I would like to use Locust for UI performance testing. How to use Locust for UI performance testing? How can I get the loading time of the HTML elements(img, lists, etc..)?
Thanks
Locust isn't a browser and doesn't parse HTML. It just does plain HTTP requests and it will not load things like images based on the response.
If you need something like that, you would need to parse the HTML in the response and do the "dependent" requests in your test script.
Locust is not made for that (as said). There are some other fancy tools which will allow to do it for you e.g:
k6.io (https://k6.io/ - previously known as LoadImpact) - allows you to perform performance checks outside of your environment and report it back to the pipeline with results. Easy to configure and integrate, great when it comes to more "clever" testing scenarios such as stress tests, load tests etc.
sitespeed.io (https://www.sitespeed.io/) - my 2nd favorite, very fun to use and easy to configure tool to track FE performance and tests (e.g. done with Selenium)
Lighthouse Reports - might be also performed as a "pointer" to the most common issues and included as a PR comments e.g. or notifications during the process (there are many Github Actions or DevOps packages doing it)
I've also gathered some of my findings in my recent talk (slides below) and is converted into the series of blogs around these topics and first of them is already published:
Slide deck from my talk on "Modern Web Performance Testing": https://slides.com/zajkowskimarcin/modern-web-performance-testing/
First blog from the series on the same topic: https://wearecogworks.com/blog/the-importance-of-modern-web-performance-testing-part-1
I need to write a WebSocket server and I am learning Node JS by reading some books I purchased. This server is for a very fast game so I need to stream small messages to groups of clients as quick as possible.
What is the difference between:
Autobahn | JS : http://autobahn.ws/js/
and
Einaros : https://github.com/einaros/ws
?
I have heard that Autobahn is very powerful and capable to deal with 200k clients without a load balancer so I was wondering if someone with more experience could advise me whether there is any advantage in opting for one or another library.
The functional difference is: Einaros is a WebSocket library, whereas Autobahn provides WebSocket implementations (e.g. AutobahnPython), plus WAMP on top of WebSocket.
WAMP provides higher-level communication for apps (RPC + PubSub - pls see the WAMP website). And AutobahnJS is a WAMP implementation for browsers (and NodeJS) on top of WebSocket.
Now, say you don't care about WAMP, and hence only need a raw WebSocket server. Then you can compare AutobahnPython with Einaros primarily based on non-functional characteristics, like protocol compliance, security and performance.
Autobahn has best-in-class protocol compliance. I dare to say that, since the Autobahn project also provides the quasi industry standard WebSocket testsuite - used by most projects - including Einaros. Autobahn has 100% strict passes on all tests. Einaros probably also - I don't know.
Performance: yes, a single AutobahnPython based WebSocket server (4GB RAM, 2 cores, PyPy, FreeBSD in a VirtualBox VM) can handle 200k connected clients. To give you some more data points: here is a post with performance benchmarks on the RaspberryPi.
In particular, this post highlights the most important (IMO) metric: 95%/99% quantile messaging latency. You shouldn't look only at average latency, since there can be big skews and massive outliers. What you want is consistent low latency.
Achieving consistent low latency is non-trivial. E.g. one factor for languages/run-times like NodeJS or PyPy (a JITted Python implementation) is the garbage collector. Every time the GC runs, it'll slow stuff done - potentially introducing large latencies in messaging. I have done extensive benchmarking (unpublished) which indicates that PyPy's incremental GC is very good in this regard. Better than HotSpot (JVM) and NodeJS (Google V8). When in doubt, and since I haven't (yet) published numbers, you shouldn't believe me, but measure yourself.
The one thing I'd strongly recommend: don't rely on average latency, measure quantiles, do histograms.
Disclose: I am original author of Autobahn and work for Tavendo.
I've recently begun hearing a great deal about Node.js. While I understand a few things that it's able to do, namely create a web server, I'm wondering if you could provide me with a solid reasoning behind using this functionality in a production environment.
This article made a solid case for using Node.js as a development tool, essentially stating that it can provide the usability of a backend environment without needing backend experience. I've also heard of many developers using Node.js in production projects (Codepen for example), but I simply don't understand how it can be useful when there are established web servers out there such as a LAMP stack.
I would love to invest the time into learning about Node.js due to its growing popularity, but because I do have a fairly solid backend foundation, I'm curious if it's necessary.
In short, what are some of the most useful aspects of Node.js and why would someone need to use in a production environment?
Perhaps the best answers can come from large companies who are using Node successfully in production.
Why Walmart is using Node.js
How we build eBay's first Node.js application
How LinkedIn used Node.js and HTML5 to build a better, faster app
Scalability is often the big reason. End to end javascript is also a popular answer. Also Node is good at making some potentially complicated "real-time" scenarios almost trivial when using a library like socket.io.
And here is a list of companies using Node with a short blurb on what they use it for.
The main company behind Node.js, Joyent, is tackling this question head-on with their recent roadshow "Node on the Road". They have collected a number of interesting testimonials of why big companies (Dow Jones, Walmart, Yahoo, etc) are using node in production. Check out their videos section for more details.
At my company we use Node in production everywhere. We typically serve the static HTML and JS files using NGINX but smaller apps will use PM2, a Node process manager, to add clustering redundancy and auto restart to a project. PM2 documentation
Example of using PM2 to maximize clusters for your Node app:
pm2 start -i max myNodeServer.js
I want to write one web application with node.js and MongoDB and I have got task to even test it. I would like to know if there are any tools like JMeter or anything else for load/stress testing of Node.js?
EDIT
My application is going to be information extraction kind of application and client expects extraction should not take more than 10 seconds for one document. Currently I have same application written in C# but its not scaling upto client's expectations. Then I came across this beautiful and fast Node.js. I think Node.js can help me alot.
Please enlighten !!!
Try nodeload: it's a collection of node.js modules for load testing HTTP services.
As a developer, you should be able to write load tests and get
informative reports without having to learn another framework. You
should be able to build by example and selectively use the parts of a
tool that fit your task. Being a library means that you can use as
much or as little of nodeload as makes sense, and you can create load
tests with the power of a full programming language. For example, if
you need to execute some function at a given rate, just use the
'nodeload/loop' module, and write the rest yourself
Just found out that this package is no longer under development so here are some active forks:
https://github.com/gamechanger/nodeload
https://github.com/Samuel29/NodeStressSuite
Why couldn't you test a node server with JMeter? For most load tests it doesn't matter what language your server is, you're just hitting it with a bunch of requests.
In any case, you could try loadtest which is implement in node.
Runs a load test on the selected HTTP or WebSockets URL. The API allows for easy integration in your own tests.
Edit:
This answer provides more options:
NodeJs stress testing tools/methods [closed]
Try artillery. Here are its features, the description of which is taken from the documentation:
Multiple protocols: Load test HTTP, WebSocket, Socket.io, Kinesis, HLS and more.
Scenarios: Support for complex scenarios to test multi-step interactions in your API or web app (great for ecommerce, transactional APIs, game servers etc).
Load testing & Functional testing: reuse the same scenario definitions to run performance tests or functional tests on your API or backend.
Performance metrics: get detailed performance metrics (latency, requests per second, concurrency, throughput).
Scriptable: write custom logic in JS, using any of the thousands of useful npm modules.
Integrations: statsd support out of the box for real-time reporting (integrate with Datadog, Librato, InfluxDB etc).
Extensible: write custom reporters, custom plugins, custom protocol engines etc.
and more! HTML reports, nice CLI, parameterization with CSV files.
As stated in the title, I would like to know if it's safe to develop a website using one of the actuals "omg" platforms that are Node.js and Ringo.js at their actual version.
Also, I would like to know if they support cookies/sessions and how do they deals with multi-fields post (fieldname[] in PHP).
Thank you
--Edit--
Thanks for all the links guys.
What can you tell me about Ringojs ?
Since I haven't figured which platform to start playing with. I must admit that the fact it can use Java seamlessly really impress me. The only available XSLT 2.0 library is in Java. I could use it as a templating system.
Is there anyone who had the chance to play with Ringojs?
From my experience using both, Ringo is more stable and "safer" for production use but you can comfortably deploy both. In addition to the ability to wrap existing Java libraries that you mention, you also get the benefit of being able to run it in an existing webapp container which manages the lifecycle of the application for you and ensures its availability.
That being said, it doesn't have to be an either or decision. By using my common-node package and assuming you don't use any Java libraries, it's perfectly feasible to maintain a project that runs on both without any changes to the code.
I've also included benchmarks that test the performance of Node.js vs. RingoJS the results of which you can find in the common-node/README.md. To summarize: RingoJS has slightly lower throughput than Node.js, but much lower variance in response times while using six times the RAM with default Java settings. The latter can be tweaked and brought down to as little as twice the memory usage of Node with e.g. my ringo-sunserver but at the expense of decreased performance.
Node.js is stable, so yes it's safe to use. Node.js is capable of handling cookies, sessions, and multiple fields but are not as easy to manage. Web frameworks solve this problem.
I recommend Express.js, it's an open-source web framework for Node.js which handles all of this and more.
You can download it here:
https://github.com/visionmedia/express
I hope this helped!
Examples of some of the bigger sites running Node.js
https://www.learnboost.com/
http://ge.tt/
https://gomockingbird.com/
https://secured.milewise.com/
http://voxer.com/
https://www.yammer.com/
http://cloud9ide.com/
http://beta.etherpad.org/
http://loggly.com/
http://wordsquared.com/
Yes. It is. https://github.com/joyent/node/wiki/Projects,-Applications,-and-Companies-Using-Node and https://github.com/joyent/node/wiki/modules
cookies/sessions/forms etc http://expressjs.com/ makes it easier
Ringojs is a framework developed by Hannes Wallnöver and uses rhino as it's scripting framework. There are webframeworks, templating-engines, orm-packages and many many more things already available. Have a look at the tutorial featuring a good subset of packages you may use for a simple web-application. It's not too long and straightforward.
Even thought some of those packages used within the tutorial (e.g. ringo-sqlstore]) are marked as 0.8 and come with the hint "consider this being beta" they are already very stable and bugs - if you find one - get fixed or commented on very fast.
And the power of uncountable java-libraries out there is at your fingertips - so if you already have java-knowledge this knowledge isn't wasted. Rhino - the scripting-engine - even enables you to implement interfaces and extend classes. It is possible a little more advanced but i've done it and i know of packages taking advantage of such features (like ringo-ftpserver which is a wrapper around Apache FtpServer written in java)
Another pro for me is - because ringojs is based on java - it works fairly well with multithreading with ringo/worker for example.