I would like to use Locust for UI performance testing. How to use Locust for UI performance testing? How can I get the loading time of the HTML elements(img, lists, etc..)?
Thanks
Locust isn't a browser and doesn't parse HTML. It just does plain HTTP requests and it will not load things like images based on the response.
If you need something like that, you would need to parse the HTML in the response and do the "dependent" requests in your test script.
Locust is not made for that (as said). There are some other fancy tools which will allow to do it for you e.g:
k6.io (https://k6.io/ - previously known as LoadImpact) - allows you to perform performance checks outside of your environment and report it back to the pipeline with results. Easy to configure and integrate, great when it comes to more "clever" testing scenarios such as stress tests, load tests etc.
sitespeed.io (https://www.sitespeed.io/) - my 2nd favorite, very fun to use and easy to configure tool to track FE performance and tests (e.g. done with Selenium)
Lighthouse Reports - might be also performed as a "pointer" to the most common issues and included as a PR comments e.g. or notifications during the process (there are many Github Actions or DevOps packages doing it)
I've also gathered some of my findings in my recent talk (slides below) and is converted into the series of blogs around these topics and first of them is already published:
Slide deck from my talk on "Modern Web Performance Testing": https://slides.com/zajkowskimarcin/modern-web-performance-testing/
First blog from the series on the same topic: https://wearecogworks.com/blog/the-importance-of-modern-web-performance-testing-part-1
Related
I am trying to figure out how (or even if) I can replace the API calls made by my app when running a Webdriver test to return stubbed output. The app uses a lot of components that are completely dependent on a time frame or 3rd party info that will not be consistent or reliable for testing. Currently I have no way to test these elements without using 'run this test if...' as an approach which is far from ideal.
My tests are written in C#.
I have found a Javascript library called xhr-mock which seems to kind of do what I want but I can't use that with my current testing solution.
The correct answer to this question may be 'that's not possible' which would be annoying but, after a whole day reading irrelevant articles on Google I fear that may be the outcome.
WebDriver tests are End to End, Black Box, User Interface tests.
If your page depends on an external gateway,
you will have a service and models to wrap that gateway for use throughout your system,
and you will likely already be referencing your models in your tests.
Given the gateway is time dependent, you should use the service consumed by your api layer in your tests as-well, and simply check that the information returned by the gateway at any time is displayed on the as page as you would expect it to be. You'll have unit tests to check the responses model correctly.
As you fear, the obligatory 'this may not be possible': Given the level of change your are subject to from your gateway, you may need to reduce your accuracy or introduce some form of refresh in your tests, as the two calls will arrive slightly apart.
You'll likely have a mock or stub api in order to develop the design, given the unpredictable gateway. It would then be up to you if you used the real or fake gateway for tests in any given environment. These tests shouldn't be run on production, so I would use a fake gateway for a ci-test environment and the real gateway for a manual-test environment, where BBT failures don't impact your release pipeline.
How do I test the behavior of a java restful web service in case of multiple concurrent requests? Is there any 3rd party tool that can be leveraged?
The service accepts POST method. It expects a couple of parameters in it's request body and produces the response in the form of JSON.
The functionality of the service is to perform database read operations using the request body parameters and populate the fetched data in the JSON.
I would recommend one of the following:
SoapUI - superior tool for web service testing. Has limited load testing capabilities. However it does not scale (no clustered mode is available) and has quite poor reporting (all you get is average, min and max response times)
Apache JMeter - multiprotocol load testing tool, supports web services load testing as well. Has better load capabilities and ways to define the load patterns and can represent load test results via HTML Reporting Dashboard. Check out Testing SOAP/REST Web Services Using JMeter article to learn how to conduct a web service load test using JMeter.
You can try Gatling to generate some load.
It has nice documentation and easy QuickStart .
For advanced usage it requires some knowledge of Scala, but it also features GUI tool for simple scenarios recording, so you can run some scripts by postman or whatever browser tool you use for debugging, record it and make that scenario automated.
After running scenarios it generates nice reports using Graphite, so you can see response times and general stats.
Later you can also use Gatling for load and performance tests of your web service, it's convenient and fast as soon as you start playing with it. It can easily generate up to 5k requests per second from my old Mac, or hold up to 1k connections.
One of the bests tools to test web services is SOAPUI.
You can use it for what you want.
Link to SOAPUI
You can check this link to see how to use SOAPUI and concurrent tests.
I would like to script benchmark of my socket.io implementation.
After some research I have identified several NodeJS modules, but they have either not been updated for past years (wsbench), or are only supporting websocket protocol (wsbench, thor) or is not testing socket.io implementation but socket.io project (socket.io-benchmark).
Since socket.io project has been highly active the past year, I wonder what is the latest and greatest tool/module to use for benchmarking?
My requirements:
Easy to script and run the tests
Test reports giving good overview of test runs
Test reports should be easy to save in order to compare with later benchmarking
Just came across this in search of some benchmarking for my Socket.IO project.
I found socket.io-benchmark, however had some additional items that I wanted to work through but found one of the forks nearly there.
https://github.com/slowthinker/socket.io-benchmark
I also forked it to add a cap on messagse/second sent to give it more realistic parameters.
Hope that helps!
I would suggest Artillery: Artillery is a modern, powerful, easy-to-use, open-source load-testing toolkit: https://github.com/shoreditch-ops/artillery
Here some feature:
Mulitple protocols: Load-test HTTP, WebSocket and Socket.io applications
Scenarios: Specify scenarios to test multi-step interactions in your API or web app
Perfomance metrics: get detailed performance metrics (latency, requests per second, concurrency, throughput)
Scriptable: write custom logic in JS to do pretty much anything
High performance: generate serious load on modest hardware
Integrations: statsd support out of the box for real-time reporting (integrate with Datadog, Librato, InfluxDB etc)
Extensible: custom reporting plugins, custom protocol engines etc
and more! HTML reports, nice CLI, parameterization with CSV files
I want to write one web application with node.js and MongoDB and I have got task to even test it. I would like to know if there are any tools like JMeter or anything else for load/stress testing of Node.js?
EDIT
My application is going to be information extraction kind of application and client expects extraction should not take more than 10 seconds for one document. Currently I have same application written in C# but its not scaling upto client's expectations. Then I came across this beautiful and fast Node.js. I think Node.js can help me alot.
Please enlighten !!!
Try nodeload: it's a collection of node.js modules for load testing HTTP services.
As a developer, you should be able to write load tests and get
informative reports without having to learn another framework. You
should be able to build by example and selectively use the parts of a
tool that fit your task. Being a library means that you can use as
much or as little of nodeload as makes sense, and you can create load
tests with the power of a full programming language. For example, if
you need to execute some function at a given rate, just use the
'nodeload/loop' module, and write the rest yourself
Just found out that this package is no longer under development so here are some active forks:
https://github.com/gamechanger/nodeload
https://github.com/Samuel29/NodeStressSuite
Why couldn't you test a node server with JMeter? For most load tests it doesn't matter what language your server is, you're just hitting it with a bunch of requests.
In any case, you could try loadtest which is implement in node.
Runs a load test on the selected HTTP or WebSockets URL. The API allows for easy integration in your own tests.
Edit:
This answer provides more options:
NodeJs stress testing tools/methods [closed]
Try artillery. Here are its features, the description of which is taken from the documentation:
Multiple protocols: Load test HTTP, WebSocket, Socket.io, Kinesis, HLS and more.
Scenarios: Support for complex scenarios to test multi-step interactions in your API or web app (great for ecommerce, transactional APIs, game servers etc).
Load testing & Functional testing: reuse the same scenario definitions to run performance tests or functional tests on your API or backend.
Performance metrics: get detailed performance metrics (latency, requests per second, concurrency, throughput).
Scriptable: write custom logic in JS, using any of the thousands of useful npm modules.
Integrations: statsd support out of the box for real-time reporting (integrate with Datadog, Librato, InfluxDB etc).
Extensible: write custom reporters, custom plugins, custom protocol engines etc.
and more! HTML reports, nice CLI, parameterization with CSV files.
I have an Web application which I have to measure the Performance from the Server response of that application while testing it manually in several machine.
How can I do it. Do we have any tool to measure the Server Response time of a Web Application while performing the manual testing on several machine.
Thanks,
Udhay S
Udhay,
JMETER provides you all necessary performance graphs from which you can get the server response. Its easy to use and you can increase the load to test the server response.
In JMETER you have a list of reports - Aggregate graph, Spline Visualizer graph and a summary report explaining the server response.
A stopwatch? Seriously, if you are running manual tests then the best, most accurate and the most purely subjective way to record how long it appears to take for a page to load is to use a stopwatch wired to the human brain. Until someone can invent a computer that can pass a Turing test this is as good as it gets.
If you feel like you need to automate things then automate the test first, and then think about automating recording response time.