In Kubernetes setup where Gatling execution and server side lies in same cluster we get exact number of requests for each endpoint passed on provided user in setup.
However in Pre-prod setup where cluster is different for Gatling execution and Server side code, getting lesser number of requests made as per Gatling report.
For Example for user load of 30 we would get random number of requests for each endpoints of parallel running scenarios like it may be 27 calls for an endpoint A, 25 calls for endpoint B and least being around 10 calls for some endpoints. Also KO is 0 for all them as per report. so what happens to the remaining calls? Also we have provided sufficient resource for Gatling execution like 4 core CPU and 4Gi memory.
Adding code snippet to inject user load
setUp(
// parallel execution of scenarios
new myScenario().scenario1.inject(rampUsers(30).during(30 seconds)),
new myScenario().scenario2.inject(nothingFor(1 second), rampUsers(30*3).during(60 seconds)),
new myScenario().scenario3.inject(nothingFor(1 second), rampUsers(30*2).during(60 seconds)),
new myScenario().scenario4.inject(nothingFor(1 second), rampUsers(30).during(30 seconds))
)
Respective Report:
Related
so my requirement is to run 90 concurrent user doing mutiple scenario (15 scenario)simultenously for 30 minutes in virtual macine.so some of the threads i use concurrent thread group and normal thread group.
now my issue is
1)after i execute all 15 scenarios, my max response for each scenario displayed very high (>40sec). is there any suggestion to reduce this high max response?
2)one of the scenario is submit web form, there is no issue if submit only one, however during the 90 concurrent user execution, some of submit web form will get 500 error code. is the error is because i use looping to achieve 30 min duration?
In order to reduce the response time you need to find the reason for this high response time, the reasons could be in:
lack of resources like CPU, RAM, etc. - make sure to monitor resources consumption using i.e. JMeter PerfMon Plugin
incorrect configuration of the middleware (application server, database, etc.), all these components need to be properly tuned for high loads, for example if you set maximum number of connections on the application server to 10 and you have 90 threads - the 80 threads will be queuing up waiting for the next available executor, the same applies to the database connection pool
use a profiler tool to inspect what's going on under the hood and why the slowest functions are that slow, it might be the case your application algorithms are not efficient enough
If your test succeeds with single thread and fails under the load - it definitely indicates the bottleneck, try increasing the load gradually and see how many users application can support without performance degradation and/or throwing errors. HTTP Status codes 5xx indicate server-side errors so it also worth inspecting your application logs for more insights
DISCLAIMER: If this post is off-topic to this site, please recommend a site where this post would be appropriate.
On Ubuntu 18.04, in bash, I am writing a network-based, threaded application that requires multiple servers. It receives files through the network and processes them, ultimately making an API call that finishes the processing and logs the results to a database for later retrieval and reporting.
So far I have written the application using non-threaded programming models and concepts. That means the files are processed one at a time in real-time. This works great if there is no sudden burst of files and/or a backlog of files to process. The main bottle neck has been the way I sequentially send files to the API one after another, waiting until the entire operation has taken place for one file and the API returns the results. The API has a rate limit of 8 calls per second. But since each call takes from .75 to 1 second, my program waits until the operation is done and only processes about 1 file per second through the API. In short, I did not have to worry about scheduling API calls because I could barely do one call per second.
Since the capacity is there to process 8 files per second, and I need more speed, I have been converting my single-threaded, sequential application into a parallel, scalable, multi-threaded application. This new version can spawn enough threads to send 8 files per second to the REST API and much more. So now I have the opposite problem. I am sending too many requests per second to the REST API and am in danger of triggering penalties, etc. Ultimately, when my traffic is higher, I will upgrade my subscription to the API and get more calls per second, but this current dilemma has got me thinking about how to schedule the API calls with different threads.
The purpose of this post is to discuss an idea about how to schedule these REST API calls across various threads. Specifically, I want to discuss how to coordinate timing and usage of the API while maintaining efficiency and yet not overloading the API. In short, I want to coordinate a group of threads so that the API is properly used. Not too fast and not too slow.
Independent of my application, this idea could be useful in a number of generically similar scenarios.
My idea is to create an "air traffic controller" ("ATC") so that the threads of the application have a centralized timing authority to check when they are ready to submit files to the REST API. The ATC would know how many time slots/calls per time period (in this case, calls per second) the API can schedule. The ATC would be listening for the threads to request a time slot ("launch code") which would give them a time slot in the future to perform their API call. The ATC would decide based on the schedule of other launch codes that it has already handed out.
In my case, from the start of the upload of the file to the API, it could take 0.75 to 1 second to complete the processing and receive a response from the API. This does not affect the count of new API calls that can be performed. It is just a consideration of how long the threads will be waiting once they call the API. It may not be relevant to this overall discussion.
Each thread would obviously have to do some error handling. If the API timed out or threw an error, then the thread would have to handle it and get back in line with the ATC -if appropriate- and ask for a new launch code. Maybe it should report the error to the ATC for centralized logging?
In situations where the file processing needs burst above 8 files per second, there would be a scheduling backlog where the threads should wait their turn as assigned by the ATC.
Here are some other considerations:
Function
The ATC would be a lightweight daemon that does the following:
- listens on some TCP port
- receives a request
security token (?), thread id, priority
- authenticates the request (?)
- examines schedule
- reserves the next available time slot
- returns the launch code
security token (?), current time, launch timing offset to current time, URL and auth token for the API
- expunged expired launch codes
The ATC would need the following:
- to know what port it is supposed to run on
- to know how many slots per time period it was set to schedule
(e.g. 8 per second)
- to have a super fast read/write access to the schedule (associative array?)
- to know the URL and corresponding auth token for the thread to use
- maybe to know multiple URLs and auth tokens for load balancing
Here are more things to consider:
Security
How could we keep the ATC secure while ensuring high performance?
Network-level security (e.g. firewalls allowing only the IP addresses of the file-processing servers?)
Auth tokens or logins and passwords?
Performance
What would the requirements be for this ATC server? Would this be taxing to a CPU and memory?
Timing
How often would an NTP call be needed? By the ATC server? By the servers which call the API?
Scalability
Being able to provide different URLs and auth tokens would allow the ATC to load balance with different API providers.
Threading of the ATC itself
Would the ATC need to spawn threads to be able to handle each new request?
How does a web server handle requests?
How would the various threads share a common schedule?
In a non-threaded environment, the ATC would possibly keep an associative array in memory to keep performance as high as possible. How would the various threads of the ATC have access to the same schedule?
So here is my question. Does this exist? If not, what are some best practices in trying to build the above?
It seems like a beanstalkd kind of network service except it only provides permission/scheduling and is extremely dependant on timing.
I am using Spring Boot and Java 8
For calling an api with 1 employee id it takes 1 miliseconds .So if I am calling API 100,000 times with 100,000 times with different employee id
why it is taking hours and not 100,000*1 millis i.e just 1.6 minutes
SpringBoot uses a thread pool to manage the workload for working on tasks. Thus, the max worker threads, is set as 200 by default.
Though this is a good number, the number of threads that can work in parallel depends upon the CPU time slicing and availability of backend resources. Assuming, that the backend resources are unlimited, the throughput will solely depend upon the CPU time available for each thread. In a multi-core CPU, it would be the maximum cores available and are able to serve the embedded tomcat container.
As Spring is a blocking framework, for a normal quad-core single CPU environment (assuming that all 4 cores are able to serve), this no. is 4. This means a maximum of 4 requests can be served in parallel. Rest all are likely to be queued and taken up when the next CPU slice is available.
Mathematical analysis:
Time taken by the API to process 1 request = 1ms
Time is taken by the API to process 4 concurrent requests = 1ms
Time taken by the API to process 1000,000 concurrent requests = 1000000 / 4 = 250 secs
This is just the best-case scenario. In real scenarios, all the CPUs are less likely to provide a time slice at the same instant. So, you are likely to see differences.
In such scenarios, it would be better to use the Spring Reactive than the conventional Spring framework in SpringBoot.
The API you're pulling from could be limit the amount of requests you can pull in a certain period of time. If you don't have access to the API source, I would attempt to run larger and larger numbers of pulls until you notice it takes significantly longer.
Well the time needed to get response from a web server depends on it's hosting machine and environment.
Usually in a single machine a limited number of thread inside a thread pool and each request is bound with one thread. So while making concurrent request every time certain number of request is processed within the available threads and rest awaits in the queue.
This can be the reason that your requests are taking a while to get response or even some of them can get a request time out.
I have a situation where I create a Node.js cluster using PM2. A single request fired at a worker would take considerable time (2+ mins) as it's doing intensive computations (in a pipeline of steps) with a couple of I/O operations at different stages (step 1 is 'download over HTTP', an intermediate and last step are 'write to disk'). The client that sends requests to the cluster throttle the requests it sends by two factors:
Frequency (how many requests per second), we use a slow pace (1 per second)
How many open requests it can make, we make this less than or equal to the number nodes we have in the cluster
For example, if the cluster is 10 nodes, then the client will only send 10 requests to the cluster at a speed of 1 per second, and won't send any more requests until one or more requests returns with either success or failure, which means that a worker or more should be free now to do more work, then the client will send more work to the cluster.
While watching the load on the server, it seems that the load balancer does not distribute work evenly as one would expect from a classic round-robin distribution schema. What happens is that a single worker (usually the 1st one) will receive a lot of requests while there're free workers in the cluster. This eventually will cash the worker to malfunction.
We implemented a mechanism to prevent a worker from proceeding with new requests if it's still working on a previous one. This prevented malfunctioning, but still, a lot of requests are denied service although the cluster has vacant workers!
Can you think of the reason why this behavior is happening, or how can improve the way PM2 does work?
In my application when i execute 2000 virtual users in thread(No: of threads) for 1 http request my response time was 30 sec , when i changed no of threads to 500 and instead of 1 http request I put 4 copies of same http request, RESPONSE TIME WAS 3 SEC . What is the difference? Is it the right way to reduce no of threads and increasing replicas of request? please help
Note: In each reqest i have changed the user id also
In terms of HTTP Request samplers your test must behave exactly like real browser behaves so artificially adding more HTTP Requests may (and will) break the logic of your workload (if it is in place).
In your case high response time seems to be caused by incorrect JMeter configuration, i.e. if JMeter is not properly configured to high load it simply will not be able to fire requests fast enough resulting in increased response time while your server will just be idle.
2000 threads sounds like quite a big number so make sure to:
Follow JMeter Best Practices
Follow recommendations from 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure especially these:
Increase JVM Heap size allocated for JMeter
Run your test in non-GUI mode
Remove all the Listeners from the Test Plan
Monitor baseline OS health metrics on the machine where JMeter is running (CPU, RAM, Disk, Network usage). You can use JMeter PerfMon Plugin for this. If you will notice the lack of any of the aforementioned resources, i.e. usage will start exceeding, say, 90% of total available capacity - JMeter is not acting at full speed and you will need to consider Distributed Testing.
To extend #Dmitri T answer, If your server response 10 times more on load, as you execute 2000 virtual users, it means there's a bottleneck that you need to identify.
Read JMeter's Best Practices
consider running multiple non-GUI JMeter instances on multiple machines using distributed mode
Also check Delay Thread creation until needed checkbox in Thread Group
JMeter has an option to delay thread creation until the thread starts sampling, i.e. after any thread group delay and the ramp-up time for the thread itself. This allows for a very large total number of threads, provided that not too many are active concurrently.
And set Thread Group Ramp-up to 2000
Start with Ramp-up = number of threads and adjust up or down as needed.