Good Afternoon Everyone,
I am load testing my .NET Web API which is hosted on a Windows 2008 Server virtual machine. I am using Visual Studio 2012 Load Test. However, once my load test reaches 780 concurrent users, the CPU % starts to decrease as shown in the image attached. The load test reaches a maximum of 1000 concurrent users but the CPU % is still decreasing at the highest user load. I cannot explain why. Is any kind of IIS limit being reached? Why does this occur? Is the maximum user load reached for this function?
Just looking for an explanation to this result and some guidance.
Thank you
IIS does have separate output cache settings which are enabled by default which does start to make sense after considering how it handles dynamic content with static response and cache worthiness:
The IIS output caching feature targets semi-dynamic content. It lets
you cache static responses for dynamic requests and increase
scalability
Configuring Cache Worthiness:
Even if you enable output caching, IIS does not immediately cache a
request. It must be requested a few times before IIS considers a
request to be “cache worthy.” Cache worthiness can be configured via
the ServerRuntime section.
Two properties determine cache worthiness:
frequentHitTimePeriod
frequentHitThreshold
A request is only cached if more than frequentHitThreshold requests for a cacheable URL arrive within the frequentHitTimePeriod.
This was a good explanation: http://www.iis.net/learn/manage/managing-performance-settings/configure-iis-7-output-caching
Related
Our IIS website has its Maximum Concurrent Connections set to 4294967295. Our Web API application is logging all the requests it serves to Application Insights and the two do not appear to match up. A call which appears to get served quickly in Insights does not appear to complete quickly in IIS's logs.
What could cause this and is this an indication that IIS is running out of connections, even if the maximum is set ridiculously high?
Phrasing this another way (after reading #zakima's comment): What should I be looking for to identify requests which are getting delayed in IIS before or after they hit the application itself?
Maximum concurrent connections defaults to 4294967295, which is a staggering number. But it does not mean that the site can have the ability to execute 4294967295 concurrent connections.
Assuming that 4294967295 concurrent connections come at the same time, IIS does not immediately start 4294967295 threads to process, because this is unrealistic. For the processing of connections, IIS has the "Maximum concurrent worker threads" limit. From some sources, this number is related to the operating system. If IIS can only start 10 worker threads in the first time to process, then the other 4294967285 must queue.
In another word, 4294967295 means the maximum amount of allowed by default concurrent connections from http.sys module to the site. Then these request will hit each module of IIS and hit application at last.
If you want to check the real max concurrent connections of IIS, please refer to this article to use Performance Monitor.
Regard to how to monitor the request getting delayed in IIS before or after, I suggest you use failed request tracing. Here is the sample of failed request tracing log of my asp.net application.
I have simple html page with two input fields no css, design nothing. Page size is 134KB. In my performance testing test case I am only trying to load page with 25 concurrent users all hitting at once. I have performed test with both Jmeter and Junit(multithread). The server CPU usage reaches to 100% when all threads are up. Is this a normal behavior? or is it an issue? Why does it happen? I have replicated same scenario with another page on same server and CPU usage is same. With 10 concurrent user CPU usage is 30 to 75%. I am new to performance testing.
its normal and depends on the server RAM/Cores, if it is simple static site , enable static file caching, specify the stack you are using , so that you can get the steps to do so.
I am running a load test using JMeter on my Azure web services.
I scale my services on S2 with 4 instances and run JMeter 4 instances with 500 threads on each.
It starts perfectly fine but after a while calls start failing and giving Timeout error (HTTP status:500).
I have checked HTTP request queue on azure and found that on 2nd instance it is very high and two instances it is very low.
Please help me to success my load test.
I assume you are using Azure App Service. If you check the settings of your App, you will notice ARR’s Instance Affinity will be enabled by default. A brief explanation:
ARR cleverly keeps track of connecting users by giving them a special cookie (known as an affinity cookie), which allows it to know, upon subsequent requests, to which server instance they were talking to. This way, we can be sure that once a client establishes a session with a specific server instance, it will keep talking to the same server as long as his session is active.
This is an important feature for session-sensitive applications, but if it's not your case then you can safely disable it to improve the load balance between your instances and avoid situations like the one you've described.
Disabling ARR’s Instance Affinity in Windows Azure Web Sites
It might be due to caching of network names resolution on JVM or OS level so all your requests are hitting only one server. If it is the case - add DNS Cache Manager to your Test Plan and it should resolve your issue.
See The DNS Cache Manager: The Right Way To Test Load Balanced Apps article for more detailed explanation and configuration instructions.
I am working on a project which is developed using plain JSF with webservice backend and our target runtime is WAS8.5. It’s an internet application and being used by more that 500 concurrent users.
We are performing NFT testing with 150% load ie. 750+ concurrent users. From Introscope, I could see web container thread pool size is set as min&max value of 50. We feel web container thread pool size is blocking the concurrent loads we are trying to achieve. We are expecting experts suggestion for ThreadPoolModule and webcontainer poolsize settings in our WAS servers. (FYI: We have two WAS servers in our cluster)
I have a intranet SiteCore website set up on IIS 7 which randomly throws the following error message
HTTP Error 503.2 - Service Unavailable
The serverRuntime#appConcurrentRequestLimit setting is being exceeded.
To fix this issue, I have made following changes
Increased the Queue Length of application pool myrjetAppPool from 1000 to 65535.
Modified Machine.Config to increase requestQueueLimit property of ProcessModel element to 100000
Increased appConcurrentRequestLimit to 10000 by running
C:\Windows\System32\inetsrv\appcmd.exe set config /section:serverRuntime /appConcurrentRequestLimit:100000
But I'm still getting the same error. ANy help is greatly appreaciated.
You might check to see where all your threads are going. We had occurrences where threads for Media Library assets were hanging and blocking up the queue.
In IIS Manager, select the server node from the tree, then the "Worker Processes" feature icon, then right-click the application pool of interest and select "View current requests". You might find something is getting stuck. I sometimes hit F5 on this screen a few dozen times in very quick succession to see the rate the requests are going through (of course Performance Monitor is better for viewing metrics but it won't tell you what URLs are being processed).
Investigate references in the linked url to 'MaxConcurrentReqeustsPerCPU' which you may need to set by creating a new registry key, depending on your OS and framework.
https://learn.microsoft.com/en-us/archive/blogs/tmarq/asp-net-thread-usage-on-iis-7-5-iis-7-0-and-iis-6-0
As already commented - check the actual concurrent request count using performance counters to determine which limit you're hitting i.e. it could be a limit of 5000 or maybe 12 (per cpu).
Edit: I realise this may look like I'm talking about a different setting entirely, but I believe there is overlap here.
We got this problem after an installation of an IIS plugin. After long investigating we saw that the config-file C:\Windows\System32\inetsrv\config\applicationHost.config had an extra location tag for the site with the problem. After removing the extra entry and an iisreset, the site/server worked normally againg. So something must went wrong during the installation....