I have a website in IIS 8.5.9600.16384, we communicate with thousands of mobile devices through cyclic synchronisation and through SignalR 2.3.0.
This morning we had an application pool reset during working hours, which caused the SignalR to call "OnReconnect" of all our mobile devices at the same time.
I though that IIS started new processes first and then killed the old, not having downtime.
Can somebody tell me exactly what happens when IIS recycles it's application pool on the SignalR side? And in which cases can there be a connection downtime? (ex : if the server is busy?)
Edited : The application pool was recycled by IIS because of the "time limit". The IT team will change this setting so that the application pools reset every day at night time when it will have a lower impact on our applications.
A worker process with process id of '8720' serving application pool 'DefaultAppPool' has requested a recycle because the worker process reached its allowed processing time limit.
Also confirmed that disallowOverlappingRotation is not set to True. Any hint would help.
A few years later, I'm still getting some problems with the application pool recycle and SignalR. We are occasionally seeing thousands of re-connections of SignalR while the application pool recycle occurs, opening more than 60k TCPIP ports and causing a crash in IIS.
We managed to have it run "okay" for quite some time but it still crashes. Any hint would help. thanks
I'd first identify how IIS was reset. If you experienced a crash or performed an IISReset, the processes would be down before a new one stood back up. If on the other hand you configured AppPool recycling, then the overlapping processes should occur as you mention. I would check the System Event Log for recycling messages. Note that not all recycle reasons are logged by default.
You may also check to make sure disallowOverlappingRotation is not set to True.
Specifies whether the WWW Service should start another worker process to replace the existing worker process while that process is shutting down. The value of this property should be set to true if the worker process loads any application code that does not support multiple worker processes.
https://learn.microsoft.com/en-us/iis/configuration/system.applicationhost/applicationpools/add/recycling/
Related
I have an application pool configured to be have maximum 1 worker process.
Other settings on the pool:
.NET : v4.0
32bit : True
StartMode : AlwaysRunning
Identity : domain\account
Idle Time-out : 0
Disable Overlapped Recycle : False
Specific Time Recycle : 02h00
This morning I noticed that the CPU of the server is on 100% constantly.
2 weeks ago the same problem happened.
I killed the process ID that was still on running, and a bit later there was only 1 pool left.
If I look to the worker processes, I see that there is one on state "running".
It looks like the new worker was started after recycle. And it is now waiting for the old one to shutdown.
But it's stuck somewhere since 24/02 ~ 25/02.
In Process Explorer on the process start timestamps. I see that the second and third worker started on the 25th.
The weird is here that the 2nd is 3 seconds before the recycle config value. And the 3rd is exact on the recycle time.
And since then each day happens the same. At this moment there are 29 exe's instead of 1 for this pool.
In the EventViewer I have found no warnings/errors related to this application pool.
When I do a manual Recycle, the issue does not happen.
I have two web servers with a loadbalancer in front. And on both the servers the same issue happened.
I did some googling, but I have not found anything similar to this.
In the pools there are multiple WCF services running.
This is installed on customers servers in their own network.
But we have this at 50+ customers, and we only had this issue at this customer.
Windows Server 2016 Standard, 1607 (14393.3443)
IIS Version 10.0.14393.0
I've a web application (MVC) that performs processes in different time intervals.
However, after a period of inactivity, the application "dies".
I guess that "kills" IIS.
What should I do that applications live permanently?
Is it possible that kills something else?
Each web application is executed by an app pool. App pools have an idle time, when that time is reached, the app pool shoots down until there is another request. You could set the idle time to a higher value.
https://technet.microsoft.com/en-us/library/cc771956
That say, it's a bad practice to have long running process in Asp.Net application. You should create a Windows Service for that.
I have a web application hosted under IIS. It is a data warehouse, and during its startup process it requires instantiating a large set of items in memory (takes roughly ~20 minutes to fully set up). Because this website is critical to our company, this system must be online 100% during the daytime, and only can be restarted during off-work hours.
For some reason, this web application seems to be "offline" when there is no usage for some time. I know this because the cache is not fully instantiated when the website is visited. This is unacceptable.
It is not clear to me why the website is shut off. The application pool is only set to recycle daily at 4:00 AM (it is 11 AM now).
Are there other settings which I'm not aware of on the IIS part that causes it to shut down the website automatically?
Addl Note: The website does not shut off automatically when running in IISExpress in Visual Studio. Only the production version hosted on IIS shuts off.
Here's a screen of the Advanced Settings for the Application Pool the web site is running under. (Not sure if it's useful.)
I'm on IIS 7.5, Server 2008 R2. It is an ASP.NET 5 Web App.
Check Idle Time-out settings under process model in screenshot. That setting is causing app pool shutting down when remain idle for 20 mins. You can set it to 0 to keep it running all time even when its idle i.e. not processing any requests.
Note: Keeping app pool running all time will consume server's precious memory. It may become critical especially if application is leaking memory.
I have a large web app that runs on our two live servers. Part of our server side C# code calls a third party app to do a task for us.
That task works most of the time, but at a certain point it stops working until the AppPool is recycled.
This all happens in w3wp.exe, so I can see it running in process monitor like this (when it is not working),
Thread Create
Access the file PreviewGenerator.exe
Hive unloaded (this is the registry)
Thread Exit
And like this when it is working,
Thread Create
Access the file PreviewGenerator.exe
Process Start
Does heaps of stuff with PreviewGenerator.exe including reading / writing / registry, etc.
Process Exit
Hive unloaded
Thread Exit
How can I debug what is going on in my AppPool and why starting a separate process is not working some of the time?
I found the best thing to do was to create a separate app pool for my application in IIS and set an upper limit for the amount of RAM it could use. Also I found it useful to turn on the 'Generate Recycle Event Log Entry' items under the app pool settings.
You can then go to the system event log and filter out the items with a source of 'WAS' to understand what is going on in the app pools, when they are restarting and when they stop from being idle etc.
I think the main problem in our case is that the IIS box was running out of memory. Tuning the app pools and adding some extra RAM seems to have solved it.
In my IIS log, I found warning level event ID 1013, which says the stop time exceeds expected stop time for worker process of a specific web application.
My question is, how could I know or track from what reason IIS worker process stops? Does this warning level event ID means worker process application pool is stopped or not?
thanks in advance,
George
If you application is an ASP.NET (2.0 or newer) you can turn on health monitoring which shoudl record details including IIS applciation pool resets.
Useful links
MSDN
4GuysFromRolla