I noticed that when you RDP to a Web Role instance on Windows Azure to make an iisreset, the World Wide Web Publishing Service shuts down, and the only way to get your role up and running again is either by restarting the aforementioned service or re-roll/restart your instance.
For reasons unknown to me, Windows Azure default the start mode of World Wide Web Publishing Service to Manual, why an iisreset sort a leave your Web Role unavailable to the WWW.
I found a solution to this - IMO - odd behavior, and answered it to the original question of this post.
However, is there an alternative to iisreset on Windows Azure - maybe programmatically where I can pinpoint the exact instance? Because that is another issue; now I have to use RDP to each instance .. it would be nice if it was possible to do a pinpoint each instance.
Think about it; i have a CNAME to www.awesome-azure.com; this is hosted by 3 instances in round-robin, and I want to reset/monitor/diagnose/heartbeat each one through a REST API (ir similiar), and not like now - through RDP.
Can this be achieved.
EDIT
Tried to make it more clear what the challenge is as well as the goal to achieve.
EDIT 2
Provided a solution to the iisreset challenge; updated the question to pinpoint instances over the Internet if possible.
Well, I still don't know why Microsoft Azure decides to set World Wide Web Publishing Service to start mode Manual, but I found a way to change it.
To the second part of the original question I still hoping for an answer, but until then, please find my solution for the first part to fix the (IMO) iisreset problem with Startup Task:
In your startup.cmd (or what ever you have named it) which I have placed in a startup folder in the root of my application, include this line of text:
powershell -ExecutionPolicy Unrestricted .\startup\w3svc.ps1
In the same folder, create a PowerShell file named w3svc.ps1 with the following content:
Set-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\Services\W3SVC -Name Start -Value 2
Voila; your IIS now works as expected - World Wide Web Publishing Service is now set to start mode Automatically.
Note: for the above to work, you need to have your osFamily property set to 2 in you ServiceConfiguration.cscfg file.
You can handle the RoleEnvironment.Changing event in WebRole.cs and set the Cancel property on the event arguments object to true. Then you just need to make a change to the configuration settings and Azure will restart all your instances in an orderly fashion.
I don't know why iisreset isn't working. As to your second question, you can use the Service Management API to reboot or reimage an instance. That might do what you want. You could also, of course, write your own code to do whatever you want. (You could have code in your web role that polls a blob called <instance ID>.txt and does an iisreset any time the blob changes.)
Related
[I'm posting this to record what I actually found out after hours of painful trial-and-error.]
I have a website that I need to be "always running" (because in this case it has a Hangfire job that's responsible for kicking off a scheduled task every 5 minutes), and by default, websites are only started up when the first request is received.
So, how can I ensure that the website is started automatically? And, how can I configure this via the TFS release management tool?
[This answer isn't specific to Hangfire, but see the Hangfire documentation's discussion of this issue for details of how it affects Hangfire, but note that the recommended work-around is somewhat involved, and much more complex than the solution below. See also a separate and quite comprehensive discussion on the Hangfire support forum that gives several alternative solutions.]
In IIS, each website is associated with an Application Pool (App Pool). You can configure your App Pool to start automatically via IIS Manager by changing the "Start Mode" to AlwaysRunning in "Advanced Settings" for the App Pool:
However, starting the App Pool doesn't start the website (or websites) associated with it. The website does not get loaded until the first request is received.
In IIS8 (or IIS7.5 with an extension), a new setting was added that allows us to work around this. You can ensure that the website gets sent a request as soon as the App Pool starts by setting "Preload Enabled" to True in "Advanced Settings" for the website:
The combination of these settings ensure that the website will automatically start up when IIS starts, and immediately after the App Pool is recycled, etc.
But, how can you get those settings applied automatically as part of a TFS release pipeline, rather than having to remember to set them manually?
In your release definition, you presumably have an "IIS Web App Management" task, which sets up the App Pool and the website. In the configuration panel for this step, there should be an "Advanced" box with an "Additional AppCmd.exe Commands" entry field. You can use AppCmd to apply the settings described above.
AppCmd has the most confusing command-line syntax I've yet seen outside of code-golf competitions, but here's the incantation that worked for me:
set config /section:applicationPools -[name='myAppPoolName'].startMode:AlwaysRunning
set app "mySiteName/" /preloadEnabled:true
Note that if you have configuration variables defined for your App Pool name and website name, then you can use those rather than hard-coding the name, such as:
set config /section:applicationPools -[name='$(appPoolName)'].startMode:AlwaysRunning
I hope this helps somebody... Thanks for listening :-)
I've setup a UNC share for IIS shared config using a specific AD service account and set to FULL CONTROL. I've also exported the config from one IIS server and set-up an additional IIS server to point to the share. When I open the applicationhost.config for example on the UNC share and remove an application pool, I can see the entry also remove in both IIS servers.
So I know:
1) I can export to the share with the specific service account
2) Both IIS servers can read the config when I edit manually
3) However when I remove an app pool from one of the IIS servers through the manager I get the above error.
I've tried using the process monitor utility to see what account is being used to write to the config and it seems it is my own AD user account rather than the shared service account. I know IIS Manager has my username e.g. ROOT\MYNAME logged on, but I wouldn't have thought it would use this to write changes to the shared config. Surely it would use the service account?
Does anyone know how to prevent this error? Why does the shared config and tied service account not come into play when making changes on one of the servers?
So, IMHO, this error is a red herring. I was publishing to a server and got a message saying I was out of space. So, I logged in, realized there was a bit of cruft in extra apps published in IIS, we didn't need. I right clicked and tried to remove one. I got the same error as you.
Having done some manual changes to applicationHost, I thought it "might be me" but it seemed very odd that editing this file would cause such a thing. However, I had recently learned that windows does some funky 32 vs 64bit machinations with this file (google it).
Deciding I had better things to do, I asked our IT to add space to the VM and guess what? I am no able to remove these apps. My guess is that I was at the end of the line on space and the backend management of these special files was not completing and throwing this not-so-helpful exception.
I'm not a 100% about this. For full disclosure, I will add that updates had been applied recently, but I'm pretty confident that this is a possible solution.
I 've found out that Azure websites (trial version) doesn't autostart my node sever process (it starts only when I load the url in the web browser); and that when there are no requests in a while, the process is killed.
I mean, when I git push my server, I would like it to start running immediately and continuously.
I read (here, for example) that this might have to do with the way iisnode manages azure websites, and that I can't do anything to change it. Is this the actual way Azure websites work? Is there any way I can deal with this?
Thanks in advance,
Bruno.
You've find the answer. There is no other answer.
The process termination because of inactivity comes from IIS - there is Idle Timeout setting. Which to my knowledge is not configurable in Azure Web Sites (at least not Free tier). Check out also this SO question and its answer to get better understanding on why you can't change this timeout on the FREE and STANDARD tiers.
And here is an interesting workaround to avoid this idle timeout. Actually if you use technique, you will also have kind-of "auto start", in terms that when your scheduler hits your site after a new deployment, it will "boot up".
This can get a little complicated, but if you don't want to use their 5-min ping service, you can keep these always on by doing the following:
Create an app setting on your website configuration tab within the portal:
WEBSITE_PRIVATE_EXTENSIONS and give it a value of 1
Create a text file named applicationhost.xdt and populate it with:
<?xml version="1.0"?><configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform"><system.applicationHost><applicationPools><add name="DefaultAppPool" managedRuntimeVersion="v4.5" startMode="AlwaysRunning"><processModel identityType="ApplicationPoolIdentity" /></add></applicationPools></system.applicationHost></configuration>
ftp into your website and create a folder on the root directory called Site Extensions. (there should now be 3 folders in your root: LogFiles, site, & SiteExtensions)
Create another folder within 'Site Extensions', named ASPLimits
Upload the applicationhost.xdt into the ASPLimits folder
Restart your website using the portal
I'm brand new to Azure. I'm trying to get a Cloud Service running with 3 web roles.
Last night I created the .cspkg and .cscfg files, exported the certificates and uploaded everything to Azure manager. The manager said my Cloud Service was successful, however for the last 10 hours when I click on "Cloud Services" in the manager it shows my service, but it says "Creating" with wait gif under "SERVICE STATUS".
Is it really still creating? Or did it fail? Is it possible to view more detailed information about the creation process and/or any log files?
Thanks,
Something "bad" has happened. Service Spin up time should be at most a few minutes.
I've seen it take up to 10-15 minutes depending on the hosting center and the number of scaled instances that need to come up but 10 hrs something has definitely gone wrong.
I would delete the service and start again. If you experience the same problem, have a look in your service start up code and make sure that there's no exceptions/infinite loops, other problems in there that might be causing problems.
It may miss any assembly references. If you included any assembly references(packages that are not part of .net) then please ensure that its copy to local attribute is set to true.
I'm currently working on an Azure project that works 100% locally with emulator resources. I'm now trying to deploy a worker role, but I'm running into an issue that I'm not sure how to troubleshoot.
Upon deploying the worker role in my Azure portal, the two instances continually loop through "recycling".
I can try to RDP into the role, but I only have about a minute to look around before the connection closes, I'm assuming due to the recycling.
After some searching it doesn't seem like this is a super common problem. Is there something trivial I'm overlooking that could be causing this issue? How would you go about troubleshooting this? Thank you for your time :)
In case of missing Reference you can troubleshoot this issue by:
Unzip your CSPKG file and then again unzip .CSSX file (just rename CSSX to zip) and match that everything references and static content is all there.. This way you can match what is on VM. Also in 2 minute windows when you RDP, try to look for Application event log for exception and get it because that would be the key to find the root cause.
IF you could see the exception in event log and look for the exception, you sure can find where it was generated. You can also use Intellitrace which might require you to redeploy the app.
Also there are ways were copying WinDBG and locking to the specific process you can debug it. I am not sure how much you would want to try but just copy the WinDBG to VM and use it would be enough (not sure how much experience you have with WinDBG though and how much time you would want to spent.)
Also been pestered by this role recycle issue numerous times. Here is the sequence of steps to debug persistent role recycles:
Debugging Azure Role Recycles
Enable Remote Access to your role - RDP login
Check eventvwr.msc (Windows Logs -> Application, App & Service Logs->Windows Azure)
Review the Azure text file logs across both C:\logs and c:\resources
Review custom logs in the Volume E: or F: for any custom role startup logging
Run AzureTools and attach to startup processes (download WinDBG, use Utils->Attach Debugger, select process - WaWorkerHost/WaIISHost, etc), use G to continue and watch debugger output for assemblies failing to load.
Installing Azure Debugging Tools via Powershell
PS> md c:\tools; Import-Module bitstransfer; Start-BitsTransfer http://dsazure.blob.core.windows.net/azuretools/AzureTools.exe c:\tools\AzureTools.exe; c:\tools\AzureTools.exe
If all items above fail - try using other tools in the AzureTools treasure trove - such as fusion logging, etc, this approach above will work!
WinDBG Sample Output - Failing to Locate Assembly (WaIISHost)
The most likely cause is that you have a missing assembly. One tactic to catch this is to wrap any startup processing in a master try/catch that manual logs the error to Azure storage.
If you added any referrences, check to make sure they're set to copylocal=true and that any external assets that were included in your service package were also set to be included.
From Avkash above:
Yes. this mean some issue in your Worker Role code is causing your Worker Role Host Process to crash.. If you look your fault stack you must see the function or the link from your code which generate this fault. IF you need help open a free Azure Support incident to Windows Azure Support team and they will help you.
Just a suggestion: Also Check the installable(if any)and any other references you use are 64bit.Azure VMs have 64bit OS. Once i was stuck up with this kind of problem due to 32/64 bit issues.
Are your worker roles exiting their work loop? A local recycle is very fast and you might not notice it, but spin-up time in the cloud can be long.
If the issue is caused by a startup batch file, I have stopped the loop by editing the batch file on the instance to include "exit /b 0" at the beginning. This will tell Azure that the startup was successful and you then have all the time you need to diagnose issues without the VM getting killed.