How to make startup tasks idempotent? - iis

I have a number of startup tasks in batch files. In particular I call IIS's appcmd.exe to configure IIS. Startup tasks in Azure are supposed to idempotent (ie, able to be run repeatedly with the same results), in case the role is restarted for some reason. Unfortunately many of my IIS configuration commands will fail the second time around, eg because they delete a configuration node the first time which is then not present on subsequent runs.
My question is, how do I make these startup tasks idempotent? Is there a way to make appcmd.exe not throw errors? Is there a way to make the shell catch the errors? Is there a way to make the Azure framework ignore the errors?
Here's an example of my startup tasks. This is all contained in a command file, configiis.cmd.
#REM Enable IIS compression for application/json MIME type
%windir%\system32\inetsrv\appcmd.exe set config -section:system.webServer/httpCompression /+"dynamicTypes.[mimeType='application/json',enabled='True']" /commit:apphost
%windir%\system32\inetsrv\appcmd.exe set config -section:system.webServer/httpCompression /+"dynamicTypes.[mimeType='application/json; charset=utf-8',enabled='True']" /commit:apphost
#REM Set IIS to automatically start AppPools
%windir%\system32\inetsrv\appcmd.exe set config -section:applicationPools -applicationPoolDefaults.startMode:AlwaysRunning /commit:apphost
#REM Set IIS to not shut down idle AppPools
%windir%\system32\inetsrv\appcmd set config -section:applicationPools -applicationPoolDefaults.processModel.idleTimeout:00:00:00 /commit:apphost
#REM But don't automatically start the AppPools that we don't use, and do shut them down when idle
%windir%\system32\inetsrv\appcmd.exe set config -section:system.applicationHost/applicationPools "/[name='Classic .NET AppPool'].startMode:OnDemand" "/[name='Classic .NET AppPool'].autoStart:False" "/[name='Classic .NET AppPool'].processModel.idleTimeout:00:01:00" /commit:apphost
%windir%\system32\inetsrv\appcmd.exe set config -section:system.applicationHost/applicationPools "/[name='ASP.NET v4.0'].startMode:OnDemand" "/[name='ASP.NET v4.0'].autoStart:False" "/[name='ASP.NET v4.0'].processModel.idleTimeout:00:01:00" /commit:apphost
%windir%\system32\inetsrv\appcmd.exe set config -section:system.applicationHost/applicationPools "/[name='ASP.NET v4.0 Classic'].startMode:OnDemand" "/[name='ASP.NET v4.0 Classic'].autoStart:False" "/[name='ASP.NET v4.0 Classic'].processModel.idleTimeout:00:01:00" /commit:apphost
#REM remove IIS response headers
%windir%\system32\inetsrv\appcmd.exe set config /section:httpProtocol /-customHeaders.[name='X-Powered-By']

Aside from #Syntaxc4's answer: Consider the use of a breadcrumb (file) locally. In your script, check for existence of a known file (that you create). If it doesn't exist, go through your startup script, also creating a breadcrumb file. Next time the vm starts up, it would again check for existence of the breadcrumb file and, if it exists, exit the cmd file. If the breadcrumb file disappears, this typically means your vm has been reconstituted somewhere else (either a new instance or a respawned instance maybe on different hardware) and IIS configuration would be needed.

You would have to check to see if the config setting is present before attempting to delete it (add conditional logic). This could be achieved by:
'appcmd.exe list config -details'
Capturing a return value would give you something to compare against, be it length of output or an actual value.

MSDN now contains an excellent guide for doing this by handling error codes from APPCMD.
http://msdn.microsoft.com/en-us/library/windowsazure/hh974418.aspx
Basically after any appcmd operation, you can do the following:
IF %ERRORLEVEL% EQU 183 DO VERIFY > NUL
and ignore any acceptable error code.

Based on David Makogon's suggestion, I added the following to the top of each of my .cmd files. This seems to do the trick. It will create a flag file (what David called a breadcrumb file) in the same directory as the executing script, then check for it on subsequent runs.
#REM A file to flag that this script has already run
#REM because if we run it twice, it errors out and prevents the Azure role from starting properly
#REM %~n0 expands to the name of the currently executing file, without the extension
SET FLAGFILE=c:\%~n0-flag.txt
IF EXIST "%FLAGFILE%" (
ECHO %FLAGFILE% exists, exiting startup script
exit /B
) ELSE (
date /t > %FLAGFILE%
)

I highly recommend using the /config:* /xml on the end of your list command. For more information on how I made iis idempotent please look at: https://github.com/opscode-cookbooks/iis
Chef is one of multiple configuration management platforms and i'm only suggesting looking at it for the code (in ruby) that does idempotent via listing the current settings and comparing them to the settings being requested to change.

Related

IIS - Setting the physical path of an Application with appcmd

In IIS, I have a web site "Default Web Site" with an application "test2". I want to set the physical path of that application to "E:\temp\out1". That directory already exists.
When I run this in Powershell (as an administrator):
appcmd.exe set app "Default Web Site/test2" -[path='/'].physicalpath:E:\temp\out1
I get this error message:
ERROR ( message:Malformed collection indexer; format is [#position,name='value',name2='value2',...]. The #position specifier is optional, and be '#start', '#end', or '#N' where N is a numeric index into the collection. )
I have no idea what that means.
Would very much appreciate a working example of using appcmd to set the physical path of an IIS application.
Turns out that the command in my question actually works on Command Prompt. However, I tried to run it from a Powershell prompt, and there I got the error message.
I couldn't find a way to run the command from Powershell. In the end, my solution was to write the command to a temporary .bat file, then execute that file from Powershell.
If I set my double quotes like this it works from Powershell prompt.
appcmd.exe set app "Default Web Site/test2" -"[path='/'].physicalpath:E:\temp\out1"

kudu custom deployment fails for asp.net core webapp

I use pretty much standard custom kudu deployment script for asp.net core web app for external git deployment.
I managed to run the script mostly successfully, through mingling with SCM_COMMAND_IDLE_TIMEOUT and .cmd files encoding. My deployment scripts look as follows:
.deployment
[config]
command = deploy.cmd
deploy.cmd
#echo off
IF "%WEBSITE_SITE_TYPE%" == "<project_name>" (
.\<project folder>\Properties\deploy.cmd
goto end
)
echo Unknown WEBSITE_SITE_TYPE: "%WEBSITE_SITE_TYPE%". Expected one of the "<project name>"
:end
exit /b
.\<project folder>\Properties\deploy.cmd - regular asp.net core deploy.cmd script, updated for proper folder structure.
The WEBSITE_SITE_TYPE is a custom environment variable I added to the web app.
All these scripts saved with utf-8 without BOM encoding.
But even though the scripts look like runs successfully, the final state of the deployment process is Failed.
When looking into the kudu deployment log last few lines of the log look as follows:
...
2018-10-14T20:50:51.0660695Z,Copying file: 'wwwroot\runtime.ec2944dd8b20ec099bf3.js',,0
2018-10-14T20:50:51.0660695Z,Copying file: 'wwwroot\styles.8ef9ef86d2a54ed3748c.css',,0
2018-10-14T20:50:51.0816988Z,Deleting app_offline.htm,,0
2018-10-14T20:50:51.0816988Z,Finished successfully.,,0
2018-10-14T20:50:51.1129431Z,\r\nD:\Program Files (x86)\SiteExtensions\Kudu\78.11002.3584\bin\Scripts\starter.cmd deploy.cmd,,2
Apparently the last line returns error code 2 and it might cause the script status to fail. But what does it mean and how do I fix it?

Azure Websites Git Deployment dropping "/" in SCM_BUILD_ARGS

Description
We are in a current project based on MVC4/Umbraco using Azure Websites to host it.
We are using SCM_BUILD_ARGS to change between different build setups depending on which site in Azure we deploy to (Test and Prod).
This is done by defining an app setting in the UI:
SCM_BUILD_ARGS = /p:Environment=Test
Earlier we used Bitbucket Integration to deploy and here this setting worked like a champ.
We have now switched to using Git Deployment, pushing the changes from our build server when tests have passed.
But when we do this, we get a lovely error.
"MSB1008: Only one project can be specified."
Trying to redeploy the same failed deployment from the UI on Azure works though.
After some trial and error I ended going into the deploy.cmd and outputting the %SCM_BUILD_ARGS% value in the script.
It looks like the / gets dropped from SCM_BUILD_ARGS but only when using Git deploy, not Bitbucket Integration or redeploy from UI.
Workaround
As workaround I have for now added a / to the deploy.cmd script in front of the %SCM_BUILD_ARGS%, but this of course breaks redeploy, since we then have //p:Environment=Test in the MSBuild command when the value of %SCM_BUILD_ARGS% has been inserted.
:: 2. Build to the temporary path
IF /I "%IN_PLACE_DEPLOYMENT%" NEQ "1" (
:: Added / to SCM_BUILD_ARGS
%MSBUILD_PATH% "%DEPLOYMENT_SOURCE%\www\www.csproj" [....] /%SCM_BUILD_ARGS%
) ELSE (
%MSBUILD_PATH% "%DEPLOYMENT_SOURCE%\www\www.csproj" [....] /%SCM_BUILD_ARGS%
)
Question
Anyone know of a better solution for this problem or is it possibly a bug in Kudu?
We would love to have both deploy from Git and Redeploy working.
Could you try changing from "/" to "-"? For instance, AppSettings from /p:Environment=Test to -p:Environment=Test, see if it helps.
-p:Environment=Test did not work for me, the setting which worked for me at the time of this writing (September 2015) was
-p:Configuration=Test
There is clearly a Kudu bug in there, and you should open an issue on https://github.com/projectkudu/kudu. But for now, I can give you a workaround.
Instead of using an App Setting, include a .deployment file at the root of your repo, containing:
[config]
SCM_BUILD_ARGS = /p:Environment=Test
I think this will work in all cases. I suspect the bug has to do with bash messing up the environment in post receive hook scenarios, which only apply to direct git push but not to Bitbucket and Redeploy scenarios.
UPDATE: In fact, it's easy to see such weird bash behavior. Try this:
Open cmd.exe
Run: set foo=/abc to set a variable
Run bash
From bash, run cmd to launch a new cmd on top of bash (so cmd -> bash -> cmd)
Run set foo to get the value of foo
Result:
FOO=C:/Program Files (x86)/git/abc
So the value gets completely messed up. The key also gets upper cases, though that's mostly harmless. Strange stuff...

Java PATH not set when creating a Azure Paas Role

I am creating an Azure PaaS role which sets the PATH variable for java.exe .
I have a background task which does that.
The startupApp.cmd looks like
setx PATH %PATH%;%CD%\jdk\bin\ /m
cscript /NoLogo util\unzip.vbs jdk.zip "%CD%"
Call the bat file to start my application.
When the VM starts I see that the PATH environment variable is correctly set and points to where the jdk\bin folder. My application however fails to start with the error "java is not recognised as an internal or external batch command".
JAVA command to start my app is
java %JAVA_OPTS% %LOG_OPTS% %LOG4J_OPTS% -cp my_app.jar %MAIN_CLASS%
Here is the confusing path,
After I log into the VM and open a command prompt window and type java I see that it works fine.
If I restart the VM, the java command to bring up my app runs fine and I and my app too starts fine.
There is a significant difference between setx and set function:
set takes effect in local cmd context. Meaning once you exit or close the cmd window, you lose the environment variable.
setx takes effect in future cmd context. So you won't see the environment variable and its value in the current cmd. You need to open a new cmd window to see it.
If you want to use it global and immediate you should use both functions side by side.
Description taken from: http://batcheero.blogspot.de/2008/02/set-and-setx.html

Azure ARR error while enabling disk cache

We have azure hosted service, and now i need to setup the ARR (application request routing) on it. I followed the blog http://robindotnet.wordpress.com/2011/07/ and ARR is working fine. Now I need to enable the diskCaching for this and I'm trying below command:
%windir%\system32\inetsrv\appcmd.exe set config -section:system.webServer/diskCache /+"[path='c:\cache',maxUsage='0']" /commit:apphost >> C:\setDiskCache.txt
But getting below error:
ERROR ( message:New driveLocation object missing required attributes. Cannot add duplicate collection entry of type 'driveLocation' with unique key attribute 'path' set to 'c:\cache'. )
and there no content getting cached in this folder. Any direction or help is appreciated.
Below is complete cmd file for reference:
cd /d "%~dp0"
start /wait msiexec.exe /i webfarm_amd64_en-US.msi /qn /log C:\installWebfarmLog.txt
start /wait msiexec.exe /i requestRouter_amd64_en-US.msi /qn /log C:\installARRLog.txt
%windir%\system32\inetsrv\appcmd.exe set config -section:system.webServer/proxy /enabled:"True" /reverseRewriteHostInResponseHeaders:"False" /preserveHostHeader:"True" /commit:apphost >> C:\setProxyLog.txt
%windir%\system32\inetsrv\appcmd.exe set config -section:applicationPools -applicationPoolDefaults.processModel.idleTimeout:00:00:00 >> C:\setAppPool.txt
%windir%\system32\inetsrv\appcmd.exe set config -section:system.webServer/diskCache /+"[path='c:\cache',maxUsage='0']" /commit:apphost >> C:\setDiskCache.txt
exit /b 0
I can find the same thing here for IIS [http://www.iis.net/learn/extensions/configuring-application-request-routing-(arr)/configure-and-enable-disk-cache-in-application-request-routing], that can be enabled manually. But we need to enable this programmatically.
As is often the case, the error message holds a hint as to the cause. The problem is you can have only one entry per drive location value. Which means that script runs fine the first time, but the second time it will throw because the value has already been applied.
You cannot remove the node using appcmd (it doesn't support clearing a collection), but you can using a text editor (this file: %windir%\System32\inetsrv\config\applicationHost.config). Or you can run a powershell script:
Import-Module WebAdministration
Remove-WebConfigurationProperty -pspath 'MACHINE/WEBROOT/APPHOST' -filter "system.webServer/diskCache" -name "."
In either case, this is the node that will be manipulated:
<driveLocation path="c:\cache" maxUsage="0" />
After that you'll be able to re-run your code.

Resources