Are Custom Script Extensions Only Temporarily Usable? - azure

Question

I am working on creating an Azure Scaleset for an application. I have it set up so that when I scale out, it downloads this powershell script I've used in the past that will:
download the CI/CD deploy agent to the new VM, install, and fully configure it.
register the new deploy target with my CI/CD pipeline, and configure it to be labeled as my app.
That's all it does. From there, my CI/CD pipeline will auto-deploy my program on to this new machine and configure it for me when it detects the new target for the app. That part works fine. When I first set it up a few weeks ago, it was working fine and it was scaling out and auto-deploying perfectly with no issues.
I noticed today that I suddenly get a 403 trying to download the extension to the machine on provision during scale-out, and it wasn't doing this a couple of days ago. It's like my extension expired or something. This puts my scaleset into a failed state until it scales in and only the original VMS that are always there as a baseline are left.
I've re-installed this and it works, but after a period of time it breaks again. I looked into the JSON of my scaleset, and it has a storage account that looks like iaasv2tempstore as it's name, so this leads me to believe that all extensions are not permanent.
This leaves me with 2 questions:
what is the average lifetime expectancy of a script extension before it's invalidated by Azure deleting it?
Is there a workaround or alternative that allows me to make this permanent, or store it somewhere I have control over to make the script extension not need to be reinstalled frequently?

Answer 1

it doesnt expire. if it gets 403 when trying to download from the blob - probably means you are using SAS token to auth and it expires. If url is accessible to the script extension it wont stop working (unless you block it from talking to Azure, for example).

Answer 2

I ended up following some of what was done here in order to fix the issue.
General Workaround for Azure's lackluster custom extension ui:
Upload the script into blob storage yourself to an account that the VMSS could access.
Run this AZ powershell script. It's not polished enough to handle updating the instances if are in the process of scaling in or out, but it will get the job done.
$fileUri = #("https://somebloburi/blob/filename.ps1")
$storageAcctName = "account"
$storageKey = "accountkey"
$settings = #{"fileUris" = $fileUri; "commandToExecute" = "powershell -ExecutionPolicy Unrestricted -File filename.ps1"};
$protectedSettings = #{"storageAccountName" = $storageAcctName; "storageAccountKey" = $storageKey};
$myVMSS = Get-AzVmss -ResourceGroupName "VMSS-RG" -VMScaleSetName "myVmss"
$myVMSS = Add-AzVmssExtension -VirtualMachineScaleSet $myVMSS -Name "CustomScriptExtension" -Publisher Microsoft.Compute -Type "CustomScriptExtension" -TypeHandlerVersion "1.7" -AutoUpgradeMinorVersion $True -Setting $settings -ProtectedSetting $protectedSettings
Update-AzVmss -ResourceGroupName "VMSS-RG" -VMScaleSetName myVmss -VirtualMachineScaleSet $myVMSS
if you don't script out the auto-upgrade, you can just do it yourself from the UI.