We have over 400 app services and we are creating a deployment tool to deploy to all of these instances as soon as possible.
We have 4 projects running in separate app service and each customer need to have all 4 projects up and running so we have distributed the instances by customer name and each instance deploy to 4 app services internally (See the code snippet below)
We've created a powershell function app with queue trigger that is calling azure powershell commands to do the deployment. The deployment steps are:
Create deployment slot
Publish to the deployment slot
Version check (Have an endpoint in all projects which returns assembly version)
Swap slot
Version check again
Delete slot
The steps are straight forward but when I deploy to single instance it runs pretty quickly but when I am trying to deploy to multiple instances (5+) then it starts becoming really slow and eventually stops execution. My assumption is that there is a memory issue. I am using premium (EP1) plan for my function so it shouldn't be that slow.
I am totally new to powershell so the question is do I need to dispose the azure powershell stuff? Or anything else to freed the memory. If yes, how can I call dispose to azure powershell commands?
Update
Adding the code snippets.
Some more context:
So the run.ps1 file has the code that downloads the ps1 script file from blob storage and dynamically executes it:
run.ps1
# Input bindings are passed in via param block.
param($QueueItem, $TriggerMetadata)
# Write out the queue message and insertion time to the information log.
Write-Host "PowerShell queue trigger function processed work item: $QueueItem"
Write-Host "Queue item insertion time: $($TriggerMetadata.InsertionTime)"
$currentContext = Get-AzContext -ListAvailable | Where-Object {$_.Name -contains 'Subscriptionname'}
$storageAccount = Get-AzStorageAccount -ResourceGroupName "<ResourceGroup>" -Name "<ResourceName>" -DefaultProfile $currentContext
$Context = $storageAccount.Context
$queue = Get-AzStorageQueue -Name logs -Context $Context
function Log {
param (
[string]$message, [string]$isDetailed, [string]$hasError = "false"
)
$jsonMessge = #"
{
"InstanceId": "$instanceId",
"TaskId": "$taskId",
"Note": "$message",
"IsDetailed": $isDetailed,
"HasError": $hasError
}
"#
# Create a new message using a constructor of the CloudQueueMessage class
$queueMessage = [Microsoft.Azure.Storage.Queue.CloudQueueMessage]::new($jsonMessge)
# # Add a new message to the queue
$queue.CloudQueue.AddMessageAsync($queueMessage)
}
try {
#Extracting data from Queue message
$queueMessage = $QueueItem
$instanceId = $queueMessage.instanceId
$taskId = $queueMessage.taskId
$siteName = $queueMessage.siteName
$buildNo = $queueMessage.buildNo
$pass = $queueMessage.password
$dbPassword = convertto-securestring "$pass" -asplaintext -force
$servicePlanName = $queueMessage.servicePlanName
$subscription = $queueMessage.subscription
$key = $queueMessage.key
$rg = $queueMessage.resourceGroup
$sqlServerName = $queueMessage.sqlServerName
Log -message "Deployment started for $($siteName)" -isDetailed "false"
$tempFolder = $env:temp
# $artifactFolderPath = "$($tempFolder)\$($siteName)\$($buildNo)"
#$artifactFilePath = "$($tempFolder)\$($siteName)\$($buildNo).zip"
$tempDownloadPath = "$($tempFolder)\$($siteName)"
$scriptFileName = "DeployScripts.ps1"
if (Test-Path $tempDownloadPath) {
Remove-Item $tempDownloadPath -Force -Recurse
}
$storageAccount = Get-AzStorageAccount -ResourceGroupName "KineticNorthEurope" -Name "KineticDeployment" -DefaultProfile $currentContext
New-Item -Path $tempFolder -Name $siteName -ItemType "directory"
$Context = $storageAccount.Context
$blobContent = #{
Blob = "DeployScripts.ps1"
Container = 'builds'
Destination = "$($tempDownloadPath)\$($scriptFileName)"
Context = $Context
}
Get-AzStorageBlobContent #blobContent -DefaultProfile $currentContext
#[System.IO.Compression.ZipFile]::ExtractToDirectory($artifactFilePath, $artifactFolderPath)
$arguments = "-rg $rg -site $siteName -buildNo $buildNo -dbPassword `$dbPassword -instanceId $instanceId -taskId $taskId -sqlServerName $sqlServerName -subscription $subscription"
$path = "$($tempDownloadPath)\$($scriptFileName)"
Unblock-File -Path $path
"$path $ScriptFilePath $arguments" | Invoke-Expression
if (Test-Path $tempDownloadPath) {
Remove-Item $tempDownloadPath -Force -Recurse
}
Log -message "Resources cleaned up" -isDetailed "false"
}
catch {
Log -message $_.Exception.message -isDetailed "false" -hasError "true"
if (Test-Path $tempDownloadPath) {
Remove-Item $tempDownloadPath -Force -Recurse
}
}
And here is the actual DeployScripts.ps1 file:
param($rg, $site, $buildNo, [SecureString] $dbPassword, $instanceId, $taskId, $sqlServerName, $subscription)
$siteNameWeb = "${site}"
$siteNamePortal = "${site}Portal"
$siteNamePortalService = "${site}PortalService"
$siteNameSyncService = "${site}SyncService"
$kineticNorthContext = Get-AzContext -ListAvailable | Where-Object {$_.Name -contains 'SubscriptionName'}
$storageAccount = Get-AzStorageAccount -ResourceGroupName "<ResourceGroup>" -Name "<ResourceName>" -DefaultProfile $kineticNorthContext
$Context = $storageAccount.Context
$queue = Get-AzStorageQueue -Name logs -Context $Context
Set-AzContext -SubscriptionName $subscription
Function CreateDeploymentSlots() {
Log -message "Creating deployment slots" -isDetailed "false"
Log -message "Creating deployment slot for web" -isDetailed "true"
New-AzWebAppSlot -ResourceGroupName $rg -name $siteNameWeb -slot develop
Log -message "Creating deployment slot for portal" -isDetailed "true"
New-AzWebAppSlot -ResourceGroupName $rg -name $siteNamePortal -slot develop
Log -message "Creating deployment slot for portal service" -isDetailed "true"
New-AzWebAppSlot -ResourceGroupName $rg -name $siteNamePortalService -slot develop
Log -message "Creating deployment slot for sync service" -isDetailed "true"
New-AzWebAppSlot -ResourceGroupName $rg -name $siteNameSyncService -slot develop
Log -message "Deployment slots created" -isDetailed "false"
}
Function DeleteDeploymentSlots() {
Log -message "Deleting deployment slots" -isDetailed "false"
Log -message "Deleting slot web" -isDetailed "true"
Remove-AzWebAppSlot -ResourceGroupName $rg -Name $siteNameWeb -Slot "develop"
Log -message "Deleting slot portal" -isDetailed "true"
Remove-AzWebAppSlot -ResourceGroupName $rg -Name $siteNamePortal -Slot "develop"
Log -message "Deleting slot portal service" -isDetailed "true"
Remove-AzWebAppSlot -ResourceGroupName $rg -Name $siteNamePortalService -Slot "develop"
Log -message "Deleting slot sync service" -isDetailed "true"
Remove-AzWebAppSlot -ResourceGroupName $rg -Name $siteNameSyncService -Slot "develop"
Log -message "Slots deployment deleted" -isDetailed "false"
}
Function SwapDeploymentSlots {
Log -message "Switching deployment slots" -isDetailed "false"
Log -message "Switch slot web" -isDetailed "true"
Switch-AzWebAppSlot -SourceSlotName "develop" -DestinationSlotName "production" -ResourceGroupName $rg -Name $siteNameWeb
Log -message "Switch slot portal" -isDetailed "true"
Switch-AzWebAppSlot -SourceSlotName "develop" -DestinationSlotName "production" -ResourceGroupName $rg -Name $siteNamePortal
Log -message "Switch slot portal service" -isDetailed "true"
Switch-AzWebAppSlot -SourceSlotName "develop" -DestinationSlotName "production" -ResourceGroupName $rg -Name $siteNamePortalService
Log -message "Switch slot sync service" -isDetailed "true"
Switch-AzWebAppSlot -SourceSlotName "develop" -DestinationSlotName "production" -ResourceGroupName $rg -Name $siteNameSyncService
Log -message "Deployment slots switched" -isDetailed "false"
}
function Log {
param (
[string]$message, [string]$isDetailed, [string]$isDeployed = "false", [string]$hasError = "false"
)
$jsonMessge = #"
{
"InstanceId": "$instanceId",
"TaskId": "$taskId",
"Note": "$message",
"IsDetailed": $isDetailed,
"IsDeployed": $isDeployed,
"HasError": $hasError
}
"#
# Create a new message using a constructor of the CloudQueueMessage class
$queueMessage = [Microsoft.Azure.Storage.Queue.CloudQueueMessage]::new($jsonMessge)
# # Add a new message to the queue
$queue.CloudQueue.AddMessageAsync($queueMessage)
}
function VersionCheckWeb() {
$tls = Invoke-WebRequest -URI "https://${site}.net/Home/Info" -UseBasicParsing
$content = $tls.Content | ConvertFrom-Json
$versionCheck = $content.VersionNo -eq $buildNo
return $versionCheck
}
function VersionCheckPortal() {
$tls = Invoke-WebRequest -URI "https://${site}portal.net/Home/Info" -UseBasicParsing
$content = $tls.Content | ConvertFrom-Json
$versionCheck = $content.VersionNo -eq $buildNo
return $versionCheck
}
function VersionCheckPortalService() {
$tls = Invoke-WebRequest -URI "https://${site}portalservice.net/Home/Info" -UseBasicParsing
$content = $tls.Content | ConvertFrom-Json
$versionCheck = $content.VersionNo -eq $buildNo
return $versionCheck
}
function VersionCheckSyncService() {
$tls = Invoke-WebRequest -URI "https://${site}syncservice.net/SyncService.svc/Info" -UseBasicParsing
$tls = $tls -replace '[?]', ""
$content = "$tls" | ConvertFrom-Json
$versionCheck = $content.VersionNo -eq $buildNo
return $versionCheck
}
function VersionCheck() {
$versionCheckWeb = VersionCheckWeb
$versionCheckPortal = VersionCheckPortal
$versionCheckPortalService = VersionCheckPortalService
$versionCheckSyncService = VersionCheckSyncService
if (($versionCheckWeb -eq "True") -and ($versionCheckPortal -eq "True") -and ($versionCheckPortalService -eq "True") -and ($versionCheckSyncService -eq "True")) {
Log -message "Version correct" -isDetailed "false"
}
else {
Log -message "Version check failed, exception not thrown" -isDetailed "false"
}
}
function VersionCheckWebSlot() {
$tls = Invoke-WebRequest -URI "https://${site}-develop.azurewebsites.net/Home/Info" -UseBasicParsing
$content = $tls.Content | ConvertFrom-Json
$versionCheck = $content.VersionNo -eq $buildNo
return $versionCheck
}
function VersionCheckPortalSlot() {
$tls = Invoke-WebRequest -URI "https://${site}portal-develop.azurewebsites.net/Home/Info" -UseBasicParsing
$content = $tls.Content | ConvertFrom-Json
$versionCheck = $content.VersionNo -eq $buildNo
return $versionCheck
}
function VersionCheckPortalServiceSlot() {
$tls = Invoke-WebRequest -URI "https://${site}portalservice-develop.azurewebsites.net/Home/Info" -UseBasicParsing
$content = $tls.Content | ConvertFrom-Json
$versionCheck = $content.VersionNo -eq $buildNo
return $versionCheck
}
function VersionCheckSyncServiceSlot() {
$tls = Invoke-WebRequest -URI "https://${site}syncservice-develop.azurewebsites.net/SyncService.svc/Info" -UseBasicParsing
$tls = $tls -replace '[?]', ""
$content = "$tls" | ConvertFrom-Json
$versionCheck = $content.VersionNo -eq $buildNo
return $versionCheck
}
function VersionCheckSlot() {
$versionCheckWeb = VersionCheckWebSlot
$versionCheckPortal = VersionCheckPortalSlot
$versionCheckPortalService = VersionCheckPortalServiceSlot
$versionCheckSyncService = VersionCheckSyncServiceSlot
if (($versionCheckWeb -eq "True") -and ($versionCheckPortal -eq "True") -and ($versionCheckPortalService -eq "True") -and ($versionCheckSyncService -eq "True")) {
Log -message "Slot version correct" -isDetailed "false"
}
else {
Log -message "Slot version check failed, exception not thrown" -isDetailed "false"
}
}
function PublishToAzure() {
$webPublishProfile = Get-AzWebAppPublishingProfile -ResourceGroupName $rg -Name $siteNameWeb
$webXml = $webPublishProfile -as [Xml]
$webUserName = $webXml.publishData.publishProfile[0].userName
$webUserPwd = $webXml.publishData.publishProfile[0].userPWD
$webpair = "$($webUserName):$($webUserPwd)"
$webencodedCreds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($webpair))
$webauthHeader = "Basic $webencodedCreds"
$webHeaders = #{
Authorization = $webauthHeader
}
$portalPublishProfile = Get-AzWebAppPublishingProfile -ResourceGroupName $rg -Name $siteNamePortal
$portalXml = $portalPublishProfile -as [Xml]
$portalUserName = $portalXml.publishData.publishProfile[0].userName
$portalUserPwd = $portalXml.publishData.publishProfile[0].userPWD
$portalpair = "$($portalUserName):$($portalUserPwd)"
$portalencodedCreds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($portalpair))
$portalauthHeader = "Basic $portalencodedCreds"
$portalHeaders = #{
Authorization = $portalauthHeader
}
$portalServicePublishProfile = Get-AzWebAppPublishingProfile -ResourceGroupName $rg -Name $siteNamePortalService
$portalServiceXml = $portalServicePublishProfile -as [Xml]
$portalServiceUserName = $portalServiceXml.publishData.publishProfile[0].userName
$portalServiceUserPwd = $portalServiceXml.publishData.publishProfile[0].userPWD
$portalServicepair = "$($portalServiceUserName):$($portalServiceUserPwd)"
$portalServiceencodedCreds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($portalServicepair))
$portalServiceauthHeader = "Basic $portalServiceencodedCreds"
$portalServiceHeaders = #{
Authorization = $portalServiceauthHeader
}
$syncServicePublishProfile = Get-AzWebAppPublishingProfile -ResourceGroupName $rg -Name $siteNameSyncService
$syncServiceXml = $syncServicePublishProfile -as [Xml]
$syncServiceUserName = $syncServiceXml.publishData.publishProfile[0].userName
$syncServiceUserPwd = $syncServiceXml.publishData.publishProfile[0].userPWD
$syncServicepair = "$($syncServiceUserName):$($syncServiceUserPwd)"
$syncServiceencodedCreds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($syncServicepair))
$syncServiceauthHeader = "Basic $syncServiceencodedCreds"
$syncServiceHeaders = #{
Authorization = $syncServiceauthHeader
}
$bodyWeb = '{"packageUri": "<blobUrl>"}'
$bodyPortal = '{"packageUri": "blobUrl"}'
$bodyPortalService = '{"packageUri": "blobUrl"}'
$bodySyncService = '{"packageUri": "blobUrl"}'
$web = Invoke-RestMethod -URI "https://${siteNameWeb}.scm.azurewebsites.net/api/publish?type=zip" -Method POST -Body $bodyWeb -Headers $webHeaders -ContentType "application/json"
Log -message "Published to Web" -isDetailed "false"
$portal = Invoke-RestMethod -URI "https://${siteNamePortal}.scm.azurewebsites.net/api/publish?type=zip" -Method POST -Body $bodyPortal -Headers $portalHeaders -ContentType "application/json"
Log -message "Published to Portal" -isDetailed "false"
$portalService = Invoke-RestMethod -URI "https://${siteNamePortalService}.scm.azurewebsites.net/api/publish?type=zip" -Method POST -Body $bodyPortalService -Headers $portalServiceHeaders -ContentType "application/json"
Log -message "Published to PortalService" -isDetailed "false"
$syncService = Invoke-RestMethod -URI "https://${siteNameSyncService}.scm.azurewebsites.net/api/publish?type=zip" -Method POST -Body $bodySyncService -Headers $syncServiceHeaders -ContentType "application/json"
Log -message "Published to SyncService" -isDetailed "false"
}
try {
CreateDeploymentSlots
PublishToAzure
VersionCheckSlot
SwapDeploymentSlots
VersionCheck
DeleteDeploymentSlots
Log -message "Instance deployed successfully" -isDetailed "false" -isDeployed "true"
}
catch {
Log -message $_.Exception.message -isDetailed "false" -hasError "true"
}
I know the code is little mess right now and that's because I've changed the implementation to faster the process but no major luck.
Question2
I actually have another question as well. We've multiple subscriptions and appservices are distributed accordingly. So when function app tries to deploy to a instance exists in another subscription, it throws error. I'm setting AzContext before starting the processing and it seems to be working fine. But i also have to put message into a queue after each step and the queue exists in a storage account that belongs to same subscription as of the function app. Currently i'm getting the AzContent and passing it as -DefaultProfile while getting the storage account. Is there a better way to handle this?
Also I am writing powershell for the first time so any suggestion would be appreciated. Thanks.
I want to stop my appservices at midnight and want to start them at morning.So i came across two things , runbooks and webjobs .So first i included a runbook which start/stop services in a resource group.But when i tested it out , i faced an error -
And also when i tried using a webjob , i used this code from here , but i was not able to see the result.The webjob was working as a script but was it actually starting/stoping the services i dont know.I am new to powershell scripts so i dont know where to make necessary changes in the code.I dont know whether i am doing it right or wrong , please help me out.Thank You.
If you want to manage Azure ARM Resource with Azure Runbook, you can create Run As accounts in your Azure automation account. When we create it, it will create a new service principal user in Azure Active Directory (AD) and assigns the Contributor role to this user at the subscription level. For more details, please refer to the document and the document.
For example
Create Run As accounts
a. Search for and select Automation Accounts.
b. On the Automation Accounts page, select your Automation account from the list.
c. In the left pane, select Run As Accounts in the account settings section.
d. Depending on which account you require, select either Azure Run As Account or Azure Classic Run As Account.
e. Depending on the account of interest, use the Add Azure Run As or Add Azure Classic Run As Account pane. After reviewing the overview information, click Create.
Create a PowerShell Workflow runbook
Script
workflow START_STOP_APP_SERVICE_BY_RESOURCE
{
Param(
[Parameter (Mandatory= $true)]
[bool]$Stop,
[Parameter (Mandatory= $true)]
[string]$ResourcegroupName
)
try
{
# use Azure As Account to log in Azure
$servicePrincipalConnection=Get-AutomationConnection -Name "AzureRunAsConnection"
Add-AzureRmAccount `
-ServicePrincipal `
-TenantId $servicePrincipalConnection.TenantId `
-ApplicationId $servicePrincipalConnection.ApplicationId `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint
}
catch {
if (!$servicePrincipalConnection)
{
$ErrorMessage = "Connection $connectionName not found."
throw $ErrorMessage
} else{
Write-Error -Message $_.Exception
throw $_.Exception
}
}
$status = 'Stopped'
if ($Stop)
{
$status = 'Running'
}
# Get Running WebApps (website_Processings_Running)
$website_Processings_Running = Get-AzureRMWebAPP -ResourceGroupName $ResourcegroupName | where-object -FilterScript{$_.state -eq $status }
foreach -parallel ($website_Processing In $website_Processings_Running)
{
if ($Stop)
{
$result = Stop-AzureRmWebApp -ResourceGroupName $ResourcegroupName -Name $website_Processing.Name
if($result)
{
Write-Output "- $($website_Processing.Name) shutdown successfully"
}
else
{
Write-Output "+ $($website_Processing.Name) did not shutdown successfully"
}
}
else
{
$result = Start-AzureRmWebApp -ResourceGroupName $ResourcegroupName -Name $website_Processing.Name
if($result)
{
Write-Output "- $($website_Processing.Name) start successfully"
}
else
{
Write-Output "+ $($website_Processing.Name) did not started successfully"
}
}
}
}
Today in 2022 you have to use Az module instead of AzureRM. You also have to switch to 7.1 Runtime if you want to succeed.
Param(
[bool]$Stop,
[string]$ResourcegroupName,
[string]$WebAppName
)
try
{
# use Azure As Account to log in Azure
$servicePrincipalConnection=Get-AutomationConnection -Name "AzureRunAsConnection"
Write-Output $servicePrincipalConnection.TenantId
Write-Output $servicePrincipalConnection.ApplicationId
Write-Output $servicePrincipalConnection.CertificateThumbprint
Add-AzAccount `
-ServicePrincipal `
-TenantId $servicePrincipalConnection.TenantId `
-ApplicationId $servicePrincipalConnection.ApplicationId `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint
}
catch {
if (!$servicePrincipalConnection)
{
$ErrorMessage = "Connection $connectionName not found."
throw $ErrorMessage
} else{
Write-Error -Message $_.Exception
throw $_.Exception
}
}
$status = 'Stopped'
if ($Stop)
{
$status = 'Running'
}
# Get Running WebApps (website_Processings_Running)
$website_Processings_Running = Get-AzWebApp `
-ResourceGroupName $ResourcegroupName] `
-Name $WebAppName | where-object -FilterScript{$_.state -eq $status}
Write-Output "- $($website_Processing.Name) WebApp to manage"
foreach ($website_Processing In $website_Processings_Running)
{
if ($Stop)
{
$result = Stop-AzWebApp -ResourceGroupName $ResourcegroupName -Name $website_Processing.Name
if($result)
{
Write-Output "- $($website_Processing.Name) shutdown successfully"
}
else
{
Write-Output "+ $($website_Processing.Name) did not shutdown successfully"
}
}
else
{
$result = Start-AzWebApp -ResourceGroupName $ResourcegroupName -Name $website_Processing.Name
if($result)
{
Write-Output "- $($website_Processing.Name) start successfully"
}
else
{
Write-Output "+ $($website_Processing.Name) did not started successfully"
}
}
}
I am trying to use Azure powershell to get VM name (Eg: demovm01, where the VM name is demovm and the suffix is 01.
I want to get the output and automatically append a new suffix of 02 if 01 already exists.
Sample script to get vm name:
$getvm = Get-AzVM -Name "$vmname" -ResourceGroupName "eodemofunction" -ErrorVariable notPresent -ErrorAction SilentlyContinue
if ($notPresent) {
Write-Output "VM not found. Creating now"
}
else {
Write-Output "VM exists."
return $true
}
I want to be able to inject this new vm name to an arm deploy to deploy
This should do it. Will increment until no VM is found and use a simple -replace to inject to your json file. Will also return all thr VM values that are already present in Azure
$i=1
$vmname_base = "vmserver"
$VMexists = #()
do {
#invert int value to double digit string
$int = $i.tostring('00')
$getvm = Get-AzVM -Name "$vmname_base$int" -ResourceGroupName "eodemofunction" -ErrorVariable notPresent -ErrorAction SilentlyContinue
if ($notPresent) {
Write-Output "VM not found. Creating now"
Write-Output "VM created name is $vmname_base$int"
#Set condition to end do while loop
VMcreated = "true"
#commands to inject to json here. I always find replace method the easiest
$JSON = Get-Content azuredeploy.parameters.json
$JSON = $JSON -replace ("Servername","$vmname_base$int")
$JSON | Out-File azuredeploy.parameters.json -Force
}
else {
Write-Output "VMexists."
# Add existing VM to array
$VMexists += "$vmname_base$int"
# Increment version, ie. 01 to 02 to 03 etc
$i++
}
} while ($VMcreated -ne "true")
return $VMexists
Your command could be like below, the $newvmname is that you want.
$vmname = "demovm01"
$getvm = Get-AzVM -Name "$vmname" -ResourceGroupName "<group name>" -ErrorVariable notPresent -ErrorAction SilentlyContinue
if ($notPresent) {
Write-Output "VM not found. Creating now"
}
else {
Write-Output "VM exists."
if($getvm.Name -like '*01'){
$newvmname = $vmname.TrimEnd('01')+"02"
}
Write-Output $newvmname
return $true
}
I am taking my first crack at making a DSC (Desired State Configuration) file to go with an ARM (Azure Resource Manager) template to deploy a Windows Server 2016 and so far everything was working great until I tried to pass a username/password so I can create a local Windows user account. I can't seem to make this function at all (see the error message below).
My question is, how do I use an ARM template to pull a password from an Azure key vault and pass it to a DSC powershell extension?
Here is my current setup:
azuredeploy.json
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"deployExecUsername": {
"type": "string",
"defaultValue": "DeployExec"
},
"deployExecPassword": {
"type": "securestring"
},
"_artifactsLocation": {
"type": "string",
"metadata": {
"description": "Auto-generated container in staging storage account to receive post-build staging folder upload"
}
},
"_artifactsLocationSasToken": {
"type": "securestring",
"metadata": {
"description": "Auto-generated token to access _artifactsLocation"
}
},
"virtualMachineName": {
"type": "string",
"defaultValue": "web-app-server"
}
},
"variables": {
"CreateLocalUserArchiveFolder": "DSC",
"CreateLocalUserArchiveFileName": "CreateLocalUser.zip"},
"resources": [
{
"name": "[concat(parameters('virtualMachineName'), '/', 'Microsoft.Powershell.DSC')]",
"type": "Microsoft.Compute/virtualMachines/extensions",
"location": "eastus2",
"apiVersion": "2016-03-30",
"dependsOn": [ ],
"tags": {
"displayName": "CreateLocalUser"
},
"properties": {
"publisher": "Microsoft.Powershell",
"type": "DSC",
"typeHandlerVersion": "2.9",
"autoUpgradeMinorVersion": true,
"settings": {
"configuration": {
"url": "[concat(parameters('_artifactsLocation'), '/', variables('CreateLocalUserArchiveFolder'), '/', variables('CreateLocalUserArchiveFileName'))]",
"script": "CreateLocalUser.ps1",
"function": "Main"
},
"configurationArguments": {
"nodeName": "[parameters('virtualMachineName')]"
}
},
"protectedSettings": {
"configurationArguments": {
"deployExecCredential": {
"UserName": "[parameters('deployExecUsername')]",
"Password": "[parameters('deployExecPassword')]"
}
},
"configurationUrlSasToken": "[parameters('_artifactsLocationSasToken')]"
}
}
}],
"outputs": {}
}
azuredeploy.parameters.json
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"deployExecPassword": {
"reference": {
"keyVault": {
"id": "/subscriptions/<GUID>/resourceGroups/<Resource Group Name>/providers/Microsoft.KeyVault/vaults/<Resource Group Name>-key-vault"
},
"secretName": "web-app-server-deployexec-password"
}
}
}
}
DSC/CreateLocalUser.ps1
Configuration Main
{
Param (
[string] $nodeName,
[PSCredential]$deployExecCredential
)
Import-DscResource -ModuleName PSDesiredStateConfiguration
Node $nodeName
{
User DeployExec
{
Ensure = "Present"
Description = "Deployment account for Web Deploy"
UserName = $deployExecCredential.UserName
Password = $deployExecCredential
PasswordNeverExpires = $true
PasswordChangeRequired = $false
PasswordChangeNotAllowed = $true
}
}
}
Deploy-AzureResourceGroup.ps1 (Default from Azure Resource Group template)
#Requires -Version 3.0
Param(
[string] [Parameter(Mandatory=$true)] $ResourceGroupLocation,
[string] $ResourceGroupName = 'AzureResourceGroup2',
[switch] $UploadArtifacts,
[string] $StorageAccountName,
[string] $StorageContainerName = $ResourceGroupName.ToLowerInvariant() + '-stageartifacts',
[string] $TemplateFile = 'azuredeploy.json',
[string] $TemplateParametersFile = 'azuredeploy.parameters.json',
[string] $ArtifactStagingDirectory = '.',
[string] $DSCSourceFolder = 'DSC',
[switch] $ValidateOnly
)
try {
[Microsoft.Azure.Common.Authentication.AzureSession]::ClientFactory.AddUserAgent("VSAzureTools-$UI$($host.name)".replace(' ','_'), '3.0.0')
} catch { }
$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 3
function Format-ValidationOutput {
param ($ValidationOutput, [int] $Depth = 0)
Set-StrictMode -Off
return #($ValidationOutput | Where-Object { $_ -ne $null } | ForEach-Object { #(' ' * $Depth + ': ' + $_.Message) + #(Format-ValidationOutput #($_.Details) ($Depth + 1)) })
}
$OptionalParameters = New-Object -TypeName Hashtable
$TemplateFile = [System.IO.Path]::GetFullPath([System.IO.Path]::Combine($PSScriptRoot, $TemplateFile))
$TemplateParametersFile = [System.IO.Path]::GetFullPath([System.IO.Path]::Combine($PSScriptRoot, $TemplateParametersFile))
if ($UploadArtifacts) {
# Convert relative paths to absolute paths if needed
$ArtifactStagingDirectory = [System.IO.Path]::GetFullPath([System.IO.Path]::Combine($PSScriptRoot, $ArtifactStagingDirectory))
$DSCSourceFolder = [System.IO.Path]::GetFullPath([System.IO.Path]::Combine($PSScriptRoot, $DSCSourceFolder))
# Parse the parameter file and update the values of artifacts location and artifacts location SAS token if they are present
$JsonParameters = Get-Content $TemplateParametersFile -Raw | ConvertFrom-Json
if (($JsonParameters | Get-Member -Type NoteProperty 'parameters') -ne $null) {
$JsonParameters = $JsonParameters.parameters
}
$ArtifactsLocationName = '_artifactsLocation'
$ArtifactsLocationSasTokenName = '_artifactsLocationSasToken'
$OptionalParameters[$ArtifactsLocationName] = $JsonParameters | Select -Expand $ArtifactsLocationName -ErrorAction Ignore | Select -Expand 'value' -ErrorAction Ignore
$OptionalParameters[$ArtifactsLocationSasTokenName] = $JsonParameters | Select -Expand $ArtifactsLocationSasTokenName -ErrorAction Ignore | Select -Expand 'value' -ErrorAction Ignore
# Create DSC configuration archive
if (Test-Path $DSCSourceFolder) {
$DSCSourceFilePaths = #(Get-ChildItem $DSCSourceFolder -File -Filter '*.ps1' | ForEach-Object -Process {$_.FullName})
foreach ($DSCSourceFilePath in $DSCSourceFilePaths) {
$DSCArchiveFilePath = $DSCSourceFilePath.Substring(0, $DSCSourceFilePath.Length - 4) + '.zip'
Publish-AzureRmVMDscConfiguration $DSCSourceFilePath -OutputArchivePath $DSCArchiveFilePath -Force -Verbose
}
}
# Create a storage account name if none was provided
if ($StorageAccountName -eq '') {
$StorageAccountName = 'stage' + ((Get-AzureRmContext).Subscription.SubscriptionId).Replace('-', '').substring(0, 19)
}
$StorageAccount = (Get-AzureRmStorageAccount | Where-Object{$_.StorageAccountName -eq $StorageAccountName})
# Create the storage account if it doesn't already exist
if ($StorageAccount -eq $null) {
$StorageResourceGroupName = 'ARM_Deploy_Staging'
New-AzureRmResourceGroup -Location "$ResourceGroupLocation" -Name $StorageResourceGroupName -Force
$StorageAccount = New-AzureRmStorageAccount -StorageAccountName $StorageAccountName -Type 'Standard_LRS' -ResourceGroupName $StorageResourceGroupName -Location "$ResourceGroupLocation"
}
# Generate the value for artifacts location if it is not provided in the parameter file
if ($OptionalParameters[$ArtifactsLocationName] -eq $null) {
$OptionalParameters[$ArtifactsLocationName] = $StorageAccount.Context.BlobEndPoint + $StorageContainerName
}
# Copy files from the local storage staging location to the storage account container
New-AzureStorageContainer -Name $StorageContainerName -Context $StorageAccount.Context -ErrorAction SilentlyContinue *>&1
$ArtifactFilePaths = Get-ChildItem $ArtifactStagingDirectory -Recurse -File | ForEach-Object -Process {$_.FullName}
foreach ($SourcePath in $ArtifactFilePaths) {
Set-AzureStorageBlobContent -File $SourcePath -Blob $SourcePath.Substring($ArtifactStagingDirectory.length + 1) `
-Container $StorageContainerName -Context $StorageAccount.Context -Force
}
# Generate a 4 hour SAS token for the artifacts location if one was not provided in the parameters file
if ($OptionalParameters[$ArtifactsLocationSasTokenName] -eq $null) {
$OptionalParameters[$ArtifactsLocationSasTokenName] = ConvertTo-SecureString -AsPlainText -Force `
(New-AzureStorageContainerSASToken -Container $StorageContainerName -Context $StorageAccount.Context -Permission r -ExpiryTime (Get-Date).AddHours(4))
}
}
# Create or update the resource group using the specified template file and template parameters file
New-AzureRmResourceGroup -Name $ResourceGroupName -Location $ResourceGroupLocation -Verbose -Force
if ($ValidateOnly) {
$ErrorMessages = Format-ValidationOutput (Test-AzureRmResourceGroupDeployment -ResourceGroupName $ResourceGroupName `
-TemplateFile $TemplateFile `
-TemplateParameterFile $TemplateParametersFile `
#OptionalParameters)
if ($ErrorMessages) {
Write-Output '', 'Validation returned the following errors:', #($ErrorMessages), '', 'Template is invalid.'
}
else {
Write-Output '', 'Template is valid.'
}
}
else {
New-AzureRmResourceGroupDeployment -Name ((Get-ChildItem $TemplateFile).BaseName + '-' + ((Get-Date).ToUniversalTime()).ToString('MMdd-HHmm')) `
-ResourceGroupName $ResourceGroupName `
-TemplateFile $TemplateFile `
-TemplateParameterFile $TemplateParametersFile `
#OptionalParameters `
-Force -Verbose `
-ErrorVariable ErrorMessages
if ($ErrorMessages) {
Write-Output '', 'Template deployment returned the following errors:', #(#($ErrorMessages) | ForEach-Object { $_.Exception.Message.TrimEnd("`r`n") })
}
}
Do note that my original template deploys the entire server, but I am able to reproduce the issue I am having with the above template and any old Windows Server 2016 VM.
I am running the template through Visual Studio 2017 Community:
The template validates and runs, but when I run it I am getting the following error message:
22:26:41 - New-AzureRmResourceGroupDeployment : 10:26:41 PM - VM has reported a failure when processing extension
22:26:41 - 'Microsoft.Powershell.DSC'. Error message: "The DSC Extension received an incorrect input: Compilation errors occurred
22:26:41 - while processing configuration 'Main'. Please review the errors reported in error stream and modify your configuration
22:26:41 - code appropriately. System.InvalidOperationException error processing property 'Password' OF TYPE 'User': Converting
22:26:41 - and storing encrypted passwords as plain text is not recommended. For more information on securing credentials in MOF
22:26:41 - file, please refer to MSDN blog: http://go.microsoft.com/fwlink/?LinkId=393729
22:26:41 - At C:\Packages\Plugins\Microsoft.Powershell.DSC\2.77.0.0\DSCWork\CreateLocalUser.4\CreateLocalUser.ps1:12 char:3
22:26:41 - + User Converting and storing encrypted passwords as plain text is not recommended. For more information on securing
22:26:41 - credentials in MOF file, please refer to MSDN blog: http://go.microsoft.com/fwlink/?LinkId=393729 Cannot find path
22:26:41 - 'HKLM:\SOFTWARE\Microsoft\PowerShell\3\DSC' because it does not exist. Cannot find path
22:26:41 - 'HKLM:\SOFTWARE\Microsoft\PowerShell\3\DSC' because it does not exist.
22:26:41 - Another common error is to specify parameters of type PSCredential without an explicit type. Please be sure to use a
22:26:41 - typed parameter in DSC Configuration, for example:
22:26:41 - configuration Example {
22:26:41 - param([PSCredential] $UserAccount)
22:26:41 - ...
22:26:41 - }.
22:26:41 - Please correct the input and retry executing the extension.".
22:26:41 - At F:\Users\Shad\Documents\Visual Studio 2017\Projects\AzureResourceGroup2\AzureResourceGroup2\bin\Debug\staging\AzureR
22:26:41 - esourceGroup2\Deploy-AzureResourceGroup.ps1:108 char:5
22:26:41 - + New-AzureRmResourceGroupDeployment -Name ((Get-ChildItem $Templat ...
22:26:41 - + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
22:26:41 - + CategoryInfo : NotSpecified: (:) [New-AzureRmResourceGroupDeployment], Exception
22:26:41 - + FullyQualifiedErrorId : Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implementation.NewAzureResourceGroupDep
22:26:41 - loymentCmdlet
What I Tried
I have already tried looking at this question:
Passing credentials to DSC script from arm template
but it seems to be using the old ARM template format to construct the DSC call and since I am not familiar with it, I am unable to work out what the extra parameters are for.
I also took a look at this question:
Securely pass credentials to DSC Extension from ARM Template
and the accepted answer is to just use PsDSCAllowPlainTextPassword = $true. While this doesn't seem like the best way, I tried adding the following configuration data file.
CreateLocalUser.psd1
#
# CreateLocalUser.psd1
#
#{
AllNodes = #(
#{
NodeName = '*'
PSDscAllowPlainTextPassword = $true
}
)
}
And changing Deploy-AzureResourceGroup.ps1 to pass these settings to the DSC configuration, as follows.
# Create DSC configuration archive
if (Test-Path $DSCSourceFolder) {
$DSCSourceFilePaths = #(Get-ChildItem $DSCSourceFolder -File -Filter '*.ps1' | ForEach-Object -Process {$_.FullName})
foreach ($DSCSourceFilePath in $DSCSourceFilePaths) {
$DSCArchiveFilePath = $DSCSourceFilePath.Substring(0, $DSCSourceFilePath.Length - 4) + '.zip'
$DSCConfigDataFilePath = $DSCSourceFilePath.Substring(0, $DSCSourceFilePath.Length - 4) + '.psd1'
Publish-AzureRmVMDscConfiguration $DSCSourceFilePath -OutputArchivePath $DSCArchiveFilePath -ConfigurationDataPath $DSCConfigDataFilePath -Force -Verbose
}
}
However, I am not getting any change in the error message when doing so.
I have been through loads of the Azure documentation to try to work this out. The link in the error message is entirely unhelpful as there are no examples of how to use the encryption with an ARM template. Most of the examples are showing running Powershell scripts rather than an ARM template. And there isn't a single example in the documentation anywhere on how to retrieve the password from a key vault and pass it into a DSC extension file from an ARM template. Is this even possible?
Note that I would be happy with simply using Visual Studio to do my deployment, if I could just get it working. But I have been working on this issue for several days and cannot seem to find a single solution that works. So, I thought I would ask here before throwing in the towel and just using the admin Windows account for web deployment.
UPDATE 2018-12-21
I noticed when running the deploy command through Visual Studio 2017 that the log contains an error message:
20:13:43 - Build started.
20:13:43 - Project "web-app-server.deployproj" (StageArtifacts target(s)):
20:13:43 - Project "web-app-server.deployproj" (ContentFilesProjectOutputGroup target(s)):
20:13:43 - Done building project "web-app-server.deployproj".
20:13:43 - Done building project "web-app-server.deployproj".
20:13:43 - Build succeeded.
20:13:43 - Launching PowerShell script with the following command:
20:13:43 - 'F:\Projects\thepath\web-app-server\bin\Debug\staging\web-app-server\Deploy-AzureResourceGroup.ps1' -StorageAccountName 'staged<xxxxxxxxxxxxxxxxx>' -ResourceGroupName 'web-app-server' -ResourceGroupLocation 'eastus2' -TemplateFile 'F:\Projects\thepath\web-app-server\bin\Debug\staging\web-app-server\web-app-server.json' -TemplateParametersFile 'F:\Projects\thepath\web-app-server\bin\Debug\staging\web-app-server\web-app-server.parameters.json' -ArtifactStagingDirectory '.' -DSCSourceFolder '.\DSC' -UploadArtifacts
20:13:43 - Deploying template using PowerShell script failed.
20:13:43 - Tell us about your experience at https://go.microsoft.com/fwlink/?LinkId=691202
After the error message occurs, it continues. I suspect it may be falling back to using another method and it is that method that is causing the failure and why others are not seeing what I am.
Sadly, no matter what I try this does not function with DSC.
Workaround
For now, I am working around this issue using a custom script extension, like this:
{
"name": "[concat(parameters('virtualMachineName'), '/addWindowsAccounts')]",
"type": "Microsoft.Compute/virtualMachines/extensions",
"apiVersion": "2018-06-01",
"location": "[resourceGroup().location]",
"dependsOn": [
"[concat('Microsoft.Compute/virtualMachines/', parameters('virtualMachineName'))]"
],
"properties": {
"publisher": "Microsoft.Compute",
"type": "CustomScriptExtension",
"typeHandlerVersion": "1.9",
"autoUpgradeMinorVersion": true,
"settings": {
"fileUris": []
},
"protectedSettings": {
"commandToExecute": "[concat('powershell -ExecutionPolicy Unrestricted -Command \"& { $secureDeployExecPassword = ConvertTo-SecureString -String ', variables('quote'), parameters('deployExecPassword'), variables('quote'), ' -AsPlainText -Force; New-LocalUser -AccountNeverExpires -UserMayNotChangePassword -Name ', variables('quote'), parameters('deployExecUsername'), variables('quote'), ' -Password $secureDeployExecPassword -FullName ', variables('quote'), parameters('deployExecUsername'), variables('quote'), ' -Description ', variables('quote'), 'Deployment account for Web Deploy', variables('quote'), ' -ErrorAction Continue ', '}\"')]"
}
}
}
And then using dependsOn to force the custom script extension to run before DSC by setting these on the DSC extension.
"dependsOn": [
"[resourceId('Microsoft.Compute/virtualMachines', parameters('virtualMachineName'))]",
"addWindowsAccounts"
],
Not the ideal solution, but it is secure and gets me past this blocking issue without resorting to using the administrator password for website deployment.
Here's one of the recent ones I'm using:
Configuration:
Param(
[System.Management.Automation.PSCredential]$Admincreds,
)
and in the template I do this:
"publisher": "Microsoft.Powershell",
"type": "DSC",
"typeHandlerVersion": "2.20",
"autoUpgradeMinorVersion": true,
"settings": {
"configuration": {
"url": "url.zip",
"script": "file.ps1",
"function": "configuration"
}
},
"protectedSettings": {
"configurationArguments": {
"adminCreds": {
"userName": "username",
"password": "password"
}
}
}
You dont need PSDscAllowPlainTextPassword because they are auto encrypted by powershell.dsc extension.
This template run on the same resource group produced the error StorageAccountAlreadyTaken but it is expected to do incremental deployment, ie. not to create any new resources. How to fix this?
{
"type": "Microsoft.Storage/storageAccounts",
"name": "[variables('prodSaName')]",
"tags": { "displayName": "Storage account" },
"apiVersion": "2016-01-01",
"location": "[parameters('location')]",
"sku": { "name": "Standard_LRS" },
"kind": "Storage",
"properties": {}
},
{
"type": "Microsoft.Storage/storageAccounts",
"name": "[ge.saName(parameters('brn'), parameters('environments')[copyIndex()])]",
"tags": { "displayName": "Storage account" },
"apiVersion": "2016-01-01",
"location": "[parameters('location')]",
"sku": { "name": "Standard_LRS" },
"kind": "Storage",
"copy": {
"name": "EnvStorageAccounts",
"count": "[length(parameters('environments'))]"
},
Run New-AzureRmResourceGroupDeployment #args -debug with this template and it outputs:
New-AzureRmResourceGroupDeployment : 15:03:38 - Error: Code=StorageAccountAlreadyTaken; Message=The storage account named
UNIQUENAMEOFTHERESOURCE is already taken.
At C:\coding\gameo\gameoemergency\provision.run.ps1:101 char:9
+ New-AzureRmResourceGroupDeployment #args -debug
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [New-AzureRmResourceGroupDeployment], Exception
+ FullyQualifiedErrorId : Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implementation.NewAzureResourceGroupDeploymentCmdle
t
PS. Somehow running from VSTS doesn't have this result. That run from local machine. Also there is no such error in Activity Log, strangely.
Update.
If I don't do these selects of a subscription as below but only for RM there are no errors.
Select-AzureRmSubscription -SubscriptionID $Cfg.subscriptionId > $null
# this seems to be the reason of the error, if removed - works:
Select-AzureSubscription -SubscriptionId $Cfg.subscriptionId > $null
# ... in order to run:
exec { Start-AzureWebsiteJob #startArgs | out-null }
Not sure if its a help but I had this issue with APIs prior to 2016-01-01 as there was a known bug. I updated to 2016-12-01 as this was the latest at the time and it worked for me.
Have you tried updating the api to the latest and trying again? The latest is 2018-02-01
This is the result of selecting a subscription twice for Resource Manager cmdlets and using Select-AzureSubscription as below:
Select-AzureRmSubscription -SubscriptionID $Cfg.subscriptionId > $null
# this seems to be the reason of the error, if removed - works:
# Select-AzureSubscription -SubscriptionId $Cfg.subscriptionId > $null
To avoid old cmdlets (before RM) we can use Invoke-AzureRmResourceAction. See example at https://stackoverflow.com/a/51712321/511144
A good thing to do before performing a deployment is to validate the resources that if it can be created with the name provided. For EventHub and Storage Account there are built in cmdlets that allows you to check it.
For EventHub
Test-AzureRmEventHubName -Namespace $eventHubNamespaceName | Select-Object -ExpandProperty NameAvailable
if ($availability -eq $true) {
Write-Host -ForegroundColor Green `r`n"Entered Event Hub Namespace name is available"
For Storage Account
while ($true) {
Write-Host -ForegroundColor Yellow `r`n"Enter Storage account name (Press Enter for default) : " -NoNewline
$storageAccountName = Read-Host
if ([string]::IsNullOrEmpty($storageAccountName)) {
$storageAccountName = $defaultstorageAccountName
}
Write-Host -ForegroundColor Yellow `r`n"Checking whether the entered name for Storage account is available"
$availability = Get-AzureRmStorageAccountNameAvailability -Name $storageAccountName | Select-Object -ExpandProperty NameAvailable
if ($availability -eq $true ) {
Write-Host -ForegroundColor Green `r`n"Entered Storage account name is available"
break
}
Write-Host `r`n"Enter valid Storage account name"
}
This will prevent deployment errors by validating before deployment and asking the user to enter a valid name again if the entered one is not present.