Deploy multiple ARM Templates in parallel with PowerShell - azure

I created a Resource Manager Project in Visual Studio 2017 and as result got Deploy-AzureResourceGroup.ps1 that deploys the template to Azure. But I have multiple templates and would like to be able to deploy them in parallel with PowerShell. My current approach iterates over each template and deploys them sequentially what takes a lot of time.
How can I achieve that?
Edit: taking the response from #4c74356b41 into account. I need to execute some more logic as part of the job. Therefore it is not enough just to execute the ResourceGroupDeployment in parallel.

First of all get all relevant templates e.g. via
$armTemplateFiles = Get-ChildItem -Path $PSScriptRoot -Include *.JobTemplate.json -Recurse;
Now iterate over each template file and create a job for each one (these jobs are then executed in parallel)
Code:
foreach ($armTemplateFile in $armTemplateFiles) {
$logic = {
Param(
[object]
[Parameter(Mandatory=$true)]
$ctx,
[object]
[Parameter(Mandatory=$true)]
$armTemplateFile,
[string]
[Parameter(Mandatory=$true)]
$resourceGroupName
)
function Format-ValidationOutput {
param ($ValidationOutput, [int] $Depth = 0)
Set-StrictMode -Off
return #($ValidationOutput | Where-Object { $_ -ne $null } | ForEach-Object { #(' ' * $Depth + ': ' + $_.Message) + #(Format-ValidationOutput #($_.Details) ($Depth + 1)) })
}
# Get related parameters file
$paramTemplateFile = Get-ChildItem -Path $armTemplateFile.FullName.Replace("JobTemplate.json", "JobTemplate.parameters.json")
# Test Deployment
$ErrorMessages = Format-ValidationOutput (Test-AzureRmResourceGroupDeployment -ResourceGroupName $resourceGroupName `
-TemplateFile $armTemplateFile.FullName `
-TemplateParameterFile $paramTemplateFile.FullName `
-DefaultProfile $ctx)
if ($ErrorMessages) {
Write-Host '', 'Validation returned the following errors:', #($ErrorMessages), '', 'Template is invalid.'
}
else { # Deploy
New-AzureRmResourceGroupDeployment -Name (($armTemplateFile.Name).Split(".")[0] + ((Get-Date).ToUniversalTime()).ToString('MMddHHmm')) `
-ResourceGroupName $resourceGroupName `
-TemplateFile $armTemplateFile.FullName `
-TemplateParameterFile $paramTemplateFile.FullName `
-Force `
-ErrorVariable ErrorMessages `
-DefaultProfile $ctx
if ($ErrorMessages) {
Write-Host '', 'Template deployment returned the following errors:', #(#($ErrorMessages) | ForEach-Object { $_.Exception.Message.TrimEnd("`r`n") })
}
}
}
Start-Job $logic -ArgumentList (Get-AzureRmContext), $armTemplateFile, $ResourceGroupName
}
While (Get-Job -State "Running")
{
Start-Sleep 10
Write-Host "Jobs still running..."
}
Get-Job | Receive-Job

Extremely bad solution, a lot of overcomplication.
New-AzureRmResourceGroupDeployment -ResourceGroup xxx -TemplateFile xxx -AsJob
and you need a loop on top of this

Related

How to set the ComputeModel property to Serverless on an Azure SQL Database using PowerShell?

I'm restoring an Azure SQL Database (Serverless) from a deleted database backup using Get-AzSqlDeletedDatabaseBackup and Restore-AzSqlDatabase PowerShell commandlets. The restore works, but the tags and ComputeModel are not restored with the database.
I've tried using Set-AzSqlDatabase:
Set-AzSqlDatabase -ResourceGroupName $resourcegroupname -DatabaseName $databasename -ServerName $servername -ComputeModel "Serverless" -AutoPauseDelayInMinutes 45
Update: I tried the following code and the Kind is set prior to using the Set-AzResource cmdlet, but it doesn't stick
$resource = Get-AzResource -ResourceGroupName $resourcegroupname -ResourceType "Microsoft.Sql/servers/databases" -Name "$servername/$databasename"
Write-Host "Setting ComputeModel to Serverless..."
$resource.Kind = "v12.0,user,vcore,serverless"
$resource
# resource.Kind is successfully set on the $resource object
Write-Host "Set-AzResource..."
$resource | Set-AzResource -Force
Anyone have any ideas?
Thank you.
Cheers,
Andy
The Get-AzSqlDeletedDatabaseBackup and Restore-AzSqlDatabase PowerShell cmdlets are doesn't contain a property to get the ComputeModel.
While Restore and Delete Backup database we don't require the ComputeModel properties. while setting database we need to require the ComputeModel.
If you want to get the compute model for the Azure SQL Database you can use, Get-AzResource command to fetch the specific information.
Thanks #joy wang SO Solution we can get the serverless Azure SQL Database.
Thanks to #holger and #Delliganesh Sevanesan for the help, I was able implement a solution that restores the most recent deleted database (SQL Database), adds some resource tags, and sets the ComputeModel to serverless.
Here's the code:
<#
Purpose: Restore the most recently deleted Azure Sql Database
Dependencies:
Az.Sql PowerShell module
#>
# Set variables first
$resourcegroupname = 'myresourcegroup'
$servername = 'myservername'
$databasename = 'mydatabasename'
[hashtable]$tags = #{
application = "testing"
}
try {
$deleteddatabases = Get-AzSqlDeletedDatabaseBackup -ResourceGroupName $resourcegroupname -ServerName $servername -DatabaseName $databasename
} catch {
Write-Error "Error getting database backups [Get-AzSqlDeletedDatabaseBackup]: " + $_.Exception.Message
exit(1)
}
# Get most recent backup in case there are multiple copies
# Assumes index and order in foreach is the same - proven in test
Write-Host "Database backups:"
$index = 0
$MostRecentBackupIndex = 0
$MostRecentBackupDate = (Get-date).AddDays(-2) # initialize variable with date from two days ago
foreach ($db in $deleteddatabases) {
if ($db.CreationDate -ge $MostRecentBackupDate) {
$MostRecentBackupIndex = $index
$MostRecentBackupDate = $db.CreationDate
Write-Host "Most Recent Database: $($db.DatabaseName) : Created: $($db.CreationDate) : DeleteDate: $($db.DeletionDate)"
}
$index++
}
$deleteddatabase = $deleteddatabases[$MostRecentBackupIndex]
Write-Host "----------------------------------------------------------------------------------"
Write-Host "Restoring: $($deleteddatabase.DatabaseName) from: $($deleteddatabase.CreationDate) backup"
Write-Host "----------------------------------------------------------------------------------"
Write-Host "Deleted database info ResourceId: "
Write-Host $deleteddatabase.ResourceId
try {
Restore-AzSqlDatabase -FromDeletedDatabaseBackup `
-DeletionDate $deleteddatabase.DeletionDate `
-ResourceGroupName $resourcegroupname `
-ServerName $servername `
-TargetDatabaseName $databasename `
-ResourceId $deleteddatabase.ResourceID `
-Edition $deleteddatabase.Edition `
-Vcore 2 `
-ComputeGeneration "Gen5"
} catch {
Write-Error "Error restoring database [Restore-AzSqlDatabase]: " + $_.Exception.Message
exit(1)
}
# Wait a few minutes to allow restore to complete before applying the tags
Start-Sleep -Seconds 180
Write-Host "Applying tags to database..."
try {
$resource = Get-AzResource -ResourceGroupName $resourcegroupname -ResourceType "Microsoft.Sql/servers/databases" -Name "$servername/$databasename"
New-AzTag -ResourceId $resource.Id -Tag $tags
} catch {
Write-Error "Error adding tags to database [Get-AzResource, New-AzTag]: " + $_.Exception.Message
exit(1)
}
Write-Host "Setting ComputeModel to Serverless..."
try {
# Important - must include -AutoPauseDelayInMinutes 60, -MinVcore, and MaxVcore parameters (thanks holger)
Set-AzSqlDatabase -ResourceGroupName $resourcegroupname -DatabaseName $databasename -ServerName $servername -ComputeModel Serverless -AutoPauseDelayInMinutes 60 -MinVcore 1 -MaxVcore 2
} catch {
Write-Error "Error setting serverless mode [Set-AzSqlDatabase]: " + $_.Exception.Message
exit(1)
}
Write-Host "Database restore complete."

Azure Automation Runbook Workflow looses AzContext

I have written the following runbook workflow, but from time to time I see the error when it try's to start or stop a VM:
Start-AzVM : Your Azure credentials have not been set up or have expired, please run Connect-AzAccount to set up your
Azure credentials.
At StartStopVmByTag:46 char:46
+
+ CategoryInfo : CloseError: (:) [Start-AzVM], ArgumentException
+ FullyQualifiedErrorId : Microsoft.Azure.Commands.Compute.StartAzureVMCommand
I have tried passing the $azContext variable in, but I still get this issue, how can I further investigate?
workflow StartStopVmByTag {
$connectionName = "AzRunAsConnection2042";
try {
# Get the connection "AzureRunAsConnection "
$servicePrincipalConnection = Get-AutomationConnection -Name $connectionName
Write-Output "Logging in to Azure..."
$null = Add-AzAccount `
-ServicePrincipal `
-TenantId $servicePrincipalConnection.TenantId `
-ApplicationId $servicePrincipalConnection.ApplicationId `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint
}
catch {
if (!$servicePrincipalConnection) {
$ErrorMessage = "Connection $connectionName not found."
throw $ErrorMessage
}
else {
Write-Error -Message $_.Exception
throw $_.Exception
}
}
[DateTime]$now = [System.TimeZoneInfo]::ConvertTimeBySystemTimeZoneId((Get-Date), 'GMT Standard Time')
$startTag = 'Start Schedule'
Write-Output "*** $now - Runbook Started ***"
# Get Subscriptions
$Subscriptions = Get-AzSubscription
ForEach ($Subscription in $Subscriptions) {
$azContext = Set-AzContext -SubscriptionId $Subscription.Id
# Get all VM's with a Start or Stop Schedule
Write-Output "$($Subscription.Name): Getting VM's..."
[Array]$taggedVms = Get-AzResource -TagName $startTag -ResourceType 'Microsoft.Compute/virtualMachines'
$taggedVms = $taggedVms | Sort-Object -Property Name -Unique
# For each VM, check if start schedule is valid for now
Foreach -Parallel ($taggedVm in $taggedVms) {
Write-Output "$($Subscription.Name): Found Tagged VM: $($taggedVm.Name), $($startTag): $($taggedVm.Tags.$startTag -replace '\s', '')"
$WORKFLOW:null = Start-AzVM -ResourceGroupName $taggedVm.ResourceGroupName -Name $taggedVm.Name -DefaultProfile $azContext -NoWait
}
}
}
I have been struggling with this issue for a while, and I've tried dozens of different workarounds and nothing has worked. I finally resolved it with these registry settings that force .NET applications to use TLS 1.2. I find it very strange that this solution works, but possibly because the TLS 1.2 set as part of any parent task doesn't get passed on to the job.
They probably aren't all required, but it seems to be a best practice these days anyway.
set-itemproperty "HKLM:\SOFTWARE\Microsoft\.NETFramework\v2.0.50727" -name SystemDefaultTlsVersions -value 1 -Type DWord
set-itemproperty "HKLM:\SOFTWARE\Microsoft\.NETFramework\v4.0.30319" -name SchUseStrongCrypto -value 1 -Type DWord
set-itemproperty "HKLM:\SOFTWARE\Wow6432Node\Microsoft\.NETFramework\v2.0.50727" -name SystemDefaultTlsVersions -value 1 -Type DWord
set-itemproperty "HKLM:\SOFTWARE\Wow6432Node\Microsoft\.NETFramework\v4.0.30319" -name SchUseStrongCrypto -value 1 -Type DWord

Azure Automation Account - Runbook Error - Object reference not set to an instance of an object

Last week, I deployed a script to backup some disks using the New-AzSnapshot cmdlet.
I scheduled this script in the Azure Automation Account and for the first 2 days, the script worked very well!
Today, I analyzed the logs and saw that the script had the error below:
Connect-AzAccount : Object reference not set to an instance of an object. At line:12 char:25 + $connectionResult = Connect-AzAccount ` + ~~~~~~~~~~~~~~~~~~~ + CategoryInfo : CloseError: (:) [Connect-AzAccount], NullReferenceException + FullyQualifiedErrorId : Microsoft.Azure.Commands.Profile.ConnectAzureRmAccountCommand
Does anyone have any idea what may be causing this error?
Below the script:
Param(
[string]$resourceGroupName
)
$connection = Get-AutomationConnection -Name AzureRunAsConnection
while (!($connectionResult) -And ($logonAttempt -le 10)) {
$LogonAttempt++
# Logging in to Azure...
$connectionResult = Connect-AzAccount `
-ServicePrincipal `
-Tenant $connection.TenantID `
-ApplicationID $connection.ApplicationID `
-CertificateThumbprint $connection.CertificateThumbprint
Start-Sleep -Seconds 30
}
# Remove old snapshots
$snapshotnames = (Get-AzSnapshot -ResourceGroupName $resourceGroupName).name
foreach($snapname in $snapshotnames)
{
Get-AzSnapshot -ResourceGroupName $resourceGroupName -SnapshotName $snapname | ?{($_.TimeCreated) -lt ([datetime]::UtcNow.AddMinutes(-10080))} | Remove-AzSnapshot -Force
}
foreach ($VMs in Get-AzVM -ResourceGroupName $resourceGroupName) {
#Set local variables
$location = $VMs.Location
#$resourceGroupName = $vmInfo.ResourceGroupName
$timestamp = Get-Date -f MM-dd-yyyy_HH_mm_ss
#Snapshot name of OS data disk
$snapshotName = "bkp-" + $VMs.Name + "-" + $timestamp
#Create snapshot configuration
$snapshot = New-AzSnapshotConfig -SourceUri $VMs.StorageProfile.OsDisk.ManagedDisk.Id -Location $location -CreateOption copy
#Take snapshot
New-AzSnapshot -Snapshot $snapshot -SnapshotName $snapshotName -ResourceGroupName $resourceGroupName
if ($VMs.StorageProfile.DataDisks.Count -ge 1) {
#Condition with more than one data disks
for ($i = 0; $i -le $VMs.StorageProfile.DataDisks.Count - 1; $i++) {
#Snapshot name of OS data disk
$snapshotName = "bkp-" + $VMs.StorageProfile.DataDisks[$i].Name + "-" + $timestamp
#Create snapshot configuration
$snapshot = New-AzSnapshotConfig -SourceUri $VMs.StorageProfile.DataDisks[$i].ManagedDisk.Id -Location $location -CreateOption copy
#Take snapshot
New-AzSnapshot -Snapshot $snapshot -SnapshotName $snapshotName -ResourceGroupName $resourceGroupName
}
}
else {
Write-Host $VMs.Name + " doesn't have any additional data disk."
}
}
You're having a nullreference exception, looking at the location inside the error message (At line:12 char:25). it shows that connection is null.
You need to declare connection first on the same page before you're able to use the variables inside.
See also: What is a NullReferenceException, and how do I fix it?
EDIT: At second notice, I think that you're already declaring the $connection at
$connection = Get-AutomationConnection -Name AzureRunAsConnection
but it looks like that's returning null. In that case, you'll need to figure out why that's returning null, because it should return a filled $connection if connected.
For a safe check to avoid errors: you should check first if the $connection is not null before entering the while, (for example $connection != null could work for Javascript). That way you won't get errors, but keep in mind that you won't get a result through this.

Azure Automation Runbook missing mandatory parameters

I'm trying to set a Tag on all virtual machines in my subscription but I keep getting errors when running the Runbook.
The error is the following:
Get-AzureRmVM : Cannot process command because of one or more missing mandatory parameters: ResourceGroupName. At line:30
Here is my Runbook:
$azureConnection = Get-AutomationConnection -Name 'AzureRunAsConnection'
#Authenticate
try {
Clear-Variable -Name params -Force -ErrorAction Ignore
$params = #{
ServicePrincipal = $true
Tenant = $azureConnection.TenantID
ApplicationId = $azureConnection.ApplicationID
CertificateThumbprint = $azureConnection.CertificateThumbprint
}
$null = Add-AzureRmAccount #params
}
catch {
$errorMessage = $_
Throw "Unable to authenticate with error: $errorMessage"
}
# Discovery of all Azure VM's in the current subscription.
$azurevms = Get-AzureRmVM | Select-Object -ExpandProperty Name
Write-Host "Discovering Azure VM's in the following subscription $SubscriptionID Please hold...."
Write-Host "The following VM's have been discovered in subscription $SubscriptionID"
$azurevms
foreach ($azurevm in $azurevms) {
Write-Host "Checking for tag $vmtagname on $azurevm"
$tagRGname = Get-AzureRmVM -Name $azurevm | Select-Object -ExpandProperty ResourceGroupName
$tags = (Get-AzureRmResource -ResourceGroupName $tagRGname -Name $azurevm).Tags
If ($tags.UpdateWindow){
Write-Host "$azurevm already has the tag $vmtagname."
}
else
{
Write-Host "Creating Tag $vmtagname and Value $tagvalue for $azurevm"
$tags.Add($vmtagname,$tagvalue)
Set-AzureRmResource -ResourceGroupName $tagRGname -ResourceName $azurevm -ResourceType Microsoft.Compute/virtualMachines -Tag $tags -Force `
}
}
Write-Host "All tagging is done"
I tried importing the right modules but this doesn't seem to affect the outcome.
Running the same commands in Cloud Shell does work correctly.
I can reproduce your issue, the error was caused by this part Get-AzureRmVM -Name $azurevm, when running this command, the -ResourceGroupName is needed.
You need to use the Az command Get-AzVM -Name $azurevm, it will work.
Running the same commands in Cloud Shell does work correctly.
In Cloud shell, azure essentially uses the new Az module to run your command, you can understand it runs the Enable-AzureRmAlias before the command, you could check that via debug mode.
Get-AzureRmVM -Name joyWindowsVM -debug
To solve your issue completely, I recommend you to use the new Az module, because the AzureRM module was deprecated and will not be updated.
Please follow the steps below.
1.Navigate to your automation account in the portal -> Modules, check if you have imported the modules Az.Accounts, Az.Compute, Az.Resources, if not, go to Browse Gallery -> search and import them.
2.After import successfully, change your script to the one like below, then it should work fine.
$azureConnection = Get-AutomationConnection -Name 'AzureRunAsConnection'
#Authenticate
try {
Clear-Variable -Name params -Force -ErrorAction Ignore
$params = #{
ServicePrincipal = $true
Tenant = $azureConnection.TenantID
ApplicationId = $azureConnection.ApplicationID
CertificateThumbprint = $azureConnection.CertificateThumbprint
}
$null = Connect-AzAccount #params
}
catch {
$errorMessage = $_
Throw "Unable to authenticate with error: $errorMessage"
}
# Discovery of all Azure VM's in the current subscription.
$azurevms = Get-AzVM | Select-Object -ExpandProperty Name
Write-Host "Discovering Azure VM's in the following subscription $SubscriptionID Please hold...."
Write-Host "The following VM's have been discovered in subscription $SubscriptionID"
$azurevms
foreach ($azurevm in $azurevms) {
Write-Host "Checking for tag $vmtagname on $azurevm"
$tagRGname = Get-AzVM -Name $azurevm | Select-Object -ExpandProperty ResourceGroupName
$tags = (Get-AzResource -ResourceGroupName $tagRGname -Name $azurevm).Tags
If ($tags.UpdateWindow){
Write-Host "$azurevm already has the tag $vmtagname."
}
else
{
Write-Host "Creating Tag $vmtagname and Value $tagvalue for $azurevm"
$tags.Add($vmtagname,$tagvalue)
Set-AzResource -ResourceGroupName $tagRGname -ResourceName $azurevm -ResourceType Microsoft.Compute/virtualMachines -Tag $tags -Force `
}
}
Write-Host "All tagging is done"

Check if Azure VM name exists before deploying arm template

I am trying to use Azure powershell to get VM name (Eg: demovm01, where the VM name is demovm and the suffix is 01.
I want to get the output and automatically append a new suffix of 02 if 01 already exists.
Sample script to get vm name:
$getvm = Get-AzVM -Name "$vmname" -ResourceGroupName "eodemofunction" -ErrorVariable notPresent -ErrorAction SilentlyContinue
if ($notPresent) {
Write-Output "VM not found. Creating now"
}
else {
Write-Output "VM exists."
return $true
}
I want to be able to inject this new vm name to an arm deploy to deploy
This should do it. Will increment until no VM is found and use a simple -replace to inject to your json file. Will also return all thr VM values that are already present in Azure
$i=1
$vmname_base = "vmserver"
$VMexists = #()
do {
#invert int value to double digit string
$int = $i.tostring('00')
$getvm = Get-AzVM -Name "$vmname_base$int" -ResourceGroupName "eodemofunction" -ErrorVariable notPresent -ErrorAction SilentlyContinue
if ($notPresent) {
Write-Output "VM not found. Creating now"
Write-Output "VM created name is $vmname_base$int"
#Set condition to end do while loop
VMcreated = "true"
#commands to inject to json here. I always find replace method the easiest
$JSON = Get-Content azuredeploy.parameters.json
$JSON = $JSON -replace ("Servername","$vmname_base$int")
$JSON | Out-File azuredeploy.parameters.json -Force
}
else {
Write-Output "VMexists."
# Add existing VM to array
$VMexists += "$vmname_base$int"
# Increment version, ie. 01 to 02 to 03 etc
$i++
}
} while ($VMcreated -ne "true")
return $VMexists
Your command could be like below, the $newvmname is that you want.
$vmname = "demovm01"
$getvm = Get-AzVM -Name "$vmname" -ResourceGroupName "<group name>" -ErrorVariable notPresent -ErrorAction SilentlyContinue
if ($notPresent) {
Write-Output "VM not found. Creating now"
}
else {
Write-Output "VM exists."
if($getvm.Name -like '*01'){
$newvmname = $vmname.TrimEnd('01')+"02"
}
Write-Output $newvmname
return $true
}

Resources