Powershell parallel or multithreading job - azure

i have the following script:
ForEach ($lista in $listas) {
$RG = $lista.rg
$VM = $lista.vm
$NIC = $lista.nic
Stop-AzVM -ResourceGroupName $RG -Name $VM -Force
$nic = Get-AzNetworkInterface -ResourceGroupName $RG -Name $NIC
$nic.EnableAcceleratedNetworking = $false
$nic | Set-AzNetworkInterface
Start-AzVM -ResourceGroupName $RG -Name $VM
}
which i can disable on azure vm accellerated network. It works fine but i would like to know if is possible to parallelize it becouse i have to do it on 20-30 vm.
Is possible to do that?
Thanks

Try this, i havnt tested it but it should hopefully work.
$ScriptBlock = {
param($RG,$VM,$NIC)
Stop-AzVM -ResourceGroupName $RG -Name $VM -Force
$nic = Get-AzNetworkInterface -ResourceGroupName $RG -Name $NIC
$nic.EnableAcceleratedNetworking = $false
$nic | Set-AzNetworkInterface
Start-AzVM -ResourceGroupName $RG -Name $VM
}
foreach($lista in $listas) {
# Execute the jobs in parallel
Start-Job $ScriptBlock -ArgumentList $lista.rg, $lista.vm, $lista.nic
}
# Wait for all to complete
While (Get-Job -State "Running") { Start-Sleep 5 }
# Display output from all jobs
$res += (Get-Job | Receive-Job)
# Cleanup
Remove-Job *

Related

Azure: Get resource for azure automation

I have runbook in Azure that i want to use in different RG,
My code
$vms = Get-AzVM -ResourceGroupName RG-TEST
foreach($vm in $vms)
{
$statuscheck = Get-AzVM -ResourceGroupName RG-TEST -Name $vm.Name -Status
if($statuscheck.Statuses.DisplayStatus[1] -eq "VM running")
{
Write-Output "Stopping virtual machine...$($vm.Name)"
Stop-AzVM -ResourceGroupName $vm.ResourceGroupName -Name $vm.Name -Force
}
else
{
Write-output "Virtual machine $($vm.Name) is already in stopped state"
}
}
How can I update the code that the script will get the name of the RG where it's located,
So that the RG is not hard coded
Dont know how to do it
Do something like this
$rgs = $(Get-AzVM)
# Get Distinct RG Names
$rgs = $rgs | Select-Object -Property ResourceGroupName | Sort-Object -Unique
foreach($rg in $rgs){
$vms = Get-AzVM -ResourceGroupName $rg.ResourceGroupName
foreach($vm in $vms)
{
$statuscheck = Get-AzVM -ResourceGroupName $rg.ResourceGroupName -Name $vm.Name -Status
if($statuscheck.Statuses.DisplayStatus[1] -eq "VM running")
{
Write-Output "Stopping virtual machine...$($vm.Name)"
Stop-AzVM -ResourceGroupName $rg.ResourceGroupName -Name $vm.Name -Force
}
else
{
Write-output "Virtual machine $($vm.Name) is already in stopped state"
}
}
}

Azure VMs fails as public ip is allocated to other resource

I am using a powershell script to create multiple Vms based on an image. The first Vm is ok but when attempting the second Vm I get an error saying that :
| Resource /subscriptions/....../networkInterfaces/xxxxx/ipConfigurations/xxxxx is referencing public IP address
| /subscriptions/xxxxxxxxx/providers/Microsoft.Network/publicIPAddresses/Microsoft.Azure.Commands.Network.Models.PSPublicIpAddress that is already allocated to
| resource /subscriptions/......./networkInterfaces/xxxxx/ipConfigurations/xxxxx.
Here is the script I am using:
param(
[string] $WeekNo="NoWeek",
[int] $VmCount=0
)
#$cred = Get-Credential -Message "Enter a username and password for the virtual machine."
## VM Account
# Credentials for Local Admin account you created in the sysprepped (generalized) vhd image
$VMLocalAdminUser = "xxxxx"
$VMLocalAdminSecurePassword = ConvertTo-SecureString "xxxxxxx" -AsPlainText -Force
$image = "/subscriptions/xxxxxxx/resourceGroups/xxxxxx/providers/Microsoft.Compute/images/xxxxxxxxx"
## Azure Account
$LocationName = "SwedenCentral"
$ResourceGroupName = "xxxx_" + $WeekNo
if( -Not( Get-AzureRmResourceGroup -Name $ResourceGroupName -Location $LocationName -ErrorAction Ignore)) {
New-AzureRmResourceGroup -Name $ResourceGroupName -Location $LocationName
Write-Host "ResourceGroup" $ResourceGroupName "created"
$VMSize = "Standard_B2ms"
## Networking
$NetworkName = "xxxxxx_" + $WeekNo + "_net" # "MyNet"
$SubnetName = "MySubnet"
$SubnetAddressPrefix = "10.0.0.0/24"
$VnetAddressPrefix = "10.0.0.0/16"
$SingleSubnet = New-AzVirtualNetworkSubnetConfig -Name $SubnetName -AddressPrefix $SubnetAddressPrefix
$Vnet = New-AzVirtualNetwork -Name $NetworkName -ResourceGroupName $ResourceGroupName -Location $LocationName -AddressPrefix $VnetAddressPrefix -Subnet $SingleSubnet
}
$Credential = New-Object System.Management.Automation.PSCredential ($VMLocalAdminUser, $VMLocalAdminSecurePassword);
$VMName = "xxxx" + $WeekNo
##New-AzVM -ResourceGroupName $ResourceGroupName -Location $LocationName -VM $VirtualMachine -Verbose -Image $image
for($i=1; $i -le $VmCount; $i++){
$VMBaseName = "iCPSEDU" + $WeekNo + $i
$StorageAccount = "xxxxx" + $WeekNo + $i
$PublicIPAddressName = $VMBaseName + "PIP$(Get-Random)"
$NICName = $VMBaseName + "NIC"
$DNSNameLabel = "xxxx" + $WeekNo + $i + "dns" # mydnsname.westus.cloudapp.azure.com
$PIP = New-AzPublicIpAddress -Name $PublicIPAddressName -DomainNameLabel $DNSNameLabel -ResourceGroupName $ResourceGroupName -Location $LocationName -AllocationMethod Dynamic
$NIC = New-AzNetworkInterface -Name $NICName -ResourceGroupName $ResourceGroupName -Location $LocationName -SubnetId $Vnet.Subnets[0].Id -PublicIpAddressId $PIP.Id
Write-Host "Creating VM " $VMBaseName
New-AzVm `
-ResourceGroupName $ResourceGroupName `
-Name $VMBaseName `
-ImageName $image `
-Location $LocationName `
-VirtualNetworkName $Vnet `
-SubnetName $SubnetName `
-SecurityGroupName "myImageNSG" `
-PublicIpAddressName $PIP -Credential $Credential -Size $VMSize -PublicIpSku Standard
Write-Host "VM " $VMBaseName " Created"
Stop-AzVM -ResourceGroupName $ResourceGroupName $VMBaseName -Force -NoWait
Write-Host "VM " $VMBaseName " Stopped"
}
Write-Host "Done."`
To me it seems that the variable used for the PIP is not "flushed" properly between the executions but I have no idea on how to do this?
Or is there something else causing the error?
I have tried adding some delays but without effect.
Create a public IP address and specify a DNS name
Create a NSG
Create a NIC and associate with created pub IP address and NSG
Create a virtual machine configuration and assign the NIC
Create the VM with the config
https://github.com/Azure/azure-docs-powershell-samples/blob/master/virtual-machine/create-vm-detailed/create-windows-vm-detailed.ps1
rough summary of important steps:
$pip = New-AzPublicIpAddress -ResourceGroupName $resourceGroup -Location $location `
-Name "mypublicdns$(Get-Random)" -AllocationMethod Static -IdleTimeoutInMinutes 4
$nsg = New-AzNetworkSecurityGroup -ResourceGroupName $resourceGroup -Location $location `
-Name myNetworkSecurityGroup -SecurityRules $nsgRuleRDP
$nic = New-AzNetworkInterface -Name myNic -ResourceGroupName $resourceGroup -Location $location `
-SubnetId $vnet.Subnets[0].Id -PublicIpAddressId $pip.Id -NetworkSecurityGroupId $nsg.Id
$vmConfig = New-AzVMConfig -VMName $vmName -VMSize Standard_D1 | `
Set-AzVMOperatingSystem -Windows -ComputerName $vmName -Credential $cred | `
Set-AzVMSourceImage -PublisherName MicrosoftWindowsServer -Offer WindowsServer -Skus 2016-Datacenter -Version latest | `
Add-AzVMNetworkInterface -Id $nic.Id
New-AzVM -ResourceGroupName $resourceGroup -Location $location -VM $vmConfig
MS is providing well tested powershell code for various tasks:
I prefer the github samples https://github.com/Azure/azure-docs-powershell-samples over the steps in learn and doc.microsoft.com
also have a deeper look at the Azure CLI examples and template based deployments. It seems to me that MS is abandoning PS a bit.

Azure-DevOps Run powershell Getting Error while running

I have below command which is used for change vm size on azure:
Install-PackageProvider -name nuget -MinimumVersion 2.8.5.201 -Force
IF(!(Get-InstalledModule PSExcel -ErrorAction SilentlyContinue)){
Install-Module -Name PSExcel -ErrorAction SilentlyContinue -Force
Import-Module PSExcel -Force
}
IF(!(Get-InstalledModule ImportExcel -ErrorAction SilentlyContinue)){
Install-Module -Name ImportExcel -ErrorAction SilentlyContinue -Force
Import-Module ImportExcel -Force
}
IF(!(Get-InstalledModule az -ErrorAction SilentlyContinue)){
Install-Module -Name az -ErrorAction SilentlyContinue -Force
Import-Module az -Force
}
Import-Module PSExcel -Force
Import-Module ImportExcel -Force
$ExcelFile = "D:\test-data (3).xlsx"
$objExcel = New-Excel -Path $ExcelFile
$WorkBook = $objExcel | Get-Workbook
$NewExcelFile ="D:\test-data1.xlsx"
copy-item $ExcelFile -Destination $NewExcelFile
$sheetName = "Sheet2"
$newExcel = New-Object -ComObject Excel.Application
$newwb = $newExcel.Workbooks.Open($NewExcelFile)
$WorkSheet = $newwb.sheets.item($sheetName)
$WorksheetRange = $workSheet.UsedRange
$RowCount = $WorksheetRange.Rows.Count
$sheet=$newwb.Worksheets.Item(1)
$q=$RowCount
write-host 260
$q1="R"+$q
$r1 = $sheet.Range("R2:$q1")
$r1.cut()
$i1="I"+$q
$r2 = $sheet.Range("I2:$i1")
$sheet.Paste($r2)
$newwb.Close($true)
$newExcel.Quit()
#Import-Module az -Force
#Connect-AzAccount
Write-Host 273
#Loop through all items in the excel
ForEach($Worksheet in #($Workbook.Worksheets))
{
$totalNoOfRecords = $Worksheet.Dimension.Rows
$totalNoOfItems = $totalNoOfRecords - 1
# Declaring starting positions first row and column names
$rowNo, $Instance_Type_Current = 1, 9
$rowNo, $Instance_Type_Recommended = 1, 18
$rowNo, $Resource_Group= 1,42
$rowNo, $Resource_Name = 1,2
$rowNo, $osname = 1,8
if ($totalNoOfRecords -gt 1)
{
#Loop to get values from excel file
for ($i = 1; $i -le $totalNoOfRecords - 1; $i++)
{
$newvmsize = $WorkSheet.Cells.Item($rowNo + $i, $Instance_Type_Recommended).text
$vmName = $WorkSheet.Cells.Item($rowNo + $i, $Resource_Name).text
$resourceGroup = $WorkSheet.Cells.Item($rowNo + $i, $Resource_Group).text
$currentvmsize = $WorkSheet.Cells.Item($rowNo + $i, $Instance_Type_Current).text
$Checkwindowsos = $WorkSheet.Cells.Item($rowNo + $i, $osname).text
$currentsizefromazure = (Get-AzVM -ResourceGroupName $resourceGroup -VMName $vmName -ErrorAction SilentlyContinue).HardwareProfile.VmSize
#If only windows OS then VM size will update on azure
IF($Checkwindowsos -match 'windows'){
IF($currentvmsize -eq $currentsizefromazure) {
Stop-AzVM -ResourceGroupName $resourceGroup -Name $vmName -Force
$vm = Get-AzVM -ResourceGroupName $resourceGroup -VMName $vmName -ErrorAction SilentlyContinue
$vm.HardwareProfile.VmSize = $newvmsize
Write-Host 304
$vmstatus = $null
while($vmstatus -notmatch 'deallocated'){
$vmstatus=((get-azvm -Name $VmName -ResourceGroupName $resourceGroup -Status).Statuses[1]).code
}
#Start VM Snapshot(Backup)
$vm1 = get-azvm -Name $VmName -ResourceGroupName $resourceGroup
write-host $VmName
$snapshotdisk = $vm1.StorageProfile
$vmlocation =$vm1.location
#backup OS DISK
$OSDiskSnapshotConfig = New-AzSnapshotConfig -SourceUri $snapshotdisk.OsDisk.ManagedDisk.id -CreateOption Copy -Location $vmlocation -OsType Windows
$snapshotNameOS = "$($snapshotdisk.OsDisk.Name)_snapshot_$(Get-Date -Format ddMMyy)"
New-AzSnapshot -ResourceGroupName $resourceGroup -SnapshotName $snapshotNameOS -Snapshot $OSDiskSnapshotConfig
Write-Output "VM-Name $($vm1.name) OS Disk Snapshot End & Snapshot name is $snapshotNameOS"
Write-Host 319
#backup Data DISK
$dataDisks = ($snapshotdisk.DataDisks).name
IF($dataDisks -ne $null){
foreach ($datadisk in $datadisks) {
$dataDisk = Get-AzDisk -ResourceGroupName $vm1.ResourceGroupName -DiskName $datadisk
Write-Output "VM $($vm1.name) data Disk Name $($datadisk.Name) Snapshot Begin"
$DataDiskSnapshotConfig = New-AzSnapshotConfig -SourceUri $dataDisk.Id -CreateOption Copy -Location $vmlocation
$snapshotNameData = "$($datadisk.name)_snapshot_$(Get-Date -Format ddMMyy)"
New-AzSnapshot -ResourceGroupName $resourceGroup -SnapshotName $snapshotNameData -Snapshot $DataDiskSnapshotConfig
Write-Output "VM $($vm1.name) data Disk $($datadisk.Name) Snapshot End"
}}
Update-AzVM -VM $vm -ResourceGroupName $resourceGroup -ErrorAction SilentlyContinue
Start-AzVM -ResourceGroupName $resourceGroup -Name $vmName -ErrorAction SilentlyContinue
$vmstatus = $null
while($vmstatus -notmatch 'running'){
$vmstatus=((get-azvm -Name $VmName -ResourceGroupName $resourceGroup -Status).Statuses[1]).code
}
Write-Host Your VM name $vmName is started and changed size from $currentvmsize changed to $newvmsize
}
elseif($currentsizefromazure -eq $newvmsize) { Write-host Your VM $vmName is already Upgraded}
else{Write-host VM current size is mismatched from azure size}
}
}
}
}
Giving below error while running this script on azure devops:
Please help to resolve error.
Unable to get the Open property of the Workbooks class
At C:\Users\azureadmin\Downloads\vsts-agent-win-x64-2.195.2\_work\1\s\Automatic VM backup.ps1:28 char:1
+ $newwb = $newExcel.Workbooks.Open($NewExcelFile)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], COMException
+ FullyQualifiedErrorId : System.Runtime.InteropServices.COMException
Note: Script is running on self hosted agent and this script is running when normal ISE also azconnection through devops azure
p.s. This script is change vm size and get backup

How to see logs for failure reason when creating VM

I am creating an Azure VM using the below PowerShell script
$username = 'xxxxxxx'
$password = ConvertTo-SecureString 'xxxxxxxx&Cx4cA' -AsPlainText -Force
$WindowsCred = New-Object System.Management.Automation.PSCredential ($username, $password)
New-AzVm `
-ResourceGroupName "1-83eb7c26-playground-sandbox" `
-Name "aznewvm23" `
-Location 'eastus' `
-VirtualNetworkName "mynewazvm23" `
-SubnetName "default" `
-SecurityGroupName "mynewvmNSG23" `
-PublicIpAddressName "mypublicip23" `
-Credential $WindowsCred `
-OpenPorts 80,3389 `
-AsJob
And when I see job status using Get-Job it's getting failed.
Id Name PSJobTypeName State HasMoreData Location Command
-- ---- ------------- ----- ----------- -------- -------
1 Long Running O… AzureLongRunni… Failed True localhost New-AzVM
I want to see the failure reason, How can I get job logs using PowerShell?
PS az204_prep> New-AzVM `
>> -ResourceGroupName 'psdemo-rg' `
>> -Name 'psdemo-win-az4' `
>> -Image 'Win2019Datacenter'`
>> -Credential $WindowsCred `
>> -OpenPorts 3389 `
>> -AsJob
Id Name PSJobTypeName State HasMoreData Location Command
-- ---- ------------- ----- ----------- -------- -------
1 Long Running O… AzureLongRunni… Running True localhost New-AzVM
PS az204_prep> Get-Job
Id Name PSJobTypeName State HasMoreData Location Command
-- ---- ------------- ----- ----------- -------- -------
1 Long Running O… AzureLongRunni… Failed True localhost New-AzVM
PS az204_prep> (Get-Job 1).JobStateInfo.State
Failed
PS az204_prep> (Get-Job 1).JobStateInfo.Reason
PS az204_prep>
To Get the Job Details, You can use Get-Job | Format-List -Property * , to get the Full details of the Job performed in Powershell.
The Operation is failing as you are creating a VM with Default Configuration and default command , So you don't have to use -As Job there. Please remove -As Job and it will successfully get created.
I tested it on my environment using the same command you are using (removing -As Job):
$username = 'xxxxxxx'
$password = ConvertTo-SecureString 'xxxxxxxx&Cx4cA' -AsPlainText -Force
$WindowsCred = New-Object System.Management.Automation.PSCredential ($username, $password)
New-AzVM -ResourceGroupName 'myresourcegroupname' -Name 'psdemo-win-az4' -Image 'Win2019Datacenter' -Credential $WindowsCred -OpenPorts 443
Outputs:
Reference:
New-AzVM (Az.Compute) | Microsoft Docs
Creating with -As Job :
$VMLocalAdminUser = "LocalAdminUser"
$VMLocalAdminSecurePassword = ConvertTo-SecureString "Password" -AsPlainText -Force
$LocationName = "westus"
$ResourceGroupName = "myresourcegroup"
$ComputerName = "mycomputername"
$VMName = "MYVMName"
$VMSize = "Standard_DS3"
$NetworkName = "Myvnet"
$NICName = "MyNIC"
$SubnetName = "MySubnet"
$SubnetAddressPrefix = "10.0.0.0/24"
$VnetAddressPrefix = "10.0.0.0/16"
$SingleSubnet = New-AzVirtualNetworkSubnetConfig -Name $SubnetName -AddressPrefix $SubnetAddressPrefix
$Vnet = New-AzVirtualNetwork -Name $NetworkName -ResourceGroupName $ResourceGroupName -Location $LocationName -AddressPrefix $VnetAddressPrefix -Subnet $SingleSubnet
$NIC = New-AzNetworkInterface -Name $NICName -ResourceGroupName $ResourceGroupName -Location $LocationName -SubnetId $Vnet.Subnets[0].Id
$Credential = New-Object System.Management.Automation.PSCredential ($VMLocalAdminUser, $VMLocalAdminSecurePassword);
$VirtualMachine = New-AzVMConfig -VMName $VMName -VMSize $VMSize
$VirtualMachine = Set-AzVMOperatingSystem -VM $VirtualMachine -Windows -ComputerName $ComputerName -Credential $Credential -ProvisionVMAgent -EnableAutoUpdate
$VirtualMachine = Add-AzVMNetworkInterface -VM $VirtualMachine -Id $NIC.Id
$VirtualMachine = Set-AzVMSourceImage -VM $VirtualMachine -PublisherName 'MicrosoftWindowsServer' -Offer 'WindowsServer' -Skus '2012-R2-Datacenter' -Version latest
New-AzVM -ResourceGroupName $ResourceGroupName -Location $LocationName -VM $VirtualMachine -Verbose -AsJob
I failed it by not satisfying the password Criteria, So, when I check the error message I can see why it failed.
After I change the Password satisfying the criteria the job succeeds.
In Portal:

Multiple VM Creation by ARM Powershell approach

I have a ps workflow(.psm file) where I am trying to create 5 vms in parallel. I am using ARM cmdlets.I am getting an error-
Error- Cannot validate argument on parameter 'SubnetId'. The argument is null or empty. Provide an argument that is not null or empty, and then try the command
again.
Here is my challange-
Even if I remove -parallel parameter from foreach even then its not making any difference.
If I run the same code NOT inside a workflow(ps1 file) removing -parralel parameter I am able to
create 5 vms
Code-
workflow Create-VMs
{
$UserName = "abc#cde.onmicrosoft.com"
$pwd = ConvertTo-SecureString "xxxxxxxx" -AsPlainText -Force
$AzureCredential = New-Object System.Management.Automation.PSCredential($UserName, $pwd)
login-azurermaccount -credential $AzureCredential
Add-AzureRmAccount -Credential $AzureCredential
Select-AzureRmSubscription -SubscriptionName "xxxxx"
$virtualNetworkName = "myvpn"
$locationName = "East US"
$ResourceGroupName = "myrg"
$user = "adminuser"
$password = "AdminPass123"
$VMSize = "Standard_D2"
$sourcevhd = "https://abc.blob.core.windows.net/vhds/windowsserver2008.vhd"
$virtualNetwork = Get-AzureRmVirtualNetwork -ResourceGroupName $ResourceGroupName -Name $virtualNetworkName
foreach -parallel($i in 1..5)
{
$VMName = "myname" + $i
$destinationVhd = "https://abc.blob.core.windows.net/vhds/windowsserver2008" + $i + ".vhd"
$staticip = "dynamicip" + $i
$virtualNetwork = Get-AzureRmVirtualNetwork -ResourceGroupName $ResourceGroupName -Name $virtualNetworkName
$publicIp = New-AzureRmPublicIpAddress -Name $staticip -ResourceGroupName $ResourceGroupName -Location $locationName -AllocationMethod Dynamic
$networkInterface = New-AzureRmNetworkInterface -ResourceGroupName $ResourceGroupName -Name $VMName -Location $locationName -SubnetId $virtualNetwork.Subnets[0].Id -PublicIpAddressId $publicIp.Id
$vmConfig = New-AzureRmVMConfig -VMName $VMName -VMSize $VMSize
$vmConfig = Set-AzureRmVMOSDisk -VM $vmConfig -Name $VMName -VhdUri $destinationVhd -CreateOption FromImage -Windows -SourceImageUri $sourcevhd
$vmConfig = Add-AzureRmVMNetworkInterface -VM $vmConfig -Id $networkInterface.Id
$securePassword = ConvertTo-SecureString $password -AsPlainText -Force
$cred = New-Object System.Management.Automation.PSCredential ($user, $securePassword)
Set-AzureRmVMOperatingSystem -VM $vmConfig -Windows -Credential $cred -ProvisionVMAgent -ComputerName $VMName
New-AzureRmVM -VM $vmConfig -Location $locationName -ResourceGroupName $ResourceGroupName
}
}
Not able to find out what is the actual problem. Any other approach for creating multiple vms in parallel using ARM ?
I've not had much luck with -parallel and the Azure cmdlets, but I can think of two options for you:
1) use PowerShell jobs - this would require you to login for each job, here's a snippet of how I do it - the "job" in this case is removing a resource group.
foreach ($AzureResourceGroup in $AzureResourceGroups) {
Start-Job -Name $AzureResourceGroup {
Param($AzureResourceGroup, $creds, $tenantID, $subscriptionName)
Login-AzureRmAccount -ServicePrincipal -Credential $creds -TenantId $tenantId
Select-AzureRmSubscription -SubscriptionName $subscriptionName
Remove-AzureRMResourceGroup -Force -Verbose -Name $AzureResourceGroup
} -ArgumentList $AzureResourceGroup, $creds, $tenantId, $subscriptionName
}
Get-Job | Wait-Job | Receive-Job
2) I think better way would be to use a JSON template, and have Azure orchestrate the deployment. There's a sample of how of the template and how to do it here: https://github.com/bmoore-msft/AzureRM-Samples/tree/master/VMFromImageMulti - there's a readme in the root to show how to deploy it.

Resources