How to detect which version of the Az PowerShell module collection is installed on an Azure DevOps agent? - azure

On Azure DevOps agents there is no Az module collection installed - Get-InstalledModule Az returns $null. But all Az modules are just available - Get-Module Az* -ListAvailable returns them all.
What is the best way to test if a particular version of the Az module collection is available? Unfortunately the Az module itself is not present in the regular module space; it only appears in the list of installed modules if installed: Get-Module Az -ListAvailable always returns $null.
Just to be sure we always test whether a particular minimum version of the Az module collection is installed. And if not, then we install it. As this easily takes a couple of minutes to complete, ideally we only do it when really necessary.

Please check the images of Microsoft-hosted agents, for example, for Windows Server 2019 image, you can get Az PowerShell module from following link:
https://github.com/actions/virtual-environments/blob/master/images/win/Windows2019-Readme.md#az-powershell-module
Screenshot:

We have investigated Microsoft's Azure PowerShell task, as this task's feature is to enable an Az version of choice or choose the latest available. The latter is done by the function Get-LatestModule in Utility.ps1 which can be found here: https://github.com/microsoft/azure-pipelines-tasks/blob/master/Tasks/AzurePowerShellV5/Utility.ps1
Our full logic in the custom task is now capable of finding any installed version when running on a self-hosted agent or picking the latest on the Microsoft agent:
# On our self-hosted agent the Az module is installed
$installedVersion = (Get-InstalledModule -Name 'Az' -AllVersions -ErrorAction SilentlyContinue).Version | Sort-Object -Desc | Select-Object -First 1
if ('2.6.0' -gt $installedVersion) {
# On Microsoft hosted agents the Az module itself is not present, but all the related Az modules are on disk in a specific folder
# This code is taken from Microsoft Azure PowerShell task (https://github.com/microsoft/azure-pipelines-tasks/blob/master/Tasks/AzurePowerShellV5/Utility.ps1)
$hostedAgentAzModulePath = Get-LatestModule -patternToMatch "^az_[0-9]+\.[0-9]+\.[0-9]+$" -patternToExtract "[0-9]+\.[0-9]+\.[0-9]+$"
if (-not $hostedAgentAzModulePath) {
# The hosted Az modules cannot be found. So proceed with installing it from the PowerShell gallery
Write-Information -MessageData "INFO --- Install module 'Az'." -InformationAction Continue
Set-PSRepository -Name 'PSGallery' -InstallationPolicy Trusted
Install-Module -Name 'Az' -AllowClobber -Force -MinimumVersion '2.6.0' -Scope CurrentUser
} else {
# Append the Az modules path to the PowerShell modules path
$env:PSModulePath = $hostedAgentAzModulePath + ";" + $env:PSModulePath
$env:PSModulePath = $env:PSModulePath.TrimStart(';')
}
}
Import-Module -MinimumVersion '2.6.0' -Name 'Az' -Force -Scope 'Global'
Write-Information `
-MessageData "INFO --- Imported Az module version $(Get-Module Az | Select-Object Version | ForEach-Object {$_.Version})." `
-InformationAction Continue
Get-LatestModule copied from Utility.ps1:
function Get-LatestModule {
[CmdletBinding()]
param([string] $patternToMatch,
[string] $patternToExtract)
$resultFolder = ""
$regexToMatch = New-Object -TypeName System.Text.RegularExpressions.Regex -ArgumentList $patternToMatch
$regexToExtract = New-Object -TypeName System.Text.RegularExpressions.Regex -ArgumentList $patternToExtract
$maxVersion = [version] "0.0.0"
$modulePath = $env:SystemDrive + "\Modules";
try {
if (-not (Test-Path -Path $modulePath)) {
return $resultFolder
}
$moduleFolders = Get-ChildItem -Directory -Path $modulePath | Where-Object { $regexToMatch.IsMatch($_.Name) }
foreach ($moduleFolder in $moduleFolders) {
$moduleVersion = [version] $($regexToExtract.Match($moduleFolder.Name).Groups[0].Value)
if($moduleVersion -gt $maxVersion) {
$modulePath = [System.IO.Path]::Combine($moduleFolder.FullName,"Az\$moduleVersion\Az.psm1")
if(Test-Path -LiteralPath $modulePath -PathType Leaf) {
$maxVersion = $moduleVersion
$resultFolder = $moduleFolder.FullName
} else {
Write-Verbose "A folder matching the module folder pattern was found at $($moduleFolder.FullName) but didn't contain a valid module file"
}
}
}
}
catch {
Write-Verbose "Attempting to find the Latest Module Folder failed with the error: $($_.Exception.Message)"
$resultFolder = ""
}
Write-Verbose "Latest module folder detected: $resultFolder"
return $resultFolder
}

Related

Azure Powershell Script Force FTPS Set-AzWebApp : Cannot convert 'System.Object[]' to the type 'System.String' required by parameter

I am currently trying to run a script in Azure that would go through all of our Web Apps and turn on FTPS.
This is what I currently have
$Subscriptions = Get-AzSubscription
foreach ($sub in $Subscriptions) {
Get-AzSubscription -SubscriptionName $sub.Name | Set-AzContext
$GetName = (Get-AzWebApp).Name
$GetRG = (Get-AzWebApp).ResourceGroup
Set-AzWebapp -Name $GetName -ResourceGroupName $GetRG -FtpsState FtpsOnly
}
Set-AzWebApp : Cannot convert 'System.Object[]' to the type 'System.String' required by parameter 'Name'. Specified
method is not supported.
I currently am getting this error, which I dont understand as .Name and .ResourceGroup, from my understanding are already strings. I am very new to powershell so any help would be greatly appreciated. Thanks everyone!
Your example calls Az-WebApp with no parameters, which gets all apps in the subscription - this is a collection - and then tries to get the Name of that result, which is what causes your error.
You need to loop through each app in the subscription as well as looping through each subscription, as in:
# Get all subscriptions and iterate them
Get-AzSubscription | ForEach-Object {
Set-AzContext -SubscriptionName $_.Name
# Get all web apps in the subscription and iterate them
Get-AzWebApp | ForEach-Object {
Set-AzWebApp -Name $_.Name -ResourceGroupName $_.ResourceGroup -FtpsState FtpsOnly
}
}

How to import automatically Az Powershell modules in azure automation account?

I am trying to install az powershell modules from my automation runbook with powershell script. But i am failing to do that. I have tried the solution in a similar topic which is here but it didn't work for me. The code is below:
$AAccName = "my-aa"
$RGName = "my-rg"
$deps1 = #("Az.Accounts","Az.Profile")
foreach($dep in $deps1){
$module = Find-Module -Name $dep
$link = $module.RepositorySourceLocation + "/package/" + $module.Name + "/" + $module.Version
New-AzAutomationModule -AutomationAccountName $AAccName -Name $module.Name -ContentLinkUri $link -ResourceGroupName $RGName
}
The error , i get:
Exception calling "ShouldContinue" with "2" argument(s): "A command
that prompts the user failed because the host program or the command
type does not support user interaction. The host was attempting to
request confirmation with the following message: PowerShellGet
requires NuGet provider version '2.8.5.201' or newer to interact with
NuGet-based repositories. The NuGet provider must be available in
'C:\Program Files\PackageManagement\ProviderAssemblies' or
'C:\Users\Client\AppData\Roaming\PackageManagement\ProviderAssemblies'.
You can also install the NuGet provider by running
'Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201
-Force'. Do you want PowerShellGet to install and import the NuGet provider now?" At C:\Program
Files\WindowsPowerShell\Modules\PowerShellGet\1.0.0.1\PSModule.psm1:7455
char:8 + if($Force -or
$psCmdlet.ShouldContinue($shouldContinueQueryMessag ... +
I have executed the command given in this message but i get this error:
Install-PackageProvider : No match was found for the specified search
criteria for the provider 'NuGet'. The package provider requires
'PackageManagement' and 'Provider' tags.
Do you have any idea how to add module in Azure Automation Account with script?
Update:
When i use Import-Module -Name Az.Profile -Force command, I get this error:
Import-Module : The specified module 'Az.Profile' was not loaded
because no valid module file was found in any module directory.
This should be because that the module is not installed on the module directory. When i manually add the module from the module gallery, it works.
Check this MS recommended PS script to update modules in Automation account. It's reference is given here: Use the update runbook to update a specific module version.
Script:
<#
Copyright (c) Microsoft Corporation. All rights reserved.
Licensed under the MIT License.
#>
<#
.SYNOPSIS
Update Azure PowerShell modules in an Azure Automation account.
.DESCRIPTION
This Azure Automation runbook updates Azure PowerShell modules imported into an
Azure Automation account with the module versions published to the PowerShell Gallery.
Prerequisite: an Azure Automation account with an Azure Run As account credential.
.PARAMETER ResourceGroupName
The Azure resource group name.
.PARAMETER AutomationAccountName
The Azure Automation account name.
.PARAMETER SimultaneousModuleImportJobCount
(Optional) The maximum number of module import jobs allowed to run concurrently.
.PARAMETER AzureModuleClass
(Optional) The class of module that will be updated (AzureRM or Az)
If set to Az, this script will rely on only Az modules to update other modules.
Set this to Az if your runbooks use only Az modules to avoid conflicts.
.PARAMETER AzureEnvironment
(Optional) Azure environment name.
.PARAMETER Login
(Optional) If $false, do not login to Azure.
.PARAMETER ModuleVersionOverrides
(Optional) Module versions to use instead of the latest on the PowerShell Gallery.
If $null, the currently published latest versions will be used.
If not $null, must contain a JSON-serialized dictionary, for example:
'{ "AzureRM.Compute": "5.8.0", "AzureRM.Network": "6.10.0" }'
or
#{ 'AzureRM.Compute'='5.8.0'; 'AzureRM.Network'='6.10.0' } | ConvertTo-Json
.PARAMETER PsGalleryApiUrl
(Optional) PowerShell Gallery API URL.
.LINK
https://learn.microsoft.com/en-us/azure/automation/automation-update-azure-modules
#>
[Diagnostics.CodeAnalysis.SuppressMessageAttribute("PSUseApprovedVerbs", "")]
param(
[Parameter(Mandatory = $true)]
[string] $ResourceGroupName,
[Parameter(Mandatory = $true)]
[string] $AutomationAccountName,
[int] $SimultaneousModuleImportJobCount = 10,
[string] $AzureModuleClass = 'AzureRM',
[string] $AzureEnvironment = 'AzureCloud',
[bool] $Login = $true,
[string] $ModuleVersionOverrides = $null,
[string] $PsGalleryApiUrl = 'https://www.powershellgallery.com/api/v2'
)
$ErrorActionPreference = "Continue"
#region Constants
$script:AzureRMProfileModuleName = "AzureRM.Profile"
$script:AzureRMAutomationModuleName = "AzureRM.Automation"
$script:GetAzureRmAutomationModule = "Get-AzureRmAutomationModule"
$script:NewAzureRmAutomationModule = "New-AzureRmAutomationModule"
$script:AzAccountsModuleName = "Az.Accounts"
$script:AzAutomationModuleName = "Az.Automation"
$script:GetAzAutomationModule = "Get-AzAutomationModule"
$script:NewAzAutomationModule = "New-AzAutomationModule"
$script:AzureSdkOwnerName = "azure-sdk"
#endregion
#region Functions
function ConvertJsonDictTo-HashTable($JsonString) {
try{
$JsonObj = ConvertFrom-Json $JsonString -ErrorAction Stop
} catch [System.ArgumentException] {
throw "Unable to deserialize the JSON string for parameter ModuleVersionOverrides: ", $_
}
$Result = #{}
foreach ($Property in $JsonObj.PSObject.Properties) {
$Result[$Property.Name] = $Property.Value
}
$Result
}
# Use the Run As connection to login to Azure
function Login-AzureAutomation([bool] $AzModuleOnly) {
try {
$RunAsConnection = Get-AutomationConnection -Name "AzureRunAsConnection"
Write-Output "Logging in to Azure ($AzureEnvironment)..."
if (!$RunAsConnection.ApplicationId) {
$ErrorMessage = "Connection 'AzureRunAsConnection' is incompatible type."
throw $ErrorMessage
}
if ($AzModuleOnly) {
Connect-AzAccount `
-ServicePrincipal `
-TenantId $RunAsConnection.TenantId `
-ApplicationId $RunAsConnection.ApplicationId `
-CertificateThumbprint $RunAsConnection.CertificateThumbprint `
-Environment $AzureEnvironment
Select-AzSubscription -SubscriptionId $RunAsConnection.SubscriptionID | Write-Verbose
} else {
Add-AzureRmAccount `
-ServicePrincipal `
-TenantId $RunAsConnection.TenantId `
-ApplicationId $RunAsConnection.ApplicationId `
-CertificateThumbprint $RunAsConnection.CertificateThumbprint `
-Environment $AzureEnvironment
Select-AzureRmSubscription -SubscriptionId $RunAsConnection.SubscriptionID | Write-Verbose
}
} catch {
if (!$RunAsConnection) {
$RunAsConnection | fl | Write-Output
Write-Output $_.Exception
$ErrorMessage = "Connection 'AzureRunAsConnection' not found."
throw $ErrorMessage
}
throw $_.Exception
}
}
# Checks the PowerShell Gallery for the latest available version for the module
function Get-ModuleDependencyAndLatestVersion([string] $ModuleName) {
$ModuleUrlFormat = "$PsGalleryApiUrl/Search()?`$filter={1}&searchTerm=%27{0}%27&targetFramework=%27%27&includePrerelease=false&`$skip=0&`$top=40"
$ForcedModuleVersion = $ModuleVersionOverridesHashTable[$ModuleName]
$CurrentModuleUrl =
if ($ForcedModuleVersion) {
$ModuleUrlFormat -f $ModuleName, "Version%20eq%20'$ForcedModuleVersion'"
} else {
$ModuleUrlFormat -f $ModuleName, 'IsLatestVersion'
}
$SearchResult = Invoke-RestMethod -Method Get -Uri $CurrentModuleUrl -UseBasicParsing
if (!$SearchResult) {
Write-Verbose "Could not find module $ModuleName on PowerShell Gallery. This may be a module you imported from a different location. Ignoring this module"
} else {
if ($SearchResult.Length -and $SearchResult.Length -gt 1) {
$SearchResult = $SearchResult | Where-Object { $_.title.InnerText -eq $ModuleName }
}
if (!$SearchResult) {
Write-Verbose "Could not find module $ModuleName on PowerShell Gallery. This may be a module you imported from a different location. Ignoring this module"
} else {
$PackageDetails = Invoke-RestMethod -Method Get -UseBasicParsing -Uri $SearchResult.id
# Ignore the modules that are not published as part of the Azure SDK
if ($PackageDetails.entry.properties.Owners -ne $script:AzureSdkOwnerName) {
Write-Warning "Module : $ModuleName is not part of azure sdk. Ignoring this."
} else {
$ModuleVersion = $PackageDetails.entry.properties.version
$Dependencies = $PackageDetails.entry.properties.dependencies
#($ModuleVersion, $Dependencies)
}
}
}
}
function Get-ModuleContentUrl($ModuleName) {
$ModuleContentUrlFormat = "$PsGalleryApiUrl/package/{0}"
$VersionedModuleContentUrlFormat = "$ModuleContentUrlFormat/{1}"
$ForcedModuleVersion = $ModuleVersionOverridesHashTable[$ModuleName]
if ($ForcedModuleVersion) {
$VersionedModuleContentUrlFormat -f $ModuleName, $ForcedModuleVersion
} else {
$ModuleContentUrlFormat -f $ModuleName
}
}
# Imports the module with given version into Azure Automation
function Import-AutomationModule([string] $ModuleName, [bool] $UseAzModule = $false) {
$NewAutomationModule = $null
$GetAutomationModule = $null
if ($UseAzModule) {
$GetAutomationModule = $script:GetAzAutomationModule
$NewAutomationModule = $script:NewAzAutomationModule
} else {
$GetAutomationModule = $script:GetAzureRmAutomationModule
$NewAutomationModule = $script:NewAzureRmAutomationModule
}
$LatestModuleVersionOnGallery = (Get-ModuleDependencyAndLatestVersion $ModuleName)[0]
$ModuleContentUrl = Get-ModuleContentUrl $ModuleName
# Find the actual blob storage location of the module
do {
$ModuleContentUrl = (Invoke-WebRequest -Uri $ModuleContentUrl -MaximumRedirection 0 -UseBasicParsing -ErrorAction Ignore).Headers.Location
} while (!$ModuleContentUrl.Contains(".nupkg"))
$CurrentModule = & $GetAutomationModule `
-Name $ModuleName `
-ResourceGroupName $ResourceGroupName `
-AutomationAccountName $AutomationAccountName
if ($CurrentModule.Version -eq $LatestModuleVersionOnGallery) {
Write-Output "Module : $ModuleName is already present with version $LatestModuleVersionOnGallery. Skipping Import"
} else {
Write-Output "Importing $ModuleName module of version $LatestModuleVersionOnGallery to Automation"
& $NewAutomationModule `
-ResourceGroupName $ResourceGroupName `
-AutomationAccountName $AutomationAccountName `
-Name $ModuleName `
-ContentLink $ModuleContentUrl > $null
}
}
# Parses the dependency got from PowerShell Gallery and returns name and version
function GetModuleNameAndVersionFromPowershellGalleryDependencyFormat([string] $Dependency) {
if ($null -eq $Dependency) {
throw "Improper dependency format"
}
$Tokens = $Dependency -split":"
if ($Tokens.Count -ne 3) {
throw "Improper dependency format"
}
$ModuleName = $Tokens[0]
$ModuleVersion = $Tokens[1].Trim("[","]")
#($ModuleName, $ModuleVersion)
}
# Validates if the given list of modules has already been added to the module import map
function AreAllModulesAdded([string[]] $ModuleListToAdd) {
$Result = $true
foreach ($ModuleToAdd in $ModuleListToAdd) {
$ModuleAccounted = $false
# $ModuleToAdd is specified in the following format:
# ModuleName:ModuleVersionSpecification:
# where ModuleVersionSpecification follows the specifiation
# at https://learn.microsoft.com/en-us/nuget/reference/package-versioning#version-ranges-and-wildcards
# For example:
# AzureRm.profile:[4.0.0]:
# or
# AzureRm.profile:3.0.0:
# In any case, the dependency version specification is always separated from the module name with
# the ':' character. The explicit intent of this runbook is to always install the latest module versions,
# so we want to completely ignore version specifications here.
$ModuleNameToAdd = $ModuleToAdd -replace '\:.*', ''
foreach($AlreadyIncludedModules in $ModuleImportMapOrder) {
if ($AlreadyIncludedModules -contains $ModuleNameToAdd) {
$ModuleAccounted = $true
break
}
}
if (!$ModuleAccounted) {
$Result = $false
break
}
}
$Result
}
# Creates a module import map. This is a 2D array of strings so that the first
# element in the array consist of modules with no dependencies.
# The second element only depends on the modules in the first element, the
# third element only dependes on modules in the first and second and so on.
function Create-ModuleImportMapOrder([bool] $AzModuleOnly) {
$ModuleImportMapOrder = $null
$ProfileOrAccountsModuleName = $null
$GetAutomationModule = $null
# Use the relevant module class to avoid conflicts
if ($AzModuleOnly) {
$ProfileOrAccountsModuleName = $script:AzAccountsModuleName
$GetAutomationModule = $script:GetAzAutomationModule
} else {
$ProfileOrAccountsModuleName = $script:AzureRmProfileModuleName
$GetAutomationModule = $script:GetAzureRmAutomationModule
}
# Get all the non-conflicting modules in the current automation account
$CurrentAutomationModuleList = & $GetAutomationModule `
-ResourceGroupName $ResourceGroupName `
-AutomationAccountName $AutomationAccountName |
?{
($AzModuleOnly -and ($_.Name -eq 'Az' -or $_.Name -like 'Az.*')) -or
(!$AzModuleOnly -and ($_.Name -eq 'AzureRM' -or $_.Name -like 'AzureRM.*' -or
$_.Name -eq 'Azure' -or $_.Name -like 'Azure.*'))
}
# Get the latest version of the AzureRM.Profile OR Az.Accounts module
$VersionAndDependencies = Get-ModuleDependencyAndLatestVersion $ProfileOrAccountsModuleName
$ModuleEntry = $ProfileOrAccountsModuleName
$ModuleEntryArray = ,$ModuleEntry
$ModuleImportMapOrder += ,$ModuleEntryArray
do {
$NextAutomationModuleList = $null
$CurrentChainVersion = $null
# Add it to the list if the modules are not available in the same list
foreach ($Module in $CurrentAutomationModuleList) {
$Name = $Module.Name
Write-Verbose "Checking dependencies for $Name"
$VersionAndDependencies = Get-ModuleDependencyAndLatestVersion $Module.Name
if ($null -eq $VersionAndDependencies) {
continue
}
$Dependencies = $VersionAndDependencies[1].Split("|")
$AzureModuleEntry = $Module.Name
# If the previous list contains all the dependencies then add it to current list
if ((-not $Dependencies) -or (AreAllModulesAdded $Dependencies)) {
Write-Verbose "Adding module $Name to dependency chain"
$CurrentChainVersion += ,$AzureModuleEntry
} else {
# else add it back to the main loop variable list if not already added
if (!(AreAllModulesAdded $AzureModuleEntry)) {
Write-Verbose "Module $Name does not have all dependencies added as yet. Moving module for later import"
$NextAutomationModuleList += ,$Module
}
}
}
$ModuleImportMapOrder += ,$CurrentChainVersion
$CurrentAutomationModuleList = $NextAutomationModuleList
} while ($null -ne $CurrentAutomationModuleList)
$ModuleImportMapOrder
}
# Wait and confirm that all the modules in the list have been imported successfully in Azure Automation
function Wait-AllModulesImported(
[Collections.Generic.List[string]] $ModuleList,
[int] $Count,
[bool] $UseAzModule = $false) {
$GetAutomationModule = if ($UseAzModule) {
$script:GetAzAutomationModule
} else {
$script:GetAzureRmAutomationModule
}
$i = $Count - $SimultaneousModuleImportJobCount
if ($i -lt 0) { $i = 0 }
for ( ; $i -lt $Count; $i++) {
$Module = $ModuleList[$i]
Write-Output ("Checking import Status for module : {0}" -f $Module)
while ($true) {
$AutomationModule = & $GetAutomationModule `
-Name $Module `
-ResourceGroupName $ResourceGroupName `
-AutomationAccountName $AutomationAccountName
$IsTerminalProvisioningState = ($AutomationModule.ProvisioningState -eq "Succeeded") -or
($AutomationModule.ProvisioningState -eq "Failed")
if ($IsTerminalProvisioningState) {
break
}
Write-Verbose ("Module {0} is getting imported" -f $Module)
Start-Sleep -Seconds 30
}
if ($AutomationModule.ProvisioningState -ne "Succeeded") {
Write-Error ("Failed to import module : {0}. Status : {1}" -f $Module, $AutomationModule.ProvisioningState)
} else {
Write-Output ("Successfully imported module : {0}" -f $Module)
}
}
}
# Uses the module import map created to import modules.
# It will only import modules from an element in the array if all the modules
# from the previous element have been added.
function Import-ModulesInAutomationAccordingToDependency([string[][]] $ModuleImportMapOrder, [bool] $UseAzModule) {
foreach($ModuleList in $ModuleImportMapOrder) {
$i = 0
Write-Output "Importing Array of modules : $ModuleList"
foreach ($Module in $ModuleList) {
Write-Verbose ("Importing module : {0}" -f $Module)
Import-AutomationModule -ModuleName $Module -UseAzModule $UseAzModule
$i++
if ($i % $SimultaneousModuleImportJobCount -eq 0) {
# It takes some time for the modules to start getting imported.
# Sleep for sometime before making a query to see the status
Start-Sleep -Seconds 20
Wait-AllModulesImported -ModuleList $ModuleList -Count $i -UseAzModule $UseAzModule
}
}
if ($i -lt $SimultaneousModuleImportJobCount) {
Start-Sleep -Seconds 20
Wait-AllModulesImported -ModuleList $ModuleList -Count $i -UseAzModule $UseAzModule
}
}
}
function Update-ProfileAndAutomationVersionToLatest([string] $AutomationModuleName) {
# Get the latest azure automation module version
$VersionAndDependencies = Get-ModuleDependencyAndLatestVersion $AutomationModuleName
# Automation only has dependency on profile
$ModuleDependencies = GetModuleNameAndVersionFromPowershellGalleryDependencyFormat $VersionAndDependencies[1]
$ProfileModuleName = $ModuleDependencies[0]
# Create web client object for downloading data
$WebClient = New-Object System.Net.WebClient
# Download AzureRM.Profile to temp location
$ModuleContentUrl = Get-ModuleContentUrl $ProfileModuleName
$ProfileURL = (Invoke-WebRequest -Uri $ModuleContentUrl -MaximumRedirection 0 -UseBasicParsing -ErrorAction Ignore).Headers.Location
$ProfilePath = Join-Path $env:TEMP ($ProfileModuleName + ".zip")
$WebClient.DownloadFile($ProfileURL, $ProfilePath)
# Download AzureRM.Automation to temp location
$ModuleContentUrl = Get-ModuleContentUrl $AutomationModuleName
$AutomationURL = (Invoke-WebRequest -Uri $ModuleContentUrl -MaximumRedirection 0 -UseBasicParsing -ErrorAction Ignore).Headers.Location
$AutomationPath = Join-Path $env:TEMP ($AutomationModuleName + ".zip")
$WebClient.DownloadFile($AutomationURL, $AutomationPath)
# Create folder for unzipping the Module files
$PathFolderName = New-Guid
$PathFolder = Join-Path $env:TEMP $PathFolderName
# Unzip files
$ProfileUnzipPath = Join-Path $PathFolder $ProfileModuleName
Expand-Archive -Path $ProfilePath -DestinationPath $ProfileUnzipPath -Force
$AutomationUnzipPath = Join-Path $PathFolder $AutomationModuleName
Expand-Archive -Path $AutomationPath -DestinationPath $AutomationUnzipPath -Force
# Import modules
Import-Module (Join-Path $ProfileUnzipPath ($ProfileModuleName + ".psd1")) -Force -Verbose
Import-Module (Join-Path $AutomationUnzipPath ($AutomationModuleName + ".psd1")) -Force -Verbose
}
#endregion
#region Main body
if ($ModuleVersionOverrides) {
$ModuleVersionOverridesHashTable = ConvertJsonDictTo-HashTable $ModuleVersionOverrides
} else {
$ModuleVersionOverridesHashTable = #{}
}
$UseAzModule = $null
$AutomationModuleName = $null
# We want to support updating Az modules. This means this runbook should support upgrading using only Az modules
if ($AzureModuleClass -eq "Az") {
$UseAzModule = $true
$AutomationModuleName = $script:AzAutomationModuleName
} elseif ( $AzureModuleClass -eq "AzureRM") {
$UseAzModule = $false
$AutomationModuleName = $script:AzureRMAutomationModuleName
} else {
Write-Error "Invalid AzureModuleClass: '$AzureModuleClass'. Must be either Az or AzureRM" -ErrorAction Stop
}
# Import the latest version of the Az automation and accounts version to the local sandbox
Update-ProfileAndAutomationVersionToLatest $AutomationModuleName
if ($Login) {
Login-AzureAutomation $UseAzModule
}
$ModuleImportMapOrder = Create-ModuleImportMapOrder $UseAzModule
Import-ModulesInAutomationAccordingToDependency $ModuleImportMapOrder $UseAzModule
#endregion
Further, if you want to refer to another approach, you can check this script.
Here is Reddit question regarding the same.
I have used azure devops to install the modules with powershell. I was trying to install it within the runbook but that didn't work so i kind of changed the method.
Here is the code:
#necessary modules list
$deps1 = #("Az.Accounts","Az.Storage","Az.Compute")
foreach($dep in $deps1){
$module = Find-Module -Name $dep
$link = $module.RepositorySourceLocation + "/package/" + $module.Name + "/" + $module.Version
New-AzAutomationModule -AutomationAccountName $AutomationAccountName -Name $module.Name -ContentLinkUri $link -ResourceGroupName $ResourceGroupName
if ($dep -eq "Az.Accounts") {
#Az.Accounts is a dependency for Az.Storage and Az.Compute modules
Write-Host "Sleeping for 180 sec in order to wait the installation of the Az.Accounts module"
Start-Sleep 180
}
}
This answer in this post helped me for the code.

Not able to access Azure FileShare Storage container from Azure Automation Runbook

I've the following Azure Automation Runbook script which goal is to take an dump/export from a REST API call which must run from a target device which is able to reach the REST API device. So Azure Automation runbook is targeting a "proxy server" then from this we're taking the REST API backup.
The approach has been working exception the fact we're able not to copy this backup file from the target server once 'cm.vm.run_command' presents output size limitation and is truncating the backup. The workaround we found for this was copying the backup file from the 'target/proxy server' directly into a Storage Account Fileshare which is mounted on the target/proxy server. My problem now is when running from Azure Automation it's not able to access the drive mounted by other user and/or is not able to mount the device or access it directly like below errors messages. Does anybody have any alternative for this ? I was able to check the runbook is having connectivity on the storage account ports 443/445 from t. That was one of the possible reasons described here https://learn.microsoft.com/en-us/azure/storage/files/storage-troubleshoot-windows-file-connection-problems
Below the commands and errors I'm receiving and the whole script used.
Copy-item -Path C:\Devicebackup.txt -Destination \\storage_account_name.file.core.windows.net\configdatafileshare\Orchestration
net use w: \\storage_account_name.file.core.windows.net\configdatafileshare\Orchestration `'/yBapkthow==`' /user:Azure\storage_account_name
Copy-item : The network path was not found
At C:\Packages\Plugins\Microsoft.CPlat.Core.RunCommandWindows\1.1.5\Downloads\s
cript9.ps1:15 char:1
+ Copy-item -Path C:\Devicebackup.txt -Destination \\storage_account_name. ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Copy-Item], IOException
+ FullyQualifiedErrorId : System.IO.IOException,Microsoft.PowerShell.Comma
nds.CopyItemCommand
The option /DL2D2QKD1OU2ZKEOJVRK4LGPIRTJKAJBZ+EDKNHWVYYEJDDYSL9CPB5T8F/9VWQBMBWC37B1NJS4YBAPKTHOW== is unknown.
The syntax of this command is:
NET USE
[devicename | *] [\\computername\sharename[\volume] [password | *]]
[/USER:[domainname\]username]
[/USER:[dotted domain name\]username]
[/USER:[username#dotted domain name]
[/SMARTCARD]
[/SAVECRED]
[[/DELETE] | [/PERSISTENT:{YES | NO}]]
NET USE {devicename | *} [password | *] /HOME
NET USE [/PERSISTENT:{YES | NO}]
Param (
[Parameter(Mandatory=$false)][string] $rgName
,[Parameter(Mandatory=$false)][string] $ProxyServerName
)
function CreatePSCommandFile {
Param(
[parameter(Mandatory=$true)][String[]]$DeviceName,
[parameter(Mandatory=$true)][String[]]$DeviceIP,
[parameter(Mandatory=$true)][String[]]$ApiToken
)
$remoteCommand =
#"
add-type #`"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
public bool CheckValidationResult(
ServicePoint srvPoint, X509Certificate certificate,
WebRequest request, int certificateProblem) {
return true;
}
}
`"#
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
Invoke-WebRequest -Uri 'www.mydownload.com' -UseBasicParsing -Headers #{ Authorization="Bearer $($ApiToken)" } | Out-file C:\Devicebackup.txt
net use w: \\storage_account_name.file.core.windows.net\configdatafileshare\Orchestration `'/STORAGE_KEY+EDknHWvyyeJDDYsL9cPB5T8F/9VwqBmbwc37B1NJS4yBapkthow==`' /user:Azure\storage_account_name
Copy-item -Path C:\Devicebackup.txt -Destination \\storage_account_name.file.core.windows.net\configdatafileshare\Orchestration
"#
Set-Content -Path .\InvokeCommand.ps1 -Value $remoteCommand
}
$connectionName = "AzureRunAsConnection"
try {
# Get the connection "AzureRunAsConnection "
$servicePrincipalConnection = Get-AutomationConnection -Name $connectionName
Write-Host "Logging in to Azure..."
$connectionResult = Connect-AzAccount `
-ServicePrincipal `
-Tenant $servicePrincipalConnection.TenantID `
-ApplicationId $servicePrincipalConnection.ApplicationID `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint
}
catch {
if (!$servicePrincipalConnection)
{
$ErrorMessage = "Connection $connectionName not found."
throw $ErrorMessage
} else{
Write-Error -Message $_.Exception
throw $_.Exception
}
}
function Backup-Device {
Param (
[Parameter(Mandatory=$false)][string] $DeviceName
,[Parameter(Mandatory=$false)][string] $DeviceIP
,[Parameter(Mandatory=$false)][string] $ApiToken
)
# Execute Backup on Fortigate Rest API
CreatePSCommandFile -DeviceName $DeviceName -DeviceIP $DeviceIP -ApiToken $ApiToken
$Output = Invoke-AzVMRunCommand -ResourceGroupName $rgName -VMName $ProxyServerName -CommandId 'RunPowerShellScript' -Scriptpath ".\InvokeCommand.ps1" -Parameter #{'api_url' = "10.29.255.212"; 'api_token' = "0p6h1rmspjf37kp80bc6ny88jw"}
($Output).Value.Message
}
Backup-Device -DeviceName "DeviceName" -DeviceIP '10.29.255.212' -ApiToken 'Api_Token'
Sharing the solution which was presented by a blessed colleague :)
Using New-SmbMapping we were able to mount the Storage Account File Share from Azure Automation PS script successfully.
if (!(Test-Path `$MapDrive)) {
New-SmbMapping -LocalPath `$MapDrive -RemotePath `$RemotePath -UserName `$UserName -Password `$Key
}
Copy-Item .\Devicebackup.txt `$MapDrive

Azure Automation Runbook missing mandatory parameters

I'm trying to set a Tag on all virtual machines in my subscription but I keep getting errors when running the Runbook.
The error is the following:
Get-AzureRmVM : Cannot process command because of one or more missing mandatory parameters: ResourceGroupName. At line:30
Here is my Runbook:
$azureConnection = Get-AutomationConnection -Name 'AzureRunAsConnection'
#Authenticate
try {
Clear-Variable -Name params -Force -ErrorAction Ignore
$params = #{
ServicePrincipal = $true
Tenant = $azureConnection.TenantID
ApplicationId = $azureConnection.ApplicationID
CertificateThumbprint = $azureConnection.CertificateThumbprint
}
$null = Add-AzureRmAccount #params
}
catch {
$errorMessage = $_
Throw "Unable to authenticate with error: $errorMessage"
}
# Discovery of all Azure VM's in the current subscription.
$azurevms = Get-AzureRmVM | Select-Object -ExpandProperty Name
Write-Host "Discovering Azure VM's in the following subscription $SubscriptionID Please hold...."
Write-Host "The following VM's have been discovered in subscription $SubscriptionID"
$azurevms
foreach ($azurevm in $azurevms) {
Write-Host "Checking for tag $vmtagname on $azurevm"
$tagRGname = Get-AzureRmVM -Name $azurevm | Select-Object -ExpandProperty ResourceGroupName
$tags = (Get-AzureRmResource -ResourceGroupName $tagRGname -Name $azurevm).Tags
If ($tags.UpdateWindow){
Write-Host "$azurevm already has the tag $vmtagname."
}
else
{
Write-Host "Creating Tag $vmtagname and Value $tagvalue for $azurevm"
$tags.Add($vmtagname,$tagvalue)
Set-AzureRmResource -ResourceGroupName $tagRGname -ResourceName $azurevm -ResourceType Microsoft.Compute/virtualMachines -Tag $tags -Force `
}
}
Write-Host "All tagging is done"
I tried importing the right modules but this doesn't seem to affect the outcome.
Running the same commands in Cloud Shell does work correctly.
I can reproduce your issue, the error was caused by this part Get-AzureRmVM -Name $azurevm, when running this command, the -ResourceGroupName is needed.
You need to use the Az command Get-AzVM -Name $azurevm, it will work.
Running the same commands in Cloud Shell does work correctly.
In Cloud shell, azure essentially uses the new Az module to run your command, you can understand it runs the Enable-AzureRmAlias before the command, you could check that via debug mode.
Get-AzureRmVM -Name joyWindowsVM -debug
To solve your issue completely, I recommend you to use the new Az module, because the AzureRM module was deprecated and will not be updated.
Please follow the steps below.
1.Navigate to your automation account in the portal -> Modules, check if you have imported the modules Az.Accounts, Az.Compute, Az.Resources, if not, go to Browse Gallery -> search and import them.
2.After import successfully, change your script to the one like below, then it should work fine.
$azureConnection = Get-AutomationConnection -Name 'AzureRunAsConnection'
#Authenticate
try {
Clear-Variable -Name params -Force -ErrorAction Ignore
$params = #{
ServicePrincipal = $true
Tenant = $azureConnection.TenantID
ApplicationId = $azureConnection.ApplicationID
CertificateThumbprint = $azureConnection.CertificateThumbprint
}
$null = Connect-AzAccount #params
}
catch {
$errorMessage = $_
Throw "Unable to authenticate with error: $errorMessage"
}
# Discovery of all Azure VM's in the current subscription.
$azurevms = Get-AzVM | Select-Object -ExpandProperty Name
Write-Host "Discovering Azure VM's in the following subscription $SubscriptionID Please hold...."
Write-Host "The following VM's have been discovered in subscription $SubscriptionID"
$azurevms
foreach ($azurevm in $azurevms) {
Write-Host "Checking for tag $vmtagname on $azurevm"
$tagRGname = Get-AzVM -Name $azurevm | Select-Object -ExpandProperty ResourceGroupName
$tags = (Get-AzResource -ResourceGroupName $tagRGname -Name $azurevm).Tags
If ($tags.UpdateWindow){
Write-Host "$azurevm already has the tag $vmtagname."
}
else
{
Write-Host "Creating Tag $vmtagname and Value $tagvalue for $azurevm"
$tags.Add($vmtagname,$tagvalue)
Set-AzResource -ResourceGroupName $tagRGname -ResourceName $azurevm -ResourceType Microsoft.Compute/virtualMachines -Tag $tags -Force `
}
}
Write-Host "All tagging is done"

Azure DSC Configuration Issue

Basically I am just trying a very basic thing of accessing the VM details using Azure DSC.
I have done the following
Added a new Credential(which holds the username and password) and Variable(which holds the subscriptionId) under Shared Resources of my
automation account
Have implemented the following DSC code for retrieving the VM details: I am able to complie this file in the portal it generates
the .MOF file as well. But when I try to apply this to a node in the
portal I get the following error:
PowerShell DSC resource
MSFT_ScriptResource failed to execute Set-TargetResource
functionality with error message: The term
'Get-AutomationPSCredential' is not recognized as the name of a
cmdlet, function, script file, or operable program. Check the
spelling of the name, or if a path was included, verify that the path
is correct and try again
Please note that the code written inside the SetScript executes successfully in a runbook!!!!!!
Configuration VMAzureDSCTasks
{
param
(
[Parameter()]
[System.String]
$NodeName = "rajeshserver",
[Parameter()]
[System.String]
$ResourceGroupName = "rajeshresourcegroup",
[Parameter()]
[System.String]
$VMSize = "Standard_D2s_v3",
[Parameter()]
[System.String]
$CredentialAssetName = "cred"
)
Import-DscResource -ModuleName 'PSDesiredStateConfiguration'
Node $NodeName
{
Script resizevm
{
SetScript = {
# Credentials and Subscription ID declaration
$Cred = Get-AutomationPSCredential -Name $using:CredentialAssetName
$null = Add-AzureRmAccount -Credential $Cred -ErrorAction Stop
$SubId = Get-AutomationVariable -Name 'SubscriptionId'
$null = Set-AzureRmContext -SubscriptionId $SubId -ErrorAction Stop
try {
$vm = Get-AzureRmVm -ResourceGroupName $using:ResourceGroupName -VMName $using:NodeName -ErrorAction Stop
} catch {
throw "Virtual Machine not found!!!!!!"
exit
}
# Output current VM Size
$currentVMSize = $vm.HardwareProfile.vmSize
Write-Verbose -Message "`nFound the specified Virtual Machine: $using:NodeName"
Write-Verbose -Message "Current size: $using:currentVMSize"
}
TestScript = {
return $false
}
GetScript = {
}
}
}
}
Command Get-AutomationPSCredential works in Azure Automation.
For DSC, pass credentials using Get-Credential.
Add parameter
[Parameter()]
[pscredential]
$Credential
And replace Get-AutomationPSCredential with Get-Credential -Credential $Credential.

Resources