Using Compress-Archive to compress multiple folders multithreadded with Powershell - multithreading

Everyday I need create archives of the following
C:.
└───1.Jan
├───20000
│ img1.bmp
│ img2.bmp
│ img3.bmp
│
├───20001
│ img1.bmp
│ img2.bmp
│ img3.bmp
│
├───20002
│ img1.bmp
│ img2.bmp
│ img3.bmp
│
├───20003
│ img1.bmp
│ img2.bmp
│ img3.bmp
│
├───20004
│ img1.bmp
│ img2.bmp
│ img3.bmp
│
├───20005
│ img1.bmp
│ img2.bmp
│ img3.bmp
│
└───Entered
I currently have the script working with creating the zip files one at a time, However I can sometimes have 200+ folders to zip and they are different sizes so I would like to get this working multithreadded.
function Zip-Folders([String] $FolderPath) {
if ($FolderPath -eq '') {
return
}
else {
$FolderNames = Get-ChildItem -Path $FolderPath -Name -Directory -Exclude Enter*
foreach ($i in $FolderNames) {
$TempPath = "$FolderPath\$i"
$TempFileName = "$i Photos"
if (-Not(Get-ChildItem -Path $TempPath | Where-Object {$_.Name -like '*.zip'})) {
Write-Host "[$TempPath] has been compressed to [$TempFileName]."
Compress-Archive -Path $tempPath\* -DestinationPath $tempPath\$TempFileName
}
Else {
Write-Host "[$i] has already been compressed."
}
}
}
}
The code asks for the folder via a Folderbrowser dialog.
If someone could either help with the code or point me in the direction where I can find the information for this, I am a beginner with PowerShell but have done some programming.
If there is any other information needed let me know.

This is how you can do it using Runspace, use the -Threshold parameter depending on how many folders you want to compress at the same time.
As commented, it is highly recommended to use the ThreadJob Module if you have the ability to install Modules in your environment.
function Zip-Folders {
[cmdletbinding()]
param(
[ValidateScript({ Test-Path $_ -PathType Container })]
[Parameter(Mandatory)]
[String] $FolderPath,
[int] $Threshold = 10
)
begin {
$RunspacePool = [runspacefactory]::CreateRunspacePool(1, $Threshold)
$RunspacePool.Open()
$subFolders = Get-ChildItem -Path $FolderPath -Directory -Exclude Enter*
}
process {
try {
$runspaces = foreach ($folder in $subFolders) {
$instance = [powershell]::Create().AddScript({
param($thisFolder)
$fileName = "{0} Photos.zip" -f $thisFolder.Name
$absolutePath = $thisFolder.FullName
$zipPath = Join-Path $absolutePath -ChildPath $fileName
if(-not (Get-ChildItem -Path $absolutePath -Filter *.zip)) {
Compress-Archive -Path $absolutePath\* -DestinationPath $zipPath
return "[$absolutePath] has been compressed to [$zipPath]."
}
"[$absolutePath] has already been compressed."
}).AddParameter('thisFolder', $folder)
$instance.RunspacePool = $RunspacePool
[pscustomobject]#{
Instance = $instance
Handle = $instance.BeginInvoke()
}
}
foreach($r in $runspaces) {
$r.Instance.EndInvoke($r.Handle)
$r.Instance.ForEach('Dispose')
}
}
catch {
$PSCmdlet.WriteError($_)
}
finally {
$runspaces.ForEach('Clear')
$RunspacePool.ForEach('Dispose')
}
}
}

Related

PowerShell: Output provider results from multiple Azure subscriptions into one .txt file - via arrayList

I am trying to get the registered providers for multiple subscriptions and output everything into one file.
For that, i am getting the subscriptions from a folder full of *.yaml files that contain information about the subscriptions, including their name.
What i cannot achieve is get the result for each subscriptions into an array and output that array into a text file. The script also allows the use of only one item from the folder in case that is the case.
Here is a sample code of a subscription and the code for it:
subscription1.yaml
name: subscription1
emailContact: email.address#domain.com
tags:
- key: "key1"
value: "Value1"
subscription2.yaml
name: subscription2
emailContact: email.address#domain.com
tags:
- key: "key1"
value: "Value1"
Folder structure where the yaml files is: ./landingZones/landingzone1/settings/dev/*.yaml
script:
param (
[Parameter(Mandatory = $false)]
[string]$Environment = 'dev',
[Parameter(Mandatory = $false)]
[string]$LandingZoneType = 'landingzone1',
[Parameter(Mandatory = $false)]
[string]$SingleSubscription
)
$scriptPath = Split-Path -parent $PSCommandPath
$subscriptionsEnvironmentDirectory = Get-ChildItem -Directory $scriptPath -Recurse -Filter "*$Environment*" | Where-Object { $_.parent.parent.Name -eq $LandingZoneType }
$subscriptions = Get-ChildItem -Path $($subscriptionsEnvironmentDirectory.FullName)
foreach ($subscription in ($subscriptions | Where-Object { ([System.String]::IsNullOrEmpty($SingleSubscription)) -or ($_.Name -replace "\.[^\.]+$", '') -eq $SingleSubscription })) {
$landingZone = Get-Content -Path $subscription.FullName | ConvertFrom-Yaml
# Set subscriptionName variable
$subscriptionName = $landingZone.name
$providers = az provider list --subscription $subscriptionName | ConvertFrom-Json
$defaultRegisteredProviders = 'Microsoft.ADHybridHealthService|Microsoft.Authorization|Microsoft.Billing|Microsoft.ClassicSubscription|Microsoft.Commerce|Microsoft.Consumption|Microsoft.CostManagement|Microsoft.Features|Microsoft.MarketplaceOrdering'
$registeredProviders = $providers | Where-Object { ($_.registrationState -eq 'Registered') -and ($_.namespace -notmatch $defaultRegisteredProviders) }
# Outputting result into txt file in the same directory where the command was executed
Write-Host ('{1}# Registered providers for subscription [{0}]' -f $subscriptionName, "`n")
$list = New-Object -TypeName 'System.Collections.ArrayList'
$sortedObjects = $registeredProviders | Sort-Object namespace | `
Format-Table `
#{l = 'Namespace'; e = { $_.namespace } }, `
#{l = "Subscription Id [$subscriptionName]"; e = { $_.id } }, `
#{l = 'Registration State'; e = { $_.registrationState } }, `
#{l = 'Registration Policy'; e = { $_.registrationPolicy } }
foreach ($i in $sortedObjects) {
$list.Add($i) | Out-Null
}
# Alternative to add into array:
# #($sortedObjects).foreach({$list.Add($_)}) | Out-Null
}
$list.Count
$list | Out-File .\registered_providers.txt -Force
The result is a file called 'registered_providers.txt' that contains only the registered providers for the first subscription in the foreach loop. I cannot get the contents of the second, third and so on in the same file, just a replaced text from the $sortedObjects
How do i create the array to contain all the info from all the subscriptions called?
Thanks
$list | Out-File Should be inside the for loop so that all the data you are fetching will be stored in the specified file.
Out-File cmdlet has -append parameter that appends the output to the existing file for every for loop happens. Otherwise, it will clean up the information/value stored in that output file.
Thanks to #jdweng for pointing the user to the right solution.
Refer to this MS Doc on Out-File -Append Parameter usage.

How to import automatically Az Powershell modules in azure automation account?

I am trying to install az powershell modules from my automation runbook with powershell script. But i am failing to do that. I have tried the solution in a similar topic which is here but it didn't work for me. The code is below:
$AAccName = "my-aa"
$RGName = "my-rg"
$deps1 = #("Az.Accounts","Az.Profile")
foreach($dep in $deps1){
$module = Find-Module -Name $dep
$link = $module.RepositorySourceLocation + "/package/" + $module.Name + "/" + $module.Version
New-AzAutomationModule -AutomationAccountName $AAccName -Name $module.Name -ContentLinkUri $link -ResourceGroupName $RGName
}
The error , i get:
Exception calling "ShouldContinue" with "2" argument(s): "A command
that prompts the user failed because the host program or the command
type does not support user interaction. The host was attempting to
request confirmation with the following message: PowerShellGet
requires NuGet provider version '2.8.5.201' or newer to interact with
NuGet-based repositories. The NuGet provider must be available in
'C:\Program Files\PackageManagement\ProviderAssemblies' or
'C:\Users\Client\AppData\Roaming\PackageManagement\ProviderAssemblies'.
You can also install the NuGet provider by running
'Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201
-Force'. Do you want PowerShellGet to install and import the NuGet provider now?" At C:\Program
Files\WindowsPowerShell\Modules\PowerShellGet\1.0.0.1\PSModule.psm1:7455
char:8 + if($Force -or
$psCmdlet.ShouldContinue($shouldContinueQueryMessag ... +
I have executed the command given in this message but i get this error:
Install-PackageProvider : No match was found for the specified search
criteria for the provider 'NuGet'. The package provider requires
'PackageManagement' and 'Provider' tags.
Do you have any idea how to add module in Azure Automation Account with script?
Update:
When i use Import-Module -Name Az.Profile -Force command, I get this error:
Import-Module : The specified module 'Az.Profile' was not loaded
because no valid module file was found in any module directory.
This should be because that the module is not installed on the module directory. When i manually add the module from the module gallery, it works.
Check this MS recommended PS script to update modules in Automation account. It's reference is given here: Use the update runbook to update a specific module version.
Script:
<#
Copyright (c) Microsoft Corporation. All rights reserved.
Licensed under the MIT License.
#>
<#
.SYNOPSIS
Update Azure PowerShell modules in an Azure Automation account.
.DESCRIPTION
This Azure Automation runbook updates Azure PowerShell modules imported into an
Azure Automation account with the module versions published to the PowerShell Gallery.
Prerequisite: an Azure Automation account with an Azure Run As account credential.
.PARAMETER ResourceGroupName
The Azure resource group name.
.PARAMETER AutomationAccountName
The Azure Automation account name.
.PARAMETER SimultaneousModuleImportJobCount
(Optional) The maximum number of module import jobs allowed to run concurrently.
.PARAMETER AzureModuleClass
(Optional) The class of module that will be updated (AzureRM or Az)
If set to Az, this script will rely on only Az modules to update other modules.
Set this to Az if your runbooks use only Az modules to avoid conflicts.
.PARAMETER AzureEnvironment
(Optional) Azure environment name.
.PARAMETER Login
(Optional) If $false, do not login to Azure.
.PARAMETER ModuleVersionOverrides
(Optional) Module versions to use instead of the latest on the PowerShell Gallery.
If $null, the currently published latest versions will be used.
If not $null, must contain a JSON-serialized dictionary, for example:
'{ "AzureRM.Compute": "5.8.0", "AzureRM.Network": "6.10.0" }'
or
#{ 'AzureRM.Compute'='5.8.0'; 'AzureRM.Network'='6.10.0' } | ConvertTo-Json
.PARAMETER PsGalleryApiUrl
(Optional) PowerShell Gallery API URL.
.LINK
https://learn.microsoft.com/en-us/azure/automation/automation-update-azure-modules
#>
[Diagnostics.CodeAnalysis.SuppressMessageAttribute("PSUseApprovedVerbs", "")]
param(
[Parameter(Mandatory = $true)]
[string] $ResourceGroupName,
[Parameter(Mandatory = $true)]
[string] $AutomationAccountName,
[int] $SimultaneousModuleImportJobCount = 10,
[string] $AzureModuleClass = 'AzureRM',
[string] $AzureEnvironment = 'AzureCloud',
[bool] $Login = $true,
[string] $ModuleVersionOverrides = $null,
[string] $PsGalleryApiUrl = 'https://www.powershellgallery.com/api/v2'
)
$ErrorActionPreference = "Continue"
#region Constants
$script:AzureRMProfileModuleName = "AzureRM.Profile"
$script:AzureRMAutomationModuleName = "AzureRM.Automation"
$script:GetAzureRmAutomationModule = "Get-AzureRmAutomationModule"
$script:NewAzureRmAutomationModule = "New-AzureRmAutomationModule"
$script:AzAccountsModuleName = "Az.Accounts"
$script:AzAutomationModuleName = "Az.Automation"
$script:GetAzAutomationModule = "Get-AzAutomationModule"
$script:NewAzAutomationModule = "New-AzAutomationModule"
$script:AzureSdkOwnerName = "azure-sdk"
#endregion
#region Functions
function ConvertJsonDictTo-HashTable($JsonString) {
try{
$JsonObj = ConvertFrom-Json $JsonString -ErrorAction Stop
} catch [System.ArgumentException] {
throw "Unable to deserialize the JSON string for parameter ModuleVersionOverrides: ", $_
}
$Result = #{}
foreach ($Property in $JsonObj.PSObject.Properties) {
$Result[$Property.Name] = $Property.Value
}
$Result
}
# Use the Run As connection to login to Azure
function Login-AzureAutomation([bool] $AzModuleOnly) {
try {
$RunAsConnection = Get-AutomationConnection -Name "AzureRunAsConnection"
Write-Output "Logging in to Azure ($AzureEnvironment)..."
if (!$RunAsConnection.ApplicationId) {
$ErrorMessage = "Connection 'AzureRunAsConnection' is incompatible type."
throw $ErrorMessage
}
if ($AzModuleOnly) {
Connect-AzAccount `
-ServicePrincipal `
-TenantId $RunAsConnection.TenantId `
-ApplicationId $RunAsConnection.ApplicationId `
-CertificateThumbprint $RunAsConnection.CertificateThumbprint `
-Environment $AzureEnvironment
Select-AzSubscription -SubscriptionId $RunAsConnection.SubscriptionID | Write-Verbose
} else {
Add-AzureRmAccount `
-ServicePrincipal `
-TenantId $RunAsConnection.TenantId `
-ApplicationId $RunAsConnection.ApplicationId `
-CertificateThumbprint $RunAsConnection.CertificateThumbprint `
-Environment $AzureEnvironment
Select-AzureRmSubscription -SubscriptionId $RunAsConnection.SubscriptionID | Write-Verbose
}
} catch {
if (!$RunAsConnection) {
$RunAsConnection | fl | Write-Output
Write-Output $_.Exception
$ErrorMessage = "Connection 'AzureRunAsConnection' not found."
throw $ErrorMessage
}
throw $_.Exception
}
}
# Checks the PowerShell Gallery for the latest available version for the module
function Get-ModuleDependencyAndLatestVersion([string] $ModuleName) {
$ModuleUrlFormat = "$PsGalleryApiUrl/Search()?`$filter={1}&searchTerm=%27{0}%27&targetFramework=%27%27&includePrerelease=false&`$skip=0&`$top=40"
$ForcedModuleVersion = $ModuleVersionOverridesHashTable[$ModuleName]
$CurrentModuleUrl =
if ($ForcedModuleVersion) {
$ModuleUrlFormat -f $ModuleName, "Version%20eq%20'$ForcedModuleVersion'"
} else {
$ModuleUrlFormat -f $ModuleName, 'IsLatestVersion'
}
$SearchResult = Invoke-RestMethod -Method Get -Uri $CurrentModuleUrl -UseBasicParsing
if (!$SearchResult) {
Write-Verbose "Could not find module $ModuleName on PowerShell Gallery. This may be a module you imported from a different location. Ignoring this module"
} else {
if ($SearchResult.Length -and $SearchResult.Length -gt 1) {
$SearchResult = $SearchResult | Where-Object { $_.title.InnerText -eq $ModuleName }
}
if (!$SearchResult) {
Write-Verbose "Could not find module $ModuleName on PowerShell Gallery. This may be a module you imported from a different location. Ignoring this module"
} else {
$PackageDetails = Invoke-RestMethod -Method Get -UseBasicParsing -Uri $SearchResult.id
# Ignore the modules that are not published as part of the Azure SDK
if ($PackageDetails.entry.properties.Owners -ne $script:AzureSdkOwnerName) {
Write-Warning "Module : $ModuleName is not part of azure sdk. Ignoring this."
} else {
$ModuleVersion = $PackageDetails.entry.properties.version
$Dependencies = $PackageDetails.entry.properties.dependencies
#($ModuleVersion, $Dependencies)
}
}
}
}
function Get-ModuleContentUrl($ModuleName) {
$ModuleContentUrlFormat = "$PsGalleryApiUrl/package/{0}"
$VersionedModuleContentUrlFormat = "$ModuleContentUrlFormat/{1}"
$ForcedModuleVersion = $ModuleVersionOverridesHashTable[$ModuleName]
if ($ForcedModuleVersion) {
$VersionedModuleContentUrlFormat -f $ModuleName, $ForcedModuleVersion
} else {
$ModuleContentUrlFormat -f $ModuleName
}
}
# Imports the module with given version into Azure Automation
function Import-AutomationModule([string] $ModuleName, [bool] $UseAzModule = $false) {
$NewAutomationModule = $null
$GetAutomationModule = $null
if ($UseAzModule) {
$GetAutomationModule = $script:GetAzAutomationModule
$NewAutomationModule = $script:NewAzAutomationModule
} else {
$GetAutomationModule = $script:GetAzureRmAutomationModule
$NewAutomationModule = $script:NewAzureRmAutomationModule
}
$LatestModuleVersionOnGallery = (Get-ModuleDependencyAndLatestVersion $ModuleName)[0]
$ModuleContentUrl = Get-ModuleContentUrl $ModuleName
# Find the actual blob storage location of the module
do {
$ModuleContentUrl = (Invoke-WebRequest -Uri $ModuleContentUrl -MaximumRedirection 0 -UseBasicParsing -ErrorAction Ignore).Headers.Location
} while (!$ModuleContentUrl.Contains(".nupkg"))
$CurrentModule = & $GetAutomationModule `
-Name $ModuleName `
-ResourceGroupName $ResourceGroupName `
-AutomationAccountName $AutomationAccountName
if ($CurrentModule.Version -eq $LatestModuleVersionOnGallery) {
Write-Output "Module : $ModuleName is already present with version $LatestModuleVersionOnGallery. Skipping Import"
} else {
Write-Output "Importing $ModuleName module of version $LatestModuleVersionOnGallery to Automation"
& $NewAutomationModule `
-ResourceGroupName $ResourceGroupName `
-AutomationAccountName $AutomationAccountName `
-Name $ModuleName `
-ContentLink $ModuleContentUrl > $null
}
}
# Parses the dependency got from PowerShell Gallery and returns name and version
function GetModuleNameAndVersionFromPowershellGalleryDependencyFormat([string] $Dependency) {
if ($null -eq $Dependency) {
throw "Improper dependency format"
}
$Tokens = $Dependency -split":"
if ($Tokens.Count -ne 3) {
throw "Improper dependency format"
}
$ModuleName = $Tokens[0]
$ModuleVersion = $Tokens[1].Trim("[","]")
#($ModuleName, $ModuleVersion)
}
# Validates if the given list of modules has already been added to the module import map
function AreAllModulesAdded([string[]] $ModuleListToAdd) {
$Result = $true
foreach ($ModuleToAdd in $ModuleListToAdd) {
$ModuleAccounted = $false
# $ModuleToAdd is specified in the following format:
# ModuleName:ModuleVersionSpecification:
# where ModuleVersionSpecification follows the specifiation
# at https://learn.microsoft.com/en-us/nuget/reference/package-versioning#version-ranges-and-wildcards
# For example:
# AzureRm.profile:[4.0.0]:
# or
# AzureRm.profile:3.0.0:
# In any case, the dependency version specification is always separated from the module name with
# the ':' character. The explicit intent of this runbook is to always install the latest module versions,
# so we want to completely ignore version specifications here.
$ModuleNameToAdd = $ModuleToAdd -replace '\:.*', ''
foreach($AlreadyIncludedModules in $ModuleImportMapOrder) {
if ($AlreadyIncludedModules -contains $ModuleNameToAdd) {
$ModuleAccounted = $true
break
}
}
if (!$ModuleAccounted) {
$Result = $false
break
}
}
$Result
}
# Creates a module import map. This is a 2D array of strings so that the first
# element in the array consist of modules with no dependencies.
# The second element only depends on the modules in the first element, the
# third element only dependes on modules in the first and second and so on.
function Create-ModuleImportMapOrder([bool] $AzModuleOnly) {
$ModuleImportMapOrder = $null
$ProfileOrAccountsModuleName = $null
$GetAutomationModule = $null
# Use the relevant module class to avoid conflicts
if ($AzModuleOnly) {
$ProfileOrAccountsModuleName = $script:AzAccountsModuleName
$GetAutomationModule = $script:GetAzAutomationModule
} else {
$ProfileOrAccountsModuleName = $script:AzureRmProfileModuleName
$GetAutomationModule = $script:GetAzureRmAutomationModule
}
# Get all the non-conflicting modules in the current automation account
$CurrentAutomationModuleList = & $GetAutomationModule `
-ResourceGroupName $ResourceGroupName `
-AutomationAccountName $AutomationAccountName |
?{
($AzModuleOnly -and ($_.Name -eq 'Az' -or $_.Name -like 'Az.*')) -or
(!$AzModuleOnly -and ($_.Name -eq 'AzureRM' -or $_.Name -like 'AzureRM.*' -or
$_.Name -eq 'Azure' -or $_.Name -like 'Azure.*'))
}
# Get the latest version of the AzureRM.Profile OR Az.Accounts module
$VersionAndDependencies = Get-ModuleDependencyAndLatestVersion $ProfileOrAccountsModuleName
$ModuleEntry = $ProfileOrAccountsModuleName
$ModuleEntryArray = ,$ModuleEntry
$ModuleImportMapOrder += ,$ModuleEntryArray
do {
$NextAutomationModuleList = $null
$CurrentChainVersion = $null
# Add it to the list if the modules are not available in the same list
foreach ($Module in $CurrentAutomationModuleList) {
$Name = $Module.Name
Write-Verbose "Checking dependencies for $Name"
$VersionAndDependencies = Get-ModuleDependencyAndLatestVersion $Module.Name
if ($null -eq $VersionAndDependencies) {
continue
}
$Dependencies = $VersionAndDependencies[1].Split("|")
$AzureModuleEntry = $Module.Name
# If the previous list contains all the dependencies then add it to current list
if ((-not $Dependencies) -or (AreAllModulesAdded $Dependencies)) {
Write-Verbose "Adding module $Name to dependency chain"
$CurrentChainVersion += ,$AzureModuleEntry
} else {
# else add it back to the main loop variable list if not already added
if (!(AreAllModulesAdded $AzureModuleEntry)) {
Write-Verbose "Module $Name does not have all dependencies added as yet. Moving module for later import"
$NextAutomationModuleList += ,$Module
}
}
}
$ModuleImportMapOrder += ,$CurrentChainVersion
$CurrentAutomationModuleList = $NextAutomationModuleList
} while ($null -ne $CurrentAutomationModuleList)
$ModuleImportMapOrder
}
# Wait and confirm that all the modules in the list have been imported successfully in Azure Automation
function Wait-AllModulesImported(
[Collections.Generic.List[string]] $ModuleList,
[int] $Count,
[bool] $UseAzModule = $false) {
$GetAutomationModule = if ($UseAzModule) {
$script:GetAzAutomationModule
} else {
$script:GetAzureRmAutomationModule
}
$i = $Count - $SimultaneousModuleImportJobCount
if ($i -lt 0) { $i = 0 }
for ( ; $i -lt $Count; $i++) {
$Module = $ModuleList[$i]
Write-Output ("Checking import Status for module : {0}" -f $Module)
while ($true) {
$AutomationModule = & $GetAutomationModule `
-Name $Module `
-ResourceGroupName $ResourceGroupName `
-AutomationAccountName $AutomationAccountName
$IsTerminalProvisioningState = ($AutomationModule.ProvisioningState -eq "Succeeded") -or
($AutomationModule.ProvisioningState -eq "Failed")
if ($IsTerminalProvisioningState) {
break
}
Write-Verbose ("Module {0} is getting imported" -f $Module)
Start-Sleep -Seconds 30
}
if ($AutomationModule.ProvisioningState -ne "Succeeded") {
Write-Error ("Failed to import module : {0}. Status : {1}" -f $Module, $AutomationModule.ProvisioningState)
} else {
Write-Output ("Successfully imported module : {0}" -f $Module)
}
}
}
# Uses the module import map created to import modules.
# It will only import modules from an element in the array if all the modules
# from the previous element have been added.
function Import-ModulesInAutomationAccordingToDependency([string[][]] $ModuleImportMapOrder, [bool] $UseAzModule) {
foreach($ModuleList in $ModuleImportMapOrder) {
$i = 0
Write-Output "Importing Array of modules : $ModuleList"
foreach ($Module in $ModuleList) {
Write-Verbose ("Importing module : {0}" -f $Module)
Import-AutomationModule -ModuleName $Module -UseAzModule $UseAzModule
$i++
if ($i % $SimultaneousModuleImportJobCount -eq 0) {
# It takes some time for the modules to start getting imported.
# Sleep for sometime before making a query to see the status
Start-Sleep -Seconds 20
Wait-AllModulesImported -ModuleList $ModuleList -Count $i -UseAzModule $UseAzModule
}
}
if ($i -lt $SimultaneousModuleImportJobCount) {
Start-Sleep -Seconds 20
Wait-AllModulesImported -ModuleList $ModuleList -Count $i -UseAzModule $UseAzModule
}
}
}
function Update-ProfileAndAutomationVersionToLatest([string] $AutomationModuleName) {
# Get the latest azure automation module version
$VersionAndDependencies = Get-ModuleDependencyAndLatestVersion $AutomationModuleName
# Automation only has dependency on profile
$ModuleDependencies = GetModuleNameAndVersionFromPowershellGalleryDependencyFormat $VersionAndDependencies[1]
$ProfileModuleName = $ModuleDependencies[0]
# Create web client object for downloading data
$WebClient = New-Object System.Net.WebClient
# Download AzureRM.Profile to temp location
$ModuleContentUrl = Get-ModuleContentUrl $ProfileModuleName
$ProfileURL = (Invoke-WebRequest -Uri $ModuleContentUrl -MaximumRedirection 0 -UseBasicParsing -ErrorAction Ignore).Headers.Location
$ProfilePath = Join-Path $env:TEMP ($ProfileModuleName + ".zip")
$WebClient.DownloadFile($ProfileURL, $ProfilePath)
# Download AzureRM.Automation to temp location
$ModuleContentUrl = Get-ModuleContentUrl $AutomationModuleName
$AutomationURL = (Invoke-WebRequest -Uri $ModuleContentUrl -MaximumRedirection 0 -UseBasicParsing -ErrorAction Ignore).Headers.Location
$AutomationPath = Join-Path $env:TEMP ($AutomationModuleName + ".zip")
$WebClient.DownloadFile($AutomationURL, $AutomationPath)
# Create folder for unzipping the Module files
$PathFolderName = New-Guid
$PathFolder = Join-Path $env:TEMP $PathFolderName
# Unzip files
$ProfileUnzipPath = Join-Path $PathFolder $ProfileModuleName
Expand-Archive -Path $ProfilePath -DestinationPath $ProfileUnzipPath -Force
$AutomationUnzipPath = Join-Path $PathFolder $AutomationModuleName
Expand-Archive -Path $AutomationPath -DestinationPath $AutomationUnzipPath -Force
# Import modules
Import-Module (Join-Path $ProfileUnzipPath ($ProfileModuleName + ".psd1")) -Force -Verbose
Import-Module (Join-Path $AutomationUnzipPath ($AutomationModuleName + ".psd1")) -Force -Verbose
}
#endregion
#region Main body
if ($ModuleVersionOverrides) {
$ModuleVersionOverridesHashTable = ConvertJsonDictTo-HashTable $ModuleVersionOverrides
} else {
$ModuleVersionOverridesHashTable = #{}
}
$UseAzModule = $null
$AutomationModuleName = $null
# We want to support updating Az modules. This means this runbook should support upgrading using only Az modules
if ($AzureModuleClass -eq "Az") {
$UseAzModule = $true
$AutomationModuleName = $script:AzAutomationModuleName
} elseif ( $AzureModuleClass -eq "AzureRM") {
$UseAzModule = $false
$AutomationModuleName = $script:AzureRMAutomationModuleName
} else {
Write-Error "Invalid AzureModuleClass: '$AzureModuleClass'. Must be either Az or AzureRM" -ErrorAction Stop
}
# Import the latest version of the Az automation and accounts version to the local sandbox
Update-ProfileAndAutomationVersionToLatest $AutomationModuleName
if ($Login) {
Login-AzureAutomation $UseAzModule
}
$ModuleImportMapOrder = Create-ModuleImportMapOrder $UseAzModule
Import-ModulesInAutomationAccordingToDependency $ModuleImportMapOrder $UseAzModule
#endregion
Further, if you want to refer to another approach, you can check this script.
Here is Reddit question regarding the same.
I have used azure devops to install the modules with powershell. I was trying to install it within the runbook but that didn't work so i kind of changed the method.
Here is the code:
#necessary modules list
$deps1 = #("Az.Accounts","Az.Storage","Az.Compute")
foreach($dep in $deps1){
$module = Find-Module -Name $dep
$link = $module.RepositorySourceLocation + "/package/" + $module.Name + "/" + $module.Version
New-AzAutomationModule -AutomationAccountName $AutomationAccountName -Name $module.Name -ContentLinkUri $link -ResourceGroupName $ResourceGroupName
if ($dep -eq "Az.Accounts") {
#Az.Accounts is a dependency for Az.Storage and Az.Compute modules
Write-Host "Sleeping for 180 sec in order to wait the installation of the Az.Accounts module"
Start-Sleep 180
}
}
This answer in this post helped me for the code.

Splitting output of a string into separate strings

I've been working on a powershell script and it's been really boggling my mind. There are 2 parts to the script.
First part is a function that gets all servers in a domain. We have 4 different domains, so I check each one individually and output the result.
Second part is a function that outputs the software on a specific remote machine. In my case, the output from the function above will be seeded into this function to see if a server has a particular piece of software installed.
The function that searches the software works properly. The function that I am getting an output of all the servers is what I am having trouble with.
The issue is, that when I output the list of servers (the output is correct), it outputs everything into a single large multiline string...
For example lets say I have 5 servers: (ServerA, ServerB, ServerC, ServerD, ServerE).
When I run the code I will get an output of all the servers for each domain like so:
TestA.com
ServerA
ServerB
ServerC
ServerD
ServerE
TestB.com
ServerA
ServerB
ServerC
ServerD
ServerE
TestC.com
ServerA
ServerB
ServerC
ServerD
ServerE
TestD.com
ServerA
ServerB
ServerC
ServerD
ServerE
However each domain output is all 1 string, so I can't seed it into the function to check software because it's trying to find it in "ServerA,ServerB,ServerC,ServerD,ServerE", instead of each server individually.
I hope this makes sense. Here is my code to get the list of servers.
#Clear Screen
CLS
function Get-Servers
{
#Variables
[array]$MyDomains="TestA.com","TestB.com","TestC.com","TestD.com"
[array]$MySearchBase="dc=TestA,dc=com","dc=TestB,dc=com","dc=TestC,dc=com","dc=TestD,dc=com"
for($i=0; $i -lt $MyDomains.Count; $i++)
{
Write-Output $($MyDomains[$i])
$MyServers = Get-ADComputer -Filter 'OperatingSystem -like "Windows*Server*"' -Properties Name -SearchBase $($MySearchBase[$i]) -Server $($MyDomains[$i]) | Format-Table Name -HideTableHeaders | out-string
foreach ($MyServer in $MyServers)
{
$MyServer
pause
}
}
}
#Get list of servers
Get-Servers
How can I get the output for each server individually to be stored in the "$MyServer" variable?
EDIT:
Here is my function to find remote software
function Get-RemoteRegistryProgram
{
<#
.Synopsis
Uses remote registry to read installed programs
.DESCRIPTION
Use dot net and the registry key class to query installed programs from a
remote machine
.EXAMPLE
Get-RemoteRegistryProgram -ComputerName Server1
#>
[CmdletBinding()]
Param
(
[Parameter(
ValueFromPipeline=$true,
ValueFromPipelineByPropertyName=$true,
Position=0)]
[string[]]
$ComputerName = $env:COMPUTERNAME
)
begin
{
$hives = #(
[Microsoft.Win32.RegistryHive]::LocalMachine,
[Microsoft.Win32.RegistryHive]::CurrentUser
)
$nodes = #(
"Software\Microsoft\Windows\CurrentVersion\Uninstall",
"Software\WOW6432Node\Microsoft\Windows\CurrentVersion\Uninstall"
)
}
process
{
$ComputerName
forEach ($computer in $ComputerName)
{
forEach($hive in $hives)
{
try
{
$registry = [Microsoft.Win32.RegistryKey]::OpenRemoteBaseKey($hive,$computer)
}
catch
{
throw $PsItem
}
forEach($node in $nodes)
{
try
{
$keys = $registry.OpenSubKey($node).GetSubKeyNames()
forEach($key in $keys)
{
$displayname = $registry.OpenSubKey($node).OpenSubKey($key).GetValue('DisplayName')
if($displayname)
{
$installedProgram = #{
# ComputerName = $computer
DisplayName = $displayname
# Version = $registry.OpenSubKey($node).OpenSubKey($key).GetValue('DisplayVersion')
}
New-Object -TypeName PSObject -Property $installedProgram
}
}
}
catch
{
$orginalError = $PsItem
Switch($orginalError.FullyQualifiedErrorId)
{
'InvokeMethodOnNull'
{
#key maynot exists
}
default
{
throw $orginalError
}
}
}
}
}
}
}
end
{
}
}
EDIT 2:
If I modify my server function like so:
for($i=0; $i -lt $MyDomains.Count; $i++)
{
Write-Output $($MyDomains[$i])
$MyServers = Get-ADComputer -Filter 'OperatingSystem -like "Windows*Server*"' -Properties Name -SearchBase $($MySearchBase[$i]) -Server $($MyDomains[$i]) | Format-Table Name -HideTableHeaders
foreach ($MyServer in $MyServers)
{
Get-RemoteRegistryProgram -ComputerName $MyServer
}
}
I get the following error:
Microsoft.PowerShell.Commands.Internal.Format.FormatStartData
Exception calling "OpenRemoteBaseKey" with "2" argument(s): "The network path was not found.
"
At line:47 char:21
+ $registry = [Microsoft.Win32.RegistryKey]::OpenRemoteBaseKey ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : IOException
Thank you in advance for any help!
Your code is converting the server names to a string
$MyServers = Get-ADComputer -Filter 'OperatingSystem -like "Windows*Server*"' -Properties Name -SearchBase $($MySearchBase[$i]) -Server $($MyDomains[$i]) | Format-Table Name -HideTableHeaders | out-string
The last part of that is out-string. Instead of piping to a format table and pushing it out as a string, keep the objects and use the properties in each object to get the names of each server.
I ended up rewriting some things and fixing my issue. To avoid the string issue, I export the results to a text file and then using get-content I read line by line from the text file and seeded each server to let me know which servers have the software I need. Here is the end result.
#Clear Screen
CLS
function Get-RemoteRegistryProgram
{
<#
.Synopsis
Uses remote registry to read installed programs
.DESCRIPTION
Use dot net and the registry key class to query installed programs from a
remote machine
.EXAMPLE
Get-RemoteRegistryProgram -ComputerName Server1
#>
[CmdletBinding()]
Param
(
[Parameter(ValueFromPipeline=$true, ValueFromPipelineByPropertyName=$true, Position=0)][string]$ComputerName = $env:COMPUTERNAME,
[Parameter(ValueFromPipeline=$true, ValueFromPipelineByPropertyName=$true, Position=1)][string]$SoftwareName
)
begin
{
$hives = #(
[Microsoft.Win32.RegistryHive]::LocalMachine,
[Microsoft.Win32.RegistryHive]::CurrentUser
)
$nodes = #(
"Software\Microsoft\Windows\CurrentVersion\Uninstall",
"Software\WOW6432Node\Microsoft\Windows\CurrentVersion\Uninstall"
)
}
process
{
$ComputerName
$skip = $false
forEach ($computer in $ComputerName)
{
forEach($hive in $hives)
{
try
{
$registry = [Microsoft.Win32.RegistryKey]::OpenRemoteBaseKey($hive,$computer)
}
catch
{
$skip = $true
}
if($skip -eq $false)
{
forEach($node in $nodes)
{
try
{
$keys = $registry.OpenSubKey($node).GetSubKeyNames()
forEach($key in $keys)
{
$displayname = $registry.OpenSubKey($node).OpenSubKey($key).GetValue('DisplayName')
#Modified by James
if(($displayname) -like "*$SoftwareName*")
{
$displayname + "`t" + $computer >> c:\scripts\sysaidServers.txt
}
<# Modified by James
if($displayname)
{
$installedProgram = #{
# ComputerName = $computer
DisplayName = $displayname
# Version = $registry.OpenSubKey($node).OpenSubKey($key).GetValue('DisplayVersion')
}
New-Object -TypeName PSObject -Property $installedProgram
}
#>
}
}
catch
{
<#
$orginalError = $PsItem
Switch($orginalError.FullyQualifiedErrorId)
{
'InvokeMethodOnNull'
{
#key maynot exists
}
default
{
throw $orginalError
}
}
#>
}
}
}
}
}
}
end
{
}
}
#Output the servers to a txt file
function Get-Servers
{
param ([Parameter( Mandatory=$true)][string]$SaveFile)
#Variables
[array]$MyDomains="DomainA.com","DomainB.com","DomainC.com","DomainD.com"
[array]$MySearchBase="dc=DomainA,dc=com","dc=DomainB,dc=com","dc=DomainC,dc=com","dc=DomainD,dc=com"
for($i=0; $i -lt $MyDomains.Count; $i++)
{
#I only want servers running Windows Server OS
$MyServers = Get-ADComputer -Filter 'OperatingSystem -like "Windows*Server*"' -Properties Name -SearchBase $($MySearchBase[$i]) -Server $($MyDomains[$i]) | Format-Table Name -HideTableHeaders | out-string
#Remove all whitespace and export to txt file
$MyServers.Trim() -replace (' ', '') >> $SaveFile
}
}
function CheckServerSoftware
{
param ([Parameter( Mandatory=$true)][string]$SaveFile)
Get-Content $SaveFile | ForEach-Object {
if($_ -match $regex)
{
$computer = $_.ToString()
Get-RemoteRegistryProgram -ComputerName $computer.Trim() $SoftwareName
Write-Output ""
}
}
}
#Path to where our exported server list is
$SaveFile = "c:\scripts\servers.txt"
$SoftwareName = "SysAid"
#If the file already exists, remove it
Remove-Item $SaveFile
#Create the text file with servers
Get-Servers $SaveFile
#Import our server list and check software on each server
CheckServerSoftware $SaveFile

Powershell script to Find folders which have "Everyone" access and how to delete it

Need a Power Shell Script to find on a remote server which folders have "Everyone" access and How do I takeout this particular access (Remove Everyone group from Security)?
Something along these lines?
# 1. Get list of folders
$FolderList = Get-ChildItem -Path c:\test\everyone -Directory -Recurse;
# 2. Iterate over folders and remove 'everyone' rules
foreach ($Folder in $FolderList) {
$Acl = Get-Acl -Path $Folder.FullName;
$Everyone = $Acl.Access.Where({ $PSItem.IdentityReference -eq 'Everyone' });
foreach ($Ace in $Everyone) {
[void] $Acl.RemoveAccessRule($Ace);
}
Set-Acl -Path $Folder.FullName -AclObject $Acl;
}
IMPORTANT: This script requires PowerShell version 4.0, as it uses the Where "method syntax."
Translating Trevor's solution to PowerShell v2/v3:
$rootFolder = '\\server\c$\some\folder'
# get locale-specific name for 'Everyone' security principal
$sid = New-Object Security.Principal.SecurityIdentifier('S-1-1-0')
$everyone = $sid.Translate([Security.Principal.NTAccount]).Value
Get-ChildItem $rootFolder -Recurse | ? { $_.PSIsContainer } | % {
$acl = Get-Acl $_.FullName
$acl.Access | ? { $_.IdentityReference -eq $everyone } | % {
$acl.RemoveAccessRule($_)
}
Set-Acl $_.FullName -Acl $acl | Out-Null
}
In PowerShell v3 you can replace | ? { $_.PSIsContainer } with -Directory.
Note that this won't remove Everyone ACEs on the root folder itself, and it also won't remove inherited Everyone ACEs.

Display all sites and bindings in PowerShell

I am documenting all the sites and binding related to the site from the IIS. Is there an easy way to get this list through a PowerShell script rather than manually typing looking at IIS?
I want the output to be something like this:
Site Bindings
TestSite www.hello.com
www.test.com
JonDoeSite www.johndoe.site
Try this:
Import-Module Webadministration
Get-ChildItem -Path IIS:\Sites
It should return something that looks like this:
Name ID State Physical Path Bindings
---- -- ----- ------------- --------
ChristophersWeb 22 Started C:\temp http *:8080:ChristophersWebsite.ChDom.com
From here you can refine results, but be careful. A pipe to the select statement will not give you what you need. Based on your requirements I would build a custom object or hashtable.
Try something like this to get the format you wanted:
Get-WebBinding | % {
$name = $_.ItemXPath -replace '(?:.*?)name=''([^'']*)(?:.*)', '$1'
New-Object psobject -Property #{
Name = $name
Binding = $_.bindinginformation.Split(":")[-1]
}
} | Group-Object -Property Name |
Format-Table Name, #{n="Bindings";e={$_.Group.Binding -join "`n"}} -Wrap
If you just want to list all the sites (ie. to find a binding)
Change the working directory to "C:\Windows\system32\inetsrv"
cd c:\Windows\system32\inetsrv
Next run "appcmd list sites" (plural) and output to a file. e.g c:\IISSiteBindings.txt
appcmd list sites > c:\IISSiteBindings.txt
Now open with notepad from your command prompt.
notepad c:\IISSiteBindings.txt
The most easy way as I saw:
Foreach ($Site in get-website) { Foreach ($Bind in $Site.bindings.collection) {[pscustomobject]#{name=$Site.name;Protocol=$Bind.Protocol;Bindings=$Bind.BindingInformation}}}
Try this
function DisplayLocalSites
{
try{
Set-ExecutionPolicy unrestricted
$list = #()
foreach ($webapp in get-childitem IIS:\Sites\)
{
$name = "IIS:\Sites\" + $webapp.name
$item = #{}
$item.WebAppName = $webapp.name
foreach($Bind in $webapp.Bindings.collection)
{
$item.SiteUrl = $Bind.Protocol +'://'+ $Bind.BindingInformation.Split(":")[-1]
}
$obj = New-Object PSObject -Property $item
$list += $obj
}
$list | Format-Table -a -Property "WebAppName","SiteUrl"
$list | Out-File -filepath C:\websites.txt
Set-ExecutionPolicy restricted
}
catch
{
$ExceptionMessage = "Error in Line: " + $_.Exception.Line + ". " + $_.Exception.GetType().FullName + ": " + $_.Exception.Message + " Stacktrace: " + $_.Exception.StackTrace
$ExceptionMessage
}
}
function Get-ADDWebBindings {
param([string]$Name="*",[switch]$http,[switch]$https)
try {
if (-not (Get-Module WebAdministration)) { Import-Module WebAdministration }
Get-WebBinding | ForEach-Object { $_.ItemXPath -replace '(?:.*?)name=''([^'']*)(?:.*)', '$1' } | Sort | Get-Unique | Where-Object {$_ -like $Name} | ForEach-Object {
$n=$_
Get-WebBinding | Where-Object { ($_.ItemXPath -replace '(?:.*?)name=''([^'']*)(?:.*)', '$1') -like $n } | ForEach-Object {
if ($http -or $https) {
if ( ($http -and ($_.protocol -like "http")) -or ($https -and ($_.protocol -like "https")) ) {
New-Object psobject -Property #{Name = $n;Protocol=$_.protocol;Binding = $_.bindinginformation}
}
} else {
New-Object psobject -Property #{Name = $n;Protocol=$_.protocol;Binding = $_.bindinginformation}
}
}
}
}
catch {
$false
}
}
I found this page because I needed to migrate a site with many many bindings to a new server. I used some of the code here to generate the powershell script below to add the bindings to the new server. Sharing in case it is useful to someone else:
Import-Module WebAdministration
$Websites = Get-ChildItem IIS:\Sites
$site = $Websites | Where-object { $_.Name -eq 'site-name-in-iis-here' }
$Binding = $Site.bindings
[string]$BindingInfo = $Binding.Collection
[string[]]$Bindings = $BindingInfo.Split(" ")
$i = 0
$header = ""
Do{
[string[]]$Bindings2 = $Bindings[($i+1)].Split(":")
Write-Output ("New-WebBinding -Name `"site-name-in-iis-here`" -IPAddress " + $Bindings2[0] + " -Port " + $Bindings2[1] + " -HostHeader `"" + $Bindings2[2] + "`"")
$i=$i+2
} while ($i -lt ($bindings.count))
It generates records that look like this:
New-WebBinding -Name "site-name-in-iis-here" -IPAddress "*" -Port 80 -HostHeader www.aaa.com
I found this question because I wanted to generate a web page with links to all the websites running on my IIS instance. I used Alexander Shapkin's answer to come up with the following to generate a bunch of links.
$hostname = "localhost"
Foreach ($Site in get-website) {
Foreach ($Bind in $Site.bindings.collection) {
$data = [PSCustomObject]#{
name=$Site.name;
Protocol=$Bind.Protocol;
Bindings=$Bind.BindingInformation
}
$data.Bindings = $data.Bindings -replace '(:$)', ''
$html = "" + $data.name + ""
$html.Replace("*", $hostname);
}
}
Then I paste the results into this hastily written HTML:
<html>
<style>
a { display: block; }
</style>
{paste PowerShell results here}
</body>
</html>

Resources