PowerShell: Output provider results from multiple Azure subscriptions into one .txt file - via arrayList - azure

I am trying to get the registered providers for multiple subscriptions and output everything into one file.
For that, i am getting the subscriptions from a folder full of *.yaml files that contain information about the subscriptions, including their name.
What i cannot achieve is get the result for each subscriptions into an array and output that array into a text file. The script also allows the use of only one item from the folder in case that is the case.
Here is a sample code of a subscription and the code for it:
subscription1.yaml
name: subscription1
emailContact: email.address#domain.com
tags:
- key: "key1"
value: "Value1"
subscription2.yaml
name: subscription2
emailContact: email.address#domain.com
tags:
- key: "key1"
value: "Value1"
Folder structure where the yaml files is: ./landingZones/landingzone1/settings/dev/*.yaml
script:
param (
[Parameter(Mandatory = $false)]
[string]$Environment = 'dev',
[Parameter(Mandatory = $false)]
[string]$LandingZoneType = 'landingzone1',
[Parameter(Mandatory = $false)]
[string]$SingleSubscription
)
$scriptPath = Split-Path -parent $PSCommandPath
$subscriptionsEnvironmentDirectory = Get-ChildItem -Directory $scriptPath -Recurse -Filter "*$Environment*" | Where-Object { $_.parent.parent.Name -eq $LandingZoneType }
$subscriptions = Get-ChildItem -Path $($subscriptionsEnvironmentDirectory.FullName)
foreach ($subscription in ($subscriptions | Where-Object { ([System.String]::IsNullOrEmpty($SingleSubscription)) -or ($_.Name -replace "\.[^\.]+$", '') -eq $SingleSubscription })) {
$landingZone = Get-Content -Path $subscription.FullName | ConvertFrom-Yaml
# Set subscriptionName variable
$subscriptionName = $landingZone.name
$providers = az provider list --subscription $subscriptionName | ConvertFrom-Json
$defaultRegisteredProviders = 'Microsoft.ADHybridHealthService|Microsoft.Authorization|Microsoft.Billing|Microsoft.ClassicSubscription|Microsoft.Commerce|Microsoft.Consumption|Microsoft.CostManagement|Microsoft.Features|Microsoft.MarketplaceOrdering'
$registeredProviders = $providers | Where-Object { ($_.registrationState -eq 'Registered') -and ($_.namespace -notmatch $defaultRegisteredProviders) }
# Outputting result into txt file in the same directory where the command was executed
Write-Host ('{1}# Registered providers for subscription [{0}]' -f $subscriptionName, "`n")
$list = New-Object -TypeName 'System.Collections.ArrayList'
$sortedObjects = $registeredProviders | Sort-Object namespace | `
Format-Table `
#{l = 'Namespace'; e = { $_.namespace } }, `
#{l = "Subscription Id [$subscriptionName]"; e = { $_.id } }, `
#{l = 'Registration State'; e = { $_.registrationState } }, `
#{l = 'Registration Policy'; e = { $_.registrationPolicy } }
foreach ($i in $sortedObjects) {
$list.Add($i) | Out-Null
}
# Alternative to add into array:
# #($sortedObjects).foreach({$list.Add($_)}) | Out-Null
}
$list.Count
$list | Out-File .\registered_providers.txt -Force
The result is a file called 'registered_providers.txt' that contains only the registered providers for the first subscription in the foreach loop. I cannot get the contents of the second, third and so on in the same file, just a replaced text from the $sortedObjects
How do i create the array to contain all the info from all the subscriptions called?
Thanks

$list | Out-File Should be inside the for loop so that all the data you are fetching will be stored in the specified file.
Out-File cmdlet has -append parameter that appends the output to the existing file for every for loop happens. Otherwise, it will clean up the information/value stored in that output file.
Thanks to #jdweng for pointing the user to the right solution.
Refer to this MS Doc on Out-File -Append Parameter usage.

Related

Powershell does not create CSV if there is no data to export in Azure

I am trying to export VM data from Azure and below script is working perfect if subscription has VMs however it does create a .csv if there is no data (VMs) and I need that even if there is no data powershell should create a blank csv. Below is my script which is working fine if subscription has VMs created in it.
function create($path) {
$exists = Test-Path -path $path
Write-Host "tried the following path: $path, it" $(If ($exists) {"Exists"} Else {"Does not Exist!"})
if (!($exists)) { New-Item $path -itemType Directory }
}
# reading file contents
$subs_file = "C:\Scrpting\Subscriptions\Subscriptions.xlsx"
$azSubs = Import-Excel $subs_file
$azSubs
$output_folder = "C:\audit-automation"
# creating folder for outputing data
create("$output_folder")
# New-Item $output_folder -itemType Directory
# iterating over subscriptions
ForEach ( $sub in $azSubs ) {
# sub
$azsub = $sub.Subscription
# app
$app = $sub.Application
$azsub
$app
# creating folder to save data for apps
# New-Item $output_folder\$app -itemType Directory
# setting config for azure
Set-AzContext -SubscriptionName $azsub
# GET VM INFO
$vms = Get-AzVM
$vmrg = Get-AzVM | Select-Object "ResourceGroupName"
$nics = get-AzNetworkInterface | Where-Object { $_.VirtualMachine -NE $null }
# creating folder to save
# New-Item $output_folder\$app\vm_info -itemType Directory
create("$output_folder\$app")
ForEach ($nic in $nics) {
$info = "" | Select VMName, ResourceGroupName, OS, PrivateIPAddress, PublicIPAddress, SubscriptionID, Status, NICName
$vm = $vms | ? -Property Id -eq $nic.VirtualMachine.id
$info.NICName = $nic.Name
$info.VMName = $vm.Name
$info.SubscriptionID = $azsub
$info.ResourceGroupName = $vm.ResourceGroupName
$info.PrivateIPAddress = $nic.IpConfigurations.PrivateIpAddress
$PublicIPAddress =
(Az vm list-ip-addresses --name $vm.Name --resource-group $vm.ResourceGroupName | ConvertFrom-Json).virtualMachine.network.publicIpAddresses.ipaddress
$info.PublicIPAddress = if ($null -eq $PublicIPAddress ) { "Not Assigned" } else { $PublicIPAddress }
$info.OS = $vm.StorageProfile.osDisk.osType
$info.Status = ((Get-AzVM -ResourceGroupName $vm.ResourceGroupName -Name $vm.Name -Status).Statuses[1]).code
$info | Export-Csv -Path $output_folder\$app\$app-vm_data$((Get-Date).ToString("yyyy-MM-dd")).csv -Append
}}
You can remove the function create and replace that with just the one line $null = New-Item $output_folder -ItemType Directory -Force.
On file system, the -Force makes the cmdlet return either the DirectoryInfo object of an existing folder or create a new folder and return that.
(don't use this on registry keys!)
Next, if you capture the output you want in the CSV file first, it is easy enough to check if there is output or not and if not, write an empty csv file (only the headers will be in there)
Something like this:
# reading file contents
$subs_file = "C:\Scrpting\Subscriptions\Subscriptions.xlsx"
$azSubs = Import-Excel $subs_file
#$azSubs
$output_folder = "C:\audit-automation"
# creating folder for outputing data
$null = New-Item $output_folder -ItemType Directory -Force
# iterating over subscriptions
foreach($sub in $azSubs) {
# sub
$azsub = $sub.Subscription
# app
$app = $sub.Application
$azsub
$app
# setting config for azure
Set-AzContext -SubscriptionName $azsub
# GET VM INFO
$vms = Get-AzVM
$vmrg = Get-AzVM | Select-Object "ResourceGroupName"
$nics = Get-AzNetworkInterface | Where-Object { $null -ne $_.VirtualMachine }
# creating folder to save
$null = New-Item (Join-Path -Path $output_folder -ChildPath $app) -ItemType Directory -Force
# capture the output of the foreach loop
$result = foreach ($nic in $nics) {
$info = "" | Select VMName, ResourceGroupName, OS, PrivateIPAddress, PublicIPAddress, SubscriptionID, Status, NICName
$vm = $vms | ? -Property Id -eq $nic.VirtualMachine.id
$info.NICName = $nic.Name
$info.VMName = $vm.Name
$info.SubscriptionID = $azsub
$info.ResourceGroupName = $vm.ResourceGroupName
$info.PrivateIPAddress = $nic.IpConfigurations.PrivateIpAddress
$PublicIPAddress = (Az vm list-ip-addresses --name $vm.Name --resource-group $vm.ResourceGroupName |
ConvertFrom-Json).virtualMachine.network.publicIpAddresses.ipaddress
$info.PublicIPAddress = if ($null -eq $PublicIPAddress ) { "Not Assigned" } else { $PublicIPAddress }
$info.OS = $vm.StorageProfile.osDisk.osType
$info.Status = ((Get-AzVM -ResourceGroupName $vm.ResourceGroupName -Name $vm.Name -Status).Statuses[1]).code
# output the object to be collected in variable $result
$info
}
# test if you actually have data now in the $result variable
if (!#($result.Count)) {
# no data; create an empty csv with just the headers
$result = "" | Select VMName, ResourceGroupName, OS, PrivateIPAddress, PublicIPAddress, SubscriptionID, Status, NICName
}
$outFile = Join-Path -Path $output_folder -ChildPath ('{0}\{0}-vm_data{1:yyyy-MM-dd}.csv' -f $app, (Get-Date))
$result | Export-Csv -Path $outFile -NoTypeInformation
}
add outside the loop, since $vms may be empty
if (!($vms)){
$info = "" | Select VMName, ResourceGroupName, OS, PrivateIPAddress, PublicIPAddress, SubscriptionID, Status, NICName
$info | Export-Csv -Path $output_folder\$app\$app-vm_data$((Get-Date).ToString("yyyy-MM-dd")).csv -Append
}

Add exceptions for file paths from azure defender to adaptive application security controls

I have a bunch of machines being monitored by adaptive application security controls that are giving warnings because the training process was not ran long enough to recognize benign executables. What's an easy way to add exceptions for the executables in active alerts to the adaptive security groups?
There's already an existing recommendation that might provide what you are trying to do:
https://learn.microsoft.com/en-us/azure/defender-for-cloud/adaptive-application-controls#respond-to-the-allowlist-rules-in-your-adaptive-application-control-policy-should-be-updated-recommendation
This script grabs the active alerts from defender, and updates the groups.
The alerts must still be dismissed manually.
function Get-ExistingRules {
Param(
$subscriptionId,
$groupName
)
$url = "https://management.azure.com/subscriptions/$subscriptionId/providers/Microsoft.Security/locations/centralus/applicationWhitelistings/${groupName}?api-version=2015-06-01-preview";
return az rest `
--method get `
--url $url `
| ConvertFrom-Json;
}
function Add-NewRules {
Param(
$subscriptionId,
$groupName,
$files
)
$url = "https://management.azure.com/subscriptions/$subscriptionId/providers/Microsoft.Security/locations/centralus/applicationWhitelistings/${groupName}?api-version=2015-06-01-preview";
$existing = Get-ExistingRules $subscriptionId $groupName;
$existing | ConvertTo-Json -Depth 100 > asd.json;
$myList = $existing.properties.pathRecommendations;
foreach ($file in $files) {
$myList += [pscustomobject]#{
path = $file.path
type = "File"
common = $true
action = "Add"
usernames = #(
[pscustomobject]#{
username = "Everyone"
recommendationAction = "Recommended"
}
)
userSids = #(
"S-1-1-0"
)
fileType = $file.type
configurationStatus = "NotConfigured"
};
}
$existing.properties.pathRecommendations = $myList;
$existing.properties = [pscustomobject]#{
protectionMode = $existing.properties.protectionMode
vmRecommendations = $existing.properties.vmRecommendations
pathRecommendations = $existing.properties.pathRecommendations
}
$existing.PSObject.properties.remove("location");
# return $existing;
$body = $existing | ConvertTo-Json -Depth 20 -Compress;
$body > temp.json;
# $body = $body -replace "`"", "\`"";
# return az rest `
# --method PUT `
# --url $url `
# --body $body `
# | ConvertFrom-Json;
# avoid max command length limit by storing body in a file
try {
return az rest `
--method PUT `
--url $url `
--body `#temp.json `
| ConvertFrom-Json;
}
catch {
Write-Warning "Encountered error adding rule";
Write-Warning "$_";
}
return $null;
}
function Format-Body {
param(
$obj
)
$body = $obj | ConvertTo-Json -Depth 100 -Compress;
$body = $body -replace "`"", "\`"";
$body = $body -replace "`r", "";
return $body;
}
Write-Host "Listing subscriptions";
# you can filter to just one subscription if you want
# $subscriptions = az account list --query "[?name=='NPRD'].id" --output tsv;
$subscriptions = az account list --query "[].id" --output tsv;
$allAlerts = New-Object System.Collections.ArrayList;
$i = 0;
foreach ($sub in $subscriptions) {
Write-Progress -Id 0 -Activity "Fetching alerts" -Status $sub -PercentComplete ($i / $subscriptions.Count * 100);
$i = $i++;
$alerts = az security alert list `
--subscription $sub `
| ConvertFrom-Json `
| Where-Object { #("VM_AdaptiveApplicationControlLinuxViolationAudited", "VM_AdaptiveApplicationControlWindowsViolationAudited") -contains $_.alertType } `
| Where-Object { $_.status -eq "Active" };
foreach ($x in $alerts) {
$allAlerts.Add($x) > $null;
}
}
Write-Progress -Id 0 "Done" -Completed;
function Get-Files {
Param(
$alert
)
if ($alert.alertType -eq "VM_AdaptiveApplicationControlLinuxViolationAudited") {
$fileType = "executable";
}
else {
$fileType = "exe";
}
$pattern = "Path: (.*?);";
$str = $alert.extendedProperties.file;
return $str `
| Select-String -Pattern $pattern -AllMatches `
| ForEach-Object { $_.Matches } `
| ForEach-Object { $_.Value } `
| ForEach-Object { [pscustomobject]#{
path = $_
type = $fileType
}};
}
$alertGroups = $allAlerts | Select-Object *, #{Name = "groupName"; Expression = { $_.extendedProperties.groupName } } | Group-Object groupName;
foreach ($group in $alertGroups) {
$groupName = $group.Name;
$group.Group[0].id -match "/subscriptions/([^/]+)/" > $null;
$subscriptionId = $matches[1];
$files = $group.Group | ForEach-Object { Get-Files $_ };
Write-Host "Adding file path rule sub=$subscriptionId group=$groupName count=$($files.Count)";
Add-NewRules $subscriptionId $groupName $files;
}

Trying to list Azure Virtual Network and export to CSV using Powershell

I am trying to create a script that can list all the Azure virtual networks and export it into Csv using Powershell.
$day = Get-Date -Format " MMM-yyyy"
$path = "C:\Users\admin-vishal.singh\Desktop\Test\Report\"+ "$day-Vnet-Report.csv"
foreach ($Sub in $Subs) {
Select-AzSubscription -SubscriptionName $Sub.Name | Out-Null
$resource_grps = Get-AzResourceGroup
foreach ($resource_grp in $resource_grps) {
$networks = Get-AzVirtualNetwork
foreach ($vnet in $networks)
{
$null = Get-AzVirtualNetwork |Select-Object SubscriptionName,ResourceGroupName,Name,AddressSpace,Subnets,SubnetAddressSpace,RouteTable | Export-CSV -Path $path -NoTypeInformation -Encoding ASCII -Append
}
}
}
I am not able to retrieve data in the right format & getting errors when retrieving data.
Below is snippet of data
Lots of values I am not able to retrieve like Subnet AddressSpace, Route Tables and Routes.
Building on what Jim Xu provided, you don't need to have a separate loop for each ResourceGroup. Get-AzVirtualNetwork will return all virtual networks for the entire subscription. Also, you'll need an expression for the SubscriptionName in the Select-Object, so the code would look like this:
foreach ($Sub in $Subs) {
Select-AzSubscription -SubscriptionName $Sub.Name | Out-Null
Get-AzVirtualNetwork |
Select-Object `
#{label='SubscriptionName'; expression={$Sub.Name}}, `
ResourceGroupName, `
Name, `
#{label='AddressSpace'; expression={$_.AddressSpace.AddressPrefixes}}, `
#{label='SubnetName'; expression={$_.Subnets.Name}}, `
#{label='SubnetAddressSpace'; expression={$_.Subnets.AddressPrefix}} |
Export-CSV -Path $path -NoTypeInformation -Encoding ASCII -Append
}
When we call export-csv command, the property values are converted to strings using the ToString() method. But the result of Get-AzVirtualNetwork are object, we cannot directly convert the value to string. For more details, please refer to here and here
So regarding the issue, I suggest you create a custom object with the information you need then save it into csv.
For exmaple
$vents =Get-AzVirtualNetwork|
Select-Object SubscriptionName,ResourceGroupName,Name, #{
label='AddressSpace'
expression={$_.AddressSpace.AddressPrefix}}, #{
label='SubnetName'
expression={$_.Subnets.Name}
}, #{
label='SubnetAddressSpace'
expression={$_.Subnets.AddressPrefix}
}
$vents | convertto-csv

How to output hash table query result into Out-GridView?

I wish to export a hashtable result into Out-GridView using the Powershell.
The purpose of the below script is to export the Azure VM tags to Out-GridView, it throws error like the below blank result:
Error on the console:
Out-GridView : Syntax error in PropertyPath 'Syntax error in Binding.Path '[ Product] ' ... '(Tag)'.'.
At line:46 char:19
+ $Output | Out-GridView #Export-Csv -Path c:\temp\1a.csv -appe ...
+ ~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [Out-GridView], InvalidOperationException
+ FullyQualifiedErrorId : ManagementListInvocationException,Microsoft.PowerShell.Commands.OutGridViewCommand
This is the actual script which was executed under the Global Administrator role:
<#
.AUTHOR: https://stackoverflow.com/users/13390556/lukasz-g
#>
$Subscription = Get-AzSubscription | Out-GridView -Title 'Select subscription' -OutputMode 'Multiple'
# Initialise output array
$Output = #()
if ($Subscription) {
foreach ($item in $Subscription) {
$item | Select-AzSubscription
# Collect all the resources or resource groups (comment one of below)
$Resource = Get-AzResource
#$Resource = Get-AzResourceGroup
# Obtain a unique list of tags for these groups collectively
$UniqueTags = $Resource.Tags.GetEnumerator().Keys | Get-Unique -AsString | Sort-Object | Select-Object -Unique | Where-Object { $_ -notlike "hidden-*" }
# Loop through the resource groups
foreach ($ResourceGroup in $Resource) {
# Create a new ordered hashtable and add the normal properties first.
$RGHashtable = New-Object System.Collections.Specialized.OrderedDictionary
$RGHashtable.Add("Name", $ResourceGroup.ResourceGroupName)
$RGHashtable.Add("Location", $ResourceGroup.Location)
$RGHashtable.Add("Id", $ResourceGroup.ResourceId)
$RGHashtable.Add("ResourceType", $ResourceGroup.ResourceType)
# Loop through possible tags adding the property if there is one, adding it with a hyphen as it's value if it doesn't.
if ($ResourceGroup.Tags.Count -ne 0) {
$UniqueTags | Foreach-Object {
if ($ResourceGroup.Tags[$_]) {
$RGHashtable.Add("[$_] (Tag)", $ResourceGroup.Tags[$_])
}
else {
$RGHashtable.Add("[$_] (Tag)", "-")
}
}
}
else {
$UniqueTags | Foreach-Object { $RGHashtable.Add("[$_] (Tag)", "-") }
}
# Update the output array, adding the ordered hashtable we have created for the ResourceGroup details.
$Output += New-Object psobject -Property $RGHashtable
}
# Sent the final output to CSV
$Output | Out-GridView #Export-Csv -Path c:\temp\1a.csv -append -NoClobber -NoTypeInformation -Encoding UTF8 -Force
}
}
$RGHashtable.Add("[$_] (Tag)"
In above code, You are trying to add something like below :
In the output
Removed everthing and I tested with simple statements
$Output = #()
$RGHashtable = New-Object System.Collections.Specialized.OrderedDictionary
$RGHashtable.Add("[Testing] (Name)", "Temporary")
$Output += New-Object psobject -Property $RGHashtable
$Output | Out-GridView
I was provided with the same error.
After couple of testing, understood the error only occurs when there is a combination "[SomeString](SomeString)" --- [...](....) in the string.
The Out-GridView is trying to parse the "[<SomeString>](<SomeString>)" and hence the error.
You could try any 1 of the below combination in your code :
$RGHashtable.Add("[$_] [Tag]", $ResourceGroup.Tags[$_])
OR
$RGHashtable.Add("{$_} (Tag)", $ResourceGroup.Tags[$_])
OR
$RGHashtable.Add("[$_] [Tag]", $ResourceGroup.Tags[$_])
This should resolve your issue.
you will have change in 3 instances in your code if I am not wrong.

In Azure, To get Load balancer details in to csv file using powershell script

I wrote a powershell script in Azure DevOps pipeline to get Load Balancer details like FrontendIPConfigurationsName,FrontendIPAddress in to csv file. AM getting those details but FrontendIPConfigurationsNames which starts with same name like "ers-A1,ers-B1,ers-C1,ers-D1" are coming in same row. But I want to get them in different rows.Please suggest
$excel = #()
LBlist = Get-AZLoadBalancer | Where-Oject {$_.ResourceGroupName -clike '$(grp-wildcard)'} | Select-Object
foreach ($LB in LBlist)
$Array =""| Select-Object ResourceGroupName, FrontendIPConfigurationsName,FrontendIPAddress
$Array.ResourceGroupName =$LB.ResourcegroupName
$Array.FrontendIPConfigurationsName = ($LB.FrontendIpConfigurationsName.name -join ',')
$Array.FrontendIPAddress =($LB.FrontendIpConfigurations.PrivateIpAddress -join ',')
}
$excel +=$Array
$excel |Format-Table ResourceGroupName, FrontendIPConfigurationsName,FrontendIPAddress
$excel | Export-Csv -NTI -Path "($Build.ArtifactStagingDirectory)/LBlist.csv
You can do something similar to this to get the objects on separate rows.
$LBlist = Get-AZLoadBalancer | Where-Object { $_.ResourceGroupName -clike '$(grp-wildcard)' }
$LBlist | Export-Csv -NTI -Path "$(Build.ArtifactStagingDirectory)/LBlist.csv"
Mock for Get-AZLoadBalancer (not sure if it's entirely accurate as I've never used Az Powershell)
function Get-AZLoadBalancer() {
$obj = #(
[PSCustomObject] #{
ResourceGroupName = '$(grp-wildcard)'
FrontendIPConfigurationsName = 'ers-A1'
FrontEndIPAddress = '1.2.3.4'
},
[PSCustomObject] #{
ResourceGroupName = '$(grp-wildcard)'
FrontendIPConfigurationsName = 'ers-B1'
FrontEndIPAddress = '1.2.3.5'
}
)
return $obj
}

Resources