Powershell - Return Azure Blob Size by first split string value - azure

So I am trying to list the contents on an Azure blog container size, I can get how much storage is being used as a whole. What I trying to do is break it down, by the first position of the filename.
And this is my sad attempt. Could someone point me in the right direction please?
$ResourceGroup = "RG"
$StorageAccountName = "SAN"
$ContainerName = "CN"
$storageAccount = Get-AzStorageAccount `
-ResourceGroupName $ResourceGroup `
-Name $StorageAccountName
$Context = $storageAccount.Context
$Blobs = Get-AzStorageBlob -Container $ContainerName -Context $Context
$length = 0
$Blobs | ForEach-Object {$length = $length + $_.Length }
#$Blobs.Name.Split("_",2)[0]
$Blobs | Select-Object Name, Length
$TotalSize = [math]::Round(($length / 1024 / 1024 / 1024 / 1024),2)
Write-Host "Total Size: $TotalSize Terabytes"
Current Output.
ABCD_History_20221127_110045 9306112
ABCD_History_20221204_110052 11010048
ABCD_History_20221211_110045 10616832
EFGH_20220327_110201 48562176
EFGH_20220403_110159 46596096
Total Size: 29.63 Terabytes
Desired Output
ABCD 30932992
EFGH 95158272
Total Size: 29.63 Terabytes

I have reproduced in my environment and got expected results as below and I followed Microsoft-Document:
$ResourceGroup = "XX"
$StorageAccountName = "rith"
$ContainerName = "rithwik"
$storageAccount = Get-AzStorageAccount `
-ResourceGroupName $ResourceGroup `
-Name $StorageAccountName
$Context = $storageAccount.Context
$Blobs = Get-AzStorageBlob -Container $ContainerName -Context $Context
$length = 0
$Blobs | ForEach-Object {$length = $length + $_.Length }
$Blobs | Select-Object Name, Length
After that used below code to get required Output:
foreach($emo in $blobs)
{
$BlobName=$Blobs.Name.Substring(0,3)
}
$Target = #()
foreach($emo in $BlobName)
{
$bn=$emo
$x=0
foreach($b in $blobs)
{
if ($b.Name -match $Bn)
{
$x=$x+$b.Length
}
}
$out = $bn + $x
$Target += $out
}
$Target | select -uniq
If you want 4 letters in output give 4 in substring command instead of 3.
If you space between name and length you use like below in place of $out = $bn + $x in for loop:
$out = $bn +" "+ $x

Related

Find/list the Unused storage accounts in azure using powershell

Trying to get the list of unused/inactive storage accounts in azure using powershell. Below is my script which im trying it will provide the storage account name and last modified date of your Azure storage accounts, but i need to list only the unused storage accounts names not all the storage accounts, for that some condition/filter i need to provide to achieve the same. Please assist me to solve this. Thanks in Advance
It will output the results into a table detailing the name and last modified date of your Azure storage accounts.
& {
foreach ($storageAccount in Get-AzStorageAccount) {
$storageAccountName = $storageAccount.StorageAccountName
$resourceGroupName = $storageAccount.ResourceGroupName
# Get storage account key
$storageAccountKey = (Get-AzStorageAccountKey -Name $storageAccountName -ResourceGroupName $resourceGroupName).Value[0]
# Create storage account context using above key
$context = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey
# Get the last modified date
$lastModified = Get-AzStorageContainer -Context $context | Sort-Object -Property #{Expression = {$_.LastModified.DateTime}} | Select-Object -Last 1 -ExpandProperty LastModified
# Collect the information to output to a table when the for loop has completed
New-Object psobject -Property #{
Name = $storageAccountName;
LastModified = $lastModified.DateTime;
ResourceGroupName = $resourceGroupName
}
}
} | Format-Table Name, LastModified, ResourceGroupName -autosize
I tried to reproduce the same in my environment and got the same result as below:
By using the same script, I got the storage account name and last modified date of the Azure storage accounts.
To get only the unused/inactive storage accounts in azure using PowerShell, I modified the script like below:
I agree with #Niclas, you need make use of get-date command.
& {
foreach ($storageAccount in Get-AzStorageAccount) {
$storageAccountName = $storageAccount.StorageAccountName
$resourceGroupName = $storageAccount.ResourceGroupName
$storageAccountKey = (Get-AzStorageAccountKey -Name $storageAccountName -ResourceGroupName $resourceGroupName).Value[0]
$context = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey
$lastModified = Get-AzStorageContainer -Context $context | Sort-Object -Property #{Expression = {$_.LastModified.DateTime}} | Select-Object -Last 1 -ExpandProperty LastModified
$unusedacc = (Get-Date).AddDays(-10)
if ($lastModified.DateTime -lt $unusedacc) {
New-Object psobject -Property #{
Name = $storageAccountName;
LastModified = $lastModified.DateTime;
ResourceGroupName = $resourceGroupName
}
}
}
} | Format-Table Name, LastModified, ResourceGroupName -autosize
Note: Based on your requirement you can change the number of days in this line $unusedacc = (Get-Date).AddDays(-10).
If there are no unused Storage accounts, then it will return blank results like below:
Use get-date
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/get-date
and use the Where-Object.
# ADD THIS
$lastModDate = (get-date).AddDays(-5).Date
$lastMod = $lastModified | Where-Object { ($_.DateTime).Date -lt $lastModDate}
# If $lastMod.DateTime is NOT empty, then:
if ($lastMod.DateTime) {
# Write-Host "variable is NOT null " + $storageAccountName # For testing purpose
# Collect the information to output to a table when the for loop has completed
New-Object psobject -Property #{
Name = $storageAccountName;
LastModified = $lastMod.DateTime; # CHANGE THIS
ResourceGroupName = $resourceGroupName
}
}
https://www.techielass.com/find-unused-storage-accounts-in-azure/
With your script:
With my changes:

How to import csv to az storage table

Am trying to upload csv file to the azure table storage, which has 1000 + records and 6 columns, as shown below as Csvheadres, unfortunately, ended up with an error.do I need to add partition key and row key columns?
the CSV columns(CsvHeaders)
res1
res2
res3
res4
res5
res6
$storageAccountKey = (Get-AzStorageAccountKey -ResourceGroupName $resourceGroupName -Name $storageAccountName).Value[0]
$ctx = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey
$table = Get-AzStorageTable -Name $tableName -Context $ctx
$CsvContents = Import-Csv -Path $Path
$CsvHeaders = ($CsvContents[0] | Get-Member -MemberType NoteProperty).Name | Where{$_ -ne "RowKey" -and $_ -ne "PartitionKey"}
Foreach($CsvContent in $CsvContents)
{
$PartitionKey = $CsvContent.PartitionKey
$RowKey = $CsvContent.RowKey
$Entity = New-Object "Microsoft.Azure.Cosmos.Table.DynamicTableEntity" "$PartitionKey", "$RowKey"
Foreach($CsvHeader in $CsvHeaders)
{
$Value = $CsvContent.$CsvHeader
$Entity.Properties.Add($CsvHeader, $Value)
}
Write-Verbose "Inserting the entity into table storage."
$result = $Table.Execute([Microsoft.Azure.Cosmos.Table.TableOperation]::Insert($Entity))
}

Powershell script to delete the .bak files older than 30 days in all containers in one single script

I am trying to use for loop for selecting all the containers using this script
param($TimerBlob)
$CleanupTime = [DateTime]::UtcNow.AddMinutes(-1)
$context = New-AzStorageContext -StorageAccountName **** -StorageAccountKey ****
$containername = #("mydbstore", "mydbstore2","mydbstore3") # List of containers -
For ($i=0; $i -lt $containername.Length; $i++) {
Get-AzStorageBlob -Container $containername[$i] -Context $context
Where-Object { $_.LastModified.UtcDateTime -lt $CleanupTime -and $_.BlobType -eq "PageBlob" -and $_.Name -like "*.bak"} |
Remove-AzStorageBlob
}
The output lists all the files in all containers but doesn't delete them
Curious, what is your output from the get-azstorageblob | ?{} without the pipe into remove-azstorageblob?
Are you getting a list of items that matches your filters?

Allocated/Unused data disk space VM Wise

(get-azurermvm -ResourceGroupName "<rgname>" -Name "vmname>").StorageProfile.DataDisks
Or
((get-azurermvm -resourcegroupname "<rgname>" -name "vmname>").StorageProfile).DataDisks.DiskSizeGB
But this is showing only the size of data disks. I want to get the Allocated and Unused data disk space. Please someone help.
It seems you want to list all blobs in that storage account, you can use this PowerShell command Get-AzureStorageBlob to list all blobs, like this:
Login-AzureRmAccount
$RGName = "ResourceGroupName"
$SAName = "storageaccountname"
$ConName = "containername"
$TempObj = New-Object -TypeName PSCustomObject
$TempObj |Add-Member -Name BlobName -MemberType NoteProperty -Value $null
$TempObj |Add-Member -Name LeaseState -MemberType NoteProperty -Value $null
$Keylist = Get-AzureRmStorageAccountKey -ResourceGroupName $RGName -StorageAccountName $SAName
$Key = $Keylist[0].Value
$Ctx = New-AzureStorageContext -StorageAccountName $SAName -StorageAccountKey $Key
$List = Get-AzureStorageBlob -Blob *.vhd -Container $ConName -Context $Ctx

How to get size of Azure Container in PowerShell

Similar to this question How to get size of Azure CloudBlobContainer
How can one get the size of the Azure Container in PowerShell. I can see a suggested script at https://gallery.technet.microsoft.com/scriptcenter/Get-Billable-Size-of-32175802 but want to know if there is a simpler way to do in PowerShell
With Azure PowerShell, you can list all blobs in the container with Get-AzureStorageBlob with Container and Context parameter like:
$ctx = New-AzureStorageContext -StorageAccountName youraccountname -storageAccountKey youraccountkey
$blobs = Get-AzureStorageBlob -Container containername -Context $ctx
Output of Get-AzureStorageBlob is an array of AzureStorageBlob, which has a property with name ICloudBlob, you can get blob length in its Properties, then you can sum length of all blobs to get content length of the container.
The following PowerShell script is a simple translation of the c# code in the accepted answer of the question How to get size of Azure CloudBlobContainer. Hope this suit your needs.
Login-AzureRmAccount
$accountName = "<your storage account name>"
$keyValue = "<your storage account key>"
$containerName = "<your container name>"
$storageCred = New-Object Microsoft.WindowsAzure.Storage.Auth.StorageCredentials ($accountName, $keyValue)
$storageAccount = New-Object Microsoft.WindowsAzure.Storage.CloudStorageAccount ($storageCred, $true)
$container = $storageAccount.CreateCloudBlobClient().GetContainerReference($containerName)
$length = 0
$blobs = $container.ListBlobs($null, $true, [Microsoft.WindowsAzure.Storage.Blob.BlobListingDetails]::None, $null, $null)
$blobs | ForEach-Object {$length = $length + $_.Properties.Length}
$length
Note: the leading Login-AzureRmAccount command will load the necessary .dll for you. If you do know the path of "Microsoft.WindowsAzure.Storage.dll", you can replace it by [Reflection.Assembly]::LoadFile("$StorageLibraryPath") | Out-Null. The path is usually like this "C:\Program Files\Microsoft SDKs\Azure.NET SDK\v2.7\ToolsRef\Microsoft.WindowsAzure.Storage.dll"
Here's my solution I just hammered through today. Above examples didn't give me what I wanted which was (1) a byte sum of all blobs in a container and (2) a list of each blob + path + size so that it can be used to compare the results to a du -b on linux (origin).
Login-AzureRmAccount
$ResourceGroupName = ""
$StorageAccountName = ""
$StorageAccountKey = ""
$ContainerName = ""
New-AzureStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey
# Don't NEED the Resource Group but, without it, fills the screen with red as it search each RG...
$size = 0
$blobs = Get-AzureRmStorageAccount -ResourceGroupName $ResourceGroupName -Name $StorageAccountName -ErrorAction Ignore | Get-AzureStorageBlob -Container $ContainerName
foreach ($blob in $blobs) {$size = $size + $blob.length}
write-host "The container is $size bytes."
$properties = #{Expression={$_.Name};Label="Name";width=180}, #{Expression={$_.Length};Label="Bytes";width=80}
$blobs | ft $properties | Out-String -width 800 | Out-File -Encoding ASCII AzureBlob_files.txt
I then moved the file to Linux to do some flip flopping of it and the find output to create a list of files to input into blobxfer. Solution to a different problem, but perhaps a suitable solution for your needs as well.

Resources