I am trying to find any file whose name begins with 'filename', but I cant work it out.
Get-AzureStorageBlob -Context $Context -Container $SourceContainer |
Get-ChildItem Where{$_.Name -like 'filename'}
Where am I going wrong? Thank you.
Try the command below, it works on my side.
Get-AzureStorageBlob -Context $Context -Container $SourceContainer |
Where-Object {$_.Name -like "filename*"}
Test result with no filter and filter:
Related
I am using the Get-AzStorageBlob command to pull out blob files
https://learn.microsoft.com/en-us/powershell/module/az.storage/get-azstorageblob?view=azps-9.2.0
I want to ignore large-size blobs as they take an eternity to process.
Do we have a way where if this command does not yield output in 10 sec should error out?
try{
$blobs = ($container | Get-AzStorageBlob -context $context -ServerTimeoutPerRequest 10 -ClientTimeoutPerRequest 10).LastModified
}
catch {write-output "$($container) timed out"}
I tried adding parameters ServerTimeoutPerRequest & ClientTimeoutPerRequest but they seem to not help at all as the command is not erroring out after 10 sec.
I tried to reproduce the same in my environment to skip the large files in Blob:
This is one approach to skip the large files using PowerShell.
Based on filesize limit condition, we can fetch the files satisfying the size limit, if any file falls under beyond this size limit will be skipped in fetching.
I have uploaded 6 Files into my blob.
Here is the script:
Connect-AzureAD
Install-Module Az.Storage
$MaxReturn = 10000
$ContainerName = "thejademo"
$Token = $Null
$StorageContext = New-AzureStorageContext -StorageAccountName 'venkatsa' -StorageAccountKey 'ky3ewvDbRRMKqVWD75mQXAeojowkYUQFzVSItmgVosrygIG+ITJsrGRgAlVcqo2sY0zlcU2QVNMu+AStJrzWDA=='
$Container = Get-AzureStorageContainer -Name 'thejademo' -Context $StorageContext
$Blobs = Get-AzStorageBlob -Container $ContainerName -MaxCount $MaxReturn -ContinuationToken $Token -Context $StorageContext |where {($_.Length -lt 71458173)}
$Total = $Blobs.count
$Skip= "Skipped Large Blob Files"
Write-Host Total Files $Total in container $ContainerName $skip
Total 6 files in my container but displayed only 4 files (skipped large files).
I am trying to use for loop for selecting all the containers using this script
param($TimerBlob)
$CleanupTime = [DateTime]::UtcNow.AddMinutes(-1)
$context = New-AzStorageContext -StorageAccountName **** -StorageAccountKey ****
$containername = #("mydbstore", "mydbstore2","mydbstore3") # List of containers -
For ($i=0; $i -lt $containername.Length; $i++) {
Get-AzStorageBlob -Container $containername[$i] -Context $context
Where-Object { $_.LastModified.UtcDateTime -lt $CleanupTime -and $_.BlobType -eq "PageBlob" -and $_.Name -like "*.bak"} |
Remove-AzStorageBlob
}
The output lists all the files in all containers but doesn't delete them
Curious, what is your output from the get-azstorageblob | ?{} without the pipe into remove-azstorageblob?
Are you getting a list of items that matches your filters?
I am trying to list all blobs that are not PDF files. How would I do that?
I know that I can list all files like this:
$StorageContext = New-AzStorageContext -StorageAccountKey XXX -StorageAccountName YYY
Get-AzStorageBlob -Container 'containerName' -Context $StorageContext
And if I want to fitler for a specific content type, png in the case below, I would change the second line to this:
Get-AzStorageBlob -Container 'containerName' -Context $StorageContext -Blob *.png
But how can I display anything besides a specific content type. In my case I would want to list everything besides PDFs?
You could use the command below.
Get-AzStorageBlob -Container 'containerName' -Context $StorageContext | Where-Object {!($_.Name -like '*.pdf')}
I extended the version from #JoyWan to apply another filter. I want to show all files less than some date:
$filterDate = (Get-Date).AddMonths(-3) # get date 3 moths before
$StorageContext = New-AzStorageContext -StorageAccountKey XXX -StorageAccountName YYY
$blobs = Get-AzStorageBlob -Container 'containerName' -Context $StorageContext | where {($_.Name -notlike '*.pdf' -and $_.LastModified -le $filterDate)}
This example would show all files older than 3 months.
I am looking to delete all files from azure storage blob which are older than 'x' days. I am trying the below code but is not working:
$StorageAccountName = '<name>'
$StorageAccountKey = '<key>'
$Ctx = New-AzureStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey
Get-AzureStorageBlob -Container "reports" -Context $Ctx -Blob *.csv
where {$_.LastModified -le (get-date).AddDays(-30) } | Remove-AzureStorageBlob
I referred the following doc but the query is not working for conditional deletion. link
I suggest you use the new azure powershell module AZ.
After install the new AZ module, try the code below:
$accountname="xx"
$accountkey="xxx"
$ctx = New-AzStorageContext -StorageAccountName $accountname -StorageAccountKey $accountkey
Get-AzStorageBlob -Container "aa1" -Blob *.jpg -Context $ctx | where {$_.LastModified -le (Get-Date).AddDays(-1)} | Remove-AzStorageBlob
After the code running, you can check on azure portal or use Get-AzStorageBlob cmdlet to see if all the specified files are deleted. In my case, all the files' date < "1 day ago" are deleted.
Azure storage have feature "Manage the Azure Blob storage lifecycle".
https://learn.microsoft.com/en-us/azure/storage/blobs/storage-lifecycle-management-concepts
For your test case you can directly refer
https://learn.microsoft.com/en-us/azure/storage/blobs/storage-lifecycle-management-concepts#powershell
$action = Add-AzStorageAccountManagementPolicyAction -BaseBlobAction Delete -daysAfterModificationGreaterThan 2555
Thank you Ivan. I compared my script with yours and found that I was missing a pipe before the where condition which was issue. After putting the pipe, I am able to delete the files based on condition. Didn't needed to go to AzureAz.
The script which is working now is :
$StorageAccountName = 'xx'
$StorageAccountKey = 'yyy'
$ctx = New-AzureStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey
Get-AzureStorageBlob -Container "abc" -Blob *.pdf -Context $ctx | where {$_.LastModified -le (Get-Date).AddDays(-4)} | Remove-AzureStorageBlob
I've got a list of virtua machines in Azure which I'm trying to convert to managed disks.
I have a list of vm's, I read from the list and export to csv capturing resourcegroupname and vm name, however I seem to get vms from the whole subscription.
Also when I attempt to import the csv, when I run $comps it returns the correct information in the csv, however I can't seem to pass them through to the next lines.
CSV format is
ResouceGroupName Name
RG-01 vm-01
RG-01 vm-02
RG-01 vm-03
RG-01 vm-04
The code I'm trying is
Login-AzureRmAccount
$sub = Get-AzureRmSubscription | ogv -PassThru
Select-AzureSubscription -SubscriptionId $sub
$virtualmachines = Get-Content C:\temp\vm.txt | % {
Get-Azurermvm | select ResourceGroupName,Name | export-csv c:\temp\vm.csv -NoClobber -NoTypeInformation -Append
}
$comps = Import-Csv c:\temp\Vm.csv |
foreach ($Comp in $comps)
{
Stop-AzureRmVM -ResourceGroupName $_.ResourceGroupName -Name $_.Name -Force
ConvertTo-AzureRmVMManagedDisk -ResourceGroupName $_.ResourceGroupName -VMName $_.Name
}
Thanks in advance..
For your issue, you export the virtual machines in a csv file and use it in the foreach code. So, it's unneccesary to use command:
$virtualmachines = Get-Content C:\temp\vm.txt | % {
Get-Azurermvm | select ResourceGroupName,Name | export-csv c:\temp\vm.csv -NoClobber -NoTypeInformation -Append
}
And your VMs all in a resourcegroup, you can get them with ResourceGroupName directly.
For the pipeline in foreach, it's unneccesary. You can use the following code that I make a little change with your code and it works well.
Login-AzureRmAccount
$sub = Get-AzureRmSubscription | ogv -PassThru
Select-AzureRmSubscription -Subscription $sub
Get-Azurermvm –ResourceGroupName RG-01 | select ResourceGroupName,Name | export-csv c:\temp\vm.csv -NoClobber -NoTypeInformation -Append
$comps = Import-Csv c:\temp\Vm.csv
foreach ($Comp in $comps)
{
Stop-AzureRmVM -ResourceGroupName $Comp.ResourceGroupName -Name $Comp.Name -Force
ConvertTo-AzureRmVMManagedDisk -ResourceGroupName $Comp.ResourceGroupName -VMName $Comp.Name
}
This is the screenshot of my result.