how to delete old files in azure container - azure

I plan to backup my azure vhd files by shutting down my vm and then copying the vhd files from the production container to a backup container. How can I automate deleting vhd files that are a week old in the backup container?

If you can accept using PowerShell, then this would do it for you. It will register a scheduled job to run daily and remove PageBlob's in the container specified.
$taskTrigger = New-ScheduledTaskTrigger -Daily -At 12:01AM
Register-ScheduledJob -Name DeleteMyOldFiles -Trigger $taskTrigger -ScriptBlock {
$isOldDate = [DateTime]::UtcNow.AddDays(-7)
Get-AzureStorageBlob -Container "[YOUR CONTAINER NAME]" |
Where-Object { $_.LastModified.UtcDateTime -lt $isOldDate -and $_.BlobType -eq "PageBlob" } |
Remove-AzureStorageBlob
}

This is something not available right out of the box. You would have to write some code yourself. Essentially the steps would be:
List all blobs in your backup container. Blob list would return blobs along with its properties. One of the properties would be LastModifiedDate (this would be in UTC).
You could then put your logic to find blobs which have been modified "x" days ago. You would then go ahead and delete those blobs.
A few other things:
You mentioned that your backup container contains some VHDs which are essentially page blobs. When you list blobs, you would also get blob type so you could further filter the list by blob type (= PageBlob)
As far as automating the process go, you could either write this in a PowerShell script and then schedule it using Windows Scheduler. If you're comfortable writing node.js, you could write the same logic using node.js and make use of Windows Azure Mobile Service Scheduler.

I found this answer when trying to delete the whole container. Using Rick's answer as a template I came up with this trial what-if to determine which containers would be deleted:
$ctx = New-AzureStorageContext -StorageAccountName $AzureAccount `
-StorageAccountKey $AzureAccountKey
$isOldDate = [DateTime]::UtcNow.AddDays(-7)
Get-AzureStorageContainer -Context $ctx `
| Where-Object { $_.LastModified.UtcDateTime -lt $isOldDate } `
| Remove-AzureStorageContainer -WhatIf
Then when I was satisfied that the list should be deleted, I used -Force so as to not have to confirm every delete:
Get-AzureStorageContainer -Context $ctx `
| Where-Object { $_.LastModified.UtcDateTime -lt $isOldDate } `
| Remove-AzureStorageContainer -Force

Related

Azure Blob Storage - Define Automation Task in Powershell

I have defined within the Azure Portal a scheduled task to delete blobs older than x days. Now I need to script this to Powershell, however I couldn't find any information about this.
At this moment, feb 2023, Azure Blob Storage Automation Tasks are in Preview mode as it can be seen in this picture.
Does anybody know if defining Automation Tasks in Powershell is possible at this moment? Thanks!
Azure Blob Storage - Define Automation Task in Powershell
You can use Azure Automation Runbook for this action:
You can use below PowerShell code to delete the blobs 10 days older:
$a="Storage account name "
$ak="ETxeSAVNp7NkvWAYwri6M7+AStOQtSZQ=="
$context = New-AzStorageContext -StorageAccountName $a -StorageAccountKey $ak
Get-AzStorageBlob -Container "name of container" -Context $context | where {$_.LastModified -le (Get-Date).AddDays(-10)}
$ak is the storage account key
If you dont want to give container name you can use below code:
$a="Storage account name "
$ak="ETxeSAVNp7NkvWAYwri6M7+AStOQtSZQ=="
$context = New-AzStorageContext -StorageAccountName $a -StorageAccountKey $ak
$c=Get-AzStorageContainer -Context $context
foreach($con in $c)
{
Get-AzStorageBlob -Container $con -Context $context | where {$_.LastModified -le (Get-Date).AddDays(-10)
}
You can also use the below command to only delete specific type of blob:
Get-AzStorageBlob -Container "name of conatiner" -Blob *.txt -Context $ctx | where {$_.LastModified -le (Get-Date).AddDays(-10)}
In place of txt you can use jpg, etc.
Here -10 means 10 days older (you can give your required number of days).
Now create a automation account and then create a powershell run book in it.
You can paste the above code in powershell run book and then you can schedule the runbook.
By this way you can create an automation task of PowerShell.

Attach and or mount unmanaged page blob snapshot to another Azure VM using powershell

We are using unmanaged disks and want to be able to locate a blob given a Live VM name and a disk name, create a snapshot of the disk and mount that disk in another (backup) VM and gain access to its file system.
Live VM may be Linux or Windows
Backup VM will typically be Linux but possibly Windows
So far, I am able to locate the live VM, and the page blob I want to snapshot and create the snapshot.
$key = (Get-AzStorageAccountKey -Name $storageAccount -ResourceGroupName $vm.ResourceGroupName | select -first 1).Value
$context = New-AzStorageContext -StorageAccountName $storageAccount -StorageAccountKey $key
$blob = Get-AzStorageBlob -Context $context -Container $containerName -Blob $blobName
$snapshot = $blob.ICloudBlob.CreateSnapshot()
I now want to mount the snapshot identified by $snapshot on the backup VM. Is this possible?
I don't mind if it requires additional steps (such as copying the snapshot to another blob) but would rather keep the steps to a minimum.
It isn't possible. Copying the snapshot to a blob, and then attaching the blob is the way to do it.

Storage account can not be deleted with resource group

We wanted to delete a resource group that contained a VM with IP and storage account etc.
Everything got deleted except the storage account because of a vhd which says it still has a lease. I can't break the lease because of the following error message:
Failed to break lease on 1 out of 1 blob(s):
VM2X-20170518-074152.vhd: This blob is being used by the system.
Is there a way to break the lease, delete the blob with the lease active, or find out where it is leased to?
Additional Info:
On the vhd on the "Edit blob" tab, I get the following message:
File size of '137.44GB' exceeds max supported file size of '2.1MB.'
This sounds like a familiar problem with classic storage accounts, if it's the problem I think it is you will need to delete image using Powershell.
Set Storage account
$storageAccountName = "your storage account"
Check OS Disk image
Get-AzureVmImage | Where-Object { $_.OSDiskConfiguration.MediaLink -ne $null -and $_.OSDiskConfiguration.MediaLink.Host.Contains($storageAccountName)`
} | Select-Object -Property ImageName, ImageLabel
Check Data Disk image
Get-AzureVmImage | Where-Object {$_.DataDiskConfigurations -ne $null `
-and ($_.DataDiskConfigurations | Where-Object {$_.MediaLink -ne $null -and $_.MediaLink.Host.Contains($storageAccountName)}).Count -gt 0 `
} | Select-Object -Property ImageName, ImageLabel
Remove any image
Remove-AzureVMImage -ImageName 'yourImageName'
Note: commands are classic/ASM, make sure you have module installed.
yes, you can use portal UI or powershell to break the lease (or SDK's). for the portal just click on the blob and there would be a button to break the lease. for powershell something like this:
$blob = Get-AzureStorageBlob -Context $ctx -Container %container% -Blob %blob%
$blob.ICloudBlob.BreakLease()

Get Container list in one azure storage account and create the same in another storage account

Team,
Thanks in Advance, I'm an IT pro and also learning PowerShell. I have 2 storage account Storage A and Storage B in different diff region and I want to create identical containers in the secondary storage account.
I've found below command which can get the list of all the containers in the primary storage account and then I can use the foreach loop to create the container on secondary storage. But I want to make sure that in the secondary storage account if it has the container name already then my command should skip that container and move on to create the next storage container.
Get-AzureStorageContainer -Context $storageAccountContext | ForEach-object { New-AzureStorageContainer -Name $_.Name -Context $Destinationcontext }
"VM" is incorrect argument against the "-Property" switch for the values we are getting from pipe(|).
Please use the correct name of the property. I have used "VM" above only as an example. VM is property in Esxi(VC) environment.
Please run the below command & check the output.
Get-AzureStorageContainer -Context $Destinationcontext
It should give you output something like below:
Property_ID Property_NAME Property_3
Value 1 Value_A Value_1A
Value 2 Value_B Value_2A
So, now the correct syntax should be:
$Vm_to_be_created|Select-object -Property "Property_ID"| ForEach-object { New-AzureStorageContainer -Name $_.Name -Context $Destinationcontext }
:)
Hope this helps!
$storageAccountContext ="Storage_Account_A"
$Destinationcontext ="Storage_Account_B"
$Vm_in_A =#(Get-AzureStorageContainer -Context $storageAccountContext )
$Vm_in_b =#(Get-AzureStorageContainer -Context $Destinationcontext)
$Vm_to_be_created =#(Compare-Object -ReferenceObject $Vm_in_A -DifferenceObject $Vm_in_b )
$Vm_to_be_created|Select-object -Property VM| ForEach-object { New-AzureStorageContainer -Name $_.Name -Context $Destinationcontext }

How do I delete an Azure storage account containing a leased blob?

I was playing with Windows Azure durable virtual machines. In the end, I deleted the virtual machine (successfully) and tried to delete the associated storage account.
The request to delete the storage account fails.
On the Preview Portal (manage.windowsazure.com) when I delete the storage account I get this error:
Failed to delete Storage account 'portalvhdscwtwycpsxxxxx'
Details:
Storage account portalvhdscwtwycpsxxxxx has 1 container(s) which have an active image and/or disk artifacts. Ensure those artifacts are removed from the image repository before deleting this storage account.
On the previous portal (windows.azure.com) I get this error:
Submit Failed
Storage account portalvhdscwtwycpsxxxxx has 1 container(s) which have an active image and/or disk artifacts. Ensure those artifacts are removed from the image repository before deleting this storage account.
Trying to delete the blob itself (a 30GB VHD) on Azure Storage Explorer I get this error:
There is currently a lease on the blob and no lease ID was specified in the request.
So my assessment is that this blob is leased (by the previous, now deleted virtual machine) and I can't delete it unless I can get this lease ID.
The question is: how can I delete this blob and, consequently, the storage account?
The key to the solution is the message that the container has an active disk artifact and the advice to remove it from the repository.
The procedure to remove the disk image from the blob repository is:
Go to the Windows Azure Management Portal.
Click on Virtual Machines.
Click on Disks.
Click on the disk.
Click on Delete Disk.
After that, the storage account can be deleted.
Notes:
This applies even if you've already deleted all of your Virtual Machines and it shows 0; there still will be artifacts under the disks tab.
Disks are detached from a deleted VM asynchronously, it may take a few minutes after the VM is deleted for this field to clear up.
See also: Unable to delete VHD, “There is currently a lease on the blob…”
Unfortunately, Fernando's answer didn't work for me, since the storage was "orphan", as I deleted its VM before deleting the storage. I couldn't find a way to do it from the portal so I've installed azure-cli, and after authentication ran the following commands:
azure storage account delete <my-account>
This fails, and the error message contains the name of culprit, e.g.:
error: Storage account <my-account> has some active image(s) and/or disk(s), e.g. <my-image>. Ensure these image(s) and/or disk(s) are removed before deleting this storage
Then I deleted the offending image
azure vm disk delete <my-image>
And tried again to delete the storage, this time successfully.
azure storage account delete <my-account>
Unfortunately there is the case where the VM was deleted but Disks shows the VM attached to the blob (a 30GB VHD) precluding the deletion. Also there is the case of using the Azure Storage Explorer you find an orfan but leased VHD blob that can't be deleted and there is no reference on the Preview Portal.
Go to virtual machines, then click on discs. Mark the disc and choose delete disc at the bottom. You can now choose if you want to keep or delete the corresponding vhd.
It is important first to delete the disc via virtual machines not to delete via storage.
You can use Iaas Management Studio : break the lease, delete the blob, then remove the orphaned image.
In my case, storage could not be deleted because of vmimages.
Use power shell command
get-azurevmimage | Where-Object -Property Category -in -Value "user"
to list all images
To delete ALL YOU IMAGES use the following script:
get-azurevmimage | Where-Object -Property Category -in -Value "user" |
foreach {
echo "remove $($_.ImageName)"
Remove-AzureVMImage –ImageName $($_.ImageName)
}
As F.M. has already stated; there is a scenario where when deleting a VM the disk still shows as attached to the VM even though the VM has been deleted.
For me this happened because I had a set a spending limit. When the spending limit is hit, your services are disabled. Any VPN gateways you have created and VMs will be deleted. Then to top it off the disks attached to the deleted VMs still think they are attached :(
I have found this blog that explains the issue and shows how to use powershell to resolve.
Hope this helps other users.
Sometimes we via the new portal to delete azure storage account, but we can’t delete it and get this error:” Failed to delete storage account 'jason1disks796'. Error: The storage account cannot be deleted due to its artifacts being in use. “
We can use PowerShell to list all the VHD blobs of the storage account(ARM module):
PS > Login-AzureRmAccount
PS > $RGName = "jason1"
PS > $SAName = "jason1disks796"
PS > $ConName = "vhds"
PS > $TempObj = New-Object -TypeName PSCustomObject
PS > $TempObj |Add-Member -Name BlobName -MemberType NoteProperty -Value $null
PS > $TempObj |Add-Member -Name LeaseState -MemberType NoteProperty -Value $null
PS > $Keylist = Get-AzureRmStorageAccountKey -ResourceGroupName $RGName -StorageAccountName $SAName
PS > $Key = $Keylist[0].Value
PS > $Ctx = New-AzureStorageContext -StorageAccountName $SAName -StorageAccountKey $Key
PS > Get-AzureStorageContainer -Context $ctx
CloudBlobContainer : Microsoft.WindowsAzure.Storage.Blob.CloudBlobContainer
Permission : Microsoft.WindowsAzure.Storage.Blob.BlobContainerPermissions
PublicAccess : Off
LastModified : 1/19/2017 1:27:21 AM +00:00
ContinuationToken :
Context : Microsoft.WindowsAzure.Commands.Common.Storage.AzureStorageContext
Name : vhds
PS > $List = Get-AzureStorageBlob -Blob *.vhd -Container $ConName -Context $Ctx
PS > $List | ForEach-Object { $TempObj.BlobName = $_.Name; $TempObj.LeaseState = $_.ICloudBlob.Properties.LeaseState; $TempObj }
BlobName LeaseState
-------- ----------
SQL20170119092405.vhd Leased
PS > Get-AzureStorageBlob -Blob * -Container $con -Context $ctx | Remove-AzureStorageBlob
PS > Remove-AzureRmStorageAccount -ResourceGroupName $RGname -Name $SAName
If your storage account is in the ASM module, you can use this script to remove storage account:
Add-AzureAccount
$SAName = "jason1161"
$ConName = "vhds"
$TempObj = New-Object -TypeName PSCustomObject
$TempObj |Add-Member -Name BlobName -MemberType NoteProperty -Value $null
$TempObj |Add-Member -Name LeaseState -MemberType NoteProperty -Value $null
$Keylist = Get-AzureStorageKey -StorageAccountName $SAName
$Key = $Keylist.primary
$Ctx = New-AzureStorageContext -StorageAccountName $SAName -StorageAccountKey $Key
$List = Get-AzureStorageBlob -Blob *.vhd -Container $ConName -Context $Ctx
$List | ForEach-Object { $TempObj.BlobName = $_.Name; $TempObj.LeaseState = $_.ICloudBlob.Properties.LeaseState; $TempObj }
PS > Get-AzureStorageBlob -Blob * -Container $con -Context $ctx | Remove-AzureStorageBlob
PS > Remove-AzureStorageAccount -Name $SAName
Besides, there is another scenario, there is no container or blob in this storage account (an empty storage account, we can’t find blob or container in this storage account via PowerShell or portal), when we use portal to delete the storage account, and the error message” Failed to delete storage account 'jason1disks796'. Error: The storage account cannot be deleted due to its artifacts being in use”. In this scenario we can create a new VM and specify the storage account to the problematic Storage Account, then delete it again.
Do check before deleting your storage account; there must be the associated virtual machine(s), Disks and Images for each storage account you created. Go to Azure portal
Select Virtual Machines tab on left pane
Click on Instances Images and Disks
Note that, Individual Virtual machines has its attached disks which show on Disks area.Before deleting a Virtual machine, delete the associated disks first and delete the virtual machine has the disk second.Then delete the storage account last. Also look out for Network in the same left-hand side pane if any associated with the account you want to delete.
In the new updated Azure portal, many of the above-stated config pages are changed. You can see Images and Disks options in "All resources" pane. In the newer version of an Azure portal, you can easily identify VMs its associated Disks and its Storage account clearly on its adjacent vertical panes all in one page with different icon images.
For those who rely on GUI to manage Azure and have no idea to use PowerShell or do the other answers, you can now delete the stuck storage account by checking "Delete unattached images" when trying to delete the storage.
It will automatically delete the storage without much hassle.

Resources