While deleting a resource group am getting the below mentioned error.
Failed to delete resource group backupone: Deletion of resource group
'backupone' failed as resources with identifiers
'Microsoft.RecoveryServices/vaults/backupone' could not be deleted.
The provisioning state of the resource group will be rolled back. The
tracking Id is 'ee14d432-edf4-48e3-ba7f-016757125909'. Please check
audit logs for more details. (Code: ResourceGroupDeletionBlocked)
Vault cannot be deleted as there are existing resources within the
vault. Please ensure there are no backup items, protected servers or
backup management servers associated with this vault. Unregister the
following containers associated with this vault before proceeding for
deletion : backuponestg Unregister all containers from the vault and
then retry to delete vault (Code:
ServiceResourceNotEmptyWithContainerDetails)
Can someone please help me understand what is wrong?
As I commented above, the error message
Vault cannot be deleted as there are existing resources within the vault
hints to me there might be a Azure Site Recovery Vault in the resource group.
Here's some more information.
If you got a Azure Site Recovery Vault inside of a resource group, and it contains some data from backed up virtual machines, then this can block the deletion of the vault.
You need to, manually, first to clean up the vault before you can delete the ASR vault. This is a mechanism set in place to prevent accidental deletion of backed up data.
Check out this link for a more detailed guide on what steps you need to take to unassociate workloads with your vault.
Related
We are using a storageclass in Kubernetes to dynamically provision azure file shares with pvc claims (https://learn.microsoft.com/en-us/azure/aks/azure-files-dynamic-pv?source=recommendations)
We use recovery service vault to backup these file shares (https://learn.microsoft.com/en-us/azure/backup/azure-file-share-backup-overview?WT.mc_id=Portal-Microsoft_Azure_Support#key-benefits-of-azure-file-share-backup).
Azure Backup locks the storage account associated with the file shares against deletion in Resource Group scope. The lock is called "AzureBackupProtectionLock" and it protects storage accouunts registered with a Recovery Services vault to guard against deletion of backups.
That seems fair, however this lock on resource group scope is blocking us from upgrading the AKS version. The screenshot with error is attached
Any idea how to prevent this behaviour ? Thanks :)
This error may occur if your resource group lock as applied as Read only access When a resource is marked as read-only, it can only be accessed; it cannot be update or deleted. By implementing this lock, all authorized users will only be granted the privileges offered by the Reader role.
Please check this Considerations before applying your locks To resolve this issue make sure to delete our locks in your resource group as below:
Go to azure portal -> your resource group ->under setting locks -> delete
You need access to Microsoft.Authorization/* or Microsoft.Authorization/locks/* operations in order to add or remove management locks. Only Owner and User Access Administrator have access to those activities among the built-in roles.
Reference: Lock your resources to protect your infrastructure
Scenario:
I had the need to move resources from one vault to another, without deleting the backups. This was due to cost, as the vault was created and everything protected in GRS, instead of LRS. I stopped all the backups in the GRS vault, moved everything from resource group (workaround that I found) and then I was able to protect everything in the new Vault. As I moved from resource group, the VM's changed their ID, and now I need to restore a backup that was in the old vault, but the restore fails because it's not able to find the VM.
Is there any way to do it?
I have an old Azure Recovery Services vault for an on-premise Windows Desktop that I am trying to remove after decommissioning said desktop. Azure is responding with this error message:
Vault cannot be deleted as there are existing resources within the vault. Please ensure there are no backup items, protected servers or backup management servers associated with this vault. Unregister the following containers associated with this vault before proceeding for deletion : VAULT-NAME. Unregister all containers from the vault and then retry to delete vault
Where VAULT-NAME is the name of my vault.
I followed the steps referenced in this answer but could not get past this step due to the fact that there is not a corresponding "Windows" or "WindowsServer" option for the WorkloadType parameter.
$item = Get-AzureRmRecoveryServicesBackupItem -Container $container -WorkloadType AzureSQLDatabase
Skipping the item retrieval and disable steps and instead trying to unregister the container and remove the vault with the following commands was of no use on account of still having backups associated with the container.
Unregister-AzureRmRecoveryServicesBackupContainer -Container $container
Remove-AzureRmRecoveryServicesVault -Vault $vault
I have not mapped this answer to the corresponding Azure commands, but I was able to find my way to a solution via the Azure Portal. The steps were as follows:
Selected my Recovery Service resource
Under the Manage section, clicked Backup Infrastructure
Under Management Servers, clicked Protected Servers
In the list that followed, clicked on the row where my Protected Server Count was greater than 0, in my case, Azure Backup Agent (because the backup agent was installed on my Windows Desktop)
Clicked on my server name in the Protected Server list
Clicked Delete in the card for my protected server
After that completed, I was able to delete the entire vault. These steps may be helpful if you have other Backup Infrastructure resources and possibly even Site Recovery Infrastructure resources associated with a vault.
Update: It seems like there's an open issue for Get-AzureRmRecoveryServicesBackupItem not having any capacity to return MARS backup items which is ultimately what the issue here was.
We are building a solution in Azure Government and we will be using Terraform to deploy the solution. Seems the preferred method is to create a Service Principal for Terraform with the Service Principal having the Contributor role scoped to the subscription.
The one issue with this we are looking to solve is that this gives the Service Principal management plane access to the Key Vault...since it is in the subscription. With Contributor role, the service principal is not able to create new access policies (assign itself or others permissions to the data plane) but we are looking for a way that can remove the service principal from having any management plane permissions.
We have tried putting a ReadOnly lock on the Key Vault before creating the Service Principal but the lock does not stop the service principal from getting the Contributor permissions on the Key Vault.
Outside of creating a new role that has Contributor for everything EXCEPT for Key Vault, does anyone have any creative ideas that might help achieve this?
Yes, the root cause of all security issues is that the Service Principal's contributor role assignment is at the subscription level/scope, which enables it to do quite damage specially if multiple applications are deployed to the same subscription (e.g. delete any resource group).
One approach would be:
Provision one resource group for the Azure Key Vault specific to the application and region (the latter in case of geo-distributed applications).
Provision the Azure Key Vault on the resource group created on the previous step.
In our case, the Security Office was responsible for the first 2 steps, where they had monitoring (e.g. email, text-messages, etc.) for any change in the Azure Key Vault (e.g. new keys/secrets/certificates added/deleted/changed, permission changes, etc.).
Provision a second resource group, which will serve as a container for the application components (e.g. Azure Function, Azure SQL Server/Database, Azure Service Bus Namespace/Queue, etc.).
Create the Service Principal and assign the Contributor role to the
application resource group only, for example:
scope =
/subscriptions/{Subscription Id}/resourceGroups/{Resource Group
Name}
Find a sample PS script to provision a Service Principal with custom scope at https://github.com/evandropaula/Azure/blob/master/ServicePrincipal/PS/Create-ServicePrincipal.ps1.
Give appropriate permissions for the Service Principal in the Azure
Key Vault. In our case, we decided to have separate Service
Principal accounts for deployment (Read-Write permissions on keys/secrets/certificates) and runtime (Read-Only permissions on keys/secrets/certificates);
Find a sample PS script to set Service Principal permission on an Azure Key Vault at https://github.com/evandropaula/Azure/blob/master/KeyVault/PS/Set-ServicePrincipalPermissions.ps1.
Having that said, there are lots of inconveniences with this approach, such as:
The process (e.g. via runbook) to provision the Azure Key Vault (including its resource group) and the application resource group will be outside of the main Terraform template responsible for the application components, which requires coordination with different teams/processes/tools/etc.
Live site involving connectivity often involves coordination among multiple teams to ensure RTO and MTTM (Mean Time To Mitigate) goals are achieved.
The Service Principal will be able to delete the application specific resource group when terraform destroy is executed, but it will fail to recreate it when running terraform apply after that due to lack of permission at the subscription level/scope. Here is the error:
provider.azurerm: Unable to list provider registration status, it is possible that this is due to invalid credentials or the service principal does not have permission to use the Resource Manager API, Azure error: resources.ProvidersClient#List: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client '' with object id '' does not have authorization to perform action 'Microsoft.Resources/subscriptions/providers/read' over scope '/subscriptions/{Subscription Id}'.".
Yeah, I know, this is a long answer, but the topic usually requires lots of cross-team discussions/brainstorming to make sure the security controls established by the Security Office are met, Developer productivity is not affected to the point that it will impact release schedules and RTO/MTTM goals are met. I hope this helps a bit!
Some times we had deleted the 'resource group' accidentally on azure cloud, its fine that if our/your resource group does not have any deployments on it and we can create immediately newer one and will proceed our stuff, but if we had deployments on it, then its becomes major problem/task.
So can we recover Resource Group? like a recovery of Virtual machine in Azure.
You can export service configurations that belong to a resource group as JSON. Within the portal, there is an option "Export Template" under the Resource Management Navigation Group.
So can we recover Resource Group? like a recovery of Virtual machine in Azure.
Unfortunately, we have no way to recovery it, if the resource group is deleted . We could get the warning if we try to delete resource group from the azure portal. Connect to Azure support team for help maybe a way, but I don't think it could be easy for restoring.
Note : based on my understanding even if we could re-create the resource with ARM template(If we have exported the ARM template before delete), it seems that just with the same resource names with deleted resource that is not equal recovery action.