I tried to deploy ARM template for azure Data Factory as part of DevOps implementation.
Mode of deployment was selected as complete in oreder to cleanup the existing adf instance and deploy only the pipelines that are available in ARM template. as shown below.
When I tried to run the deployment, It failed with an error as
##[error]The deployment failed because some resources could not be deleted. This might be due to not having permissions to delete resources in the targeted scope. Please see https://aka.ms/arm-debug for usage details.
2020-11-02T05:33:34.5795133Z ##[error]Check out the troubleshooting guide to see if your issue is addressed: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment?view=azure-devops#troubleshooting
When I did a debug on this issue, i could understand that the deployment scope is selected as Resource Group and deployment task tried to delete all the resources under this resource group and failed because it couldnt delete resources other than ADF instance because of access issues.
Since I do not have access to other resources, they were not deleted, otherwise i could have messed up everything by deleting all other resources like ADLS, databricks, sql....
Since I am deploying ADF ARM template is there any way to restrict the deployment scope only for ADF instance, which will not affect other resources.
Any leads appreciated!
I have a query about the What-If feature provided by azure for ARM template deployment. Can we use this in our Release pipeline as a powershell task?
You cannot restrict the deployment scope lower than the ResourceGroup as you noticed.
The only way for you to do this would be either putting the ADF in a seperate resource group but i asume that is not possible.
A second way of doing this would be deleting the ADF through the portal or powershell and then do an incremental deployment of you ARM template that has only the ADF definition in it.
Microsoft rolled out a new feature for ARM deployments called What-If. This is a super nice feature to check what changes to what resources will happen when deploying your template. Note, it only works with powershell core at the moment. If you work with ARM templates this could help you and catch resource deletion before you deploy anything.
When deploying a data factory, do not select Complete as your deployment mode. This will overwrite all resources in the resource group including non-ADF entities.
Selecting Incremental will deploy only the resources located in the ARM template. If generated from the adf_publish branch, then this will contain all of the resources in your factory.
Related
I've got a bicep file that has made the following resources: Function app, storage account, app insights, key vault and app service plan, via a build pipeline in Azure Dev Ops Pipelines. I need to populate these resources with artifacts built from source code e.g. adding functions to the function app. I'm not sure how to do this though. Does anyone have any advice or links to tutorials? I'm new to Azure and would appreciate any help. Thanks.
Basically, Azure pipelines working with tasks to deploy artifacts to the resources. These resources can be AKS, IIS or Azure functions as you mentioned. If you particularly looking for Azure function deployment task you can use the AzureFunction#1 task for that.
Here is some additional useful links;
https://www.azuredevopslabs.com/labs/vstsextend/azurefunctions/#:~:text=Azure%20Functions%20is%20an%20event,well%20as%20on%2Dpremises%20systems.
https://learn.microsoft.com/en-us/azure/azure-functions/functions-how-to-azure-devops?tabs=dotnet-core%2Cyaml%2Ccsharp
In my ADF CI/CD setup mentioned here, when I deploy to ACC or PRD after I have deleted some resources (linked service or pipeline), it is not getting deleted in ACC or PRD. But those things I add or edit are updated correctly. How can I fix it ?
I'm assuming the deployment task is set to Incremental as in your documentation. If this is the case then the deployment is doing a delta on the existing defined resources. Since the ones deleted out of your ARM template the Deployment task does not know to evaluate them. You can set it to "deployment" mode as stated in the link you provided:
Keep in mind this will delete and recreate and potentially involve some downtime while recreation occurs.
For more information check out the different Azure Resource Manager deployment modes.
I am new to Azure as well as Devops, ARM template etc..I have deployed an ARM template for key valut with a name "ABC" via DevOPs. Now I would like to change some parameters. I don't have access to do it via portal and I am forced to use only ARM template. How can this be achieved via devops? Should I delete that resource and start from the scratch or any SOP to update resource parameters via ARM templates. For example I want to enable "“enabledForDeployment” for already deployed kay-vault. Thanks.
generally speaking you just re-run the same template with the changes you've introduced to the template. there are many ways of deploying the template (az cli, powershell, built-in steps, various sdk's).
just keep in mind, some properties are settable only at creation time, so effectively read only, you wouldn't be able to update those after the resource is created, some resource providers do not play well with idempotent behavior and this will also lead to issues when you try to redeploy the same template, some providers offer dodgy behavior when you need to use a completely different api call to update something compared to creating it.
So there's no single answer, but for what you are doing, just updating the template and running it again should work just fine.
You can do it through PowerShell,
Update a resource in an Azure Resource Manager template
I was doing an arm template deployment of Azure data factory from dev-ops pipelines. Mistakenly I have chosen deployment mode as complete because of that all the resources inside my target resource group are deleted(DW, ADF, Databricks and all other resources).
Is there any way to rollback?
unless everything was deployed by a single template - no. your only option is to redeploy individual templates. but that wont bring back any data you had.
if everything was deployed with a single template there is a -previous switch in powershell, but assuming all the extra things got deleted, this doesnt apply to your case.
We are trying to establish continuous deployment. We want to do network swaps when deploying new resources.
Sometimes the Azure infrastructure is not responsive provisioning the resources. As we can have a left over conflicting resources when a deployment fails due to unsuccessful deployments.
Question : Should we name our resources uniquely every time we deploy ? Such as giving our build number naming to our infrastructure.
PS:I have noticed that azure team already does this. As they have to host a multi tenant architecture.
Arm is designed to be idempotent. That means you should be able to redeploy any template any time. When doing infrastructure as code you probably provision all resources in a resource group at the same time. Do the deployments in complete mode, then resources that no longer are linked in the templates are removed. Here is are article on versioning ARM templates that can help you:
versioning-arm-template-deployments