CI/CD Pipeline is failing for Synapse - azure

I am getting below error when deploying Synapse CI/CD pipeline using Azure DevOps. The deployment was working fine until I added a few more notebooks and it started failing. Notebooks run fine on lower environment. All paths in the notebooks exist in higher environment. There is no concept of linkedTemplates in Synapse. I thought it was failing due to the length of ARMtemplate has exceeded.
Below is the error log, I have removed subscription name and resource group name and pipeline names.
2022-07-06T01:53:27.9739484Z Error during execution: Error: Failed to fetch the deployment status {"code":"400","message":"CreateOrUpdateNotebook failed: [statusCode from ADF:BadRequest, ErrorMessage: {\"code\":\"BadRequest\",\"message\":\"The document creation or update failed because of invalid reference ''.\",\"target\":\"/subscriptions//resourceGroups/rg-name/providers/Microsoft.Synapse/workspaces/workspace-02/notebooks/SilverToGoldPipeline\",\"details\":null,\"error\":null}, workspace: workspace-02, notebook: SilverToGoldPipeline, ArtifactId: ]"}
2022-07-06T01:53:27.9811549Z ##[error]Encountered with exception:Error: Failed to fetch the deployment status {"code":"400","message":"CreateOrUpdateNotebook failed: [statusCode from ADF:BadRequest, ErrorMessage: {\"code\":\"BadRequest\",\"message\":\"The document creation or update failed because of invalid reference ''.\",\"target\":\"/subscriptions/<>/resourceGroups/rg-name/providers/Microsoft.Synapse/workspaces/workspace-02/notebooks/SilverToGoldPipeline\",\"details\":null,\"error\":null}, workspace: workspace-02, notebook: SilverToGoldPipeline, ArtifactId: ]"}
2022-07-06T01:53:27.9954864Z ##[section]Finishing: Synapse deployment task for workspace: Stage

I was able to resolve the issue. The issue was that parameters file for higher environment was missing values for the pipelines. When we publish from master/main branch, if a new notebook is added and it is parameterized then an entry is created in the parameters file.

In my case, the error was because I had to create the Apache Spark pool first, before deploying.
Apparently, the deployment was not creating it automatically.

Related

Devops release pipeline azure resource group deployment task with spaces in the template parameter

In a release pipeline I have an Azure Resource Group Deployment task.
I have a bicep file when I am trying to deploy, so The "template" parameter is set to "**\Deploy.bicep". When it runs, that resolves to D:\a\r1\a_POC.ResearchDay.2023-01-26.Azure Web App and DB\drop\Infrastructure\Deploy.bicep, which has spaces, and this causes the task to fail:
2023-02-14T19:34:59.5345426Z ##[error]Check out the troubleshooting guide to see if your issue is addressed: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment?view=azure-devops#troubleshooting
2023-02-14T19:34:59.5366080Z ##[error]Error: Command failed: az bicep build --file D:\a\r1\a\_POC.ResearchDay.2023-01-26.Azure Web App and DB\drop\Infrastructure\Deploy.bicep
ERROR: unrecognized arguments: Web App and DB\drop\Infrastructure\Deploy.bicep
I've tried surrounding the template parameter value in single and double quotes, and I get the same error. I think I could rename my git repo, but I want to know if there is another solution.

BadRequest: The document creation or update failed because of invalid reference

I'm trying to deploy an ADF pipeline using CICD (Azure DevOps Deployment) using a release pipeline.
Here I'm trying to merge stuff from my Collaboration branch to master (usig GIT).
I'm also using:
Azure Deployment:
Create Or Update Resource Group action on SiteOpsConsolidatedProd (Agent Job) as a part of the Continuous Deployment Process
I'm getting the following error:
[error]At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.
[error]Details:
##[error]BadRequest: The document creation or update failed because of invalid reference 'irslinked'.
##[error]Check out the troubleshooting guide to see if your issue is addressed: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment?view=azure-devops#troubleshooting
##[error]Task failed while creating or updating the template deployment.
irslinked is the name of a self-hosted IR (Linked)
this IR: irslinked is also included in the overwrite template parameters list.
As suggested by Microsoft in their ADF CICD best practices, the self-hosted IR should be hosted on a dedicated ADF instance and should be of type shared and linked to the dev/test/prod instances.
Link to the documentation: https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-deployment#best-practices-for-cicd
Please respond if it works.
This may be helpful for someone. I had a similar invalid reference error. Actually, my ADF was imported from git. I have reconfigured the git with "import existing resource to repository" option.
Azure documentation https://learn.microsoft.com/en-us/azure/data-factory/ci-cd-github-troubleshoot-guide#recover-from-a-deleted-data-factory
If customer had a Self-hosted Integration Runtime in deleted ADF, they
will have to create a new instance in new ADF, also uninstall and
reinstall the instance on their On-prem machine/VM with the new key
obtained. After setup of IR is completed, customer will have to change
the Linked Service to point to new IR and test the connection or it
will fail with error invalid reference.
I had this error message but it referenced a resource that didn't even exist.
I was able to solve it using steps from here:
https://learn.microsoft.com/en-us/azure/data-factory/source-control#troubleshooting-git-integration
Notably:
Disconnect from existing git repo.
Reconnect back to the same repo, but select "import existing resources" into a "Create new git branch".
Then use git to create more commits on top of that branch, which remove all the extra crud that got generated, until the codebase is back to the desired state.
Note that I had to have sequential commits deleting stuff in dependency order, before it would work. i.e. Pipelines, then dataflows, then datasets, then linked services, then Int.Runtimes.

Arm template validation fails through Azure Devops release, works from powershell and test-AzResourceGroupDeployment

I have a release setup that I'm trying get out through Azure Devops release pipelines. This is based on another release that I've cloned and works fine.
The issue is that the template is failing a validation check and not going any further. Strange thing is i'm able to check the syntax successfully with new-AzResourceGroupDeployment and test-AzResourceGroupDeployment. It reports no errors and has been deploying fine.
Using Azure Devops release the error coming back is:
2020-02-26T12:26:16.2632844Z ==============================================================================
2020-02-26T12:26:16.2633634Z Task : ARM template deployment
2020-02-26T12:26:16.2634204Z Description : Deploy an Azure Resource Manager (ARM) template to all the deployment scopes
2020-02-26T12:26:16.2634534Z Version : 3.1.19
2020-02-26T12:26:16.2634945Z Author : Microsoft Corporation
2020-02-26T12:26:16.2635504Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment
2020-02-26T12:26:16.2635948Z ==============================================================================
2020-02-26T12:26:16.8677026Z ARM Service Conection deployment scope - Subscription
2020-02-26T12:26:16.8760315Z Checking if the following resource group exists: myResourceGroup.
2020-02-26T12:26:17.2234188Z Resource group exists: true.
2020-02-26T12:26:17.2259290Z Creating deployment parameters.
2020-02-26T12:26:17.2558066Z The detected encoding for file 'd:\a\r1\a\_Azure-Infrastructure\myResourceGroup\deployment\azuredeploy.json' is 'utf-8'
2020-02-26T12:26:17.2561303Z The detected encoding for file 'd:\a\r1\a\_Azure-Infrastructure\myResourceGroup\deployment\param.dev.json' is 'utf-8'
2020-02-26T12:26:17.5304032Z Starting template validation.
2020-02-26T12:26:17.5304834Z Deployment name is Release-vstfs:///ReleaseManagement/Release/95
2020-02-26T12:26:17.5877973Z ##[warning]Validation errors were found in the Azure Resource Manager template. This can potentially cause template deployment to fail. Template validation failed. Error: {"message":"No HTTP resource was found that matches the request URI 'https://management.azure.com/subscriptions/mySubscription/resourcegroups/myResourceGroup/providers/Microsoft.Resources/deployments/Release-vstfs:/ReleaseManagement/Release/95/validate?api-version=2017-05-10'."}.. Please follow https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/template-syntax
2020-02-26T12:26:17.5890527Z Starting Deployment.
2020-02-26T12:26:17.5891348Z Deployment name is Release-vstfs:///ReleaseManagement/Release/95
2020-02-26T12:26:17.6289831Z There were errors in your deployment. Error code: undefined.
2020-02-26T12:26:17.6291819Z ##[error][object Object]
2020-02-26T12:26:17.6295387Z ##[error]Task failed while creating or updating the template deployment.
2020-02-26T12:26:17.6399260Z ##[section]Finishing: ARM Template deployment: Resource Group scope
​Looking the log above, I'm not sure if it's failing on the validation errors or on the error near the end with a undefined error code.
The task is from another subscription and it's listed in the Azure Resource Manager connection on the task and also lists the subscription and the resource groups in the subscription.
Anyone any ideas on how I can get this working or where to start troubleshooting?
your deployment name is:
Release-vstfs:/ReleaseManagement/Release/95
fairly certain that's not allowed (/). so you need to set your deployment name to something reasonable.
ps. : might not be allowed as well
Did you look at the Activity Log in Azure portal?
All deployment attempts are logged here. Look in particular at the JSON pane, often the real issue is only displayed here.
Otherwise, if it doesn't reach Azure, can you double check the service connection in Azure DevOps? (in particular the related user permission in Azure)

Azure DevOps - Custom Build Task Extension is published, shared and installed in my private organization; but cannot be used in pipeline

I'm learning Azure DevOps. I'm interested in creating a custom build task to get a build artifact from a pipeline and send it to a server. I tried creating a simple custom build task extension and installing it in my organization.
The extension is displayed as installed, with a version of "1.0.4". I tried using it in a pipeline, in YAML file as "- task: ExtentionTest#1", but I'm getting an error "An error occurred while loading the YAML build pipeline. wrong number of segments".
The custom build task doesn't show in the last of tasks next to the YAML file either. Is there a correct method to use a custom build task in a pipeline?

Azure Devops integration error in Azure DataBricks

I am getting following error while syncing a azure databricks notebook with an Azure DevOps repo I've linked it to:
Error while syncing Git history: Numeric value (3204746134) out of range of int? at [Source: {"id":
The repository branch specified in azure databricks is successfully created over on Azure DevOps.
I tried removing the revision history of the notebook that is no longer required to reduce the number of changes to be checked-in. This did not help to fix the error.
There was error while syncing git repo with Azure databricks when repo size is above 3 GB. This was confirmed by the products team and a fix has been deployed.

Resources