ARM template - bad request failed when deploying a linked service - azure

Question

I am trying to deploy a synapse instance via an ARM template and the deployment is successful via the Azure DevOps portal, but when I try to deploy the same template with an Azure Keyvault linked service I encounter the following error:
##[error]At least one resource deployment operation failed. Please list deployment
operations for details. Please see https://aka.ms/DeployOperations for usage details.
##[error]Details:
##[error]BadRequest:
After inspecting the activity logs from the Synapse instance I found out the following:
"resourceGroupName": "platform-test-group",
"resourceProviderName": {
"value": "Microsoft.Synapse",
"localizedValue": "Microsoft.Synapse"
},
"resourceType": {
"value": "Microsoft.Synapse/workspaces/linkedservices",
"localizedValue": "Microsoft.Synapse/workspaces/linkedservices"
},
"resourceId": "/subscriptions/xxxx-xxxx-xxxx-xxxx/resourcegroups/platform-test-group/providers/Microsoft.Synapse/workspaces/synapsedataapp/linkedservices/AzureKeyVault",
"status": {
"value": "Failed",
"localizedValue": "Failed"
},
"subStatus": {
"value": "NotFound",
"localizedValue": "Not Found (HTTP Status Code: 404)"
},
"submissionTimestamp": "2022-02-01T02:30:31.1471914Z",
"subscriptionId": "xxxx-xxxx-xxxx-xxxx",
"tenantId": "0f44c5d4-xxxx-xxxx-xxxxx",
"properties": {
"statusCode": "NotFound",
"serviceRequestId": null,
"statusMessage": "{\"error\":{\"code\":\"BadRequest\",\"message\":\"\"}}",
"eventCategory": "Administrative",
"entity": "/subscriptions/xxxx-xxxx-xxxx-xxxx/resourcegroups/platform-test-group/providers/Microsoft.Synapse/workspaces/synapsedataapp/linkedservices/AzureKeyVault",
"message": "Microsoft.Synapse/workspaces/linkedservices/write",
"hierarchy": "xxxx-xxxx-xxxx-xxxx/Enterprise/Group/Group-Test/xxxx-xxxx-xxxx-xxxx"
},
"relatedEvents": []
}
As you can see, the 404 error appears when the template tries to deploy to the tenant id which is not found, however, when I deploy the keyvault via the synapse UI I encounter no error.
Below is the code snippet that I use in my ARM template to deploy the keyvault to the synapse instance:
{
"name": "[concat(variables('workspaceName'), '/AzureKeyVault')]",
"type": "Microsoft.Synapse/workspaces/linkedservices",
"apiVersion": "2021-06-01-preview",
"properties": {
"annotations": [],
"type": "AzureKeyVault",
"typeProperties": {
"baseUrl": "https://data-test-kv.vault.azure.net/"
}
},
"dependsOn": [
"[variables('workspaceName')]"
]
}
Am I missing some kind of permission or connection that I need to enable? Why am I able to deploy successfully through the UI but not through the ARM template? Any comment or suggestion is greatly valued, so please feel free to comment or improve this question.

Answer 1

I had to contact Microsoft support and their reply was the following:
ARM templates cannot be used to create a linked service. This is due to the fact that linked services are not ARM resources, for examples, synapse workspaces, storage account, virtual networks, etc. Instead, a linked service is classified as an artifact. To still complete the task at hand, you will need to use the Synapse REST API or PowerShell. Below is the link that provides guidance on how to use the API. https://learn.microsoft.com/en-us/powershell/module/az.synapse/set-azsynapselinkedservice?view=azps-7.1.0
This limitation in ARM is applied only to Synapse and they might fix this in the future.
Additional references:
https://feedback.azure.com/d365community/idea/05e41bf1-0925-ec11-b6e6-000d3a4f07b8
https://feedback.azure.com/d365community/idea/48f1bf78-2985-ec11-a81b-6045bd7956bb

Answer 2

In Synapse unlike ADF, linked-services are not part of arm-templates. They are called artifacts and it comprises: Note Books, Spark Definitions, Linked Services, Pipelines etc.
You can find the full article here: https://techcommunity.microsoft.com/t5/azure-synapse-analytics-blog/how-to-use-ci-cd-integration-to-automate-the-deploy-of-a-synapse/ba-p/2248060
In short, first, deploy Synapse using arm templates. And then set up the linked services:
- task: Synapse workspace deployment#1
displayName: 'Setup:Synapse KeyVault Linked Service'
inputs:
TemplateFile: '$(Build.Repository.LocalPath)/TemplateForWorkspace.json'
ParametersFile: '$(Build.Repository.LocalPath)/TemplateParametersForWorkspace.json'
azureSubscription: '${{ parameters.environments.serviceConnectionId }}'
ResourceGroupName: '$(computeResourceGroupName)'
TargetWorkspaceName: '$(synapseWorkspaceName)'
DeleteArtifactsNotInTemplate: true
OverrideArmParameters: |
synapseLinkedServiceKV: $(synapseLinkedServiceKV)
workspaceName: $(synapseWorkspaceName)
Environment: 'prod'
TemplateForWorkspace.json:
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"workspaceName": {
"type": "string"
},
"synapseLinkedServiceKV": {
"type": "string"
}
},
"variables": {
"workspaceId": "[concat('Microsoft.Synapse/workspaces/', parameters('workspaceName'))]"
},
"resources": [
{
"name": "[concat(parameters('workspaceName'), '/' , parameters('synapseLinkedServiceKV'))]",
"type": "Microsoft.Synapse/workspaces/linkedServices",
"apiVersion": "2019-06-01-preview",
"properties": {
"type": "AzureKeyVault",
"typeProperties": {
"baseUrl": "[concat('https://', parameters('synapseLinkedServiceKV'), '.vault.azure.net/')]"
},
"annotations": [],
"description": "Linked Service to Azure KeyVault. KeyVault is used to primarily fetch secrets"
},
"dependsOn": []
}
]
}
TemplateParametersForWorkspace.json:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"workspaceName": {
"value": ""
},
"synapseLinkedServiceKV": {
"value": ""
}
}
}
It deletes the existing artifacts and deploys the one above. You would first need to install the task extension on your Azure Devops for Synapse workspace deployment#1
Note above template was auto-generated. In synapse studio, goto Git Configuration and point it to your repo. It will submit the changes to the branch workspace_publish. You can copy and build on top of the specific artifact code.