ARM Template - auto approval of managed private endpoint - azure

Question

I am developing an ARM template for Azure Data Factory with managed private endpoints to SQL Server and Azure Datalake. However, when the ARM template completes execution, the managed private endpoints are in "Pending" state. How can I provision the managed private endpoint so that it gets provisioned as "Approved" once ADF is completely provisioned using ARM Template. Following is my template.json file:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"environment": {
"type": "string",
"metadata": {
"description": "name of environment for deployment"
}
},
"project": {
"type": "string",
"metadata": {
"description": "name of the project for building the name of resources"
}
},
"location": {
"defaultValue": "eastus",
"type": "string"
},
"adfFactoryName": {
"type": "string"
},
"adfVersion": {
"type": "string"
},
"tags": {
"type": "object",
"metadata": {
"description": "tags to add to resources"
}
},
"adfVNetEnabled": {
"type": "bool"
},
"adfPublicNetworkAccess": {
"type": "bool"
},
"adfAzureDatabricksDomain": {
"type": "string",
"metadata": "Azure Databricks existing cluster Id"
},
"adfAzureDatabricksExistingClusterId": {
"type": "string",
"metadata": "Azure Databricks existing cluster Id"
},
"adfDataLakeConnectionString": {
"type": "string",
"metadata": "Azure Data Lake connection string"
},
"adfDataFactoryIdentity": {
"type": "string",
"metadata": "Identity type for data factory"
},
"adfLSASDBConnectionString": {
"type": "string",
"metadata": "SQL DB connection string"
},
"adfKVBaseURL": {
"type": "string",
"metadata": "Keyvault connection string"
},
"adfDataLakeStorageName": {
"type": "string",
"metadata": "Azure Data lake Storage Name"
},
"adfDataLakeStorageGroupId": {
"type": "string",
"metadata": "Azure Data lake Storage Group ID"
},
"adfSqlServerName": {
"type": "string",
"metadata": "Azure SQL Server name"
},
"adfSqlServerGroupId": {
"type": "string",
"metadata": "Azure SQL Server Group ID"
}
},
"variables": {
"factoryId": "[concat('Microsoft.DataFactory/factories/', parameters('adfFactoryName'))]",
"managedVirtualNetworkName": "[concat(parameters('adfFactoryName'), '/default')]"
},
"resources": [
{
"condition": "[equals(parameters('adfVersion'), 'V2')]",
"type": "Microsoft.DataFactory/factories",
"apiVersion": "2018-06-01",
"name": "[parameters('adfFactoryName')]",
"location": "[parameters('location')]",
"identity": {
"type": "[parameters('adfDataFactoryIdentity')]"
},
"properties": {
"publicNetworkAccess": "[if(bool(parameters('adfPublicNetworkAccess')), 'Enabled', 'Disabled')]"
},
"tags": "[parameters('tags')]",
"resources": [
{
"condition": "[and(equals(parameters('adfVersion'), 'V2'), parameters('adfVNetEnabled'))]",
"name": "[concat(parameters('adfFactoryName'), '/default')]",
"type": "Microsoft.DataFactory/factories/managedVirtualNetworks",
"apiVersion": "2018-06-01",
"properties": {},
"dependsOn": [
"[concat('Microsoft.DataFactory/factories/', parameters('adfFactoryName'))]"
]
},
{
"condition": "[and(equals(parameters('adfVersion'), 'V2'), parameters('adfVNetEnabled'))]",
"name": "[concat(parameters('adfFactoryName'), '/DDIR')]",
"type": "Microsoft.DataFactory/factories/integrationRuntimes",
"apiVersion": "2018-06-01",
"properties": {
"type": "Managed",
"managedVirtualNetwork": {
"referenceName": "default",
"type": "ManagedVirtualNetworkReference"
},
"typeProperties": {
"computeProperties": {
"location": "[parameters('location')]"
}
}
},
"dependsOn": [
"[concat('Microsoft.DataFactory/factories/', parameters('adfFactoryName'))]",
"[concat('Microsoft.DataFactory/factories/', parameters('adfFactoryName'), '/managedVirtualNetworks/default')]"
]
},
{
"name": "[concat(parameters('adfFactoryName'), '/AzureKeyVault')]",
"type": "Microsoft.DataFactory/factories/linkedServices",
"apiVersion": "2018-06-01",
"properties": {
"annotations": [],
"type": "AzureKeyVault",
"typeProperties": {
"baseUrl": "[parameters('adfKVBaseURL')]"
}
},
"dependsOn": [
"[parameters('adfFactoryName')]"
]
},
{
"name": "[concat(parameters('adfFactoryName'), '/AzureDatabricks_LinkedService')]",
"type": "Microsoft.DataFactory/factories/linkedServices",
"apiVersion": "2018-06-01",
"properties": {
"annotations": [],
"type": "AzureDatabricks",
"typeProperties": {
"domain": "[parameters('adfAzureDatabricksDomain')]",
"accessToken": {
"type": "AzureKeyVaultSecret",
"store": {
"referenceName": "AzureKeyVault",
"type": "LinkedServiceReference"
},
"secretName": "[concat('kvs-databricks-',parameters('environment'), 'aue', parameters('project'))]"
},
"existingClusterId": "[parameters('adfAzureDatabricksExistingClusterId')]"
},
"connectVia": {
"referenceName": "DDIR",
"type": "IntegrationRuntimeReference"
}
},
"dependsOn": [
"[parameters('adfFactoryName')]",
"[concat(variables('factoryId'), '/integrationRuntimes/DDIR')]",
"[concat(variables('factoryId'), '/linkedServices/AzureKeyVault')]"
]
},
{
"name": "[concat(parameters('adfFactoryName'), '/AzureDatalake_DDIR')]",
"type": "Microsoft.DataFactory/factories/linkedServices",
"apiVersion": "2018-06-01",
"properties": {
"annotations": [],
"type": "AzureBlobFS",
"typeProperties": {
"url": "[parameters('adfDataLakeConnectionString')]"
},
"connectVia": {
"referenceName": "DDIR",
"type": "IntegrationRuntimeReference"
}
},
"dependsOn": [
"[parameters('adfFactoryName')]",
"[concat(variables('factoryId'), '/integrationRuntimes/DDIR')]"
]
},
{
"name": "[concat(parameters('adfFactoryName'), '/LS_ASDB')]",
"type": "Microsoft.DataFactory/factories/linkedServices",
"apiVersion": "2018-06-01",
"properties": {
"annotations": [],
"type": "AzureSqlDatabase",
"typeProperties": {
"connectionString": "[parameters('adfLSASDBConnectionString')]",
"password": {
"type": "AzureKeyVaultSecret",
"store": {
"referenceName": "AzureKeyVault",
"type": "LinkedServiceReference"
},
"secretName": "kvs-synapsepwd-devauegteng"
}
},
"connectVia": {
"referenceName": "DDIR",
"type": "IntegrationRuntimeReference"
}
},
"dependsOn": [
"[parameters('adfFactoryName')]",
"[concat(variables('factoryId'), '/integrationRuntimes/DDIR')]",
"[concat(variables('factoryId'), '/linkedServices/AzureKeyVault')]"
]
}
]
},
{
"name": "[concat(parameters('adfFactoryName'), '/default/',parameters('adfDataLakeStorageName'))]",
"type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints",
"apiVersion": "2018-06-01",
"properties": {
"privateLinkResourceId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('adfDataLakeStorageName'))]",
"groupId": "[parameters('adfDataLakeStorageGroupId')]"
},
"dependsOn": [
"[concat('Microsoft.DataFactory/factories/', parameters('adfFactoryName'), '/managedVirtualNetworks/default')]"
]
},
{
"name": "[concat(parameters('adfFactoryName'), '/default/',parameters('adfSqlServerName'))]",
"type": "Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints",
"apiVersion": "2018-06-01",
"properties": {
"privateLinkResourceId": "[resourceId('Microsoft.Sql/servers', parameters('adfSqlServerName'))]",
"groupId": "[parameters('adfSqlServerGroupId')]"
},
"dependsOn": [
"[concat('Microsoft.DataFactory/factories/', parameters('adfFactoryName'), '/managedVirtualNetworks/default')]"
]
}
]
}

Answer 1

I ran in the same problem and created a Az Powershell script function to help me out.
function ApprovePrivateEndpointConnections {
param (
[string] $ResourceGroupName,
[string] $ResourceName
)
$Resource = Get-AzResource -Name $ResourceName -ResourceGroupName $ResourceGroup
if ($Resource) {
$ResourcePendingConnection = Get-AzPrivateEndpointConnection -PrivatelinkResourceId $Resource.ResourceId | Where-Object -FilterScript {$_.PrivateLinkServiceConnectionState.Status -EQ 'Pending'}
if ($ResourcePendingConnection) {
foreach ($Connection in $ResourcePendingConnection)
{
Approve-AzPrivateEndpointConnection -ResourceId $Connection.Id
}
}
else
{
Write-Output 'No pending connections for Resource: ' $ResourceName
}
}
else
{
Write-Output 'No resource found with name: ' $ResourceName
}
}

Answer 2

I found Toofle's script an excellent start here, but was slightly nervous about the security risk of simply approving all pending requests.
I have therefore modified the script to take additional parameters relating to the private endpoint I am expecting, and to ignore any endpoints which do not match.
param(
[Parameter(Mandatory)]
[string] $DataFactoryName,
[Parameter(Mandatory)]
[string] $PrivateEndpointName,
[Parameter(Mandatory)]
[string] $TargetResourceId
)
$Resource = Get-AzResource -ResourceId $TargetResourceId
if ($Resource) {
$ResourcePendingConnection = Get-AzPrivateEndpointConnection -PrivatelinkResourceId $Resource.ResourceId `
| Where-Object -FilterScript {$_.PrivateLinkServiceConnectionState.Status -EQ 'Pending'} `
| Where-Object -FilterScript {$_.PrivateEndpoint.Id -like "*/providers/Microsoft.Network/privateEndpoints/$DataFactoryName.$PrivateEndpointName"}
if ($ResourcePendingConnection) {
foreach ($Connection in $ResourcePendingConnection)
{
Approve-AzPrivateEndpointConnection -ResourceId $Connection.Id
}
}
else
{
Write-Output 'No pending connections for Resource: ' $TargetResourceId
}
}
else
{
Write-Output 'No resource found with ID: ' $TargetResourceId
}

Answer 3

Create private endpoint resource using Azure portal or using ARM template with the following JSON template and mention status as Approved in ConnectionState.
Example:
{
"name": "string",
"type": "Microsoft.Network/privateEndpoints",
"apiVersion": "2020-07-01",
"location": "string",
"tags": {},
"properties": {
"subnet": {
"id": "string",
"name": "string"
},
"privateLinkServiceConnections": [
{
"id": "string",
"properties": {
"privateLinkServiceId": "string",
"groupIds": [
"string"
],
"requestMessage": "string",
"privateLinkServiceConnectionState": {
"status": "string",
"description": "string",
"actionsRequired": "string"
}
},
"name": "string"
}
],
"manualPrivateLinkServiceConnections": [
{
"id": "string",
"properties": {
"privateLinkServiceId": "string",
"groupIds": [
"string"
],
"requestMessage": "string",
"privateLinkServiceConnectionState": {
"status": "string",
"description": "string",
"actionsRequired": "string"
}
},
"name": "string"
}
],
"customDnsConfigs": [
{
"fqdn": "string",
"ipAddresses": [
"string"
]
}
]
},
"resources": []
}
Refer - privateendpoints for string property values.

Answer 4

I couldn't find a way to get it done in ARM itself. My feeling that if it's possible to do so, you have to be on the receiving side of the link (storage account, database etc.), whether it's ARM, Bicep or CLI.
What we are doing now is:
az network private-endpoint-connection list -g <RESOURCE_GROUP> -n <SA_NAME> --type Microsoft.Storage/storageAccounts --query "[?properties.privateLinkServiceConnectionState.status == 'Pending'].name" > output.json
jq -c '.[]' output.json | xargs -r -L 1 az network private-endpoint-connection approve -g <RESOURCE_GROUP> --resource-name <SA_NAME> --type Microsoft.Storage/storageAccounts --description "Approved" -n
You may put this in an Azure CLI step of the release pipeline.

Answer 5

Approval needs to be automated with a script, there is a good example in the official templates.