Pass Service Principal Client Id and Secret to ARM Template - azure

I have an ARM template that creates an Azure Key Vault followed by an Azure Kubernetes service. The problem is that the Azure Kubernetes service needs a Service Principle's Client ID and Client Secret passed in the first time I create it. So I run the application.json without the kubernetes_servicePrincipalClientId and kubernetes_servicePrincipalClientSecret parameters in the production.parameters.json file:
application.json
{
"comments": "Kubernetes Service Principal Client ID",
"type": "Microsoft.KeyVault/vaults/secrets",
"name": "[concat(parameters('key_vault_name'), '/KubernetesServicePrincipalClientId')]",
"apiVersion": "2018-02-14",
"properties": {
"contentType": "text/plain",
"value": "[parameters('kubernetes_servicePrincipalClientId')]"
}
},
{
"comments": "Kubernetes Service Principal Client Secret",
"type": "Microsoft.KeyVault/vaults/secrets",
"name": "[concat(parameters('key_vault_name'), '/KubernetesServicePrincipalClientSecret')]",
"apiVersion": "2018-02-14",
"properties": {
"contentType": "text/plain",
"value": "[parameters('kubernetes_servicePrincipalClientSecret')]"
}
}
The second time I run the ARM template, I add the following lines to my production.parameters.json file, so that the Client ID and Client Secret are retrieved from Azure Key Vault where they were stored the first time I ran the ARM template.
production.parameters.json
"kubernetes_servicePrincipalClientId": {
"reference": {
"keyVault": {
"id": "/subscriptions/[Subscription Id]/resourcegroups/[Resource Group Name]/providers/Microsoft.KeyVault/vaults/[Vault Name]"
},
"secretName": "KubernetesServicePrincipalClientId"
}
},
"kubernetes_servicePrincipalClientSecret": {
"reference": {
"keyVault": {
"id": "/subscriptions/[Subscription Id]/resourcegroups/[Resource Group Name]/providers/Microsoft.KeyVault/vaults/[Vault Name]"
},
"secretName": "KubernetesServicePrincipalClientSecret"
}
}
Unfortunately it looks like you can't create a service principal in an ARM template. Is there a better way to configure all this in an automated way, so that regardless of whether or not I'm running the template the first time or second time, I don't have to perform any manual steps?

no, those apis are not exposed to arm, there is no way of managing a service principal with an ARM Template. you can however create a script that will provision the service principal and pass its details to the arm template, or you can use some sort of tool to handle all of this for you (pulumi\terraform\ansible)

Related

What fields am I supposed to populate in test_configurations.json?

I am very new to Azure and cloud services in general. I am trying to experiment with the azure storage sdk (https://github.com/Azure/azure-storage-cpp) to use in my application and am trying to get used to it. I'm trying to run the tests to make sure that I built the library correctly.
Looking at the README it says to fill in the test_configurations.json file but I don't know what should be filled in or how to obtain the info that should actually be in there. I signed up for the free Azure account and create a storage account but I don't see client_id, tenant_id or client secret anywhere in the console. Here is the config file:
{
"target": "production",
"premium_target": "premium_account",
"blob_storage_target": "blob_storage_account",
"tenants": [
{
"name": "devstore",
"type": "devstore",
"connection_string": "UseDevelopmentStorage=true"
},
{
"name": "production",
"type": "cloud",
"connection_string": "DefaultEndpointsProtocol=https;"
},
{
"name": "premium_account",
"type": "cloud",
"connection_string": "DefaultEndpointsProtocol=https;"
},
{
"name": "blob_storage_account",
"type": "cloud",
"connection_string": "DefaultEndpointsProtocol=https;"
}
],
"token_information": {
"account_name": "",
"tenant_id": "",
"client_id": "",
"client_secret": "",
"resource": "https://storage.azure.com"
}
}
How do I obtain the ids and secrets that I need?
You need to register an application as storage client and then you can obtain those details
as mentioned here
The first step in using Azure AD to authorize access to storage
resources is registering your client application with an Azure AD

Azure Pipeline Key vault issue while creating DevTest Labs VM

I want to create an Azure DevTest labs VM via pipeline process . However, when I start to deploy, it fails on the step "Create Azure DevTest Lab VM" and it seems to be caused by my key vault is missing. But after check my key vault, my secret is store in it and should be no problem with the permission.
Not sure the message "A secret store was not set up for the user." means, does it mean that there's a problem with the association between my account and the key vault? How can I verify this? Thanks.
Agent job: Create DevTest Labs VM
Win7entx64.json
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json",
"contentVersion": "1.0.0.0",
"parameters": {
"newVMName": {
"defaultValue": "win7entx64",
"type": "String"
},
"labName": {
"defaultValue": "my_lab",
"type": "String"
},
"size": {
"defaultValue": "Standard_A3",
"type": "String"
},
"userName": {
"defaultValue": "trendkj",
"type": "String"
},
"password": {
"type": "securestring",
"defaultValue": "[[[VmPassword]]"
}
},
"variables": {
"labSubnetName": "[concat(variables('labVirtualNetworkName'), 'Subnet')]",
"labVirtualNetworkId": "[resourceId('Microsoft.DevTestLab/labs/virtualnetworks', parameters('labName'), variables('labVirtualNetworkName'))]",
"labVirtualNetworkName": "[concat('Dtl', parameters('labName'))]",
"vmId": "[resourceId ('Microsoft.DevTestLab/labs/virtualmachines', parameters('labName'), parameters('newVMName'))]",
"vmName": "[concat(parameters('labName'), '/', parameters('newVMName'))]"
},
"resources": [
{
"type": "Microsoft.DevTestLab/labs/virtualmachines",
"apiVersion": "2018-10-15-preview",
"name": "[variables('vmName')]",
"location": "westus",
"properties": {
"labVirtualNetworkId": "[variables('labVirtualNetworkId')]",
"notes": "Windows 7 Enterprise",
"galleryImageReference": {
"offer": "windows-7",
"publisher": "microsoftwindowsdesktop",
"sku": "win7-enterprise",
"osType": "Windows",
"version": "latest"
},
"size": "[parameters('size')]",
"userName": "[parameters('userName')]",
"password": "[parameters('password')]",
"isAuthenticationWithSshKey": false,
"labSubnetName": "[variables('labSubnetName')]",
"disallowPublicIpAddress": true,
"storageType": "Standard",
"allowClaim": false
}
}
],
"outputs": {
"labVMId": {
"type": "String",
"value": "[variables('vmId')]"
}
}
}
My secret in the Key Vault:
My Access Policy in the Key Vault
According to your reference to key valut in arm template "defaultValue": "[[[VmPassword]]", you should use the secret saved in DevTest Lab. Page as shown below.
If you want to use the secret in the Key vault page as shown in the question, the reference in the arm template should be as follows:
"sqlAdministratorLoginPassword": {
"reference": {
"keyVault": {
"id": "/subscriptions/25abd97b-44a3-4092-8337-xxxx/resourceGroups/mykeyvault/providers/Microsoft.KeyVault/vaults/cmkvtest"
},
"secretName": "sqlpassword"
}
}
About this, please refer to "3.ADD KEY VAULT SECRETS TO YOU ARM TEMPLATES" part of this artifact.
It has been a time since this was posted.
I am facing the same issue. The secret that you are setting in the Devtest lab is only visible for the user creating it.
The api gets more insights abaot the secret. For getting the secret is the following:
GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DevTestLab/labs/{labName}/users/{userName}/secrets/{name}?api-version=2018-09-15
As you see there is also the username in the path.
So if you have UserA and UserB, UserA sets a secret in DTL lab, when UserB goes in My secrets it will not see the secret of UserA. He can even create a secret with the same name with a different value.
It is a shame that you can not have common secrets for all the users in a DevtestLab (maybe there is but i did not find it yet).
In other words you need to log in into the DTL with the user defined in the azure connection from Azure Devops and create the secret manually. Or create it from the pipeline under his context.
Reference:
https://learn.microsoft.com/en-us/rest/api/dtl/secrets/get
https://learn.microsoft.com/en-us/rest/api/dtl/secrets/create-or-update
Are you missing a ']'?
[[[VmPassword]] --> [[[VmPassword]]]

LogicApp OAuth API connectors and ARM deployments

Looking for some advice here. We try to do our ARM deployments in "complete mode" most of the time. But with API connectors such as onedriveforbusiness we noticed that oauth is invalidated and someone has to go into the portal to re authorize the api connector again.
Is there a workaround available that solves this issue? Or should I just seperate the logic apps into a seperate deployment that runs in Incremental mode?
You could configure to use the service principal credentials passed to the ARM template.
The ARM Template used here was:
{
"type": "microsoft.web/connections",
"apiVersion": "2016-06-01",
"name": "[variables('Principal Data Factory Connection Name')]",
"location": "[resourceGroup().location]",
"dependsOn": [],
"properties": {
"api": {
"id": "[concat(subscription().id, '/providers/Microsoft.Web/locations/', resourceGroup().location, '/managedApis/azuredatafactory')]"
},
"displayName": "Service Princiapl Data Factory Connection",
"parameterValues": {
"token:clientId": "[parameters('Service Principal App Id')]",
"token:clientSecret": "[parameters('Service Principal Secret')]",
"token:TenantId": "[parameters('Service Principal Tenant')]",
"token:resourceUri": "https://management.core.windows.net/",
"token:grantType": "client_credentials"
}
}
}
Credentials are passed in the configuration. There is no need to authorize / authenticate.

Failed to edit Key Vault API connection

I am using Azure Key Vault connector in the logic apps and for deploying the logic apps using ARM templates.
In the ARM templates I have added the Microsoft.Web/connections resource to include Key Vault API connection.
The API Connection gets successfully deployed, but when I open it on the portal to Authorize it I get an error :- "Failed to edit Api connection "keyvault"".
The resource template of the key vault looks like below :-
{
"type": "Microsoft.Web/connections",
"apiVersion": "2016-06-01",
"name": "[parameters('connections_keyvault_name')]",
"location": "[parameters('location')]",
"properties": {
"api": {
"id": "[concat('/subscriptions/',subscription().subscriptionId,'/providers/Microsoft.Web/locations/', parameters('location'), '/managedApis/', parameters('connections_keyvault_name'))]"
},
"displayName": "",
"customParameterValues": {}
}
}
The status of the API connection after deployment always shows "Error". However, I am using office365 API connection as well which works fine after deployment i.e. when I authorize it, it allows me to save it.
Can anyone please help me with this issue? This is blocking us to move this logic app to production.
Thanks,
Archana Kolte
You have to add the key vault name to connect to by setting the vaultName property under properties.paramaterValues.
Also, the end of the properties.api.id is the same for all Key Vault API Connections and should be /managedApis/keyvault.
Your ARM template would look something like this
{
"type": "Microsoft.Web/connections",
"apiVersion": "2016-06-01",
"name": "[parameters('connections_keyvault_name')]",
"location": "[parameters('location')]",
"properties": {
"api": {
"id": "[concat('/subscriptions/',subscription().subscriptionId,'/providers/Microsoft.Web/locations/', parameters('location'), '/managedApis/keyvault')]"
},
"displayName": "",
"parameterValues": {
"vaultName": "<name-of-your-key-vault>"
}
}
}

Enable HTTPS on Azure Front Door custom domain with ARM template deployment

I am deploying an Azure Front Door via an ARM template, and attempting to enable HTTPS on a custom domain.
According to the Azure documentation for Front Door, there is a quick start template to "Add a custom domain to your Front Door and enable HTTPS traffic for it with a Front Door managed certificate generated via DigiCert." However, while this adds a custom domain, it does not enable HTTPS.
Looking at the ARM template reference for Front Door, I can't see any obvious way to enable HTTPS, but perhaps I'm missing something?
Notwithstanding the additional information below, I'd like to be able to enable HTTPS on a Front Door custom domain via an ARM template deployment. Is this possible at this time?
Additional information
Note that there is a REST operation to enable HTTPS, but this does not seem to work with a Front Door managed certificate -
POST https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/frontDoors/{frontDoorName}/frontendEndpoints/{frontendEndpointName}/enableHttps?api-version=2019-05-01
{
"certificateSource": "FrontDoor",
"protocolType": "ServerNameIndication",
"minimumTLSVersion": "1.2"
}
There is also a Az PowerShell cmdlet to enable HTTP, which does work.
Enable-AzFrontDoorCustomDomainHttps -ResourceGroupName "lmk-bvt-accounts-front-door" -FrontDoorName "my-front-door" -FrontendEndpointName "my-front-door-rg"
UPDATE: This implementation currently seems to be unstable and is working only intermittently, which indicates it may not be production ready yet.
This now actually seems to be possible with ARM templates, after tracking down the latest Front Door API (2020-01-01) specs (which don't appear to be fully published in the MS reference websites yet):
https://github.com/Azure/azure-rest-api-specs/tree/master/specification/frontdoor/resource-manager/Microsoft.Network/stable/2020-01-01
There's a new customHttpsConfiguration property in the frontendEndpoint properties object:
"customHttpsConfiguration": {
"certificateSource": "AzureKeyVault" // or "FrontDoor",
"minimumTlsVersion":"1.2",
"protocolType": "ServerNameIndication",
// Depending on "certificateSource" you supply either:
"keyVaultCertificateSourceParameters": {
"secretName": "<secret name>",
"secretVersion": "<secret version>",
"vault": {
"id": "<keyVault ResourceID>"
}
}
// Or:
"frontDoorCertificateSourceParameters": {
"certificateType": "Dedicated"
}
}
KeyVault Managed SSL Certificate Example
Note: I have tested this and appears to work.
{
"type": "Microsoft.Network/frontdoors",
"apiVersion": "2020-01-01",
"properties": {
"frontendEndpoints": [
{
"name": "[variables('frontendEndpointName')]",
"properties": {
"hostName": "[variables('customDomain')]",
"sessionAffinityEnabledState": "Enabled",
"sessionAffinityTtlSeconds": 0,
"webApplicationFirewallPolicyLink": {
"id": "[variables('wafPolicyResourceId')]"
},
"resourceState": "Enabled",
"customHttpsConfiguration": {
"certificateSource": "AzureKeyVault",
"minimumTlsVersion":"1.2",
"protocolType": "ServerNameIndication",
"keyVaultCertificateSourceParameters": {
"secretName": "[parameters('certKeyVaultSecret')]",
"secretVersion": "[parameters('certKeyVaultSecretVersion')]",
"vault": {
"id": "[resourceId(parameters('certKeyVaultResourceGroupName'),'Microsoft.KeyVault/vaults',parameters('certKeyVaultName'))]"
}
}
}
}
}
],
...
}
}
Front Door Managed SSL Certificate Example
Looks like for a FrontDoor managed certificate you would need to set:
Note: I have not tested this
{
"type": "Microsoft.Network/frontdoors",
"apiVersion": "2020-01-01",
"properties": {
"frontendEndpoints": [
{
"name": "[variables('frontendEndpointName')]",
"properties": {
"hostName": "[variables('customDomain')]",
"sessionAffinityEnabledState": "Enabled",
"sessionAffinityTtlSeconds": 0,
"webApplicationFirewallPolicyLink": {
"id": "[variables('wafPolicyResourceId')]"
},
"resourceState": "Enabled",
"customHttpsConfiguration": {
"certificateSource": "FrontDoor",
"minimumTlsVersion":"1.2",
"protocolType": "ServerNameIndication",
"frontDoorCertificateSourceParameters": {
"certificateType": "Dedicated"
}
}
}
}
],
...
}
}
I was able to successfully make an enableHttps REST Call using the Azure Management API.
I got a successful response and can see the resource results in the portal.azure.com and resource.azure.com sites.
However I am pretty sure the Management API, and PowerShell methods are the only ways supported right now. Since there is likely some validation required on the Certificate and Handling, they didn't include that yet in the ARM Templates. Given validation can be quite important, it is best you confirm your configuration is workable in the UI first, before automating it (IMHO).
According to this discussion this seems only possible via the REST API (see e.g. this answer) and not (yet) via ARM.
I managed to get this working with an ARM template. The below link shows you how to do this using Azure Front Door as a certificate source:
https://github.com/Azure/azure-quickstart-templates/blob/master/101-front-door-custom-domain/azuredeploy.json
I drew inspiration from this for deploying a certificate from Azure Key Vault for a custom domain. Here are the relevant elements from the ARM template that I am using:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"hubName": {
"type": "string",
"metadata": {
"description": "Name to assign to the hub. This name will prefix all resources contained in the hub."
}
},
"frontdoorName": {
"type": "string",
"metadata": {
"description": "Name to assign to the Frontdoor instance"
}
},
"frontdoorCustomDomain": {
"type": "string",
"metadata": {
"description": "The custom domain name to be applied to the provisioned Azure Frontdoor instance"
}
},
"keyVaultCertificateName": {
"type": "string",
"metadata": {
"description": "Name of the TLS certificate in the Azure KeyVault to be deployed to Azure Frontdoor for supporting TLS over a custom domain",
"assumptions": [
"Azure KeyVault containing the TLS certificate is deployed to the same resource group as the resource group where Azure Frontdoor will be deployed to",
"Azure KeyVault name is the hub name followed by '-keyvault' (refer to variable 'keyVaultName' in this template)"
]
}
},
...
},
"variables": {
"frontdoorName": "[concat(parameters('hubName'), '-', parameters('frontdoorName'))]",
"frontdoorEndpointName": "[concat(variables('frontdoorName'), '-azurefd-net')]",
"customDomainFrontdoorEndpointName": "[concat(variables('frontdoorName'), '-', replace(parameters('frontdoorCustomDomain'), '.', '-'))]",
"keyVaultName": "[concat(parameters('hubName'), '-keyvault')]",
"frontdoorHostName": "[concat(variables('frontdoorName'), '.azurefd.net')]",
...
},
"resources": [
{
"type": "Microsoft.Network/frontdoors",
"apiVersion": "2020-05-01",
"name": "[variables('frontdoorName')]",
"location": "Global",
"properties": {
"resourceState": "Enabled",
"backendPools": [...],
"healthProbeSettings": [...],
"frontendEndpoints": [
{
"id": "[concat(resourceId('Microsoft.Network/frontdoors', variables('frontdoorName')), concat('/FrontendEndpoints/', variables('frontdoorEndpointName')))]",
"name": "[variables('frontdoorEndpointName')]",
"properties": {
"hostName": "[variables('frontdoorHostName')]",
"sessionAffinityEnabledState": "Enabled",
"sessionAffinityTtlSeconds": 0,
"resourceState": "Enabled"
}
},
{
"id": "[concat(resourceId('Microsoft.Network/frontdoors', variables('frontdoorName')), concat('/FrontendEndpoints/', variables('customDomainFrontdoorEndpointName')))]",
"name": "[variables('customDomainFrontdoorEndpointName')]",
"properties": {
"hostName": "[parameters('frontdoorCustomDomain')]",
"sessionAffinityEnabledState": "Enabled",
"sessionAffinityTtlSeconds": 0,
"resourceState": "Enabled"
}
}
],
"loadBalancingSettings": [...],
"routingRules": [...],
"backendPoolsSettings": {
"enforceCertificateNameCheck": "Enabled",
"sendRecvTimeoutSeconds": 30
},
"enabledState": "Enabled",
"friendlyName": "[variables('frontdoorName')]"
}
},
{
"type": "Microsoft.Network/frontdoors/frontendEndpoints/customHttpsConfiguration",
"apiVersion": "2020-07-01",
"name": "[concat(variables('frontdoorName'), '/', variables('customDomainFrontdoorEndpointName'), '/default')]",
"dependsOn": [
"[resourceId('Microsoft.Network/frontdoors', variables('frontdoorName'))]"
],
"properties": {
"protocolType": "ServerNameIndication",
"certificateSource": "AzureKeyVault",
"minimumTlsVersion": "1.2",
"keyVaultCertificateSourceParameters": {
"secretName": "[parameters('keyVaultCertificateName')]",
"vault": {
"id": "[resourceId(resourceGroup().name, 'Microsoft.KeyVault/vaults', variables('keyVaultName'))]"
}
}
}
}
]
}
Azure Front Door classic now seems to support both managed certificates and custom certificates for custom domains. At least there are quickstart templates in the official repo from Microsoft exactly for these cases:
managed certificate
custom certificate
They both use Microsoft.Network/frontdoors/frontendEndpoints/customHttpsConfiguration subresource of the Front Door, currently with API version 2020-07-01. Only the parent subresource is documented in the templates reference, though.
The name of the customHttpsConfiguration resource is "default", so when the resource is specified as a top-level resource in the template, its complete name is something like "myfrontdoorafd/www-example-com/default".
Using Bicep (which transpiles to JSON ARM templates and which I highly recommend), the important part of the template looks like this:
param frontDoorName string
param customDomainName string
var frontEndEndpointCustomName = replace(customDomainName, '.', '-')
resource frontDoor 'Microsoft.Network/frontDoors#2020-01-01' = {
name: frontDoorName
properties: {
frontendEndpoints: [
{
name: frontEndEndpointCustomName
properties: {
hostName: customDomainName
...
}
}
...
]
...
}
...
resource frontendEndpoint 'frontendEndpoints' existing = {
name: frontEndEndpointCustomName
}
}
// This resource enables a Front Door-managed TLS certificate on the frontend.
resource customHttpsConfiguration 'Microsoft.Network/frontdoors/frontendEndpoints/customHttpsConfiguration#2020-07-01' = {
parent: frontDoor::frontendEndpoint
name: 'default'
properties: {
protocolType: 'ServerNameIndication'
certificateSource: 'FrontDoor'
frontDoorCertificateSourceParameters: {
certificateType: 'Dedicated'
}
minimumTlsVersion: '1.2'
}
}
Note that the deployment will be in progress till the certificate is actually issued and deployed to all points of presence (PoP) of Azure. This may take really long and even fail due to RequestTimeout. If you want to just start the operation and let it complete asynchronously, use e.g. the enable-https subcommand in Azure CLI. Even after the failure, the customHttpsProvisioningState is Pending and the certificate provisioning process may complete successfully.
Also note that when you have many frontend endpoints and changes happen frequently but most frontend endpoints stay unchanged, the pattern from this template cannot be generalized just by specifying multiple customHttpsConfiguration instances for multiple frontend endpoints. Such a generalization is not efficient and likely hits the rate limit of the underlying API (429 TooManyRequests) because the API is called even when the endpoint already has the HTTPS configuration.
In such a case, I was able to use nested templates and conditional deployment to deploy the customHttpsConfiguration subresource only when the frontend endpoint's property customHttpsProvisioningState has the value of Disabled. This works OK even with tens of frontend endpoints when a new frontend endpoint is added (and it should get a managed certificate). Even in deployment mode Complete, the once-applied configuration persists.

Resources