I am very new to Azure and cloud services in general. I am trying to experiment with the azure storage sdk (https://github.com/Azure/azure-storage-cpp) to use in my application and am trying to get used to it. I'm trying to run the tests to make sure that I built the library correctly.
Looking at the README it says to fill in the test_configurations.json file but I don't know what should be filled in or how to obtain the info that should actually be in there. I signed up for the free Azure account and create a storage account but I don't see client_id, tenant_id or client secret anywhere in the console. Here is the config file:
{
"target": "production",
"premium_target": "premium_account",
"blob_storage_target": "blob_storage_account",
"tenants": [
{
"name": "devstore",
"type": "devstore",
"connection_string": "UseDevelopmentStorage=true"
},
{
"name": "production",
"type": "cloud",
"connection_string": "DefaultEndpointsProtocol=https;"
},
{
"name": "premium_account",
"type": "cloud",
"connection_string": "DefaultEndpointsProtocol=https;"
},
{
"name": "blob_storage_account",
"type": "cloud",
"connection_string": "DefaultEndpointsProtocol=https;"
}
],
"token_information": {
"account_name": "",
"tenant_id": "",
"client_id": "",
"client_secret": "",
"resource": "https://storage.azure.com"
}
}
How do I obtain the ids and secrets that I need?
You need to register an application as storage client and then you can obtain those details
as mentioned here
The first step in using Azure AD to authorize access to storage
resources is registering your client application with an Azure AD
I want to create an Azure DevTest labs VM via pipeline process . However, when I start to deploy, it fails on the step "Create Azure DevTest Lab VM" and it seems to be caused by my key vault is missing. But after check my key vault, my secret is store in it and should be no problem with the permission.
Not sure the message "A secret store was not set up for the user." means, does it mean that there's a problem with the association between my account and the key vault? How can I verify this? Thanks.
Agent job: Create DevTest Labs VM
Win7entx64.json
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json",
"contentVersion": "1.0.0.0",
"parameters": {
"newVMName": {
"defaultValue": "win7entx64",
"type": "String"
},
"labName": {
"defaultValue": "my_lab",
"type": "String"
},
"size": {
"defaultValue": "Standard_A3",
"type": "String"
},
"userName": {
"defaultValue": "trendkj",
"type": "String"
},
"password": {
"type": "securestring",
"defaultValue": "[[[VmPassword]]"
}
},
"variables": {
"labSubnetName": "[concat(variables('labVirtualNetworkName'), 'Subnet')]",
"labVirtualNetworkId": "[resourceId('Microsoft.DevTestLab/labs/virtualnetworks', parameters('labName'), variables('labVirtualNetworkName'))]",
"labVirtualNetworkName": "[concat('Dtl', parameters('labName'))]",
"vmId": "[resourceId ('Microsoft.DevTestLab/labs/virtualmachines', parameters('labName'), parameters('newVMName'))]",
"vmName": "[concat(parameters('labName'), '/', parameters('newVMName'))]"
},
"resources": [
{
"type": "Microsoft.DevTestLab/labs/virtualmachines",
"apiVersion": "2018-10-15-preview",
"name": "[variables('vmName')]",
"location": "westus",
"properties": {
"labVirtualNetworkId": "[variables('labVirtualNetworkId')]",
"notes": "Windows 7 Enterprise",
"galleryImageReference": {
"offer": "windows-7",
"publisher": "microsoftwindowsdesktop",
"sku": "win7-enterprise",
"osType": "Windows",
"version": "latest"
},
"size": "[parameters('size')]",
"userName": "[parameters('userName')]",
"password": "[parameters('password')]",
"isAuthenticationWithSshKey": false,
"labSubnetName": "[variables('labSubnetName')]",
"disallowPublicIpAddress": true,
"storageType": "Standard",
"allowClaim": false
}
}
],
"outputs": {
"labVMId": {
"type": "String",
"value": "[variables('vmId')]"
}
}
}
My secret in the Key Vault:
My Access Policy in the Key Vault
According to your reference to key valut in arm template "defaultValue": "[[[VmPassword]]", you should use the secret saved in DevTest Lab. Page as shown below.
If you want to use the secret in the Key vault page as shown in the question, the reference in the arm template should be as follows:
"sqlAdministratorLoginPassword": {
"reference": {
"keyVault": {
"id": "/subscriptions/25abd97b-44a3-4092-8337-xxxx/resourceGroups/mykeyvault/providers/Microsoft.KeyVault/vaults/cmkvtest"
},
"secretName": "sqlpassword"
}
}
About this, please refer to "3.ADD KEY VAULT SECRETS TO YOU ARM TEMPLATES" part of this artifact.
It has been a time since this was posted.
I am facing the same issue. The secret that you are setting in the Devtest lab is only visible for the user creating it.
The api gets more insights abaot the secret. For getting the secret is the following:
GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DevTestLab/labs/{labName}/users/{userName}/secrets/{name}?api-version=2018-09-15
As you see there is also the username in the path.
So if you have UserA and UserB, UserA sets a secret in DTL lab, when UserB goes in My secrets it will not see the secret of UserA. He can even create a secret with the same name with a different value.
It is a shame that you can not have common secrets for all the users in a DevtestLab (maybe there is but i did not find it yet).
In other words you need to log in into the DTL with the user defined in the azure connection from Azure Devops and create the secret manually. Or create it from the pipeline under his context.
Reference:
https://learn.microsoft.com/en-us/rest/api/dtl/secrets/get
https://learn.microsoft.com/en-us/rest/api/dtl/secrets/create-or-update
Are you missing a ']'?
[[[VmPassword]] --> [[[VmPassword]]]
Looking for some advice here. We try to do our ARM deployments in "complete mode" most of the time. But with API connectors such as onedriveforbusiness we noticed that oauth is invalidated and someone has to go into the portal to re authorize the api connector again.
Is there a workaround available that solves this issue? Or should I just seperate the logic apps into a seperate deployment that runs in Incremental mode?
You could configure to use the service principal credentials passed to the ARM template.
The ARM Template used here was:
{
"type": "microsoft.web/connections",
"apiVersion": "2016-06-01",
"name": "[variables('Principal Data Factory Connection Name')]",
"location": "[resourceGroup().location]",
"dependsOn": [],
"properties": {
"api": {
"id": "[concat(subscription().id, '/providers/Microsoft.Web/locations/', resourceGroup().location, '/managedApis/azuredatafactory')]"
},
"displayName": "Service Princiapl Data Factory Connection",
"parameterValues": {
"token:clientId": "[parameters('Service Principal App Id')]",
"token:clientSecret": "[parameters('Service Principal Secret')]",
"token:TenantId": "[parameters('Service Principal Tenant')]",
"token:resourceUri": "https://management.core.windows.net/",
"token:grantType": "client_credentials"
}
}
}
Credentials are passed in the configuration. There is no need to authorize / authenticate.
I am using Azure Key Vault connector in the logic apps and for deploying the logic apps using ARM templates.
In the ARM templates I have added the Microsoft.Web/connections resource to include Key Vault API connection.
The API Connection gets successfully deployed, but when I open it on the portal to Authorize it I get an error :- "Failed to edit Api connection "keyvault"".
The resource template of the key vault looks like below :-
{
"type": "Microsoft.Web/connections",
"apiVersion": "2016-06-01",
"name": "[parameters('connections_keyvault_name')]",
"location": "[parameters('location')]",
"properties": {
"api": {
"id": "[concat('/subscriptions/',subscription().subscriptionId,'/providers/Microsoft.Web/locations/', parameters('location'), '/managedApis/', parameters('connections_keyvault_name'))]"
},
"displayName": "",
"customParameterValues": {}
}
}
The status of the API connection after deployment always shows "Error". However, I am using office365 API connection as well which works fine after deployment i.e. when I authorize it, it allows me to save it.
Can anyone please help me with this issue? This is blocking us to move this logic app to production.
Thanks,
Archana Kolte
You have to add the key vault name to connect to by setting the vaultName property under properties.paramaterValues.
Also, the end of the properties.api.id is the same for all Key Vault API Connections and should be /managedApis/keyvault.
Your ARM template would look something like this
{
"type": "Microsoft.Web/connections",
"apiVersion": "2016-06-01",
"name": "[parameters('connections_keyvault_name')]",
"location": "[parameters('location')]",
"properties": {
"api": {
"id": "[concat('/subscriptions/',subscription().subscriptionId,'/providers/Microsoft.Web/locations/', parameters('location'), '/managedApis/keyvault')]"
},
"displayName": "",
"parameterValues": {
"vaultName": "<name-of-your-key-vault>"
}
}
}
I am deploying an Azure Front Door via an ARM template, and attempting to enable HTTPS on a custom domain.
According to the Azure documentation for Front Door, there is a quick start template to "Add a custom domain to your Front Door and enable HTTPS traffic for it with a Front Door managed certificate generated via DigiCert." However, while this adds a custom domain, it does not enable HTTPS.
Looking at the ARM template reference for Front Door, I can't see any obvious way to enable HTTPS, but perhaps I'm missing something?
Notwithstanding the additional information below, I'd like to be able to enable HTTPS on a Front Door custom domain via an ARM template deployment. Is this possible at this time?
Additional information
Note that there is a REST operation to enable HTTPS, but this does not seem to work with a Front Door managed certificate -
POST https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/frontDoors/{frontDoorName}/frontendEndpoints/{frontendEndpointName}/enableHttps?api-version=2019-05-01
{
"certificateSource": "FrontDoor",
"protocolType": "ServerNameIndication",
"minimumTLSVersion": "1.2"
}
There is also a Az PowerShell cmdlet to enable HTTP, which does work.
Enable-AzFrontDoorCustomDomainHttps -ResourceGroupName "lmk-bvt-accounts-front-door" -FrontDoorName "my-front-door" -FrontendEndpointName "my-front-door-rg"
UPDATE: This implementation currently seems to be unstable and is working only intermittently, which indicates it may not be production ready yet.
This now actually seems to be possible with ARM templates, after tracking down the latest Front Door API (2020-01-01) specs (which don't appear to be fully published in the MS reference websites yet):
https://github.com/Azure/azure-rest-api-specs/tree/master/specification/frontdoor/resource-manager/Microsoft.Network/stable/2020-01-01
There's a new customHttpsConfiguration property in the frontendEndpoint properties object:
"customHttpsConfiguration": {
"certificateSource": "AzureKeyVault" // or "FrontDoor",
"minimumTlsVersion":"1.2",
"protocolType": "ServerNameIndication",
// Depending on "certificateSource" you supply either:
"keyVaultCertificateSourceParameters": {
"secretName": "<secret name>",
"secretVersion": "<secret version>",
"vault": {
"id": "<keyVault ResourceID>"
}
}
// Or:
"frontDoorCertificateSourceParameters": {
"certificateType": "Dedicated"
}
}
KeyVault Managed SSL Certificate Example
Note: I have tested this and appears to work.
{
"type": "Microsoft.Network/frontdoors",
"apiVersion": "2020-01-01",
"properties": {
"frontendEndpoints": [
{
"name": "[variables('frontendEndpointName')]",
"properties": {
"hostName": "[variables('customDomain')]",
"sessionAffinityEnabledState": "Enabled",
"sessionAffinityTtlSeconds": 0,
"webApplicationFirewallPolicyLink": {
"id": "[variables('wafPolicyResourceId')]"
},
"resourceState": "Enabled",
"customHttpsConfiguration": {
"certificateSource": "AzureKeyVault",
"minimumTlsVersion":"1.2",
"protocolType": "ServerNameIndication",
"keyVaultCertificateSourceParameters": {
"secretName": "[parameters('certKeyVaultSecret')]",
"secretVersion": "[parameters('certKeyVaultSecretVersion')]",
"vault": {
"id": "[resourceId(parameters('certKeyVaultResourceGroupName'),'Microsoft.KeyVault/vaults',parameters('certKeyVaultName'))]"
}
}
}
}
}
],
...
}
}
Front Door Managed SSL Certificate Example
Looks like for a FrontDoor managed certificate you would need to set:
Note: I have not tested this
{
"type": "Microsoft.Network/frontdoors",
"apiVersion": "2020-01-01",
"properties": {
"frontendEndpoints": [
{
"name": "[variables('frontendEndpointName')]",
"properties": {
"hostName": "[variables('customDomain')]",
"sessionAffinityEnabledState": "Enabled",
"sessionAffinityTtlSeconds": 0,
"webApplicationFirewallPolicyLink": {
"id": "[variables('wafPolicyResourceId')]"
},
"resourceState": "Enabled",
"customHttpsConfiguration": {
"certificateSource": "FrontDoor",
"minimumTlsVersion":"1.2",
"protocolType": "ServerNameIndication",
"frontDoorCertificateSourceParameters": {
"certificateType": "Dedicated"
}
}
}
}
],
...
}
}
I was able to successfully make an enableHttps REST Call using the Azure Management API.
I got a successful response and can see the resource results in the portal.azure.com and resource.azure.com sites.
However I am pretty sure the Management API, and PowerShell methods are the only ways supported right now. Since there is likely some validation required on the Certificate and Handling, they didn't include that yet in the ARM Templates. Given validation can be quite important, it is best you confirm your configuration is workable in the UI first, before automating it (IMHO).
According to this discussion this seems only possible via the REST API (see e.g. this answer) and not (yet) via ARM.
I managed to get this working with an ARM template. The below link shows you how to do this using Azure Front Door as a certificate source:
https://github.com/Azure/azure-quickstart-templates/blob/master/101-front-door-custom-domain/azuredeploy.json
I drew inspiration from this for deploying a certificate from Azure Key Vault for a custom domain. Here are the relevant elements from the ARM template that I am using:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"hubName": {
"type": "string",
"metadata": {
"description": "Name to assign to the hub. This name will prefix all resources contained in the hub."
}
},
"frontdoorName": {
"type": "string",
"metadata": {
"description": "Name to assign to the Frontdoor instance"
}
},
"frontdoorCustomDomain": {
"type": "string",
"metadata": {
"description": "The custom domain name to be applied to the provisioned Azure Frontdoor instance"
}
},
"keyVaultCertificateName": {
"type": "string",
"metadata": {
"description": "Name of the TLS certificate in the Azure KeyVault to be deployed to Azure Frontdoor for supporting TLS over a custom domain",
"assumptions": [
"Azure KeyVault containing the TLS certificate is deployed to the same resource group as the resource group where Azure Frontdoor will be deployed to",
"Azure KeyVault name is the hub name followed by '-keyvault' (refer to variable 'keyVaultName' in this template)"
]
}
},
...
},
"variables": {
"frontdoorName": "[concat(parameters('hubName'), '-', parameters('frontdoorName'))]",
"frontdoorEndpointName": "[concat(variables('frontdoorName'), '-azurefd-net')]",
"customDomainFrontdoorEndpointName": "[concat(variables('frontdoorName'), '-', replace(parameters('frontdoorCustomDomain'), '.', '-'))]",
"keyVaultName": "[concat(parameters('hubName'), '-keyvault')]",
"frontdoorHostName": "[concat(variables('frontdoorName'), '.azurefd.net')]",
...
},
"resources": [
{
"type": "Microsoft.Network/frontdoors",
"apiVersion": "2020-05-01",
"name": "[variables('frontdoorName')]",
"location": "Global",
"properties": {
"resourceState": "Enabled",
"backendPools": [...],
"healthProbeSettings": [...],
"frontendEndpoints": [
{
"id": "[concat(resourceId('Microsoft.Network/frontdoors', variables('frontdoorName')), concat('/FrontendEndpoints/', variables('frontdoorEndpointName')))]",
"name": "[variables('frontdoorEndpointName')]",
"properties": {
"hostName": "[variables('frontdoorHostName')]",
"sessionAffinityEnabledState": "Enabled",
"sessionAffinityTtlSeconds": 0,
"resourceState": "Enabled"
}
},
{
"id": "[concat(resourceId('Microsoft.Network/frontdoors', variables('frontdoorName')), concat('/FrontendEndpoints/', variables('customDomainFrontdoorEndpointName')))]",
"name": "[variables('customDomainFrontdoorEndpointName')]",
"properties": {
"hostName": "[parameters('frontdoorCustomDomain')]",
"sessionAffinityEnabledState": "Enabled",
"sessionAffinityTtlSeconds": 0,
"resourceState": "Enabled"
}
}
],
"loadBalancingSettings": [...],
"routingRules": [...],
"backendPoolsSettings": {
"enforceCertificateNameCheck": "Enabled",
"sendRecvTimeoutSeconds": 30
},
"enabledState": "Enabled",
"friendlyName": "[variables('frontdoorName')]"
}
},
{
"type": "Microsoft.Network/frontdoors/frontendEndpoints/customHttpsConfiguration",
"apiVersion": "2020-07-01",
"name": "[concat(variables('frontdoorName'), '/', variables('customDomainFrontdoorEndpointName'), '/default')]",
"dependsOn": [
"[resourceId('Microsoft.Network/frontdoors', variables('frontdoorName'))]"
],
"properties": {
"protocolType": "ServerNameIndication",
"certificateSource": "AzureKeyVault",
"minimumTlsVersion": "1.2",
"keyVaultCertificateSourceParameters": {
"secretName": "[parameters('keyVaultCertificateName')]",
"vault": {
"id": "[resourceId(resourceGroup().name, 'Microsoft.KeyVault/vaults', variables('keyVaultName'))]"
}
}
}
}
]
}
Azure Front Door classic now seems to support both managed certificates and custom certificates for custom domains. At least there are quickstart templates in the official repo from Microsoft exactly for these cases:
managed certificate
custom certificate
They both use Microsoft.Network/frontdoors/frontendEndpoints/customHttpsConfiguration subresource of the Front Door, currently with API version 2020-07-01. Only the parent subresource is documented in the templates reference, though.
The name of the customHttpsConfiguration resource is "default", so when the resource is specified as a top-level resource in the template, its complete name is something like "myfrontdoorafd/www-example-com/default".
Using Bicep (which transpiles to JSON ARM templates and which I highly recommend), the important part of the template looks like this:
param frontDoorName string
param customDomainName string
var frontEndEndpointCustomName = replace(customDomainName, '.', '-')
resource frontDoor 'Microsoft.Network/frontDoors#2020-01-01' = {
name: frontDoorName
properties: {
frontendEndpoints: [
{
name: frontEndEndpointCustomName
properties: {
hostName: customDomainName
...
}
}
...
]
...
}
...
resource frontendEndpoint 'frontendEndpoints' existing = {
name: frontEndEndpointCustomName
}
}
// This resource enables a Front Door-managed TLS certificate on the frontend.
resource customHttpsConfiguration 'Microsoft.Network/frontdoors/frontendEndpoints/customHttpsConfiguration#2020-07-01' = {
parent: frontDoor::frontendEndpoint
name: 'default'
properties: {
protocolType: 'ServerNameIndication'
certificateSource: 'FrontDoor'
frontDoorCertificateSourceParameters: {
certificateType: 'Dedicated'
}
minimumTlsVersion: '1.2'
}
}
Note that the deployment will be in progress till the certificate is actually issued and deployed to all points of presence (PoP) of Azure. This may take really long and even fail due to RequestTimeout. If you want to just start the operation and let it complete asynchronously, use e.g. the enable-https subcommand in Azure CLI. Even after the failure, the customHttpsProvisioningState is Pending and the certificate provisioning process may complete successfully.
Also note that when you have many frontend endpoints and changes happen frequently but most frontend endpoints stay unchanged, the pattern from this template cannot be generalized just by specifying multiple customHttpsConfiguration instances for multiple frontend endpoints. Such a generalization is not efficient and likely hits the rate limit of the underlying API (429 TooManyRequests) because the API is called even when the endpoint already has the HTTPS configuration.
In such a case, I was able to use nested templates and conditional deployment to deploy the customHttpsConfiguration subresource only when the frontend endpoint's property customHttpsProvisioningState has the value of Disabled. This works OK even with tens of frontend endpoints when a new frontend endpoint is added (and it should get a managed certificate). Even in deployment mode Complete, the once-applied configuration persists.