Applying cert to ASE ILB via ARM template fails - azure

I am using an ARM template to deploy an ILB ASE which works fine but I am now trying to put a self-signed certificate on the ILB ASE as part of that deployment and it keeps failing with the error The specified network password is not correct.
I am actually applying the ARM template via Terraform. I store the certificate and the password in Key Vault. I pull the certificate from Key Vault using the PowerShell line below and then pass it into Terraform as a variable. This is the cert in Base64 encoded format:
$aseCertBase64 = (Get-AzureKeyVaultSecret -VaultName $kvName -Name $kvASECertName).SecretValueText
I first tried obtaining the Cert password using a Terraform data resource but that deployment failed with the The specified network password is not correct error. In an attempt to troubleshoot the password issue, I put the clear text password into the ARM template directly and re-ran the deployment. The deployment failed again with the same error so now I am not sure what it is looking for.
I validated that the password was correct by pulling the cert and password from Key Vault with PowerShell, converting the cert and then importing it into my local store successfully.
Below is the ARM Template I am using:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"ilbase_name": {
"type": "string",
"metadata": {
"description": "The name of the ILBASE"
}
},
"ilbase_domain_name": {
"type": "string",
"metadata": {
"description": "The prviate domain name inside ILBASE"
}
},
"ilbase_subnet_name": {
"type": "string",
"metadata": {
"description": "The name of the subnet assigned to the ILBASE"
}
},
"ilbase_rglocation": {
"defaultValue": "East US",
"type": "string",
"metadata": {
"description": "The region where the ILBASE will be deployed"
}
},
"vnet_name": {
"type": "string",
"metadata": {
"description": "The name of the vnet the ILBASE subnet is part of"
}
},
"vnet_rg": {
"type": "string",
"metadata": {
"description": "The name of the resource group the ILBASE vnet is in"
}
},
"vnet_id": {
"type": "string",
"metadata": {
"description": "The resource id of the ILBASE vnet"
}
},
"aseCert": {
"type": "string",
"metadata": {
"description": "The Base64 encoded string containing the cert to be applied to the ILBASE"
}
},
"aseCertPwd": {
"defaultValue": "XNKVTzCell637BNl",
"type": "string",
"metadata": {
"description": "The password for the ILBASE certificate"
}
},
"aseCertName": {
"defaultValue": "aseCert",
"type": "string",
"metadata": {
"description": "The password for the ILBASE certificate"
}
}
},
"resources": [
{
"apiVersion": "2015-08-01",
"type": "Microsoft.Web/certificates",
"name": "[parameters('aseCertName')]",
"location": "[parameters('ilbase_rglocation')]",
"properties": {
"pfxBlob": "[parameters('aseCert')]",
"password": "[parameters('aseCertPwd')]",
"hostingEnvironmentProfile": {
"id": "[resourceId('Microsoft.Web/hostingEnvironments',parameters('ilbase_name'))]"
}
},
"dependsOn": [
"[concat('Microsoft.Web/hostingEnvironments/',parameters('ilbase_name'))]"
]
},
{
"apiVersion": "2018-02-01",
"type": "Microsoft.Web/hostingEnvironments",
"name": "[parameters('ilbase_name')]",
"kind": "ASEV2",
"location": "[parameters('ilbase_rglocation')]",
"properties": {
"name": "[parameters('ilbase_name')]",
"location": "[parameters('ilbase_rglocation')]",
"vnetName": "[parameters('vnet_name')]",
"vnetResourceGroup": "[parameters('vnet_rg')]",
"vnetSubnetName": "[parameters('ilbase_subnet_name')]",
"virtualNetwork": {
"Id": "[parameters('vnet_id')]",
"Subnet": "[parameters('ilbase_subnet_name')]"
},
"dnsSuffix": "[parameters('ilbase_domain_name')]",
"internalLoadBalancingMode": "Web, Publishing",
"multiSize": "Medium",
"multiRoleCount": 2,
"ipsslAddressCount": 0,
"networkAccessControlList": [],
"frontEndScaleFactor": 15,
"suspended": false
}
}
]
}

Consider looking at calling the thumbPrint parameter for the cert. I believe that's required based on the default ARM template Microsoft have on GitHub here. Some further reference here at learn.microsoft.com.

Related

Azure ARM template give scoping access to different subscription

I have the following ARM template and I am trying to give scoping access to a subscription/resource group that is different from the subscription that I am currently deploying the ARM template. I read this documentation that talks about the scoping but I am not sure how to add the correct properties.
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Location for all resources."
}
},
"publicIpName": {
"type": "string"
},
"publicIpSku": {
"type": "string",
"defaultValue": "Standard"
},
"publicIPPrefixResourceId": {
"type": "string",
"metadata": {
"description": "Resource Id of the PublicIpPrefix to create VM VIP"
}
}
},
"resources": [
{
"apiVersion": "2019-02-01",
"type": "Microsoft.Network/publicIPAddresses",
"name": "[parameters('publicIpName')]",
"location": "[parameters('location')]",
"sku": {
"name": "[parameters('publicIpSku')]"
},
"properties": {
"publicIPAllocationMethod": "Static",
"publicIPPrefix": {
"Id": "[parameters('publicIPPrefixResourceId')]"
}
}
}
]
}
Below is the sample code where you can check properties related Arm Template here
ARM template
"parameters": {
"vnetAName": {
"type": "string",
"metadata": {
"description": "Name of the first VNET"
}
},
"vnetBName": {
"type": "string",
"metadata": {
"description": "Name of the Second VNET"
}
},
"vnetAPrefix": {
"type": "string",
"metadata": {
"description": "Prefix of the first VNET"
}
},
"vnetBPrefix": {
"type": "string",
"metadata": {
"description": "Prefix of the Second VNET"
}
},
"subscriptionAID": {
"type": "string",
"metadata": {
"description": "the Subscription ID for the first VNET"
}
},
"resourceGroupAName": {
"type": "string",
"metadata": {
"description": "the resource group name for the first VNET"
}
},
Go through this document for complete information.

How do I deploy this ARM template with Terraform? Please Advise

When I deploy this template via Terraform and Azure Devops, I get an Invalid template error while the template deploys normally on the portal. This is the error:
'The template resource '' of type 'microsoft.insights/workbooks' at
line '1' and column '1512' is not valid. The name property cannot be
null or empty. Please see https://aka.ms/arm-template/#resources for
usage details.'."
AdditionalInfo=[{"info":{"lineNumber":1,"linePosition":1512,"path":"properties.template.resources[0]"},"type":"TemplateViolation"}]
What modification should I make to deploy via Terraform?
{
"contentVersion": "1.0.0.0",
"parameters": {
"workbookDisplayName": {
"type": "string",
"defaultValue": "Azure Firewall Workbook",
"metadata": {
"description": "The friendly name for the workbook that is used in the Gallery or Saved List. This name must be unique within a resource group."
}
},
"workbookType": {
"type": "string",
"allowedValues": [
"workbook",
"sentinel"
],
"defaultValue": "workbook",
"metadata": {
"description": "The gallery that the workbook will been shown under. Supported values include workbook, tsg, etc. Usually, this is 'workbook'"
}
},
"DiagnosticsWorkspaceName": {
"type": "string",
"defaultValue": "WorkspaceName",
"metadata": {
"description": "Provide the workspace name for your Network Diagnostic logs"
}
},
"DiagnosticsWorkspaceSubscription": {
"type": "string",
"defaultValue": "WorkspaceSubscriptionID",
"metadata": {
"description": "Provide the workspace subscription GUID for your Network Diagnostic logs"
}
},
"DiagnosticsWorkspaceResourceGroup": {
"type": "string",
"defaultValue": "ResourceGroupName",
"metadata": {
"description": "Provide the workspace resourcegroupname for your Network Diagnostic logs"
}
},
"workbookId": {
"type": "string",
"defaultValue": "[newGuid()]",
"metadata": {
"description": "The unique guid for this workbook instance"
}
}
},
"variables": {
"workbookSourceId": "[concat('/subscriptions/',parameters('DiagnosticsWorkspaceSubscription'),'/resourcegroups/', parameters('DiagnosticsWorkspaceResourceGroup'), '/providers/Microsoft.OperationalInsights/workspaces/',parameters('DiagnosticsWorkspaceName'))]"
},
"resources": [
{
"name": "[parameters('workbookId')]",
"type": "microsoft.insights/workbooks",
"location": "[resourceGroup().location]",
"apiVersion": "2018-06-17-preview",
"dependsOn": [],
"kind": "shared",
"properties": {
"displayName": "[parameters('workbookDisplayName')]"}",
"version": "1.0",
"sourceId": "[variables('workbookSourceId')]",
"category": "[parameters('workbookType')]"
}
}
],
"outputs": {
"workbookId": {
"type": "string",
"value": "[resourceId( 'microsoft.insights/workbooks', parameters('workbookId'))]"
}
},
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#"
}
I don't know ARM templates but I have really good experience on Terraform AWS & Terraform Azure providers.
First of all, you better take a look Terraform resource page which is here. That would be helpful in understanding resource needs and outcomes.
I might be wrong, because your Terraform script is not visible in the question section. Nevertheless, I guess you might have an issue in Terraform side. As I understood, you are getting error from ARM templates. It is complaining about missing name parameter which is mandatory. You may forget passing parameter names from Terraform to ARM template. I might be wrong, this is just a suggestion, the correct way would be reviewing Terraform azurerm_template_deployment resource.
// ARM Template part
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"storageAccountType": {
"type": "string",
"defaultValue": "Standard_LRS",
"allowedValues": [
"Standard_LRS",
"Standard_GRS",
"Standard_ZRS"
],
"metadata": {
"description": "Storage Account type"
}
}
},
// Terraform resource provisioning
parameters = {
"storageAccountType" = "Standard_GRS"
}

How to manage Azure Api Management CA Certificates through ARM templates

I'm trying to manage CA certificates in Azure APIM through ARM but everything I tried gave no positive result.
For visualization, this is what I'm talking about:
When I look at the schema Microsoft.ApiManagement/service, there's a section for certificates where I can set the storeName variable but without results.
For sanity, I tried to upload it though Powershell plus manually and both option worked but that CA Certificate got wiped from the APIM at each deployment of my ARM template even if I used the "Incremental" option.
First I tried to modify the APIM ARM template by adding that block to the "properties" section:
"certificates": [
{
"encodedCertificate": "[parameters('RootCertificateBase64Content')]",
"certificatePassword": "[parameters('RootCertificatePassword')]",
"storeName": "Root"
}]
Here's my first test snippet for complete traceability:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"apimName": {
"type": "string",
"metadata": {
"description": "Name of the apimanagement"
}
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Location for all resources."
}
},
"sku": {
"type": "string",
"allowedValues": [
"Developer",
"Standard",
"Premium"
],
"defaultValue": "Developer",
"metadata": {
"description": "The pricing tier of this API Management service"
}
},
"skuCapacity": {
"type": "string",
"allowedValues": [
"1",
"2"
],
"defaultValue": "1",
"metadata": {
"description": "The instance size of this API Management service."
}
},
"subnetResourceId": {
"type": "string",
"metadata": {
"description": ""
}
},
"RootCertificateBase64Content": {
"type": "string",
"metadata": {
"description": "The Root certificate content"
}
},
"RootCertificatePassword": {
"type": "string",
"metadata": {
"description": "The Root certificate password"
}
}
},
"variables": {
"publisherEmail": "whatever#heyho.com",
"publisherName": "Whatever Team",
"notificationSenderEmail": "whatever#heygo.com"
},
"resources": [
{
"apiVersion": "2019-12-01",
"name": "[parameters('apimName')]",
"type": "Microsoft.ApiManagement/service",
"location": "[parameters('location')]",
"sku": {
"name": "[parameters('sku')]",
"capacity": "[parameters('skuCapacity')]"
},
"properties": {
"notificationSenderEmail": "[variables('notificationSenderEmail')]",
"publisherEmail": "[variables('publisherEmail')]",
"publisherName": "[variables('publisherName')]",
"virtualNetworkConfiguration": {
"subnetResourceId": "[parameters('subnetResourceId')]"
},
"virtualNetworkType": "Internal",
"certificates": [
{
"encodedCertificate": "[parameters('RootCertificateBase64Content')]",
"certificatePassword": "[parameters('RootCertificatePassword')]",
"storeName": "Root"
}]
},
"identity": {
"type": "SystemAssigned"
}
}
],
"outputs": {
"apiManagementPrivateHostIp": {
"type": "string",
"value": "[reference(concat(resourceId('Microsoft.ApiManagement/service', parameters('apimName')))).privateIPAddresses[0]]"
}
}
}
Second alternative I tried was to use the Microsoft.ApiManagement/service/certificates schema. There is no option there to specify the StoreName so I assumed it wasn't the right schema but I tried anyway. All attempts generated a certificate in the built-in Certificates store instead of the CA Certificates store.
Here's my second attempt's snippet:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"apimName": {
"type": "string",
"metadata": {
"description": "The parent APIM name"
}
},
"certificateName": {
"type": "string",
"metadata": {
"description": "The certificate name"
}
},
"CertificateBase64Content": {
"type": "string",
"metadata": {
"description": "The content of the certificate"
}
},
"CertificatePassword": {
"type": "string",
"metadata": {
"description": "The certificate password"
}
}
},
"resources": [
{
"name": "[concat(parameters('apimName'), '/Root/', parameters('certificateName'))]",
"type": "Microsoft.ApiManagement/service/certificates",
"apiVersion": "2019-01-01",
"properties": {
"data": "[parameters('CertificateBase64Content')]",
"password": "[parameters('CertificatePassword')]"
}
}
],
"outputs": {}
}
While looking at terraform documentation, it seems that it's possible to manage these certificates through the base schema and I confirmed that through the terraform azurerm provider source code (Unfortunately I cannot use Terraform and I MUST use ARM in that scenario).
Any clues on how to manage CA certificates in Azure APIM through ARM?
I assume you want to update CA certificate authority sections of already existing APIM? If yes then just provide all required properties for Microsoft.ApiManagement/service but for name use already existing APIM name that you want to update and choose the same resource group.
Thanks to this, it will just update existing APIM with properties you provided, instead of creating new APIM. The required properties are name, type, apiVersion, location, sku, properties. For properties you need to provide publisherEmail and publisherName, and of course certificates - this is what you want to update after all. So the absolute minimum for update will look like this:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters":{
"base64EncodedCertificate":{
"defaultValue":"base64 encoded certificate",
"type":"String"
},
"certificatePassword":{
"defaultValue":"certificate password",
"type":"String"
}
},
"variables": {},
"resources": [
{
"name": "existing-apim-name",
"type": "Microsoft.ApiManagement/service",
"apiVersion": "2021-01-01-preview",
"location": "West Europe",
"sku": {
"name": "Developer",
"capacity": 1
},
"properties": {
"publisherEmail": "publisher#gmail.com",
"publisherName": "Publisher Name",
"certificates": [
{
"encodedCertificate": "[parameters('base64EncodedCertificate')]",
"certificatePassword": "[parameters('certificatePassword')]",
"storeName": "Root"
}
]
}
}
]
}
Watch out. certificates array must contain all certificates that you want to have on this APIM. All existing CA certs that are not in this array will be deleted.

Deploy an AKS cluster inside an existing Vnet/sub-net using ARM templates

I have an existing Virtual Network created with two subnets: aks-subnet and persistence-subnet.
My goal is to create an Azure Kubernetes Cluster inside the aks-subnet
I am creating resource groups and resources at the subscription level, using the New-AzDeployment command from PowerShell core.
Like my idea is create a resource group and deploy resources to it, I have a nested template defining the resources to create in the resource group.
So I have the resource group created from the ARM template
please find this json definition in the entire template showed below at the end of this thread
"type": "Microsoft.Resources/resourceGroups",
And I am using a Deployment resource in order to nest the template that contains the resources that I want to create inside the resource group.
please find inmediately after:
"type": "Microsoft.Resources/deployments"
So inside, this Microsoft.Resources/deployments I am creating a Vnet with the two subnets previously mentioned,
and It works!, the Vnet and the subnet is created inside the resource group created in the same template.
Please find in the entire template showed below at the end of this thread
"type": "Microsoft.Network/virtualNetworks",
Now I want to add an Azure Kubernetes cluster inside the aks-vnet.
Please find in the entire template:
"dependsOn": [
"Microsoft.Network/virtualNetworks/AssessmentVNet"
],
"type": "Microsoft.ContainerService/managedClusters",
And then when I am associating the defaultpool (or virtual machine) to the aks-subnet,
Please find vnetSubnetID attribute of this way in the entire template showed below at the end of this thread:
"vnetSubnetID": "[resourceId(parameters('resourceGroupName'),'Microsoft.Network/virtualNetworks/subnets',parameters('vnetName'),parameters('subnet1Name'))]",
I am trying to access to the VnetSubnetID of this way, according to this AKS advanced networking official link suggest:
According to the immediately above I am doing here the following:
I got the resourceId from the resourceGroupName where is located the Vnet.
I am indicating the type of resource subnet Microsoft.Network/virtualNetworks/subnets
And I am passing like parameters the name of the Vnet which have the subnet and the name of the subnet as well parameters('vnetName'),parameters('subnet1Name'))
But when I execute the template from Power shell I got the following error:
PS /home/bgarcial/projects/my-project/Deployments/ARMTemplates/ResourceGroup> New-AzDeployment `
>> -Name SentiaAssessment `
>> -location westeurope `
>> -TemplateUri $templateUri `
>> -resourceGroupName $resourceGroupName `
>> -environmentName accp `
>> -dnsPrefix WordpressSentiaAssessment-dns `
>> -servicePrincipalClientId $servicePrincipalClientId `
>> -servicePrincipalClientSecret $servicePrincipalClientSecret
New-AzDeployment : 10:20:02 PM - Resource Microsoft.Resources/deployments 'storageDeployment' failed with message '{
"error": {
"code": "InvalidTemplate",
"message": "Unable to process template language expressions for resource '/subscriptions/9148bd11-f32b-4b5d-a6c0-5ac5317f29ca/resourceGroups/sentia-assessment/providers/Microsoft.Resources/deployments/storageDeployment' at line '150' and column '9'. 'The provided value 'sentia-assessment' is not valid subscription identifier. Please see https://aka.ms/arm-template-expressions/#resourceid for usage details.'",
"additionalInfo": [
{
"type": "TemplateViolation",
"info": {
"lineNumber": 150,
"linePosition": 9,
"path": ""
}
}
]
}
}'
At line:1 char:1
+ New-AzDeployment `
+ ~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [New-AzDeployment], Exception
+ FullyQualifiedErrorId : Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implementation.NewAzureDeploymentCmdlet
New-AzDeployment : 10:20:02 PM - Template output evaluation skipped: at least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-debug for usage details.
At line:1 char:1
+ New-AzDeployment `
+ ~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [New-AzDeployment], Exception
+ FullyQualifiedErrorId : Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implementation.NewAzureDeploymentCmdlet
New-AzDeployment : 10:20:02 PM - Template output evaluation skipped: at least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-debug for usage details.
At line:1 char:1
+ New-AzDeployment `
+ ~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [New-AzDeployment], Exception
+ FullyQualifiedErrorId : Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implementation.NewAzureDeploymentCmdlet
DeploymentName : MyDeployment
Location : westeurope
ProvisioningState : Failed
Timestamp : 10/23/19 8:19:57 PM
Mode : Incremental
TemplateLink :
Parameters :
Name Type Value
============================== ========================= ==========
resourceGroupName String sentia-assessment
location String West Europe
vnetName String AssessmentVNet
vnetAddressPrefix String 10.0.0.0/8
subnet1Prefix String 10.240.0.0/16
subnet1Name String aks-subnet
subnet2Prefix String 10.241.0.0/27
subnet2Name String persistence-subnet
k8s_cluster_name String WordpressSentiaAssessment
kubernetesVersion String 1.14.7
dnsPrefix String WordpressSentiaAssessment-dns
nodeCount Int 3
agentVMSize String Standard_D2_v2
servicePrincipalClientId SecureString
servicePrincipalClientSecret SecureString
serviceCidr String 100.0.0.0/16
dnsServiceIP String 100.0.0.10
dockerBridgeCidr String 172.17.0.1/16
environmentName String accp
Outputs :
DeploymentDebugLogLevel :
PS /home/bgarcial/projects/my-project/Deployments/ARMTemplates/ResourceGroup>
Looks like I would need to include the suscriptionId value inside the resourceId template function that I am using, but currently is not clear for me how to do it despite that I am querying the template reference
By the way, other detail is that I am using as a networkPlugin, kubenet.
Do I need to use the Azure CNI as a network plugin in order to get it?
This is the complete ARM template. I put it here in order to give an idea that what I am doing and how, and also to someone can find the json sections code definitions that I explain above.
I would be very grateful if someone can point me in the right direction
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"resourceGroupName": {
"type": "string"
},
"location": {
"type": "string",
"defaultValue": "West Europe",
"metadata": {
"description": "Geographic Location for all resources."
}
},
"vnetName": {
"type": "string",
"defaultValue": "AssessmentVNet",
"metadata": {
"description": "Sentia Wordpress Assesment Vnet"
}
},
"vnetAddressPrefix": {
"type": "string",
"defaultValue": "10.0.0.0/8",
"metadata": {
"description": "Address prefix"
}
},
"subnet1Prefix": {
"type": "string",
"defaultValue": "10.240.0.0/16",
"metadata": {
"description": "AKS Subnet"
}
},
"subnet1Name": {
"type": "string",
"defaultValue": "aks-subnet",
"metadata": {
"description": "aks-subnet"
}
},
"subnet2Prefix": {
"type": "string",
"defaultValue": "10.241.0.0/27",
"metadata": {
"description": "Persistence subnet"
}
},
"subnet2Name": {
"type": "string",
"defaultValue": "persistence-subnet",
"metadata": {
"description": "persistence-subnet"
}
},
"k8s_cluster_name":{
"type": "string",
"defaultValue": "WordpressSentiaAssessment",
"metadata": {
"description": "The name of the Azure Kubernetes Service Cluster"
}
},
"kubernetesVersion": {
"type": "string",
"defaultValue": "1.14.7",
"metadata": {
"description": "The version of the Azure Kubernetes Service Cluster"
}
},
"dnsPrefix": {
"type": "string",
"metadata": {
"description": "Optional DNS prefix to use with hosted Kubernetes API server FQDN."
}
},
"nodeCount": {
"type": "int",
"defaultValue": 3,
"metadata": {
"description": "The number of nodes that should be created along with the cluster."
},
"minValue": 1,
"maxValue": 100
},
"agentVMSize": {
"type": "string",
"defaultValue": "Standard_D2_v2",
"metadata": {
"description": "The size of the Virtual Machine."
}
},
"servicePrincipalClientId": {
"metadata": {
"description": "Client ID (used by cloudprovider)."
},
"type": "securestring"
},
"servicePrincipalClientSecret": {
"metadata": {
"description": "The Service Principal Client Secret."
},
"type": "securestring"
},
"serviceCidr": {
"type": "string",
"metadata": {
"description": "A CIDR notation IP range from which to assign service cluster IPs."
},
"defaultValue": "100.0.0.0/16"
},
"dnsServiceIP": {
"type": "string",
"metadata": {
"description": "Containers DNS server IP address."
},
"defaultValue": "100.0.0.10"
},
"dockerBridgeCidr": {
"type": "string",
"metadata": {
"description": "A CIDR notation IP for Docker bridge."
},
"defaultValue": "172.17.0.1/16"
},
"environmentName": {
"type": "string",
"metadata": {
"description": "Environment name for tagging purposes, e.g. dev, accp, prod"
}
}
},
"variables": {
"osDiskSizeGB": 0,
"osType": "Linux",
"maxPods": 110,
"networkPlugin": "kubenet"
},
"resources": [
{
"type": "Microsoft.Resources/resourceGroups",
"apiVersion": "2018-05-01",
"location": "[parameters('location')]",
"name": "[parameters('resourceGroupName')]",
"properties": {}
},
{
"type": "Microsoft.Resources/deployments",
"apiVersion": "2018-05-01",
"name": "storageDeployment",
"resourceGroup": "[parameters('resourceGroupName')]",
"dependsOn": [
"[resourceId('Microsoft.Resources/resourceGroups/', parameters('resourceGroupName'))]"
],
"properties": {
"mode": "Incremental",
"template": {
"$schema":"https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [
{
"apiVersion": "2018-10-01",
"type": "Microsoft.Network/virtualNetworks",
"name": "[parameters('vnetName')]",
"location": "[parameters('location')]",
"properties": {
"subnets": [
{
"name": "[parameters('subnet1Name')]",
"properties": {
"addressPrefix": "[parameters('subnet1Prefix')]"
}
},
{
"name": "[parameters('subnet2Name')]",
"properties": {
"addressPrefix": "[parameters('subnet2Prefix')]",
"serviceEndpoints": [
{
"service": "Microsoft.Storage",
"locations": "[parameters('location')]"
},
{
"service": "Microsoft.Sql",
"locations": "[parameters('location')]"
}
]
}
}
],
"addressSpace": {
"addressPrefixes": [
"[parameters('vnetAddressPrefix')]"
]
}
},
"tags": {
"Environment": "[parameters('environmentName')]"
}
},
{
"apiVersion": "2019-06-01",
"dependsOn": [
"Microsoft.Network/virtualNetworks/AssessmentVNet"
],
"type": "Microsoft.ContainerService/managedClusters",
"location":"[parameters('location')]",
"name": "[concat(parameters('k8s_cluster_name'), '-aks')]",
"properties":{
"kubernetesVersion":"[parameters('kubernetesVersion')]",
"enableRBAC": true,
"dnsPrefix": "[concat(parameters('k8s_cluster_name'),'-dns')]",
"agentPoolProfiles":[
{
"name":"defaultpool",
"osDiskSizeGB": "[variables('osDiskSizeGB')]",
"count":"[parameters('nodeCount')]",
"vmSize": "[parameters('agentVMSize')]",
"osType": "[variables('osType')]",
"storageProfile": "ManagedDisks",
"type": "VirtualMachineScaleSets",
"vnetSubnetID": "[resourceId(parameters('resourceGroupName'),'Microsoft.Network/virtualNetworks/subnets',parameters('vnetName'),parameters('subnet1Name'))]",
"maxPods": "[variables('maxPods')]"
}
],
"servicePrincipalProfile": {
"ClientId": "[parameters('servicePrincipalClientId')]",
"Secret": "[parameters('servicePrincipalClientSecret')]"
},
"networkProfile": {
"networkPlugin": "[variables('networkPlugin')]",
"serviceCidr": "[parameters('serviceCidr')]",
"dnsServiceIP": "[parameters('dnsServiceIP')]",
"dockerBridgeCidr": "[parameters('dockerBridgeCidr')]"
}
},
"tags": {
"Environment": "[parameters('environmentName')]"
}
},
{
}
]
}
}
}
]
}

Cannot change agent VM count

I have an ACS Kubernetes cluster that was created with an agent count of 1. I went to the portal to increase the agent count to 2 and received a generic error saying the provisioning of resource(s) for container service failed.
Looking at the activity logs, there is a bit more information.
Write ContainerServices - PreconditionFailed - Provisioning of resource(s) for container service 'xxxxxxx' in
resource group 'xxxxxxxx' failed.
Validate - InvalidTemplate - Deployment template validation failed: 'The resource 'Microsoft.Network/networkSecurityGroups/k8s-master-3E4D5818-nsg' is not defined in the template. Please see https://aka.ms/arm-template for usage details.'.
Trying to change it via the Azure CLI 2.0 also returns the same error.
Update: The cluster was stood up using an ARM template with a single container service resource based on the sample in the quickstart templates repo.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"dnsNamePrefix": {
"type": "string",
"metadata": {
"description": "Sets the Domain name prefix for the cluster. The concatenation of the domain name and the regionalized DNS zone make up the fully qualified domain name associated with the public IP address."
}
},
"agentCount": {
"type": "int",
"defaultValue": 1,
"metadata": {
"description": "The number of agents for the cluster. This value can be from 1 to 100 (note, for Kubernetes clusters you will also get 1 or 2 public agents in addition to these seleted masters)"
},
"minValue":1,
"maxValue":100
},
"agentVMSize": {
"type": "string",
"defaultValue": "Standard_D2_v2",
"allowedValues": [
"Standard_A0", "Standard_A1", "Standard_A2", "Standard_A3", "Standard_A4", "Standard_A5",
"Standard_A6", "Standard_A7", "Standard_A8", "Standard_A9", "Standard_A10", "Standard_A11",
"Standard_D1", "Standard_D2", "Standard_D3", "Standard_D4",
"Standard_D11", "Standard_D12", "Standard_D13", "Standard_D14",
"Standard_D1_v2", "Standard_D2_v2", "Standard_D3_v2", "Standard_D4_v2", "Standard_D5_v2",
"Standard_D11_v2", "Standard_D12_v2", "Standard_D13_v2", "Standard_D14_v2",
"Standard_G1", "Standard_G2", "Standard_G3", "Standard_G4", "Standard_G5",
"Standard_DS1", "Standard_DS2", "Standard_DS3", "Standard_DS4",
"Standard_DS11", "Standard_DS12", "Standard_DS13", "Standard_DS14",
"Standard_GS1", "Standard_GS2", "Standard_GS3", "Standard_GS4", "Standard_GS5"
],
"metadata": {
"description": "The size of the Virtual Machine."
}
},
"linuxAdminUsername": {
"type": "string",
"defaultValue": "azureuser",
"metadata": {
"description": "User name for the Linux Virtual Machines."
}
},
"orchestratorType": {
"type": "string",
"defaultValue": "Kubernetes",
"allowedValues": [
"Kubernetes",
"DCOS",
"Swarm"
],
"metadata": {
"description": "The type of orchestrator used to manage the applications on the cluster."
}
},
"masterCount": {
"type": "int",
"defaultValue": 1,
"allowedValues": [
1
],
"metadata": {
"description": "The number of Kubernetes masters for the cluster."
}
},
"sshRSAPublicKey": {
"type": "string",
"metadata": {
"description": "Configure all linux machines with the SSH RSA public key string. Your key should include three parts, for example 'ssh-rsa AAAAB...snip...UcyupgH azureuser#linuxvm'"
}
},
"servicePrincipalClientId": {
"metadata": {
"description": "Client ID (used by cloudprovider)"
},
"type": "securestring",
"defaultValue": "n/a"
},
"servicePrincipalClientSecret": {
"metadata": {
"description": "The Service Principal Client Secret."
},
"type": "securestring",
"defaultValue": "n/a"
}
},
"variables": {
"adminUsername":"[parameters('linuxAdminUsername')]",
"agentCount":"[parameters('agentCount')]",
"agentsEndpointDNSNamePrefix":"[concat(parameters('dnsNamePrefix'),'agents')]",
"agentVMSize":"[parameters('agentVMSize')]",
"masterCount":"[parameters('masterCount')]",
"mastersEndpointDNSNamePrefix":"[concat(parameters('dnsNamePrefix'),'mgmt')]",
"orchestratorType":"[parameters('orchestratorType')]",
"sshRSAPublicKey":"[parameters('sshRSAPublicKey')]",
"servicePrincipalClientId": "[parameters('servicePrincipalClientId')]",
"servicePrincipalClientSecret": "[parameters('servicePrincipalClientSecret')]",
"useServicePrincipalDictionary": {
"DCOS": 0,
"Swarm": 0,
"Kubernetes": 1
},
"useServicePrincipal": "[variables('useServicePrincipalDictionary')[variables('orchestratorType')]]",
"servicePrincipalFields": [
null,
{
"ClientId": "[parameters('servicePrincipalClientId')]",
"Secret": "[parameters('servicePrincipalClientSecret')]"
}
]
},
"resources": [
{
"apiVersion": "2016-09-30",
"type": "Microsoft.ContainerService/containerServices",
"location": "[resourceGroup().location]",
"name":"[resourceGroup().name]",
"properties": {
"orchestratorProfile": {
"orchestratorType": "[variables('orchestratorType')]"
},
"masterProfile": {
"count": "[variables('masterCount')]",
"dnsPrefix": "[variables('mastersEndpointDNSNamePrefix')]"
},
"agentPoolProfiles": [
{
"name": "agentpools",
"count": "[variables('agentCount')]",
"vmSize": "[variables('agentVMSize')]",
"dnsPrefix": "[variables('agentsEndpointDNSNamePrefix')]"
}
],
"linuxProfile": {
"adminUsername": "[variables('adminUsername')]",
"ssh": {
"publicKeys": [
{
"keyData": "[variables('sshRSAPublicKey')]"
}
]
}
},
"servicePrincipalProfile": "[variables('servicePrincipalFields')[variables('useServicePrincipal')]]"
}
}
],
"outputs": {
"masterFQDN": {
"type": "string",
"value": "[reference(concat('Microsoft.ContainerService/containerServices/', resourceGroup().name)).masterProfile.fqdn]"
},
"sshMaster0": {
"type": "string",
"value": "[concat('ssh ', variables('adminUsername'), '#', reference(concat('Microsoft.ContainerService/containerServices/', resourceGroup().name)).masterProfile.fqdn, ' -A -p 22')]"
},
"agentFQDN": {
"type": "string",
"value": "[reference(concat('Microsoft.ContainerService/containerServices/', resourceGroup().name)).agentPoolProfiles[0].fqdn]"
}
}
}
This is a known service issue for old clusters. A fix is currently rolling out and is being tracked in this github issue, https://github.com/Azure/ACS/issues/16
Jack (a dev on the ACS team)
I had test in my lab with this template, but I can't reproduce your error.
please try to use azure resource explorer to edit the count of agent pool:

Resources