AKS has recently released support for node pools https://learn.microsoft.com/en-us/azure/aks/use-multiple-node-pools . Are node pools supported in ARM templates? If so what is the syntax for using them? I was not able to find any documentation about ARM template support online.
here's a working template example:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"type": "Microsoft.ContainerService/ManagedClusters",
"apiVersion": "2019-04-01",
"name": "aks-test",
"location": "eastus",
"properties": {
"kubernetesVersion": "1.13.5",
"dnsPrefix": "xxx",
"agentPoolProfiles": [
{
"name": "nodepool1",
"count": 1,
"vmSize": "Standard_DS2_v2",
"osDiskSizeGB": 100,
"storageProfile": "ManagedDisks",
"maxPods": 110,
"osType": "Linux",
"enable_auto_scaling": true,
"min_count": 1,
"max_count": 3,
"type": "VirtualMachineScaleSets"
},
{
"name": "nodepool2",
"count": 1,
"vmSize": "Standard_DS2_v2",
"osDiskSizeGB": 100,
"storageProfile": "ManagedDisks",
"maxPods": 110,
"osType": "Linux",
"enable_auto_scaling": true,
"min_count": 1,
"max_count": 3,
"type": "VirtualMachineScaleSets"
}
],
"linuxProfile": {
"adminUsername": "azureuser",
"ssh": {
"publicKeys": [
{
"keyData": "key"
}
]
}
},
"servicePrincipalProfile": {
"clientId": "yyy",
"secret": "zzz"
},
"enableRBAC": true,
"networkProfile": {
"networkPlugin": "kubenet",
"podCidr": "10.244.0.0/16",
"serviceCidr": "10.0.0.0/16",
"dnsServiceIP": "10.0.0.10",
"dockerBridgeCidr": "172.17.0.1/16"
}
}
}
]
}
you'd need to enable vmss preview before running this.
Unfortunately, I'm afraid you cannot use the Azure Template to create AKS with multiple node pools currently. In the document that you provide, you need to enable the VMSS to create AKS with multiple node pools. It's the agent type which you just can enable it in the CLI preview version for AKS. And you cannot find it in the template.
There is no difference in both templates for single node pool and multiple node pools when you create it except the elements in the property agentPoolProfiles:
"agentPoolProfiles": [
{
"name": "nodepool1",
"count": 1,
"vmSize": "Standard_DS2_v2",
"osDiskSizeGB": 100,
"storageProfile": "ManagedDisks",
"maxPods": 110,
"osType": "Linux"
},
{
"name": "secnodepool",
"count": 1,
"vmSize": "Standard_DS2_v2",
"osDiskSizeGB": 100,
"storageProfile": "ManagedDisks",
"maxPods": 110,
"osType": "Linux"
}
],
I think the multiple node pools will be available in the template when the it really publish, not the preview version. So you just need to wait patiently.
Update
Apologize for the above wrong answer. In the "2019-02-01" "apiVersion", you can already set the agent type as "VirtualMachineScaleSets" in the property "type" in "agentPoolProfiles". The mistake that I test it in "2018-03-31" "apiVersion".
Related
I am new to Azure. I have this existing VM. It was built by my colleague and I think he built it through the Marketplace with the Azure portal.
Now I want to build a new one with the same settings (that have the same performance spec) such as the VM sku, OS disk, and data disk. I don't want to keep any existing data. It will be built in a different RG with a different VNet and subnet. What is the best way to do it?
I tried to "export template" on the current VM but I think the JSON file just specifies the existing disks and NIC to use, instead of creating new ones. Here is what it looks like
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"virtualMachines_myCurrentVM_name": {
"defaultValue": "myCurrentVM",
"type": "String"
},
"disks_myCurrentVM_OsDisk_1_xxxxxx_externalid": {
"defaultValue": "/subscriptions/12345678-abcd-abcd-abcd-12345678/resourceGroups/nmtprdarmrgp001/providers/Microsoft.Compute/disks/myCurrentVM_OsDisk_1_xxxxxx",
"type": "String"
},
"disks_myCurrentVM_DataDisk_0_externalid": {
"defaultValue": "/subscriptions/12345678-abcd-abcd-abcd-12345678/resourceGroups/nmtprdarmrgp001/providers/Microsoft.Compute/disks/myCurrentVM_DataDisk_0",
"type": "String"
},
"networkInterfaces_myCurrentVM290_externalid": {
"defaultValue": "/subscriptions/12345678-abcd-abcd-abcd-12345678/resourceGroups/nmtprdarmrgp001/providers/Microsoft.Network/networkInterfaces/myCurrentVM290",
"type": "String"
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.Compute/virtualMachines",
"apiVersion": "2021-11-01",
"name": "[parameters('virtualMachines_myCurrentVM_name')]",
"location": "westus2",
"tags": {
"a": "1",
"b": "2"
},
"plan": {
"name": "f5-bigiq-virtual-edition-byol",
"product": "f5-big-iq",
"publisher": "f5-networks"
},
"properties": {
"hardwareProfile": {
"vmSize": "Standard_D4_v3"
},
"storageProfile": {
"imageReference": {
"publisher": "f5-networks",
"offer": "f5-big-iq",
"sku": "f5-bigiq-virtual-edition-byol",
"version": "latest"
},
"osDisk": {
"osType": "Linux",
"name": "[concat(parameters('virtualMachines_myCurrentVM_name'), '_OsDisk_1_xxxxxx')]",
"createOption": "FromImage",
"caching": "ReadWrite",
"managedDisk": {
"storageAccountType": "StandardSSD_LRS",
"id": "[parameters('disks_myCurrentVM_OsDisk_1_xxxxxx_externalid')]"
},
"deleteOption": "Detach",
"diskSizeGB": 120
},
"dataDisks": [
{
"lun": 0,
"name": "[concat(parameters('virtualMachines_myCurrentVM_name'), '_DataDisk_0')]",
"createOption": "Attach",
"caching": "ReadOnly",
"writeAcceleratorEnabled": false,
"managedDisk": {
"storageAccountType": "StandardSSD_LRS",
"id": "[parameters('disks_myCurrentVM_DataDisk_0_externalid')]"
},
"deleteOption": "Detach",
"diskSizeGB": 128,
"toBeDetached": false
}
]
},
"osProfile": {
"computerName": "[parameters('virtualMachines_myCurrentVM_name')]",
"adminUsername": "azureuser",
"linuxConfiguration": {
"disablePasswordAuthentication": true,
"ssh": {
"publicKeys": [
{
"path": "/home/azureuser/.ssh/authorized_keys",
"keyData": "ssh-rsa <some key here>"
}
]
},
"provisionVMAgent": true,
"patchSettings": {
"patchMode": "ImageDefault",
"assessmentMode": "ImageDefault"
}
},
"secrets": [],
"allowExtensionOperations": true,
"requireGuestProvisionSignal": true
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[parameters('networkInterfaces_myCurrentVM290_externalid')]"
}
]
},
"diagnosticsProfile": {
"bootDiagnostics": {
"enabled": true
}
}
}
}
]
}
Is it the best way to edit and modify the JSON file, or there is another way to do this? Thanks!
Thank you Matan Shabtay. Posting your suggestion as answer to help other community members.
From deployment Section (of the resource group where the VM is currently residing) you would get deployment history. Use that templet to replicate your your VM.
Select the resource group you want to examine.
Select the link under Deployments.
Select one of the deployments from the deployment history.
You can use the view templet option
Reference: https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/deployment-history?tabs=azure-portal
I'm creating AKS cluster using ARM script.
The goal is to create cluster from 0 with all needed configurations VNs, NGS and AgentPool.
Here is mine ManagedCluster ARM template:
{
"apiVersion": "2020-03-01",
"name": "[parameters('clusterName')]",
"type": "Microsoft.ContainerService/managedClusters",
"location": "[parameters('templateSettings').location]",
"properties": {
"kubernetesVersion": "[parameters('kubernetesVersion')]",
"enableRBAC": true,
"dnsPrefix": "[parameters('dnsPrefix')]",
"agentPoolProfiles": [
{
"name": "nodepool",
"count": "[parameters('agentCount')]",
"vmSize": "[parameters('agentVMSize')]",
"osDiskSizeGB": "[parameters('osDiskSizeGB')]",
"vnetSubnetID": "[parameters('vnetSubnetID')]",
"maxPods": 110,
"minCount": 1,
"maxCount": 100,
"enableAutoScaling": true,
"type": "VirtualMachineScaleSets",
"storageProfile": "ManagedDisks"
}
],
"servicePrincipalProfile": {
"ClientId": "[parameters('servicePrincipalClientId')]",
"Secret": "[parameters('servicePrincipalClientSecret')]"
},
"networkProfile": {
"loadBalancerSku": "standard",
"networkPlugin": "kubenet"
},
"addonProfiles": {
"httpApplicationRouting": {
"enabled": true
},
"omsagent": {
"enabled": true,
"config": {
"logAnalyticsWorkspaceResourceID": "[resourceId('Microsoft.OperationalInsights/workspaces', parameters('workspaceName'))]"
}
}
},
"nodeResourceGroup": "[concat(resourceGroup().name, '-nodes')]"
}
}
On deployment to Azure I receive this error which is not mentioned anywhere in docs:
{
"error": {
"code": "InvalidTemplateDeployment",
"message": "The template deployment 'AksDeployment' is not valid according to the validation procedure. The tracking id is 'b68b569d-**********************'. See inner errors for details.",
"details": [
{
"code": "MustDefineAtLeastOneSystemPool",
"message": "Provisioning of resource(s) for container service aks-arm-test in resource group rg-arm-test failed. Message: {\n \"code\": \"MustDefineAtLeastOneSystemPool\",\n \"message\": \"Must define at least one system pool.\"\n }. Details: "
}
]
}
}
actually, you can just mark it as a system:
{
"name": "nodepool",
"count": "[parameters('agentCount')]",
"vmSize": "[parameters('agentVMSize')]",
"osDiskSizeGB": "[parameters('osDiskSizeGB')]",
"vnetSubnetID": "[parameters('vnetSubnetID')]",
"maxPods": 110,
"minCount": 1,
"maxCount": 100,
"enableAutoScaling": true,
"type": "VirtualMachineScaleSets",
"storageProfile": "ManagedDisks",
"mode" : "System"
}
https://learn.microsoft.com/en-us/azure/aks/use-system-pools#show-details-for-your-node-pool
older api version should also work
There seems to be a az cli version to create a autoscaling kubernetes cluster in Azure AKS. https://learn.microsoft.com/en-us/azure/aks/cluster-autoscaler
Can this be done via Azure ARM templates too? If yes, can you please point me to some examples?
its not documented anywhere, as far as I can tell, reference here. working example:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"location": "eastus",
"name": "name",
"type": "Microsoft.ContainerService/ManagedClusters",
"apiVersion": "2019-04-01",
"properties": {
"kubernetesVersion": "1.13.5",
"dnsPrefix": "xxx",
"agentPoolProfiles": [
{
"name": "nodepool1",
"count": 1,
"vmSize": "Standard_DS2_v2",
"osDiskSizeGB": 100,
"storageProfile": "ManagedDisks",
"maxPods": 110,
"osType": "Linux",
"enable_auto_scaling": true,
"min_count": 1,
"max_count": 3,
"type": "VirtualMachineScaleSets"
}
],
"linuxProfile": {
"adminUsername": "azureuser",
"ssh": {
"publicKeys": [
{
"keyData": "xxx"
}
]
}
},
"servicePrincipalProfile": {
"clientId": "yyy",
"secret": "xxx"
},
"enableRBAC": true,
"networkProfile": {
"networkPlugin": "kubenet",
"podCidr": "10.244.0.0/16",
"serviceCidr": "10.0.0.0/16",
"dnsServiceIP": "10.0.0.10",
"dockerBridgeCidr": "172.17.0.1/16"
}
}
}
]
}
ps. its probably not supported.
I have problem when creating my custom template. I am trying to create VM from vhd as vhd but in deployment it fail with error osProfile missing. It is interesting because in other template from internet I see there is no osprofile section and it is deploying without problems.
So I added osProfile with computerName parameter. But now deployment failing with error there is adminUsername and adminPassword needed. I don´t understand how is possible that in another script this is not requied and it will create VM without problems.
There is also fact that my template is creating VM using vhd but that other template is creating VM using managed disk. Is this possibly problematic?
My piece of code:
"apiVersion": "2017-03-30",
"type": "Microsoft.Compute/virtualMachines",
"name": "[parameters('vmName')]",
"location": "[parameters('location')]",
"dependsOn": [
"[concat('Microsoft.Network/networkInterfaces/', variables('nicName'))]"
],
"properties": {
"hardwareProfile": {
"vmSize": "[parameters('vmSize')]"
},
"osProfile": {
"computerName": "[concat(parameters('vmName'))]",
"adminUsername": "",
"adminPassword": ""
},
"storageProfile": {
"osDisk": {
"name": "[concat(parameters('vmName'),'-osDisk')]",
"osType": "[parameters('osType')]",
"caching": "ReadWrite",
"image": {
"uri": "[parameters('osVhdUri')]"
},
"vhd": {
"uri": "[variables('osDiskVhdName')]"
},
"createOption": "FromImage"
}
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces',variables('nicName'))]"
}
]
},
"diagnosticsProfile": {
"bootDiagnostics": {
"enabled": true,
"storageUri": "[concat(reference(concat('Microsoft.Storage/storageAccounts/', parameters('userDiagStorageAccountName')), '2016-01-01').primaryEndpoints.blob)]"
}
}
I know adminUsername and adminPassword can´t be empty but I don´t want this parameters in creating VM from existing vhd.
Piece of template code from deployment successful:
"apiVersion": "2017-03-30",
"type": "Microsoft.Compute/virtualMachines",
"name": "[parameters('vmName')]",
"location": "[parameters('location')]",
"tags": {
"displayName": "VirtualMachine"
},
"dependsOn": [
"[concat('Microsoft.Network/networkInterfaces/', variables('nicName'))]",
"[concat(parameters('vmName'), '_OSdisk')]",
"[concat(parameters('vmName'), '_Datadisk')]"
],
"properties": {
"hardwareProfile": {
"vmSize": "[parameters('vmSize')]"
},
"storageProfile": {
"osDisk": {
"osType": "[parameters('osType')]",
"caching": "ReadWrite",
"createOption": "Attach",
"managedDisk": {
"id": "[resourceId('Microsoft.Compute/disks', concat(parameters('vmName'), '_OSdisk'))]"
}
},
"dataDisks": [
{
"lun": 0,
"managedDisk": {
"id": "[resourceId('Microsoft.Compute/disks', concat(parameters('vmName'), '_Datadisk'))]"
},
"caching": "ReadOnly",
"createOption": "Attach"
}
]
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces',variables('nicName'))]"
}
]
},
"diagnosticsProfile": {
"bootDiagnostics": {
"enabled": true,
"storageUri": "[concat(reference(concat('Microsoft.Storage/storageAccounts/', variables('diagStorageAccountName')), '2016-01-01').primaryEndpoints.blob)]"
}
}
Here is printscreen of failed deployment without osProfile form my template.
change "createOption": "FromImage" to "createOption": "Attach". You are trying to create a VM from marketplace image, not from existing VHD.
in this case you can remove osProfile completely
Below mentioned API is used to invoke all virtual machines :
https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Compute/virtualMachines?api-version=2015-06-15
In the response of virtual machines, virtual machine Id as mentioned here
"id": "/subscriptions/subscriptonId/resourceGroups/AGILITY/providers/Microsoft.Compute/virtualMachines/ProxyDontDelete10001",
in which resource group name is in Capital Letters (AGILITY) and if we invoke another rest api using this ID to get the instance view details it's not working.
/providers/Microsoft.Compute/virtualMachines/i-00000009/InstanceView
{
"value": [
{
"properties": {
"vmId": "7eb8dca3-dacf-4c51-b079-a508bf6d02b9",
"hardwareProfile": {
"vmSize": "Basic_A0"
},
"storageProfile": {
"osDisk": {
"osType": "Linux",
"name": "ProxyDontDelete10001",
"createOption": "FromImage",
"image": {
"uri": "https://blob.blob.core.windows.net/vhd/SM-RHEL6.7s-x64-9.2.r1664-20150801.vhd"
},
"vhd": {
"uri": "https://blob.blob.core.windows.net/vhds/ProxyDontDelete10001_ee751938-8d5c-468b-a36f-63e5332405cf.vhd"
},
"caching": "ReadWrite"
},
"dataDisks": [],
},
"osProfile": {
"computerName": "ProxyDontDelete10001",
"adminUsername": "admin",
"linuxConfiguration": {
"disablePasswordAuthentication": false
},
"secrets": [],
},
"networkProfile": {
"networkInterfaces": [
{
"id": "/subscriptions/{subscriptionId}/resourceGroups/test/providers/Microsoft.Network/networkInterfaces/testProxyDontDelete10001_ee751938-8d5c-468b-a36f-63e5332405cf"
}
],
},
"provisioningState": "Succeeded"
},
"type": "Microsoft.Compute/virtualMachines",
"location": "westus",
"id": "/subscriptions/subscriptonId/resourceGroups/AGILITY/providers/Microsoft.Compute/virtualMachines/ProxyDontDelete10001",
"name": "ProxyDontDelete10001"
},
Ok, I'm not sure I entirely understand the question, but resource group name is not case sensitive, so doing to:
/subscriptions/subscriptonId/resourceGroups/agility/providers/Microsoft.Compute/virtualMachines/ProxyDontDelete10001
should also work.