Related
Hopefully someone can help there.
I am trying to integrate Azure Application Gateway with Key Vault using ARM template and getting an issue:
SecretIdSpecifiedIsInvalid: SecretId '==' specified in '/subscriptions/xxxx-xxxx-xxxx-xxxx-xxxx/resourceGroups/rg-ProjectX-dev-infra/providers/Microsoft.Network/applicationGateways/appgw-ProjectX-dev/sslCertificates/appGwSslCert' is invalid. []
User assigned managed identity of Application Gateway has proper permissions ('Get' and 'List' under secrets and certificates) in Azure Key Vault.
Certificate is self-signed and generated in Azure Key Vault. It works as expected if I add certificate using Azure portal but it fails to add using ARM template.
The following guides were used during deployment:
TLS termination with Key Vault certificates and Pass sensitive values
Parameters.json file:
"app-gateway-httpsvaultCert": {
"reference": {
"keyVault": {
"id": "/subscriptions/xxxx-xxxx-xxxx-xxxx-xxxx/resourceGroups/rg-projeX-dev-infra/providers/Microsoft.KeyVault/vaults/kv-ProjectX-dev"
},
"secretName": "zzz-zzz-zzz-zzz"
}
},
Defined parameter as secure string in the template file:
"app-gateway-httpsvaultCert": {
"type": "securestring",
"metadata": {
"description": "Secure access string from Azure Application Gateway to Key Vault."
}
},
Template.json file:
{
"type": "Microsoft.Network/applicationGateways",
"apiVersion": "2020-11-01",
"name": "[variables('app-gateway-name')]",
"location": "[parameters('location')]",
"tags": "[parameters('resource-Tags')]",
"dependsOn": [
"[resourceId('Microsoft.Insights/components', variables('app-insights-name'))]",
"[resourceId('Microsoft.Network/publicIPAddresses', variables('public-ip-name'))]",
"[resourceId('Microsoft.Network/virtualNetworks', variables('vnet-name'))]",
"[resourceId('Microsoft.KeyVault/vaults', variables('kv-name'))]"
],
"identity": {
"type": "UserAssigned",
"userAssignedIdentities": {
"[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', variables('appgw-managed-id'))]": {
}
}
},
"properties": {
"sku": {
"name": "Standard_v2",
"tier": "Standard_v2",
"capacity": "[parameters('app-gateway-capacity')]"
},
"gatewayIPConfigurations": [
{
"name": "appGatewayIpConfig",
"properties": {
"subnet": {
"id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('vnet-name'), 'appgw-subnet')]"
}
}
}
],
"sslCertificates": [
{
"name": "appGwSslCert",
"properties": {
"keyVaultSecretId": "[parameters('app-gateway-httpsvaultCert')]"
}
}
],
"trustedRootCertificates": [],
"frontendIPConfigurations": [
{
"name": "appGwPublicFrontendIp",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"publicIPAddress": {
"id": "[resourceId('Microsoft.Network/publicIPAddresses', variables('public-ip-name'))]"
}
}
}
],
"frontendPorts": [
{
"name": "port_443",
"properties": {
"port": 443
}
}
],
"backendAddressPools": [
{
"name": "gatewayBackEnd",
"properties": {
"backendAddresses": [
{
"fqdn": "[concat(variables('apim-name'), '.azure-api.net')]"
},
{
"fqdn": "[concat(variables('fr-name'), '.cognitiveservices.azure.com')]"
}
]
}
}
],
"backendHttpSettingsCollection": [
{
"name": "global-gateway-https-setting",
"properties": {
"port": 443,
"protocol": "Https",
"cookieBasedAffinity": "Disabled",
"pickHostNameFromBackendAddress": true,
"requestTimeout": 20,
"probe": {
"id": "[resourceId('Microsoft.Network/applicationGateways/probes', variables('app-gateway-name'), 'global-gateway-probe')]"
}
}
}
],
"httpListeners": [
{
"name": "global-listener-https",
"properties": {
"frontendIPConfiguration": {
"id": "[resourceId('Microsoft.Network/applicationGateways/frontEndIPConfigurations', variables('app-gateway-name'), 'appGwPublicFrontendIp')]"
},
"frontendPort": {
"id": "[resourceId('Microsoft.Network/applicationGateways/frontEndPorts', variables('app-gateway-name'), 'port_443')]"
},
"protocol": "Https",
"sslCertificate": {
"id": "[resourceId('Microsoft.Network/applicationGateways/sslCertificates', variables('app-gateway-name'), 'appGwSslCert')]"
},
"hostNames": [],
"requireServerNameIndication": false
}
}
],
"urlPathMaps": [],
"requestRoutingRules": [
{
"name": "global-routing-rule",
"properties": {
"ruleType": "Basic",
"httpListener": {
"id": "[resourceId('Microsoft.Network/applicationGateways/httpListeners', variables('app-gateway-name'), 'global-listener-https')]"
},
"backendAddressPool": {
"id": "[resourceId('Microsoft.Network/applicationGateways/backendAddressPools', variables('app-gateway-name'), 'gatewayBackEnd')]"
},
"backendHttpSettings": {
"id": "[resourceId('Microsoft.Network/applicationGateways/backendHttpSettingsCollection', variables('app-gateway-name'), 'global-gateway-https-setting')]"
}
}
}
],
"probes": [
{
"name": "global-gateway-probe",
"properties": {
"protocol": "Https",
"port": 443,
"path": "/status-0123456789abcdef",
"interval": 30,
"timeout": 30,
"unhealthyThreshold": 3,
"pickHostNameFromBackendHttpSettings": true,
"minServers": 0
}
}
],
"rewriteRuleSets": [],
"redirectConfigurations": [],
"privateLinkConfigurations": [],
"sslPolicy": {
"policyType": "Predefined",
"policyName": "AppGwSslPolicy20170401S"
},
"enableHttp2": true
}
},
An issue has been fixed changing template.json and parameters.json files accordingly:
Template.json:
"app-gateway-httpsvaultCert": {
"type": "String",
"defaultValue": "https://[KeyVaultName].vault.azure.net/secrets/[CertName]",
"metadata": {
"description": "The base-64 encoded SSL certificate PFX data. Must be supplied via a parameters file references to a Key Vault / Secret Name."
}
}
Parameters.json:
"app-gateway-httpsvaultCert": {
"value":
"https://[KeyVaultName].vault.azure.net/secrets/[CertName]"
}
Getting error while deploying App gateway waf_v2 with more then one listener. For single listner it is working fine.
Error :
{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"BadRequest","message":"{\r\n "error": {\r\n "code": "ApplicationGatewayHttpListenersUsingSameFrontendPortAndFrontendIpConfig",\r\n "message": "Two Http Listeners of Application Gateway journal-listner and attachmentmgmt-listner are using the same Frontend Port /subscriptions/77585cb5-cc1b-47a6-b60d-4c1ec4b078fc/resourceGroups/drgr001appgatewaydevtest-au/providers/Microsoft.Network/applicationGateways/bo-appgw-waf-v2-entdev1/frontendPorts/frontendPorts and FrontendIpConfiguration /subscriptions/77585cb5-cc1b-47a6-b60d-4c1ec4b078fc/resourceGroups/drgr001appgatewaydevtest-au/providers/Microsoft.Network/applicationGateways/bo-appgw-waf-v2-entdev1/frontendIPConfigurations/PrivateFrontendIp.",\r\n "details": []\r\n }\r\n}"}]}
template code :
// Application Gateway
{
"name": "[parameters('applicationGatewayName')]",
"type": "Microsoft.Network/applicationGateways",
"apiVersion": "2020-03-01",
"location": "[parameters('location')]",
"zones": "[parameters('availabilityZones')]",
"properties": {
"sku": {
"name": "WAF_v2",
"tier": "WAF_v2",
"capacity": "[parameters('capacity')]"
},
"sslCertificates": [
{
"name": "[parameters('certName')]",
"properties": {
"data": "[parameters('certData')]",
"password": "[parameters('certPassword')]"
}
}
],
"gatewayIPConfigurations": [
{
"name": "gatewayIp",
"properties": {
"subnet": {
"id": "[variables('subnetRef')]"
}
}
}
],
"trustedRootCertificates": "[parameters('trustedRootCertificates')]",
"frontendIPConfigurations": [
{
"name": "PublicFrontendIp",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"publicIPAddress": {
"id": "[parameters('publicIpResourceId')]"
}
}
},
{
"name": "PrivateFrontendIp",
"properties": {
"privateIPAddress": "[parameters('privateIPAddress')]",
"privateIPAllocationMethod": "Static",
"subnet": {
"id": "[variables('subnetRef')]"
}
}
}
],
"frontendPorts": [
{
"name": "frontendPorts",
"properties": {
"Port": 443
}
}
],
"backendAddressPools": "[parameters('backendAddressPools')]",
"probes": "[parameters('probes')]",
"copy": [
{
"name": "backendHttpSettingsCollection",
"count": "[length(parameters('backendHttpSettings'))]",
"input": {
"name": "[parameters('backendHttpSettings')[copyIndex('backendHttpSettingsCollection')].name]",
"properties": {
"port": 443,
"pickHostNameFromBackendAddress": true,
"protocol": "Https",
"probeEnabled": "[parameters('backendHttpSettings')[copyIndex('backendHttpSettingsCollection')].probeEnabled]",
"probe": {
"id": "[resourceId('Microsoft.Network/applicationGateways/probes', parameters('applicationGatewayName'), parameters('backendHttpSettings')[copyIndex('backendHttpSettingsCollection')].probe)]"
}
}
}
},
{
"name": "httpListeners",
"count": "[length(parameters('httpListeners'))]",
"input": {
"name": "[parameters('httpListeners')[copyIndex('httpListeners')].name]",
"properties": {
"protocol": "Https",
// Set hostname if it exists
"hostName": "[if(contains(parameters('httpListeners')[copyIndex('httpListeners')], 'hostName'), parameters('httpListeners')[copyIndex('httpListeners')].hostName, '')]",
"sslCertificate": {
"id": "[concat(variables('applicationGatewayId'), '/sslCertificates/',parameters('httpListeners')[copyIndex('httpListeners')].sslCertificateName)]"
},
"frontendIPConfiguration": {
"id": "[concat(variables('applicationGatewayId'), '/frontendIPConfigurations/PrivateFrontendIp')]"
},
"frontendPort": {
"id": "[concat(variables('applicationGatewayId'), '/frontendPorts/frontendPorts')]"
}
}
}
},
{
"name": "requestRoutingRules",
"count": "[length(parameters('requestRoutingRules'))]",
"input": {
"name": "[parameters('requestRoutingRules')[copyIndex('requestRoutingRules')].name]",
"properties": {
"ruleType": "Basic",
"backendAddressPool": {
"id": "[concat(variables('applicationGatewayId'), '/backendAddressPools/',parameters('requestRoutingRules')[copyIndex('requestRoutingRules')].backendpoolName)]"
},
"backendHttpSettings": {
"id": "[concat(variables('applicationGatewayId'), '/backendHttpSettingsCollection/',parameters('requestRoutingRules')[copyIndex('requestRoutingRules')].backendHttpSetting)]"
},
"httpListener": {
"id": "[concat(variables('applicationGatewayId'), '/httpListeners/',parameters('requestRoutingRules')[copyIndex('requestRoutingRules')].httpListener)]"
}
}
}
}
],
"redirectConfigurations": "[parameters('redirectConfigurations')]",
"enableHttp2": "[parameters('enableHttp2')]",
"webApplicationFirewallConfiguration": "[variables('webApplicationFirewallConfiguration')]",
"urlPathMaps": "[parameters('urlPathMaps')]",
"authenticationCertificates": "[parameters('authenticationCertificates')]",
"sslPolicy": {
"policyType": "Predefined",
"policyName": "AppGwSslPolicy20170401S"
},
"rewriteRuleSets": "[parameters('rewriteRuleSets')]"
}
}
Parameter used :
"backendHttpSettings": {
"value": [
{
"name": "https-attachment",
"probeEnabled": true,
"probe": "attachment-probe"
},
{
"name": "https-journal",
"probeEnabled": true,
"probe": "journal-probe"
}
]
},
"backendAddressPools": {
"value": [
{
"name": "AttachmentServicePool",
"properties": {
"backendAddresses": [
{
"fqdn": "attachmentmgmt-svc-api-dev-euw.aseentdev.sys.dom"
}
]
}
},
{
"name": "journalServicePool",
"properties": {
"backendAddresses": [
{
"fqdn": "journalmgmt-svc-api-dev-euw.aseentdev.sys.dom"
}
]
}
}
]
},
"availabilityZones": {
"value": [
1,
2
]
},
"probes": {
"value": [
{
"name": "attachment-probe",
"properties": {
"protocol": "Https",
"PickHostNameFromBackendHttpSettings": true,
"path": "/index.htm",
"interval": 5,
"timeout": 10,
"match": {
"statusCodes": [
200
],
"body": "SUCCESS"
}
}
},
{
"name": "journal-probe",
"properties": {
"protocol": "Https",
"PickHostNameFromBackendHttpSettings": true,
"path": "/index.htm",
"interval": 5,
"timeout": 10,
"match": {
"statusCodes": [
200
],
"body": "SUCCESS"
}
}
}
]
},
"httpListeners": {
"value": [
{
"name": "attachmentmgmt-listner",
"sslCertificateName": "abc"
},
{
"name": "journal-listner",
"sslCertificateName": "abc"
}
]
},
"urlPathMaps": {
"value": []
},
"requestRoutingRules": {
"value": [
{
"name": "attachment-routing-rule",
"backendpoolName": "AttachmentServicePool",
"backendHttpSetting": "https-attachment",
"httpListener": "attachmentmgmt-listner"
},
{
"name": "journal-routing-rule",
"backendpoolName": "journalServicePool",
"backendHttpSetting": "https-journal",
"httpListener": "journal-listner"
}
]
},
You can't create two basic listeners on the same port. Either use different ports or two different hostnames with Multi site listener.
change in parameter file with passing host name for httplistner worked fine.
"httpListeners": {
"value": [
{
"name": "listner1",
"sslCertificateName": "ABC",
"hostName": "wb.abc.dom"
},
{
"name": "listner2",
"sslCertificateName": "ABC",
"hostName": "wb1.abc.dom"
}
]
}
I am trying to configure Azure Application Gateway with Basic Rule. For my Frontend IP, I have created set DNS name to whatever.canadacentral.cloudapp.azure.com and uploaded a self-signed certificate. When I hit https:// everything works correctly however when I go to https://whatever.canadacentral.cloudapp.azure.com it returns 307 redirecting me to my backend pool https://whatever.azurewebsites.net/
Is this something to do with canadacentral.cloudapp.azure.com and I need to provide custom DNS?
Here's my template for Application Gateway:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"applicationGateways_ExampleDev_name": {
"defaultValue": "ExampleDev",
"type": "String"
},
"virtualNetworks_Ex_DEV_externalid": {
"defaultValue": "/subscriptions/xxx/resourceGroups/Example-Ex-DEV/providers/Microsoft.Network/virtualNetworks/Ex-DEV",
"type": "String"
},
"publicIPAddresses_ExampleDevIP_externalid": {
"defaultValue": "/subscriptions/xxx/resourceGroups/Example-Ex-DEV/providers/Microsoft.Network/publicIPAddresses/ExampleDevIP",
"type": "String"
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.Network/applicationGateways",
"apiVersion": "2019-09-01",
"name": "[parameters('applicationGateways_ExampleDev_name')]",
"location": "canadacentral",
"properties": {
"sku": {
"name": "WAF_v2",
"tier": "WAF_v2"
},
"gatewayIPConfigurations": [
{
"name": "appGatewayIpConfig",
"properties": {
"subnet": {
"id": "[concat(parameters('virtualNetworks_Ex_DEV_externalid'), '/subnets/default')]"
}
}
}
],
"sslCertificates": [
{
"name": "ApplicationGateway",
"properties": {}
}
],
"trustedRootCertificates": [],
"frontendIPConfigurations": [
{
"name": "appGwPublicFrontendIp",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"publicIPAddress": {
"id": "[parameters('publicIPAddresses_ExampleDevIP_externalid')]"
}
}
}
],
"frontendPorts": [
{
"name": "port_80",
"properties": {
"port": 80
}
},
{
"name": "port_443",
"properties": {
"port": 443
}
}
],
"backendAddressPools": [
{
"name": "ExampleApiDev",
"properties": {
"backendAddresses": [
{
"fqdn": "Exampleapi-dev.azurewebsites.net"
}
]
}
},
{
"name": "ExampleAuthDev",
"properties": {
"backendAddresses": [
{
"fqdn": "Exampleauth-dev.azurewebsites.net"
}
]
}
},
{
"name": "ExampleAppDev",
"properties": {
"backendAddresses": [
{
"fqdn": "Exampleapp-dev.azurewebsites.net"
}
]
}
}
],
"backendHttpSettingsCollection": [
{
"name": "default",
"properties": {
"port": 80,
"protocol": "Http",
"cookieBasedAffinity": "Disabled",
"pickHostNameFromBackendAddress": true,
"affinityCookieName": "ApplicationGatewayAffinity",
"requestTimeout": 20,
"probe": {
"id": "[concat(resourceId('Microsoft.Network/applicationGateways', parameters('applicationGateways_ExampleDev_name')), '/probes/defaultxxx')]"
}
}
}
],
"httpListeners": [
{
"name": "public-https",
"properties": {
"frontendIPConfiguration": {
"id": "[concat(resourceId('Microsoft.Network/applicationGateways', parameters('applicationGateways_ExampleDev_name')), '/frontendIPConfigurations/appGwPublicFrontendIp')]"
},
"frontendPort": {
"id": "[concat(resourceId('Microsoft.Network/applicationGateways', parameters('applicationGateways_ExampleDev_name')), '/frontendPorts/port_443')]"
},
"protocol": "Https",
"sslCertificate": {
"id": "[concat(resourceId('Microsoft.Network/applicationGateways', parameters('applicationGateways_ExampleDev_name')), '/sslCertificates/ApplicationGateway')]"
},
"hostNames": [],
"requireServerNameIndication": false
}
}
],
"urlPathMaps": [],
"requestRoutingRules": [
{
"name": "basic",
"properties": {
"ruleType": "Basic",
"httpListener": {
"id": "[concat(resourceId('Microsoft.Network/applicationGateways', parameters('applicationGateways_ExampleDev_name')), '/httpListeners/public-https')]"
},
"backendAddressPool": {
"id": "[concat(resourceId('Microsoft.Network/applicationGateways', parameters('applicationGateways_ExampleDev_name')), '/backendAddressPools/ExampleApiDev')]"
},
"backendHttpSettings": {
"id": "[concat(resourceId('Microsoft.Network/applicationGateways', parameters('applicationGateways_ExampleDev_name')), '/backendHttpSettingsCollection/default')]"
}
}
}
],
"probes": [
{
"name": "default07a3e3ac-3c07-40f6-ad80-837f4cdd1009",
"properties": {
"protocol": "Http",
"path": "/swagger/index.html",
"interval": 30,
"timeout": 30,
"unhealthyThreshold": 3,
"pickHostNameFromBackendHttpSettings": true,
"minServers": 0,
"match": {
"statusCodes": [
"200-399"
]
}
}
}
],
"rewriteRuleSets": [],
"redirectConfigurations": [],
"webApplicationFirewallConfiguration": {
"enabled": true,
"firewallMode": "Prevention",
"ruleSetType": "OWASP",
"ruleSetVersion": "3.0",
"disabledRuleGroups": [],
"exclusions": [],
"requestBodyCheck": true,
"maxRequestBodySizeInKb": 128,
"fileUploadLimitInMb": 50
},
"enableHttp2": false,
"autoscaleConfiguration": {
"minCapacity": 0,
"maxCapacity": 2
}
}
}
]
}
In this case, for application gateway V2, you have two solutions from this document.
Rewrite the location header
Set the host name in the location header to the application gateway's
domain name. To do this, create a rewrite rule with a condition that
evaluates if the location header in the response contains
azurewebsites.net. It must also perform an action to rewrite the
location header to have the application gateway's host name.
Use a custom domain name
In this way, you must own a custom domain and add custom domain in app servvice, see Map an existing custom DNS name to Azure App Service. You could follow this process:
I've created a cluster with a master and 5 nodes with flannel for the POD network and that is working fine.
What is not working is that after installing kubeDNS (kubedns, dnsmasq and sidecar) but I can't get the new nameserver to be injected into HOST /etc/resolv.conf, because of that I can't resolve any hostnames.
Everything else works fine, all KubeDNS containers are running and with no errors
My kube-proxy ARGS
KUBE_PROXY_ARGS="--cluster-cidr=10.254.0.0/16"
My Kubelet configs
KUBELET_DNS="--cluster-dns=10.254.0.253"
KUBELET_DOMAIN="--cluster-domain=cluster.local"
Here are my configs for the DNS POD:
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "kube-dns-4073989832-f7g5g",
"generateName": "kube-dns-4073989832-",
"namespace": "kube-system",
"selfLink": "/api/v1/namespaces/kube-system/pods/kube-dns-4073989832-f7g5g",
"uid": "6f76055c-5b1e-11e7-b0c5-0050568fc023",
"resourceVersion": "3974782",
"creationTimestamp": "2017-06-27T09:53:13Z",
"labels": {
"k8s-app": "kube-dns",
"pod-template-hash": "4073989832"
},
"annotations": {
"kubernetes.io/created-by": "{\"kind\":\"SerializedReference\",\"apiVersion\":\"v1\",\"reference\":{\"kind\":\"ReplicaSet\",\"namespace\":\"kube-system\",\"name\":\"kube-dns-4073989832\",\"uid\":\"8afa7fce-5a9e-11e7-b714-0050568fc023\",\"apiVersion\":\"extensions\",\"resourceVersion\":\"3974404\"}}\n",
"scheduler.alpha.kubernetes.io/critical-pod": ""
},
"ownerReferences": [
{
"apiVersion": "extensions/v1beta1",
"kind": "ReplicaSet",
"name": "kube-dns-4073989832",
"uid": "8afa7fce-5a9e-11e7-b714-0050568fc023",
"controller": true
}
]
},
"spec": {
"volumes": [
{
"name": "kube-dns-config",
"configMap": {
"name": "kube-dns",
"defaultMode": 420
}
}
],
"containers": [
{
"name": "kubedns",
"image": "vvcelparti01:443/k8s-dns-kube-dns-amd64:1.14.2",
"args": [
"--domain=cluster.local",
"--dns-port=10053",
"--config-dir=/kube-dns-config",
"--kube-master-url=http://10.64.146.26:8080",
"--v=2"
],
"ports": [
{
"name": "dns-local",
"containerPort": 10053,
"protocol": "UDP"
},
{
"name": "dns-tcp-local",
"containerPort": 10053,
"protocol": "TCP"
},
{
"name": "metrics",
"containerPort": 10055,
"protocol": "TCP"
}
],
"env": [
{
"name": "PROMETHEUS_PORT",
"value": "10055"
}
],
"resources": {
"limits": {
"memory": "170Mi"
},
"requests": {
"cpu": "100m",
"memory": "70Mi"
}
},
"volumeMounts": [
{
"name": "kube-dns-config",
"mountPath": "/kube-dns-config"
}
],
"livenessProbe": {
"httpGet": {
"path": "/healthcheck/kubedns",
"port": 10054,
"scheme": "HTTP"
},
"initialDelaySeconds": 60,
"timeoutSeconds": 5,
"periodSeconds": 10,
"successThreshold": 1,
"failureThreshold": 5
},
"readinessProbe": {
"httpGet": {
"path": "/readiness",
"port": 8081,
"scheme": "HTTP"
},
"initialDelaySeconds": 3,
"timeoutSeconds": 5,
"periodSeconds": 10,
"successThreshold": 1,
"failureThreshold": 3
},
"terminationMessagePath": "/dev/termination-log",
"imagePullPolicy": "IfNotPresent"
},
{
"name": "dnsmasq",
"image": "vvcelparti01:443/k8s-dns-dnsmasq-nanny-amd64:1.14.2",
"args": [
"-v=2",
"-logtostderr",
"-configDir=/etc/k8s/dns/dnsmasq-nanny",
"-restartDnsmasq=true",
"--",
"-k",
"--cache-size=1000",
"--log-facility=-",
"--server=/cluster.local/127.0.0.1#10053",
"--server=/in-addr.arpa/127.0.0.1#10053",
"--server=/ip6.arpa/127.0.0.1#10053"
],
"ports": [
{
"name": "dns",
"containerPort": 53,
"protocol": "UDP"
},
{
"name": "dns-tcp",
"containerPort": 53,
"protocol": "TCP"
}
],
"resources": {
"requests": {
"cpu": "150m",
"memory": "20Mi"
}
},
"volumeMounts": [
{
"name": "kube-dns-config",
"mountPath": "/etc/k8s/dns/dnsmasq-nanny"
}
],
"livenessProbe": {
"httpGet": {
"path": "/healthcheck/dnsmasq",
"port": 10054,
"scheme": "HTTP"
},
"initialDelaySeconds": 60,
"timeoutSeconds": 5,
"periodSeconds": 10,
"successThreshold": 1,
"failureThreshold": 5
},
"terminationMessagePath": "/dev/termination-log",
"imagePullPolicy": "IfNotPresent"
},
{
"name": "sidecar",
"image": "vvcelparti01:443/k8s-dns-sidecar-amd64:1.14.2",
"args": [
"--v=2",
"--logtostderr",
"--probe=kubedns,127.0.0.1:10053,kubernetes.default.svc.cluster.local,5,A",
"--probe=dnsmasq,127.0.0.1:53,kubernetes.default.svc.cluster.local,5,A"
],
"ports": [
{
"name": "metrics",
"containerPort": 10054,
"protocol": "TCP"
}
],
"resources": {
"requests": {
"cpu": "10m",
"memory": "20Mi"
}
},
"livenessProbe": {
"httpGet": {
"path": "/metrics",
"port": 10054,
"scheme": "HTTP"
},
"initialDelaySeconds": 60,
"timeoutSeconds": 5,
"periodSeconds": 10,
"successThreshold": 1,
"failureThreshold": 5
},
"terminationMessagePath": "/dev/termination-log",
"imagePullPolicy": "IfNotPresent"
}
],
"restartPolicy": "Always",
"terminationGracePeriodSeconds": 30,
"dnsPolicy": "Default",
"serviceAccountName": "kube-dns",
"serviceAccount": "kube-dns",
"nodeName": "gopher01",
"securityContext": {}
},
"status": {
"phase": "Running",
"conditions": [
{
"type": "Initialized",
"status": "True",
"lastProbeTime": null,
"lastTransitionTime": "2017-06-27T09:52:45Z"
},
{
"type": "Ready",
"status": "True",
"lastProbeTime": null,
"lastTransitionTime": "2017-06-27T09:52:55Z"
},
{
"type": "PodScheduled",
"status": "True",
"lastProbeTime": null,
"lastTransitionTime": "2017-06-27T09:53:13Z"
}
],
"hostIP": "10.64.146.24",
"podIP": "172.30.18.4",
"startTime": "2017-06-27T09:52:45Z",
"containerStatuses": [
{
"name": "dnsmasq",
"state": {
"running": {
"startedAt": "2017-06-27T09:52:47Z"
}
},
"lastState": {},
"ready": true,
"restartCount": 0,
"image": "vvcelparti01:443/k8s-dns-dnsmasq-nanny-amd64:1.14.2",
"imageID": "docker-pullable://vvcelparti01:443/k8s-dns-dnsmasq-nanny-amd64#sha256:5a9dda0fdf5bf548eb6a63260c3f5e6f5cdc3d0917279e38a435c00967c6c57c",
"containerID": "docker://682fa7e0ffb28f26aee97a8ac7fe564096ece3ef3d7fe14fd9ed6857526d2d2f"
},
{
"name": "kubedns",
"state": {
"running": {
"startedAt": "2017-06-27T09:52:47Z"
}
},
"lastState": {},
"ready": true,
"restartCount": 0,
"image": "vvcelparti01:443/k8s-dns-kube-dns-amd64:1.14.2",
"imageID": "docker-pullable://vvcelparti01:443/k8s-dns-kube-dns-amd64#sha256:c78ed83587e42e7fc21f07756364c568c5c0fe10289f4f7f19d03a97f15b7a60",
"containerID": "docker://20b729004655a43efd384f8dded1f97d898a3b54092e190aba3d2031e72da056"
},
{
"name": "sidecar",
"state": {
"running": {
"startedAt": "2017-06-27T09:52:47Z"
}
},
"lastState": {},
"ready": true,
"restartCount": 0,
"image": "vvcelparti01:443/k8s-dns-sidecar-amd64:1.14.2",
"imageID": "docker-pullable://vvcelparti01:443/k8s-dns-sidecar-amd64#sha256:8d8c0e03e5f91ae85be7402ac88f804c52431dac32491c7a2557fd462fd2695b",
"containerID": "docker://bbaec6e9d0aa933daaee7c33b6d64d0f37f1a57213fabd2aa1c686c61a356f7f"
}
]
}
}
Here is my throubleshooting session:
$ kubectl get svc --namespace=kube-system
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns 10.254.0.253 <none> 53/UDP,53/TCP 24d
kubernetes-dashboard 10.254.170.86 <none> 80/TCP 29d
$ kubectl get ep kube-dns --namespace=kube-system
NAME ENDPOINTS AGE
kube-dns 172.30.18.4:53,172.30.18.4:53 24d
I have created template1 which will deploy HDI cluster and template2 which will deploy Azure VM seperately.
Now I want to get the Head-node Private IP from cluster and pass it to Azure VM template for processing using ARM template.
How can I do so?
Considering this is the object you are getting from HDcluster:
{
"id": "xxx",
"name": "xxx",
"type": "Microsoft.HDInsight/clusters",
"location": "East US",
"etag": "xxx",
"tags": null,
"properties": {
"clusterVersion": "3.5.1000.0",
"osType": "Linux",
"clusterDefinition": {
"blueprint": "https://blueprints.azurehdinsight.net/spark-3.5.1000.0.9865375.json",
"kind": "SPARK",
"componentVersion": {
"Spark": "1.6"
}
},
"computeProfile": {
"roles": [
{
"name": "headnode",
"targetInstanceCount": 2,
"hardwareProfile": {
"vmSize": "ExtraLarge"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "sshuser"
}
}
},
{
"name": "workernode",
"targetInstanceCount": 1,
"hardwareProfile": {
"vmSize": "Large"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "sshuser"
}
}
},
{
"name": "zookeepernode",
"targetInstanceCount": 3,
"hardwareProfile": {
"vmSize": "Medium"
},
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "sshuser"
}
}
}
]
},
"provisioningState": "Succeeded",
"clusterState": "Running",
"createdDate": "2017-04-11T09:07:44.68",
"quotaInfo": {
"coresUsed": 20
},
"connectivityEndpoints": [
{
"name": "SSH",
"protocol": "TCP",
"location": "xxx.azurehdinsight.net",
"port": 22
},
{
"name": "HTTPS",
"protocol": "TCP",
"location": "xxx.azurehdinsight.net",
"port": 443
}
],
"tier": "standard"
}
}
I'm guessing this is the best output you can get, so you can use something like:
"outputs": {
"test": {
"type": "Object",
"value": "[reference(parameters('clusterName'),'2015-03-01-preview').connectivityEndpoints[0].location]"
}
}
This will get you an output of xxx.azurehdinsight.net
And you can either create a new deployment with this data or (just like I said) add RHEL VM to the same template and make it dependOn on the HDCluster deployment and reference the same thing as an input to VMextension.