Pulumi and AKS : Public IP not linked to AKS load-balancer - azure

Question

trying, using pulumi, to setup and AKS cluster with a public ip as load-balancer and a domain name.
I tried several options using the following project : https://github.com/hostettler/pulumi-az-test
I end up with a working AKS cluster (great!) , that has a load-balancer (amazing!), on a public ip (fantastic!), but with an fqdn that is not linked to the public ip of the load-balancer (huh!).
I tried to
extract the public ip to force the domain but I did not manage to do it. I get the resource id of the public id created but I cannot cast it
as a public ip object to set the domain. Meaning how to load a public ip object from a public ip resource-id.
"/subscriptions/XXXXXX-XXX-XX-XXXx-XXXXXXXXXXXX/resourceGroups/MC_XXXXX_XXXXX_aksCluster5632b82_eastus/providers/Microsoft.Network/publicIPAddresses/XXXXX-XXXX-XXX-XXX-XXXXXXXXX"
tries to add a public ip to the cluster but that ip, while having a good fqdn is not used by the load-balancer
Any idea on how to address? Solving either would be ok with me.
Thanks a lot in advance

Answer 1

found the problem (or rather a work around). The problem stems from the helm chart I was using. The bitnami nginx chart seemed to ignored the controller.service.loadBalanceIP. Therefore, I replaced it by the ingress-nginx.
Thus, replacing
chart: "nginx",
version: "13.2.10",
fetchOpts: {
repo: "https://charts.bitnami.com/bitnami",
},
by
chart: "ingress-nginx",
version: "4.3.0",
fetchOpts: {
repo: "https://kubernetes.github.io/ingress-nginx",
},
solved the problem. Instead of recreating an external ip address for each deployment, it used the one provided in controller.service.loadBalanceIP that is static and with the proper A record.
import * as azure from "#pulumi/azure";
import * as k8s from "#pulumi/kubernetes";
import * as pulumi from "#pulumi/pulumi";
import * as config from "./config";
import * as azuread from "#pulumi/azuread";
const current = azuread.getClientConfig({});
// Create a Virtual Network for the cluster
const aksVnet = new azure.network.VirtualNetwork("aks-net", {
resourceGroupName: config.resourceGroup.name,
addressSpaces: ["10.2.0.0/16"],
});
// Create a Subnet for the cluster
const aksSubnetId = new azure.network.Subnet("aks-net", {
resourceGroupName: config.resourceGroup.name,
virtualNetworkName: aksVnet.name,
addressPrefixes: ["10.2.1.0/24"],
serviceEndpoints : ["Microsoft.Sql"],
},
{dependsOn: [aksVnet]}
);
// Now allocate an AKS cluster.
export const k8sCluster = new azure.containerservice.KubernetesCluster("aksCluster", {
resourceGroupName: config.resourceGroup.name,
location: config.location,
defaultNodePool: {
name: "aksagentpool",
nodeCount: config.nodeCount,
vmSize: config.nodeSize,
vnetSubnetId: aksSubnetId.id,
},
dnsPrefix: `${pulumi.getStack()}-reg-engine`,
linuxProfile: {
adminUsername: "aksuser",
sshKey: {
keyData: config.sshPublicKey,
},
},
networkProfile: {
networkPlugin : "azure",
loadBalancerProfile: {
managedOutboundIpCount: 1,
},
loadBalancerSku: "standard",
outboundType: "loadBalancer",
},
identity: {
type: "SystemAssigned",
},
},
{dependsOn: [aksSubnetId]}
);
const publicIp = new azure.network.PublicIp("app-engine-ip", {
resourceGroupName: k8sCluster.nodeResourceGroup,
allocationMethod: "Static",
domainNameLabel : "app-engine",
sku : "Standard",
tags: {
service: "kubernetes-api-loadbalancer",
},
},
{dependsOn: [k8sCluster]}
);
// Expose a K8s provider instance using our custom cluster instance.
export const k8sProvider = new k8s.Provider("aksK8s", {
kubeconfig: k8sCluster.kubeConfigRaw,
});
const clusterSvcsNamespace = new k8s.core.v1.Namespace(
config.namespace,
undefined,
{ provider: k8sProvider });
export const clusterSvcsNamespaceName = clusterSvcsNamespace.metadata.name;
const regEngineIngress = new k8s.helm.v3.Chart(
"sample-engine-ingress",
{
repo: "kubernetes.github.io",
chart: "ingress-nginx",
version: "4.3.0",
fetchOpts: {
repo: "https://kubernetes.github.io/ingress-nginx",
},
namespace: clusterSvcsNamespace.metadata.name,
values: {
controller: {
replicaCount: 2,
nodeSelector: {
"kubernetes\.io/os": "linux",
},
service: {
loadBalancerIP: publicIp.ipAddress,
}
}
},
},
{ provider: k8sProvider },
);
export let cluster = k8sCluster.name;
export let kubeConfig = k8sCluster.kubeConfigRaw;
export let k8sFqdn = k8sCluster.fqdn;
export let externalIp = publicIp.ipAddress;