I'm working with Azure Kubernetes Service and would like to manage my infrastructure using Terraform.
When you create a new AKS cluster in Azure, a separate resource group is created to manage the resources that the cluster depends on (e.g. virtual machine sets, load balancer, etc.)
This is no different when creating an AKS cluster using the Terraform azurerm_kubernetes_cluster resource.
However, I'd like to be able to work with the resources created in this resource group within Terraform. For example, when using the Application Gateway Ingress Controller, I'd like to be able to grab the public IP address that is created in this resource group so I can assign a DNS A record in my DNS zone. This is one such example, but the scope of my question includes any resources created in this AKS-managed resource group.
I have attempted to reference these as data resources in Terraform that depend on the creation of the AKS cluster, however this requires a role assignment to the new resource group, which my service principal will not have. Assigning the Terraform service principal to the entire subscription also feels like too much of a sledgehammer approach.
It seems I must be missing something, as this seems like a big flaw in the current approach with Terraform. Can anyone enlighten me to something I am missing?
Related
what is the difference between "resource group" and "container group" in ms azure?
I tried ms docs but could not find a clear distinction, seems like resource group is a single container while container group is a collection of containers but I am not sure if I got it right.
thank you!
The top-level resource in Azure Container Instances is the container group. It's a collection of containers (Docker containers) that get scheduled on the same host machine. The containers in a container group share a lifecycle, resources, local network, and storage volumes. It's similar in concept to a pod in Kubernetes.
https://learn.microsoft.com/en-us/azure/container-instances/container-instances-container-groups
A resource group is a logical folder (not a Docker container), that groups resources (other Azure services) for an Azure solution.
https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-portal
Resource group
A resource group is a container that holds related resources for an Azure solution.
The resource group can include all the resources for the solution, or only those resources that you want to manage as a group.
You decide how you want to allocate resources to resource groups based on what makes the most sense for your organization.
Generally, add resources that share the same lifecycle to the same resource group so you can easily deploy, update, and delete them as a group.
The resource group stores metadata about the resources. Therefore, when you specify a location for the resource group, you are specifying where that metadata is stored.
For compliance reasons, you may need to ensure that your data is stored in a particular region.
Resource: https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-portal
Container group
A set of containers that all run on the same host computer is known to be as a container group in Microsoft azure.
A container group's lifespan, resources, local network, and storage volumes are all shared by the containers in the group.
It's similar to a Kubernetes pod in terms of idea.
A Microsoft's Azure resource's network configuration template is a network profile.
This network profile is the one which provides the resource's network attributes, such as the subnet into which it should be deployed.
If we are going to deploy the container group into a subnet and then to a virtual network, then the azure produces a network profile for us when we use the az container create command.
Resource: https://www.javatpoint.com/what-is-a-container-group-in-microsoft-azure
Is there a way we can alter the resource names of the resources provisioned by AKS itself (screenshot below). I know I can change the node resource group name as per the documentation but cannot find any reference (or documentation) if we can change the AKS managed resource names. The resources for which I want to have custom naming specifically are:
Load balancer
AKS Virtual Machine Scale Set
You cannot change the resource names of the resources provisioned by AKS itself. Because it is managed by AKS only. you can give your own name of node resource group at the time of creation using IAC tool like Terraform, Biceps etc. But you can’t change the node resource name of AKS once created.
From Portal you can not assign your own name of node_resource_group
Note : node_resource_group - (Optional) The name of the Resource Group where the Kubernetes Nodes should exist. Changing this forces a new resource to be created.
Azure requires that a new, non-existent Resource Group is used, as otherwise the provisioning of the Kubernetes Service will fail.
Please Refer these document : azurerm_kubernetes_cluster | Resources | hashicorp/azurerm | Terraform Registry
Microsoft.ContainerService/managedClusters - Bicep & ARM template reference | Microsoft Docs
While creating aks cluster by using terraform and using existing values like resource group, vnet, subnet, network security group,
Deployment creating additional resource group, network security group, dns zone.
Do you have any idea how to stop that or disable?
Creating new RG with name MC_RG_XXXXXXXX
Good things:
Cluster getting created by using existing Vnet, subnet, cluster linked with existing RG but its creating network security Group and DNS Zones with new RG.
Any idea?
This is an automatic resource group created to house the cluster https://learn.microsoft.com/en-us/azure/aks/faq#why-are-two-resource-groups-created-with-aks , there is an option in preview to specify the name of this resource group using Azure CLI and within an ARM template https://learn.microsoft.com/en-us/azure/aks/faq#can-i-provide-my-own-name-for-the-aks-node-resource-group.
I'm not sure how this will map in Terraform though.
Some times we had deleted the 'resource group' accidentally on azure cloud, its fine that if our/your resource group does not have any deployments on it and we can create immediately newer one and will proceed our stuff, but if we had deployments on it, then its becomes major problem/task.
So can we recover Resource Group? like a recovery of Virtual machine in Azure.
You can export service configurations that belong to a resource group as JSON. Within the portal, there is an option "Export Template" under the Resource Management Navigation Group.
So can we recover Resource Group? like a recovery of Virtual machine in Azure.
Unfortunately, we have no way to recovery it, if the resource group is deleted . We could get the warning if we try to delete resource group from the azure portal. Connect to Azure support team for help maybe a way, but I don't think it could be easy for restoring.
Note : based on my understanding even if we could re-create the resource with ARM template(If we have exported the ARM template before delete), it seems that just with the same resource names with deleted resource that is not equal recovery action.
Is it possible to share a particular resource (redis cache in my case) across multiple resource group?
Resource group is just logical container. It doesn't matter in which resource group resource is. You can use resources from any resource group.
Example:
Imagine that you create App Service Plan in RG1 and Web App Service in RG2. This Web App Service can use App Service Plan from RG1.
Your redis cache could be used from any code you write, whether in the cloud, on-premises, or wherever, given proper uri + access key.
As long as you have access keys/passwords/etc. to your resources, you can use your resources no matter what resource group they're in. This includes ssh keys, vm usernames/passwords, redis cache keys, storage keys, cosmos db keys, sql database logins, etc.
The only thing access-specific, with resource groups, is granted user permissions. That is: if you add someone as a contributor to resource group A, and not to resource group B, they won't be able to manipulate the settings in resource group B via the portal or any of the Management APIs. Still, they'd be able to work with all of the resources in resource group B, assuming they had the login details / keys / etc.