Setting up a azurerm_container_registry with terraform, I was wondering how I can change the permissions for certain users (e.g. ReadOnly), or perhaps to create a access_key which can be used from my CI-Pipeline, but does not require a user at all.
This Terraform configuration creates an ACR registry, and Azure Service Principal,
and grants the SP contributor access to the ACR registry. This can be updated to reader.
More information can be found on ACR auth with service principals here.
resource "azurerm_resource_group" "acr-rg" {
name = "acr-rg-007"
location = "West US"
}
resource "azurerm_container_registry" "acr" {
name = "acr00722"
resource_group_name = "${azurerm_resource_group.acr-rg.name}"
location = "${azurerm_resource_group.acr-rg.location}"
sku = "standard"
}
resource "azurerm_azuread_application" "acr-app" {
name = "acr-app"
}
resource "azurerm_azuread_service_principal" "acr-sp" {
application_id = "${azurerm_azuread_application.acr-app.application_id}"
}
resource "azurerm_azuread_service_principal_password" "acr-sp-pass" {
service_principal_id = "${azurerm_azuread_service_principal.acr-sp.id}"
value = "Password12"
end_date = "2020-01-01T01:02:03Z"
}
resource "azurerm_role_assignment" "acr-assignment" {
scope = "${azurerm_container_registry.acr.id}"
role_definition_name = "Contributor"
principal_id = "${azurerm_azuread_service_principal_password.acr-sp-pass.service_principal_id}"
}
output "docker" {
value = "docker login ${azurerm_container_registry.acr.login_server} -u ${azurerm_azuread_service_principal.acr-sp.application_id} -p ${azurerm_azuread_service_principal_password.acr-sp-pass.value}"
}
Related
I have setup a AKS cluster through terraform and in the same Resource group, I have create an Application gateway. As such
resource "azurerm_kubernetes_cluster" "cluster" {
name = var.aks_name
tags = var.tags
location = var.vnet_location
resource_group_name = local.aks_resource_group
dns_prefix = "k8s"
oidc_issuer_enabled = true
role_based_access_control_enabled = true
azure_policy_enabled = true
#workload_identity_enabled = true
default_node_pool {
name = var.node_pool_name
node_count = var.node_count
vm_size = var.node_type
os_disk_size_gb = var.os_disk_size_gb
vnet_subnet_id = local.aks_subnet_id
}
identity {
type = "SystemAssigned"
}
ingress_application_gateway {
gateway_id = var.application_gateway_id
}
...
During creation, Azure creates a MC_******* resource group for the managed cluster and I use a service principal to deploy all Terraform resources
The ingress controller for the application "ingress_application_gateway" acquires automatically a managed entity deployed inside the MC_**** resource group.
I need to access that managed entity in order to set some needed access policies on dependent resources.
Ingress controller need access to both the application gateway aswell as the original Resource group, as such;
#----------------------------------------------------------------
# Ingress controller running on AKS
#-----------------------------------------------------------------
data "azurerm_resource_group" "aks_resource_group" {
name = local.aks_resource_group
}
#Gateway ingress controller should have reader access on resource group
resource "azurerm_role_assignment" "igc_reader_access" {
scope = data.azurerm_resource_group.aks_resource_group.id
role_definition_name = "Reader"
principal_id = data.azurerm_user_assigned_identity.ingress_controller_identity.principal_id
}
#Gateway ingress controller should have contributor access on application gateway
resource "azurerm_role_assignment" "ag_contributor_access" {
scope = var.application_gateway_id
role_definition_name = "Contributor"
principal_id = data.azurerm_user_assigned_identity.ingress_controller_identity.principal_id
}
data "azurerm_user_assigned_identity" "ingress_controller_identity" {
name = "ingressapplicationgateway-qa"
resource_group_name = "MC_RG-Digital-Service-qa_westeurope"
}
To the question, how can I setup so my Service principal (that is used to create the azurerm_kubernetes_cluster resource) to have reader access on the MC_****** resource group created by azure for aks.
I dont want to give complete subscription contributor access to my Service Principal.
You could do smtg like this, when your execute Terraform as the Service Principal (We could reduce the data resources if the Application Gateway would be in the same Terraform project):
# get current client ( your Service principal)
data "azurerm_client_config" "current" {
}
# get MC Resource Group
data "azurerm_resource_group" "aks_mc_rg" {
depends_on = [azurerm_kubernetes_cluster.cluster]
name = azurerm_kubernetes_cluster.aks.node_resource_group
}
# get Application Gateway
data "azurerm_application_gateway" "aks_igc" {
name = "existing-app-gateway"
resource_group_name = "existing-resources"
}
# read Managed identity
data "azurerm_user_assigned_identity" "ingress_controller_identity" {
name = "ingressapplicationgateway-qa"
resource_group_name = data.azurerm_resource_group.aks_mc_rg.id
}
# assign reader to current client
resource "azurerm_role_assignment" "rg_aks_nodes_owners" {
scope = data.azurerm_resource_group.aks_mc_rg.id
role_definition_name = "Reader"
principal_id = data.azurerm_client_config.current.object_id
}
#Gateway ingress controller should have reader access on resource group
resource "azurerm_role_assignment" "igc_reader_access" {
scope = data.azurerm_resource_group.aks_mc_rg.id
role_definition_name = "Reader"
principal_id = data.azurerm_user_assigned_identity.ingress_controller_identity.principal_id
}
#Gateway ingress controller should have contributor access on application gateway
resource "azurerm_role_assignment" "ag_contributor_access" {
scope = data.azurerm_application_gateway.aks_igc.id
role_definition_name = "Contributor"
principal_id = data.azurerm_user_assigned_identity.ingress_controller_identity.principal_id
}
It seems my question is related to this post but since there is no answer I will ask again.
I have an Azure Devops project which I use to deploy static content into a container inside a Storage Account via Pipelines. I've recently decided to deploy my infrastructure using Terraform as well as my code but I'm running into an issue. I managed to create all my infrastructure with Terraform inside my Pipeline except for the Role Assignment.
I basically need to add a new Role Assignment to my Storage Account, through Azure it goes :
Go to my Storage Account
Go to Access Control (IAM)
Add a new Role Assignments
Select Storage Blob Data Contributor
Click on Select members
Select my Azure Devops Project
Review + assign
From what I understand in the Terraform documentation I should do something like this :
resource "azurerm_resource_group" "resource_group" {
name = var.resource_group_name
location = var.location
}
resource "azurerm_storage_account" "storage_account" {
name = var.storage_account_name
resource_group_name = azurerm_resource_group.resource_group.name
location = azurerm_resource_group.resource_group.location
account_tier = "Standard"
account_replication_type = "LRS"
}
resource "azurerm_role_assignment" "role_assignment" {
scope = azurerm_storage_account.storage_account.id
role_definition_id = "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/providers/Microsoft.Authorization/roleDefinitions/ba92f5b4-2d11-453d-a403-e96b0029c9fe" # Which is the Storage Blob Data Contributor role if I'm not mistaken.
principal_id = "yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy" # Which should be the Application ID ?
}
Except it doesn't work, when I try to run it in local without the Azure Pipeline to check if this works, the process is stuck in the "Still creating..." state for more than 10 minutes, which seems weird since when you do it manually it only takes up to a few seconds. I don't have any error I just end up canceling the command.
What am I missing / doing wrong here ?
I've found what was the issue. For the principal_id you need to put the Object_ID of your Service Principal and not your Application_ID. You end up with something like :
main.tf
...
locals {
sub = "/subscription"
permission_storage_blob_data_contributor = "providers/Microsoft.Authorization/roleDefinitions/ba92f5b4-2d11-453d-a403-e96b0029c9fe"
}
data "azurerm_subscription" "primary" { }
resource "azurerm_resource_group" "resource_group" {
name = var.resource_group_name
location = var.location
}
resource "azurerm_storage_account" "storage_account" {
name = var.storage_account_name
resource_group_name = azurerm_resource_group.resource_group.name
location = azurerm_resource_group.resource_group.location
account_tier = "Standard"
account_replication_type = "LRS"
}
resource "azurerm_role_assignment" "role_assignment" {
scope = azurerm_storage_account.storage_account.id
role_definition_id = join("/", [local.sub, data.azurerm_subscription.primary.subscription_id, local.permission_storage_blob_data_contributor])
principal_id = var.devops_project_object_id
}
...
variables.tf
...
variable "location" {
type = string
description = "Location for the deployment"
default = "West Europe"
}
variable "resource_group_name" {
type = string
description = "Resource Group Name"
}
variable "storage_account_name" {
type = string
description = "Storage Account Name"
}
# yyyyyyyy-yyyy-yyyy-yyyyyyyyyyyy format
variable "devops_project_object_id" {
type = string
description = "Object ID (principal_id) for the Devops Project linked to the Azure Subscription in the Azure Active Directory."
}
...
Role assignment can be simplified to this call:
resource "azurerm_role_assignment" "blob_contributor" {
scope = azurerm_storage_account.storage_account.id
role_definition_name = "Storage Blob Data Contributor"
principal_id = var.devops_project_object_id
}
Looking at terraform documentation I have trouble determining how to assign UAMI as kubelet_identity for aks cluster.
The identity { ... } block which sets controlPlane UAMI as described here is not what I am looking for.
The question is - is there a terraform way I can assign additional UAMI apart from the one in identity {..} block and use it to access ACR?
I want to set a separate UAMI as a kubelet identity as described here
is there a terraform way I can assign additional UAMI apart from the
one in identity {..} block and use it to access ACR?
According to the details you provided, you can create an additional UAMI and associate it with the AKS cluster kubelet identity, then assign the role to the UAMI, example code here:
resource "azurerm_kubernetes_cluster" "example" {
name = "example-aks1"
location = azurerm_resource_group.example.location
resource_group_name = azurerm_resource_group.example.name
dns_prefix = "exampleaks1"
default_node_pool {
name = "default"
node_count = 1
vm_size = "Standard_D2_v2"
}
identity {
type = "SystemAssigned"
}
kubelet_identity {
client-id = azurerm_user_assigned_identity.kubelet.client_id
object-id = azurerm_user_assigned_identity.kubelet.principal_id
user_assigned_identity_id = azurerm_user_assigned_identity.kubelet.id
}
...
}
resource "azurerm_role_assignment" "acr_for_kubelet" {
principal_id = azurerm_user_assigned_identity.kubelet.client_id
scope = azurerm_container_registry.container_registry.id
role_definition_name = "AcrPull"
}
Update:
Actually, when you create the AKS and enable the system-assigned managed identity, then it will create the two user-assigned identities for the AKS cluster, one is to access other resources, and one is to manage the AKS cluster itself and this one is the kubelet identity.
It doesn't make sense to assign the kubelet identity permission to access the ACR. What you need to do is to assign the AKS identity permission to access the ACR. Or use the secret and service account inside the Kubernetes to access the ACR.
You need something like this,
resource "azurerm_kubernetes_cluster" "kube_cluster" {
name = local.cluster_name
dns_prefix = local.cluster_name
location = var.location
resource_group_name = local.resource_group
default_node_pool {
name = "default"
node_count = 2
vm_size = "Standard_DS2_v2"
}
identity {
type = "UserAssigned"
user_assigned_identity_id = data.azurerm_user_assigned_identity.managed_identity.id
}
}
resource "azurerm_role_assignment" "acr_role_assignment" {
principal_id = azurerm_kubernetes_cluster.kube_cluster.kubelet_identity[0].object_id
scope = data.azurerm_container_registry.container_registry.id
role_definition_name = "AcrPull"
}
You can view the whole script here
I have the following terraform module to setup app services under the same plan:
provider "azurerm" {
}
variable "env" {
type = string
description = "The SDLC environment (qa, dev, prod, etc...)"
}
variable "appsvc_names" {
type = list(string)
description = "The names of the app services to create under the same app service plan"
}
locals {
location = "eastus2"
resource_group_name = "app505-dfpg-${var.env}-web-${local.location}"
acr_name = "app505dfpgnedeploycr88836"
}
resource "azurerm_app_service_plan" "asp" {
name = "${local.resource_group_name}-asp"
location = local.location
resource_group_name = local.resource_group_name
kind = "Linux"
reserved = true
sku {
tier = "Basic"
size = "B1"
}
}
resource "azurerm_app_service" "appsvc" {
for_each = toset(var.appsvc_names)
name = "${local.resource_group_name}-${each.value}-appsvc"
location = local.location
resource_group_name = local.resource_group_name
app_service_plan_id = azurerm_app_service_plan.asp.id
site_config {
linux_fx_version = "DOCKER|${local.acr_name}/${each.value}:latest"
}
app_settings = {
DOCKER_REGISTRY_SERVER_URL = "https://${local.acr_name}.azurecr.io"
}
}
output "hostnames" {
value = {
for appsvc in azurerm_app_service.appsvc: appsvc.name => appsvc.default_site_hostname
}
}
I am invoking it through the following configuration:
terraform {
backend "azurerm" {
}
}
locals {
appsvc_names = ["gateway"]
}
module "web" {
source = "../../modules/web"
env = "qa"
appsvc_names = local.appsvc_names
}
output "hostnames" {
description = "The hostnames of the created app services"
value = module.web.hostnames
}
The container registry has the images I need:
C:\> az acr login --name app505dfpgnedeploycr88836
Login Succeeded
C:\> az acr repository list --name app505dfpgnedeploycr88836
[
"gateway"
]
C:\> az acr repository show-tags --name app505dfpgnedeploycr88836 --repository gateway
[
"latest"
]
C:\>
When I apply the terraform configuration everything is created fine, but inspecting the created app service resource in Azure Portal reveals that its Container Settings show no docker image:
Now, I can manually switch to another ACR and then back to the one I want only to get this:
Cannot perform credential operations for /subscriptions/0f1c414a-a389-47df-aab8-a351876ecd47/resourceGroups/app505-dfpg-ne-deploy-eastus2/providers/Microsoft.ContainerRegistry/registries/app505dfpgnedeploycr88836 as admin user is disabled. Kindly enable admin user as per docs: https://learn.microsoft.com/en-us/azure/container-registry/container-registry-authentication#admin-account
This is confusing me. According to https://learn.microsoft.com/en-us/azure/container-registry/container-registry-authentication#admin-account the admin user should not be used and so my ACR does not have one. On the other hand, I understand that I need somehow configure the app service to authenticate with the ACR.
What is the right way to do it then?
So this is now possible since the v2.71 version of the Azure RM provider. A couple of things have to happen...
Assign a Managed Identity to the application (can also use User Assigned but a bit more work)
Set the site_config.acr_use_managed_identity_credentials property to true
Grant the application's identity ACRPull rights on the container.
Below is a modified version of the code above, not tested but should be ok
provider "azurerm" {
}
variable "env" {
type = string
description = "The SDLC environment (qa, dev, prod, etc...)"
}
variable "appsvc_names" {
type = list(string)
description = "The names of the app services to create under the same app service plan"
}
locals {
location = "eastus2"
resource_group_name = "app505-dfpg-${var.env}-web-${local.location}"
acr_name = "app505dfpgnedeploycr88836"
}
resource "azurerm_app_service_plan" "asp" {
name = "${local.resource_group_name}-asp"
location = local.location
resource_group_name = local.resource_group_name
kind = "Linux"
reserved = true
sku {
tier = "Basic"
size = "B1"
}
}
resource "azurerm_app_service" "appsvc" {
for_each = toset(var.appsvc_names)
name = "${local.resource_group_name}-${each.value}-appsvc"
location = local.location
resource_group_name = local.resource_group_name
app_service_plan_id = azurerm_app_service_plan.asp.id
site_config {
linux_fx_version = "DOCKER|${local.acr_name}/${each.value}:latest"
acr_use_managed_identity_credentials = true
}
app_settings = {
DOCKER_REGISTRY_SERVER_URL = "https://${local.acr_name}.azurecr.io"
}
identity {
type = "SystemAssigned"
}
}
data "azurerm_container_registry" "this" {
name = local.acr_name
resource_group_name = local.resource_group_name
}
resource "azurerm_role_assignment" "acr" {
for_each = azurerm_app_service.appsvc
role_definition_name = "AcrPull"
scope = azurerm_container_registry.this.id
principal_id = each.value.identity[0].principal_id
}
output "hostnames" {
value = {
for appsvc in azurerm_app_service.appsvc: appsvc.name => appsvc.default_site_hostname
}
}
EDITED 21 Dec 2021
The MS documentation issue regarding the value being reset by Azure has now been resolved and you can also control Managed Identity via the portal.
So you can use service principal auth with App Service, but you'd have to create service principal grant it ACRpull permissions over the registry and use service principal login\password in App Service site_config
DOCKER_REGISTRY_SERVER_USERNAME
DOCKER_REGISTRY_SERVER_PASSWORD
Trying to assign a service principal to an Azure container registry to no avail. The Azure docs state the following...
You can assign a service principal to your registry, and your application or service can use it for headless authentication
I wish to do this through terraform, but I do not see any sort of service principal option in the Azure provider azurerm_container_registry docs (managed)...
resource "azurerm_resource_group" "rg" {
name = "resourceGroup1"
location = "West US"
}
resource "azurerm_container_registry" "acr" {
name = "containerRegistry1"
resource_group_name = "${azurerm_resource_group.rg.name}"
location = "${azurerm_resource_group.rg.location}"
sku = "Premium"
admin_enabled = false
georeplication_locations = ["East US", "West Europe"]
}
I was hoping to find a service_principal stanza such as what is available in the azurerm_kubernetes_cluster resource...
resource "azurerm_kubernetes_cluster" "test" {
// [...]
service_principal {
client_id = "00000000-0000-0000-0000-000000000000"
client_secret = "00000000000000000000000000000000"
}
}
Am I thinking about this the wrong way? There seems to be a disconnect with a lot of the Azure resources on how service principals play a part. I assume evolving APIs are to blame, but this seems like a simple ask... Perhaps I'm totally off base here. Thoughts?
There is something different between the AKS and ACR with the Service Principal.
For AKS, it needs to create a secret for the pods or services to access the ACR and the secret is set in the yaml file. So it needs both client_id and client_secret at the same time.
But for ACR, you need a Service Principal with specific permission associated with it for others to access it. So you just need to assign the Service Principal to the ACR with the permission setting and client_id. The terraform code would like this:
resource "azurerm_resource_group" "rg" {
name = "resourceGroup1"
location = "West US"
}
resource "azurerm_container_registry" "acr" {
name = "containerRegistry1"
resource_group_name = "${azurerm_resource_group.rg.name}"
location = "${azurerm_resource_group.rg.location}"
sku = "Premium"
admin_enabled = false
georeplication_locations = ["East US", "West Europe"]
}
resource "azurerm_azuread_service_principal" "test" {
application_id = "${azurerm_azuread_application.test.application_id}"
}
resource "azurerm_azuread_service_principal_password" "test" {
service_principal_id = "${azurerm_azuread_service_principal.test.id}"
value = "VT=uSgbTanZhyz#%nL9Hpd+Tfay_MRV#"
end_date = "2020-01-01T01:02:03Z"
}
resource "azurerm_role_assignment" "test" {
scope = "${azurerm_container_registry.acr.id}"
role_definition_id = "Contributor"
principal_id = "${azurerm_azuread_service_principal_password.test.service_principal_id}"
}
For more details, see azurerm_role_assignment. Hope this will help you. If you need more help please let me know.