I was wondering if someone could help me with setting up Vnet Peerings across subscriptions in Azure using Terraform. Each subscription is within the same tenant, but they have different service principals. I keep getting errors suggesting that the service principal cannot see the resource group in the other subscription. This is despite giving that service principal contributor access to the other subscription.
This is an example of the code I have:
resource "azurerm_virtual_network_peering" "dev-to-test" {
name = "dev-to-test"
resource_group_name = "gl-dev-rg"
virtual_network_name = "gl-dev-vnet"
remote_virtual_network_id = "/subscriptions/subscriptionid/resourceGroups/gl-test-rg/providers/Microsoft.Network/virtualNetworks/gl-test-vnet"
allow_virtual_network_access = true
allow_forwarded_traffic = true
}
resource "azurerm_virtual_network_peering" "test-to-dev" {
name = "test-to-dev"
resource_group_name = "gl-test-rg"
virtual_network_name = "gl-test-vnet"
remote_virtual_network_id = "/subscriptions/subscriptionid/resourceGroups/gl-dev-rg/providers/Microsoft.Network/virtualNetworks/gl-dev-vnet"
allow_virtual_network_access = true
allow_forwarded_traffic = true
}
Any help would be really appreciated!
Further information can be found here:
https://github.com/terraform-providers/terraform-provider-azurerm/issues/1253
The question was asked and answered in this issue. The TL;DR is terraform has an alias parameter for providers. This allows two separate service principals to reference different resources in a single terraform run.
Related
When i am trying to create cloud endpoint from terraform script in azure i am getting following error,
Error: waiting for creation of Storage Sync Cloud Endpoint: (Cloud Endpoint Name “azbackup001zscallerc-file-sync-grp-CE” / Sync Group Name “azbackup001zscallerc-file-sync-grp” / Storage Sync Service Name “azbackup001zscallerc-file-sync” / Resource Group “RG”): Code=“-2134364065” Message=“Unable to read specified storage account. Please check the permissions and try again after some time.”
however when i am creating the same from azure portal i am able to create without any issues. I have checked all my permissions and even from global admin account as well, i am unable to do so. Please assist the possible solution
Please assist on checking permission issue as i can do same thing from az cli as well as powershell.
As it is even having issues with global admin account,Check When creation of Cloud Endpoint setup permission to that storage sync service that cloud sync is dependent on.
See Storage Sync service errors : make sure Azure File Sync has access to the storage account.
resource "azurerm_storage_sync" "example" {
name = "kaexample-ss"
resource_group_name = data.azurerm_resource_group.example.name
location = data.azurerm_resource_group.example.location
}
resource "azurerm_storage_sync_group" "example" {
name = "kaexample-ss-group"
storage_sync_id = azurerm_storage_sync.example.id
}
resource "azurerm_storage_account" "example" {
name = "kaaexample"
resource_group_name = data.azurerm_resource_group.example.name
location = data.azurerm_resource_group.example.location
account_tier = "Standard"
account_replication_type = "LRS"
}
resource "azurerm_storage_share" "example" {
name = "kaexample-share"
storage_account_name = azurerm_storage_account.example.name
quota = 50
acl {
id = "GhostedRecall"
access_policy {
permissions = "r"
}
}
}
resource "azurerm_storage_sync_cloud_endpoint" "example" {
name = "example-ss-ce"
storage_sync_group_id = azurerm_storage_sync_group.example.id
file_share_name = azurerm_storage_share.example.name
storage_account_id = azurerm_storage_account.example.id
}
Please check this Az.StorageSync: Cloud endpoint creation access rigths failure issue · GitHub
Looking at terraform documentation I have trouble determining how to assign UAMI as kubelet_identity for aks cluster.
The identity { ... } block which sets controlPlane UAMI as described here is not what I am looking for.
The question is - is there a terraform way I can assign additional UAMI apart from the one in identity {..} block and use it to access ACR?
I want to set a separate UAMI as a kubelet identity as described here
is there a terraform way I can assign additional UAMI apart from the
one in identity {..} block and use it to access ACR?
According to the details you provided, you can create an additional UAMI and associate it with the AKS cluster kubelet identity, then assign the role to the UAMI, example code here:
resource "azurerm_kubernetes_cluster" "example" {
name = "example-aks1"
location = azurerm_resource_group.example.location
resource_group_name = azurerm_resource_group.example.name
dns_prefix = "exampleaks1"
default_node_pool {
name = "default"
node_count = 1
vm_size = "Standard_D2_v2"
}
identity {
type = "SystemAssigned"
}
kubelet_identity {
client-id = azurerm_user_assigned_identity.kubelet.client_id
object-id = azurerm_user_assigned_identity.kubelet.principal_id
user_assigned_identity_id = azurerm_user_assigned_identity.kubelet.id
}
...
}
resource "azurerm_role_assignment" "acr_for_kubelet" {
principal_id = azurerm_user_assigned_identity.kubelet.client_id
scope = azurerm_container_registry.container_registry.id
role_definition_name = "AcrPull"
}
Update:
Actually, when you create the AKS and enable the system-assigned managed identity, then it will create the two user-assigned identities for the AKS cluster, one is to access other resources, and one is to manage the AKS cluster itself and this one is the kubelet identity.
It doesn't make sense to assign the kubelet identity permission to access the ACR. What you need to do is to assign the AKS identity permission to access the ACR. Or use the secret and service account inside the Kubernetes to access the ACR.
You need something like this,
resource "azurerm_kubernetes_cluster" "kube_cluster" {
name = local.cluster_name
dns_prefix = local.cluster_name
location = var.location
resource_group_name = local.resource_group
default_node_pool {
name = "default"
node_count = 2
vm_size = "Standard_DS2_v2"
}
identity {
type = "UserAssigned"
user_assigned_identity_id = data.azurerm_user_assigned_identity.managed_identity.id
}
}
resource "azurerm_role_assignment" "acr_role_assignment" {
principal_id = azurerm_kubernetes_cluster.kube_cluster.kubelet_identity[0].object_id
scope = data.azurerm_container_registry.container_registry.id
role_definition_name = "AcrPull"
}
You can view the whole script here
i would like to deploy Azure landingzone using terraform in multiple subscriptions, Hub network should have azure firewall in subscription1 and each spoke have different subscriptions, i need 4 spokes which would be deployed in 4 separate subscriptions.
can some one help me with logic, how to write terraform.
For your requirements, here is the architecture that you can follow. The Hub and the spoke are connected via the VNet Peering. According to the description:
The virtual networks can be in the same, or different subscriptions.
When you peer virtual networks in different subscriptions, both
subscriptions can be associated to the same or different Azure Active
Directory tenant.
So you can peer VNets in two different subscriptions. I assume you use the Azure CLI as the authentication your account already login and has enough permission in both two subscriptions. Here is an example code:
provider "azurerm" {
features {}
alias = "subscription1"
subscription_id = "xxxxxxx"
}
provider "azurerm" {
features {}
alias = "subscription2"
subscription_id = "xxxxxxx"
}
data "azurerm_virtual_network" "remote" {
provider = azurerm.subscription1
name = "remote_vnet_name"
resource_group_name = "remote_group_name"
}
data "azurerm_virtual_network" "vnet" {
provider = azurerm.subscription2
name = "vnet_name"
resource_group_name = "group_name"
}
resource "azurerm_virtual_network_peering" "peering" {
provider = azurerm.subscription2
name = "${data.azurerm_virtual_network.vnet.name}-to-${data.azurerm_virtual_network.remote.name}"
resource_group_name = "group_name"
virtual_network_name = data.azurerm_virtual_network.vnet.name
remote_virtual_network_id = data.azurerm_virtual_network.remote.id
allow_virtual_network_access = true
allow_forwarded_traffic = true
# `allow_gateway_transit` must be set to false for vnet Global Peering
allow_gateway_transit = false
}
resource "azurerm_virtual_network_peering" "peering1" {
provider = azurerm.subscription1
name = "${data.azurerm_virtual_network.remote.name}-to-${data.azurerm_virtual_network.vnet.name}"
resource_group_name = "remote_group_name"
virtual_network_name = data.azurerm_virtual_network.remote.name
remote_virtual_network_id = data.azurerm_virtual_network.vnet.id
allow_virtual_network_access = true
allow_forwarded_traffic = true
# `allow_gateway_transit` must be set to false for vnet Global Peering
allow_gateway_transit = false
}
The VNet peering always comes with a pair. So you need to create a peering for each VNet that in the different subscriptions in a peering. This example just shows you how to create a peering for the two VNets in different subscriptions. Then you can complete the whole architecture as you wish in Terraform.
Trying to assign a service principal to an Azure container registry to no avail. The Azure docs state the following...
You can assign a service principal to your registry, and your application or service can use it for headless authentication
I wish to do this through terraform, but I do not see any sort of service principal option in the Azure provider azurerm_container_registry docs (managed)...
resource "azurerm_resource_group" "rg" {
name = "resourceGroup1"
location = "West US"
}
resource "azurerm_container_registry" "acr" {
name = "containerRegistry1"
resource_group_name = "${azurerm_resource_group.rg.name}"
location = "${azurerm_resource_group.rg.location}"
sku = "Premium"
admin_enabled = false
georeplication_locations = ["East US", "West Europe"]
}
I was hoping to find a service_principal stanza such as what is available in the azurerm_kubernetes_cluster resource...
resource "azurerm_kubernetes_cluster" "test" {
// [...]
service_principal {
client_id = "00000000-0000-0000-0000-000000000000"
client_secret = "00000000000000000000000000000000"
}
}
Am I thinking about this the wrong way? There seems to be a disconnect with a lot of the Azure resources on how service principals play a part. I assume evolving APIs are to blame, but this seems like a simple ask... Perhaps I'm totally off base here. Thoughts?
There is something different between the AKS and ACR with the Service Principal.
For AKS, it needs to create a secret for the pods or services to access the ACR and the secret is set in the yaml file. So it needs both client_id and client_secret at the same time.
But for ACR, you need a Service Principal with specific permission associated with it for others to access it. So you just need to assign the Service Principal to the ACR with the permission setting and client_id. The terraform code would like this:
resource "azurerm_resource_group" "rg" {
name = "resourceGroup1"
location = "West US"
}
resource "azurerm_container_registry" "acr" {
name = "containerRegistry1"
resource_group_name = "${azurerm_resource_group.rg.name}"
location = "${azurerm_resource_group.rg.location}"
sku = "Premium"
admin_enabled = false
georeplication_locations = ["East US", "West Europe"]
}
resource "azurerm_azuread_service_principal" "test" {
application_id = "${azurerm_azuread_application.test.application_id}"
}
resource "azurerm_azuread_service_principal_password" "test" {
service_principal_id = "${azurerm_azuread_service_principal.test.id}"
value = "VT=uSgbTanZhyz#%nL9Hpd+Tfay_MRV#"
end_date = "2020-01-01T01:02:03Z"
}
resource "azurerm_role_assignment" "test" {
scope = "${azurerm_container_registry.acr.id}"
role_definition_id = "Contributor"
principal_id = "${azurerm_azuread_service_principal_password.test.service_principal_id}"
}
For more details, see azurerm_role_assignment. Hope this will help you. If you need more help please let me know.
I'm trying to execute a sample terraform plan given below.
# Configure the Microsoft Azure Provider
provider "azurerm" {
subscription_id = "..."
client_id = "..."
client_secret = "..."
tenant_id = "..."
}
# Create a resource group
resource "azurerm_resource_group" "production" {
name = "production"
location = "West US"
}
# Create a virtual network in the web_servers resource group
resource "azurerm_virtual_network" "network" {
name = "productionNetwork"
address_space = ["10.0.0.0/16"]
location = "West US"
resource_group_name = "${azurerm_resource_group.production.name}"
subnet {
name = "subnet1"
address_prefix = "10.0.1.0/24"
}
subnet {
name = "subnet2"
address_prefix = "10.0.2.0/24"
}
subnet {
name = "subnet3"
address_prefix = "10.0.3.0/24"
}
}`enter code here`
I followed [1] to generate credentials via creating Active Directory application and used the correct subscription_id, client_id, client_secret, tenant_id in the above plan and executed 'terraform plan' against it. But I'm getting below error.
Error refreshing state: 1 error(s) occurred:
Credentials for acessing the Azure Resource Manager API are likely to be incorrect, or
the service principal does not have permission to use the Azure Service Management
API.
[1] https://azure.microsoft.com/en-us/documentation/articles/resource-group-create-service-principal-portal/
Any idea on this?
It seems like in terraform documentation, they haven't included the step of assigning role to the service principal. Follow these steps and it works.
1) Create the service principal through Azure CLI by following this link https://azure.microsoft.com/en-us/documentation/articles/resource-group-authenticate-service-principal-cli/ which assigns the role as well to the service principal
2) Go to Azure RM portal-->Active Directory -->App registration --> Create the key
3) Use the appropriate values from above in .tf file.
Then run the command terraform plan.