terraform nested for each loop in azure storage account - azure

I would want to create multiple storage acounts and inside each of those sotrage accounts some containers.
If I would want 3 storage account i would always want to create container-a and container-b in those 3 storage accounts
So for example would be. Storage account list ["sa1","sa2","sa3"].
resource "azurerm_storage_account" "storage_account" {
count = length(var.list)
name = var.name
resource_group_name = module.storage-account-resource-group.resource_group_name[0]
location = var.location
account_tier = var.account_tier
account_kind = var.account_kind
then container block
resource "azurerm_storage_container" "container" {
depends_on = [azurerm_storage_account.storage_account]
count = length(var.containers)
name = var.containers[count.index].name
container_access_type = var.containers[count.index].access_type
storage_account_name = azurerm_storage_account.storage_account[0].name
container variables:
variable "containers" {
type = list(object({
name = string
access_type = string
}))
default = []
description = "List of storage account containers."
}
list variable
variable "list" {
type = list(string)
description = "the env to deploy. ['dev','qa','prod']"
This code will create only one container in the first storage account "sa1" but not in the others two "sa2" and "sa3". I read I need to use 2 for each to iterate in both list of storage account and continaers, but not sure how should be the code for it.

It would be better to use for_each:
resource "azurerm_storage_account" "storage_account" {
for_each = toset(var.list)
name = var.name
resource_group_name = module.storage-account-resource-group.resource_group_name[0]
location = var.location
account_tier = var.account_tier
account_kind = var.account_kind
}
then you need an equivalent of a double for loop, which you can get using setproduct:
locals {
flat_list = setproduct(var.list, var.containers)
}
and then you use local.flat_list for containers:
resource "azurerm_storage_container" "container" {
for_each = {for idx, val in local.flat_list: idx => val}
name = each.value.name[1].name
container_access_type = each.value.name[1].access_type
storage_account_name = azurerm_storage_account.storage_account[each.value[0]].name
}
p.s. I haven't run the code, thus it may require some adjustments, but the idea remains valid.

Related

Terraform Azurerm: Create blob if not exists

I got Terrafrom code that creates storage account, container and block blob. Is it possible to configure that block blob is created only if it doesn't already exist?
In case of re-running terraform I wouldn't like to replace blob if it is already there as the content might have been manually modified and i would like to keep it.
Any tips? Only alternative I could think of is running powershell/bash script during further deployment steps that would create file if needed, but I am curious if this can be done just with Terraform.
locals {
storage_account_name_teast = format("%s%s", local.main_pw_prefix_short, "teast")
}
resource "azurerm_storage_account" "teaststorage" {
name = local.storage_account_name_teast
resource_group_name = azurerm_resource_group.main.name
location = var.location
account_tier = var.account_tier
account_replication_type = var.account_replication_type
allow_nested_items_to_be_public = false
min_tls_version = "TLS1_2"
network_rules {
default_action = "Deny"
bypass = [
"AzureServices"
]
virtual_network_subnet_ids = []
ip_rules = local.ip_rules
}
tags = var.tags
}
resource "azurerm_storage_container" "teastconfig" {
name = "config"
storage_account_name = azurerm_storage_account.teaststorage.name
container_access_type = "private"
}
resource "azurerm_storage_blob" "teastfeaturetoggle" {
name = "featureToggles.json"
storage_account_name = azurerm_storage_account.teaststorage.name
storage_container_name = azurerm_storage_container.teastconfig.name
type = "Block"
source = "vars-pr-default-toggles.json"
}
After scanning through terraform plan I figured out it was forcing a blob replacement because of:
content_md5 = "9a95db04fb1ff3abcd7ff81fcfb96307" -> null # forces replacement
I added lifecycle hook to blob resource to prevent it:
resource "azurerm_storage_blob" "teastfeaturetoggle" {
name = "featureToggles.json"
storage_account_name = azurerm_storage_account.teaststorage.name
storage_container_name = azurerm_storage_container.teastconfig.name
type = "Block"
source = "vars-pr-default-toggles.json"
lifecycle {
ignore_changes = [
content_md5,
]
}
}

Accessing specific storage account id created using terraform module

I am creating an infrastructure with terraform modules. Some of the common and repeatitive infra are created using module
and other resources are created independently outside of the module. The structure of my code is described as below.
-terraform\module\storage.tf
-terraform\main.tf
-terraform\mlws.tf
This is my code for /module/storage.tf where I am createing a storage account like this
resource "azurerm_storage_account" "storage" {
name = var.storage_account_name
resource_group_name = var.rg_name
location = var.location
account_tier = "Standard"
account_replication_type = "GRS"
min_tls_version = "TLS1_2"
}
module "m1" {
source = "./modules"
storage_account_name = "m1storage"
rg_name = "rg1"
location = "USCentral"
}
module "m2" {
source = "./modules"
storage_account_name = "m2storage"
rg_name = "rg2"
location = "USCentral"
}
module "m3" {
source = "./modules"
storage_account_name = "m3storage"
rg_name = "rg3"
location = "USCentral"
}
resource "azurerm_machine_learning_workspace" "mlws" {
name = "mlws"
location = ""USCentral"
resource_group_name = "mlws-rg1"
application_insights_id = azurerm_application_insights.mlops_appins.id
key_vault_id = data.azurerm_key_vault.kv.id
storage_account_id = **<Mandatory to be filled>**
container_registry_id = azurerm_container_registry.acr.id
identity {
type = "SystemAssigned"
}
depends_on = [
module.m2
]
}
The code for storage account is under \terraform\module\storage.tf, The code for calling the module is under \terraform\main.tf, The code for machine learning workspace is under \terraform\mlws.tf.
Since my mlws.tf code is outside the module but it need to be associated with storage account id created under module m2 in above code.
I am struggling to fetch the id of "m2storage" storage account. Can you please provide solution on how can I access the id of specific storage account created through module and attach it with my code which is outside the module.
This is how it normally works. You run module m2 and it should give output something like this (should include storage_account_id):
output "storage_account_id" {
description = "M2 storage account id."
value = m2.storage_account.storage_account_id
}
Now you have the output and you want to use it you will refer to it as:
resource "azurerm_machine_learning_workspace" "mlws" {
name = "mlws"
location = ""USCentral"
resource_group_name = "mlws-rg1"
application_insights_id = azurerm_application_insights.mlops_appins.id
key_vault_id = data.azurerm_key_vault.kv.id
storage_account_id = module.m2.storage_account_id
container_registry_id = azurerm_container_registry.acr.id
identity {
type = "SystemAssigned"
}
depends_on = [
module.m2
]
}
Let me know if you need more help.

Terraform how to define a variable to let user choose existing resource or create new one

I am new to Terraform.
I am having this question, would like to ask here.
So I am creating a azurerm_storage_account with all these input variables
#ie:
resource "azurerm_storage_account" "example" {
name = "storageaccountname"
resource_group_name = azurerm_resource_group.example.name
location = azurerm_resource_group.example.location
account_tier = "Standard"
account_replication_type = "GRS"
tags = {
environment = "staging"
}
}
How to define a variable can allow user to choose the existing storage account or creating a new one.
I am thinking to do this:
variable "choice" {
description = "Allow user to choose the existing storage account or creating a new one"
type = string
default = "create"
}
Then I don't know where to use this variable to work with the azurerm_storage_account resource.
Any pointer would be appreciate!
Thank you.
You can use count for that. Thus if your variable is "create", a new resource is going to be created. If its not, then a azurerm_storage_account data source will be used to query for details of the existing resource.
resource "azurerm_storage_account" "example" {
count = var.choice == "create" ? 1 : 0
name = "storageaccountname"
resource_group_name = azurerm_resource_group.example.name
location = azurerm_resource_group.example.location
account_tier = "Standard"
account_replication_type = "GRS"
tags = {
environment = "staging"
}
}
data "azurerm_storage_account" "example" {
count = var.choice == "create" ? 0 : 1
#...
}

Terraform - Reference for_each from a module output

I am trying to use Terraform to create multiple storage containers, and then upload multiple blobs to each container.
I have the part of creating multiple containers working, but can't figure out how to reference the for_each output of each container when uploading the blobs.
Storage Container Module (Works)
resource "azurerm_storage_container" "azure" {
for_each = toset(var.storage_containers)
name = each.value
storage_account_name = var.storage_account_name
container_access_type = var.storage_account_container_access_type
}
output "azurerm_storage_container_name" {
value = toset(keys(azurerm_storage_container.azure))
}
Child Module (Works)
module "storage_container" {
source = "C:/TerraformModules/modules/azurerm/azurerm_storage_container"
storage_account_name = module.storage_account.azurerm_storage_account_name
storage_containers = var.STORAGE_CONTAINER_NAMES
tags = var.TAGS
}
Code to upload blob (doesn't work for trying to upload into each container)
**In a variables.tf file**
variable "STORAGE_CONTAINER_DEFAULT_BLOBS" {
description = "The default blobs in each storage container"
type = list(string)
}
**In a vars.auto.tfvars file**
STORAGE_CONTAINER_DEFAULT_BLOBS = ["one", "two", "three"]
**In a main.tf file**
resource "azurerm_storage_blob" "storage_blob" {
for_each = toset(var.STORAGE_CONTAINER_DEFAULT_BLOBS)
name = each.value
storage_account_name = module.storage_account.azurerm_storage_account_name
storage_container_name = module.storage_container[each.value].azurerm_storage_container_name
type = "Block"
source_content = "blob file"
}
If I were to set the container name in storage_container_name, it works and the container gets each blob. But I'm not able to reference the container from the module.
I have this error:
Error: Invalid index
on storage_blobs.tf line 5, in resource "azurerm_storage_blob" "storage_blob":
5: storage_container_name = module.storage_container[each.value].azurerm_storage_container_name
|----------------
| each.value is "two"
| module.storage_container is object with 1 attribute "azurerm_storage_container_name"
The given key does not identify an element in this collection value.
What I need to achieve:
resource "azurerm_storage_blob" "storage_blob" {
for_each = toset(var.STORAGE_CONTAINER_DEFAULT_BLOBS)
name = each.value
storage_account_name = module.storage_account.azurerm_storage_account_name
storage_container_name = # How to reference the storage accounts created with the `storage_container ` module? #
type = "Block"
source_content = "blob file"
}
storage_container_name takes only one value, not a list of values. So if you have 3 STORAGE_CONTAINER_DEFAULT_BLOBS and n number of var.storage_containers you have to iterate n*3 times in azurerm_storage_blob.
Thus, you can try the following with setproduct:
resource "azurerm_storage_blob" "storage_blob" {
for_each = {for idx, val in setproduct(module.storage_container.azurerm_storage_container_name, var.STORAGE_CONTAINER_DEFAULT_BLOBS): idx=>val}
name = each.value[1]
storage_account_name = module.storage_account.azurerm_storage_account_name
storage_container_name = each.value[0]
type = "Block"
source_content = "blob file"
}

Terraform: Error: Invalid index operation

I am using Terraform 0.14 for to automate the creation of some Azure resources.
I am trying to create assign a pull role to an Azure Kubernetes cluster to pull images from an Azure container registry using a Managed system identity
Here is my code
Azure Kubernetes cluster (main.tf file)
resource "azurerm_kubernetes_cluster" "akc" {
name = var.cluster_name
location = var.location
resource_group_name = var.resource_group_name
dns_prefix = var.dns_prefix
kubernetes_version = var.kubernetes_version
api_server_authorized_ip_ranges = var.api_server_authorized_ip_ranges
identity {
type = "SystemAssigned"
}
}
Azure Kubernetes cluster (outputs.tf file)
output "principal_id" {
value = azurerm_kubernetes_cluster.akc.identity[0]["principal_id"]
}
Azure role assignment (main.tf file)
# Create a role assignment
resource "azurerm_role_assignment" "ara" {
scope = var.scope
role_definition_name = var.role_definition_name
principal_id = var.principal_id
}
Test environment (main.tf file)
# Create azure kubernetes cluster
module "azure_kubernetes_cluster" {
source = "../modules/azure-kubernetes-cluster"
cluster_name = var.cluster_name
location = var.location
dns_prefix = var.dns_prefix
resource_group_name = var.resource_group_name
kubernetes_version = var.kubernetes_version
node_count = var.node_count
min_count = var.min_count
max_count = var.max_count
os_disk_size_gb = "100"
max_pods = "110"
vm_size = var.vm_size
aad_group_name = var.aad_group_name
vnet_subnet_id = var.vnet_subnet_id
}
# Create azure container registry
module "azure_container_registry" {
source = "../modules/azure-container-registry"
container_registry_name = var.container_registry_name
resource_group_name = var.resource_group_name
location = var.location
sku = var.sku
admin_enabled = var.admin_enabled
}
# Create azure role assignment
module "azure_role_assignment" {
source = "../modules/azure-role-assignment"
scope = module.azure_container_registry.acr_id
role_definition_name = var.role_definition_name
principal_id = module.azure_kubernetes_cluster.principal_id
}
However, when I run the terraform plan command, I get the error below:
Error: Invalid index operation
on ../modules/aks-cluster/outputs.tf line 14, in output "principal_id":
14: value = azurerm_kubernetes_cluster.cluster.identity[0]["principal_id"]
Only attribute access is allowed here. Did you mean to access attribute
"principal_id" using the dot operator?
Trying to figure out the solution to this.
I later figured out the solution to the error. Some modifications were made in Terraform 0.12 and later versions on how index operations are called. So rather than this:
Azure Kubernetes cluster (outputs.tf file)
output "principal_id" {
value = azurerm_kubernetes_cluster.akc.identity[0]["principal_id"]
}
It will be this:
Azure Kubernetes cluster (outputs.tf file)
output "principal_id" {
value = azurerm_kubernetes_cluster.akc.identity.*.principal_id
}
And also instead of this:
Test environment (main.tf file)
# Create azure role assignment
module "azure_role_assignment" {
source = "../modules/azure-role-assignment"
scope = module.azure_container_registry.acr_id
role_definition_name = var.role_definition_name
principal_id = module.azure_kubernetes_cluster.principal_id
}
It will be this:
Test environment (main.tf file)
# Create azure role assignment
module "azure_role_assignment" {
source = "../modules/azure-role-assignment"
scope = module.azure_container_registry.acr_id
role_definition_name = var.role_definition_name
principal_id = module.azure_kubernetes_cluster.principal_id[0]
}
Resources: Invalid index when referencing output from module in 0.12
That's all
I hope this helps

Resources