Local Merge Argument definition not found - terraform

I am attempting to merge my tfvars values into a locals block, but it appears the merge is not working. For example, I put the location value in the tfvars to null and based on the code, in that case, it should instead choose the location of the Resource Group. I get an error of "The argument location is required, but no definition was found." Makes me feel that the whole merge is not happening at all.
Similarly, the merge of the tags is not working.
Does the merge only work for values that don't exist in the tfvars or can it take existing values, run logic on them, and produce the output? Maybe my whole idea of the merge is wrong if it cannot do the latter.
Any help appreciated.
According to the documentation: "If more than one given map or object defines the same key or attribute, then the one that is later in the argument sequence takes precedence."
##Data Source for Resource Groups
data "azurerm_resource_group" "resource_groups" {
for_each = { for law in var.log_analytics_workspace : law.name => law }
name = each.value.resource_group_name
}
###LOCALS###
locals {
laws = { for law in var.log_analytics_workspace : law.name => merge(
{
location = law.location != null ? law.location : data.azurerm_resource_group.resource_groups[law.name].location
sku = coalesce(law.sku, "PerGB2018")
retention_in_days = coalesce(law.retention_in_days, 30)
daily_quota_gb = coalesce(law.daily_quota_gb, 1)
internet_ingestion_enabled = false
internet_query_enabled = false
cmk_for_query_forced = false
reservation_capacity_in_gb_per_day = law.sku != "CapacityReservation" ? null : law.reservation_capacity_in_gb_per_day
tags = merge(data.azurerm_resource_group.resource_groups[law.name].tags, law.tags)
}, law)
}
}
#Log Analytics Resource
resource "azurerm_log_analytics_workspace" "log_analytics_workspace" {
for_each = local.laws
name = each.value.name
resource_group_name = each.value.resource_group_name
location = each.value.location
sku = each.value.sku
retention_in_days = each.value.retention_in_days
daily_quota_gb = each.value.daily_quota_gb
internet_ingestion_enabled = each.value.internet_ingestion_enabled
internet_query_enabled = each.value.internet_query_enabled
cmk_for_query_forced = each.value.cmk_for_query_forced
reservation_capacity_in_gb_per_day = each.value.reservation_capacity_in_gb_per_day
tags = each.value.tags
}
log_analytics_workspace = [
{
name = "law-eastus-dev-01"
resource_group_name = "rg-eastus-dev-01"
location = "eastus"
sku = "PerGB2018"
retention_in_days = 7
daily_quota_gb = 1
reservation_capacity_in_gb_per_day = 100
tags = {
law = "dev-east"
}
}
]
Changes to Outputs:
+ laws = {
+ law-eastus-dev-01 = {
+ cmk_for_query_forced = false
+ daily_quota_gb = 1
+ internet_ingestion_enabled = false
+ internet_query_enabled = false
+ location = "eastus"
+ name = "law-eastus-dev-01"
+ reservation_capacity_in_gb_per_day = 100
+ resource_group_name = "rg-eastus-dev-01"
+ retention_in_days = 7
+ sku = "PerGB2018"
+ tags = {
+ "law" = "dev-east"
}
}
}

Related

Create VPS in GCP via Terraform module using count

getting stuck with problem.
Need a terraform expert help. I want to create VPS in GCP with count using module. How to correct create and attach google_compute_address and google_compute_disk to each VPS with different names
Any help, please
Module code:
resource "google_compute_instance" "vps" {
count = var.server_count
name = var.server_count > 1 ? "${var.server_name}-${count.index}" : var.server_name
description = var.server_description
machine_type = var.server_type
zone = var.server_datacenter
deletion_protection = var.server_delete_protection
labels = var.server_labels
metadata = var.server_metadata
tags = var.server_tags
boot_disk {
auto_delete = false
initialize_params {
size = var.boot_volume_size
type = var.boot_volume_type
image = var.boot_volume_image
labels = var.boot_volume_labels
}
}
dynamic "attached_disk" {
for_each = var.volumes
content {
source = attached_disk.value["volume_name"]
}
}
dynamic "network_interface" {
for_each = var.server_network
content {
subnetwork = network_interface.value["subnetwork_name"]
network_ip = network_interface.value["subnetwork_ip"]
dynamic "access_config" {
for_each = network_interface.value.nat_ip ? [1] : []
content {
nat_ip = google_compute_address.static_ip.address
}
}
}
}
}
resource "google_compute_disk" "volume" {
for_each = var.volumes
name = each.value["volume_name"]
type = each.value["volume_type"]
size = each.value["volume_size"]
zone = var.server_datacenter
labels = each.value["volume_labels"]
}
resource "google_compute_address" "static_ip" {
count = var.server_count
name = var.server_count > 1 ? "${var.server_name}-${count.index}" : var.server_name
region = var.server_region
}
Using example:
module "vps-test" {
source = "../module"
credentials_file = "../../../../main/vault/prod/.tf/terraform-bb-prod-ground.json"
server_count = 2
server_name = "example-vps"
server_description = "simple vps for module testing"
server_type = "e2-small"
server_region = "europe-west4"
server_datacenter = "europe-west4-c"
server_labels = {
project = "terraform"
environment = "test"
}
server_metadata = {
groups = "parent_group.child_group"
}
boot_volume_image = "debian-cloud/debian-11"
boot_volume_size = 30
boot_volume_labels = {
environment = "production"
project = "v3"
type = "system"
}
server_tags = ["postgres", "production", "disable-gce-firewall"]
server_delete_protection = true
server_network = {
common_network = {
subnetwork_name = "${data.terraform_remote_state.network.outputs.subnetwork_vpc_production_common_name}"
subnetwork_ip = ""
nat_ip = true
} # },
# custom_network = {
# subnetwork_name = (data.terraform_remote_state.network.outputs.subnetwork_vpc_production_k8s_name)
# subnetwork_ip = ""
# nat_ip = false
# }
}
volumes = {
volume_data1 = {
volume_name = "v3-postgres-saga-import-test-storage"
volume_size = "40"
volume_type = "pd-ssd"
volume_labels = {
environment = "production"
project = "v3"
type = "storage"
}
},
volume_data2 = {
volume_name = "volume-vpstest2"
volume_size = "20"
volume_type = "pd-ssd"
volume_labels = {
environment = "production"
project = "v2"
type = "storage"
}
}
}
}
Now error is: Because google_compute_address.static_ip has "count" set, its attributes must be accessed on specific instances And i know, error with same disk name will come

Variables definition for optional parameters options - Terraform

I'm trying to write code for AKS with all possible arguments, and then in variables I specify map. Problem is I don't want to specify all of the optional arguments in my variables each time, I wonder if there is a possibility to skip them in variables.
Here is my example:
main.tf
resource "azurerm_kubernetes_cluster" "this" {
for_each = var.AKS_Config
name = each.value.AKS_Name
location = each.value.AKS_Location
resource_group_name = each.value.AKS_ResourceGroupName
dns_prefix = each.value.dns_prefix
#dns_prefix_private_cluster = # dns_prefix or dns_prefix_private_cluster must be specified
kubernetes_version = each.value.kubernetes_version
automatic_channel_upgrade = each.value.automatic_channel_upgrade
api_server_authorized_ip_ranges = each.value.api_server_authorized_ip_ranges
dynamic "identity" {
for_each = each.value.identity
content {
type = identity.value.type
user_assigned_identity_id = can(identity.value.user_assigned_identity_id) == true ? identity.value.user_assigned_identity_id : {}
}
}
}
variables.tf
variable "AKS_Config" {
default = {
"AKS_1" = {
AKS_Name = "AKSTest001"
AKS_Location = "West Europe"
AKS_ResourceGroupName = "SDSADAS"
dns_prefix = "AKSTESTPREFIX"
default_node_pool_name = "test-name"
node_count = 1
vm_sku = "Standard_D2_v2"
kubernetes_version = "1.21.7"
automatic_channel_upgrade = "stable"
api_server_authorized_ip_ranges = ["16.0.0.0/16"]
identity = {
type = "SystemAssigned"
}
default_node_pool = {
name = "nodetest01"
node_count = 3
vm_size = "Standard_D2_v2"
availability_zones = ["1", "2", "3"]
auto_scaling_enabled = true
enable_host_encryption = false
enable_node_public_ip = true
fips_enabled = true
kubelet_disk_type = "OS"
max_pods = 2
}
}
}
}
As you can see there are bunch of optional arguments for AKS, but I don't want to specify each one of them as 'null' every deployment.
Is there a option to achieve that? Some kind of function 'if key does not exist skip it'
Thank you
EDIT:
What do you think about this "workaround"?
dynamic "identity" {
for_each = can(each.value.identity) == true ? each.value.identity : {}
content {
type = each.value.identity.type
user_assigned_identity_id = can(each.value.identity.user_assigned_identity_id) == true ? each.value.identity.user_assigned_identity_id : null
}
}
Using can it's working, or at least terraform validate doesn't throw any problems and I don't need to specify optional parameters in map, but I don't know if this should be avoided in my case
Generally there are two options I see to handle this problem:
Split the default map to different vaiables, and assign to each variable a default value.
Use the merge() terraform function in order to override specific values in the default map you declared to be used in the for_each.

Using dynamic values for Kubernetes namespace labels

I am managing my on-prem Kubernetes cluster namespaces with Terraform and want to include some custom labels/annotations on them. This is to make auditing easier and also we have mutating webhooks that rely on labels/annotations.
I am trying to do something like this (pseudo code)
resource "kubernetes_namespace" "namespaces" {
for_each = {for k, v in var.namespaces: k => v}
metadata {
name = each.value.name
annotations = {
"linkerd.io/inject" = each.value.linkerd
{{loop over each.value.custom_annotations}}
}
labels = {
"apps.kubernetes.io/app" = each.value.app
"k8s.domain.co/managed-by" = each.value.managed
"k8s.domain.co/owner" = each.value.owner
{{loop over each.value.custom.labels}}
}
}
}
I have my var.namespaces variable constructed like
description = "List of namespaces controlled by Terraform"
type = list(object({
name = string
linkerd = string
app = string
owner = string
managed = string
custom_annotations = list(object({
label = string
value = string
}))
custom_labels = list(object({
label = string
value = string
}))
}))
I am trying to end up with
namespaces = [
{
name = foo
...
custom_annotations = {
label = "myannotation"
value = "myvalue"
custom_labels = {
label = "mylabel"
value = "myvalue"
}]
resource "kubernetes_namespace" "namespaces" {
for_each = {for k, v in var.namespaces: k => v}
metadata {
name = each.value.name
annotations = {
"linkerd.io/inject" = each.value.linkerd
myannotation = myvalue
}
labels = {
"apps.kubernetes.io/app" = each.value.app
"k8s.domain.co/managed-by" = each.value.managed
"k8s.domain.co/owner" = each.value.owner
mylabel = myvalue
}
}
}
I have a feeling some mix of locals and dynamic blocks would be the solution but I can't seem to pin them together in a way that works
Any advice please?
I managed to get this almost working for myself without using locals or dynamic blocks. However I can't include the default labels and annotations
resource "kubernetes_namespace" "namespaces" {
for_each = { for k, v in var.namespaces: k => v} //loop over the namespaces
metadata {
name = each.value.name
annotations = {
for annotation in each.value.custom_annotations: annotation.label => annotation.value
}
labels = {
for label in each.value.custom_labels: label.label => label.value
}
}
}
With this input
namespaces = [
{
name = "metallb-system"
linkerd = "enabled"
app = "metallb"
owner = "KPE"
managed = "Terraform"
custom_annotations = []
custom_labels = [{label="foo.io/bar", value="foobar"}, {label="bar.io/foo", value="barfoo"}]
},
{ name = "test-ns"
linkerd = "enabled"
app = "myapp"
owner = "Me"
managed = "Terraform"
custom_annotations = [{label="foo.io/annotation", value="test"}]
custom_labels = [{label="app.io/label", value="value"}]
}
]
It gives me this output
Changes to Outputs:
+ namespaces = {
+ 0 = {
+ id = "metallb-system"
+ metadata = [
+ {
+ annotations = {}
+ generate_name = ""
+ generation = 0
+ labels = {
+ "bar.io/foo" = "barfoo"
+ "foo.io/bar" = "foobar"
}
+ name = "metallb-system"
+ resource_version = "410142"
+ uid = "02d6b1e1-707a-49cf-9a2d-3f28c9ce1e5a"
},
]
+ timeouts = null
}
+ 1 = {
+ id = (known after apply)
+ metadata = [
+ {
+ annotations = {
+ "foo.io/annotation" = "test"
}
+ generate_name = null
+ generation = (known after apply)
+ labels = {
+ "app.io/label" = "value"
}
+ name = "test-ns"
+ resource_version = (known after apply)
+ uid = (known after apply)
},
]
+ timeouts = null
}
}
I found a way to add default labels and annotations using setunion
locals {
default_annotations = [{label = "foo", value = "bar"}]
default_labels = [{label = "terraform", value = true}]
}
resource "kubernetes_namespace" "namespaces" {
for_each = { for k, v in var.namespaces: k => v} //loop over the namespaces
metadata {
name = each.value.name
annotations = {
for annotation in setunion(each.value.custom_annotations, local.default_annotations) : annotation.label => annotation.value
}
labels = {
for label in setunion(each.value.custom_labels, local.default_labels) : label.label => label.value
}
}
}
I know this doesn't exactly solve your use case, as you are wanting to read the value from your list of namesapces, however I do think it is one step closer!

How to fetch id of a variable using name in terraform?

I have a terraform resource in which I am trying to make a subnet_id variable dynamic. So I have varibles defined below in which subnet_id = "worker-subnet-1". I want to pass the name of the subnet and fetch the subnet id as I have multiple subnets. How can I do that.
resource "oci_containerengine_node_pool" "node_pool" {
for_each = var.nodepools
cluster_id = oci_containerengine_cluster.cluster[0].id
compartment_id = var.compartment_id
depends_on = [oci_containerengine_cluster.cluster]
kubernetes_version = var.cluster_kubernetes_version
name = each.value["name"]
node_config_details {
placement_configs {
availability_domain = var.availability_domain
subnet_id = oci_core_subnet.each.value["subnet_name"].id
}
size = each.value["size"]
}
node_shape = each.value["node_shape"]
node_shape_config {
#Optional
memory_in_gbs = each.value["memory"]
ocpus = each.value["ocpus"]
}
node_source_details {
image_id = each.value["image_id"]
source_type = "IMAGE"
}
ssh_public_key = file(var.ssh_public_key_path)
}
These are my variables:
nodepools = {
np1 = {
name = "np1"
size = 3
ocpus = 8
memory = 120
image_id = "test"
node_shape = "VM.Standard2.8"
subnet_name = "worker-subnet-1"
}
np2 = {
name = "np2"
size = 2
ocpus = 8
memory = 120
image_id = "test"
node_shape = "VM.Standard2.8"
subnet_name = "worker-subnet-1"
}
}
any suggestions?
resource "oci_core_subnet" "snet-workers" {
cidr_block = lookup(var.subnets["snet-workers"], "subnet_cidr")
compartment_id = var.compartment_id
vcn_id = oci_core_virtual_network.base_vcn.id
display_name = lookup(var.subnets["snet-workers"], "display_name")
dns_label = lookup(var.subnets["snet-workers"], "dns_label")
prohibit_public_ip_on_vnic = true
security_list_ids = [oci_core_security_list.private_worker_nodes.id]
route_table_id = oci_core_route_table.rt-nat.id
}
You have to use like below where change <local resource name> to the name you have given for your resource
subnet_id = oci_core_subnet.<local resource name>[each.value.subnet_id].id

looping over local map of bojects with for expression on a resouce that already has a for_loop

How could i achieve looping over local.daily map so i could create the backup policy name based on the map value on a resource that already uses a for_each loop ?
On the following example, on the resource azurerm_backup_policy_file_share i would like to populate the name field with the value of local.daily["name"] value.
locals {
regions = [
"centralus",
"northeurope"
]
}
resource "azurerm_resource_group" "recovery_vault" {
name = "recovery-vault-${terraform.workspace}-rg"
location = var.azure_region
tags = {
environment = terraform.workspace
source = "terraform"
service = "Backup Vault"
}
}
resource "azurerm_recovery_services_vault" "vaults" {
for_each = toset(local.regions)
name = "recovery-vault-${terraform.workspace}-${each.key}"
location = each.key
resource_group_name = azurerm_resource_group.recovery_vault.name
sku = "Standard"
soft_delete_enabled = true
}
locals {
daily = [{
name = "Every23h"
frequency = "Daily"
time = "23:00"
count = 30
}
]
}
resource "azurerm_backup_policy_file_share" "daily" {
for_each = azurerm_recovery_services_vault.vaults
name = "need this field to be name retrieved from local.daily"
resource_group_name = each.value["resource_group_name"]
recovery_vault_name = each.value["name"]
timezone = "UTC"
dynamic "backup" {
for_each = local.daily
content {
frequency = backup.value["frequency"]
time = backup.value["time"]
}
}
dynamic "retention_daily" {
for_each = local.daily
content {
count = retention_daily.value["count"]
}
}
}
I guess what you need is setproduct.
It should look something like this:
resource "azurerm_backup_policy_file_share" "daily" {
for_each = toset(setproduct(azurerm_recovery_services_vault.vaults, local.daily))
name = each.value[1].name
resource_group_name = each.value[0]["resource_group_name"]
# ...

Resources