Please help to understand how to correctly build dynamic rules for resource
In input I want to send vars like this :
role_rules = {
rule01 = {
"api_groups" = ["apps"]
"resources" = ["pods"]
"resource_names" = ["foo"]
"verbs" = ["get", "list", "watch"]
}
rule02 = {
"api_groups" = ["apps2"]
"resources" = ["services"]
"resource_names" = ["foo2"]
"verbs" = ["*"]
}
}
And in a result have two rules for my resource.
I tried to do this in a way like :
resource "kubernetes_role" "this" {
metadata {
name = var.role_name
labels = local.metadata_labels
}
dynamic "rule" {
for_each = local.role_permission_rules
content {
api_groups = try(role.value["api_groups"], "")
resources = try(role.value["resources"], "")
resource_names = try(role.value["resource_names"], "")
verbs = try(role.value["verbs"], "")
}
}
}
locals {
role_permission_rules = {
for rule in keys(var.role_rules):
rule => lookup(var.role_rules, rule)
}
}
But unfortunately, it's not working with a lot of errors that no value on the root module.
Any ideas on how to correct realize such stuff?
I would recommend using lookup instead of try. However, I think you just need to throw it into a list by containing the item in brackets []. Also I would recommend referencing rule.value and not role.value
For example:
dynamic "rule" {
for_each = local.role_permission_rules
content {
api_groups = [lookup(rule.value, "api_groups", null)]
resources = [lookup(rule.value, "resources", null)]
resource_names = [lookup(rule.value, "resource_names", null)]
verbs = [lookup(rule.value, "verbs", null)]
}
}
Related
I am trying to define SES rule sets with an order defined by a collection of rules in a variable.
I have tried the solution in https://github.com/hashicorp/terraform-provider-aws/issues/24067 to use the after property of the resource, and It does create the first rule, and fails when creating the second, and all subsequent rules, because the first rule does not exist yet (the one with after=null). I guess it needs some time to finalize. depends_on does not work with dynamic dependencies, as far as i know, so this will not make it either.
If I re-run the apply, then the second rule is created, but all the other rules fail.
my recipients_by_position map is indexed by 0-padded position (i.e. "01", "02", etc):
This is my code
locals {
recipients = {
"mail1-recipient1" = {
short_name = "mail1"
domain = "mail1.domain.com"
recipient = "recipient1"
position = 1
target_bucket = "bucket1"
}
"mail1-recipient2" = {
short_name = "mail1"
domain = "mail1.domain.com"
recipient = "recipient2"
position = 2
target_bucket = "bucket1"
}
"mail2-recipient1" = {
short_name = "mail2"
domain = "mail2.domain.com"
recipient = "recipient1"
position = 3
target_bucket = "bucket2"
}
}
spec_by_domain = {
"mail1.domain.com" = {
irrelevant ={}
}
"mail2.domain.com" = {
irrelevant ={}
}
}
recipients_by_position = {for r in local.recipients: "${format("%02s",r.position)}" => r}
}
resource "aws_ses_domain_identity" "domains" {
for_each = local.spec_by_domain
domain = each.key
}
resource "aws_ses_receipt_rule_set" "main" {
rule_set_name = "new-rules"
}
# store it in S3
resource "aws_ses_receipt_rule" "store" {
for_each = local.recipients_by_position
after = each.value.position == 1 ? null : "${format("%02s",each.value.position - 1)}"
# name = "${each.value.short_name}-store_on_s3-${each.value.recipient}"
name = each.key
rule_set_name = aws_ses_receipt_rule_set.main.rule_set_name
recipients = ["${each.value.recipient}#${each.value.domain}"]
enabled = true
scan_enabled = true
s3_action {
bucket_name = aws_s3_bucket.mailboxes[each.value.domain].bucket
object_key_prefix = each.value.recipient
position = 1
}
}
apply fails with a bunch of
Error: Error creating SES rule: RuleDoesNotExist: Rule does not exist: xx
with xx from 01 to whatever number of rules were defined
Updated with a more illustrative example.
My end goal is to have Terraform create instances of a resource generated with the for_each meta argument in a specific sequence. HCL is known to be a declarative language and when Terraform applies a configuration it can create resources randomly unless you use the depends_on argument or refer from one resource (instance) to another. However, the depends_on argument does not take values that are "calculated", so I don't know how to use it in modules.
For this reason, in order to force Terraform to create instances of a resource in a specific sequence, I decided to try to make the value of a certain argument in an instance it creates "calculated" based on the values of the same argument from another instance.
Below you can find a more practical example based on using one of the providers, but the question is more general and pertains to Terraform as such.
Let's take a test module that instantiates the cloudflare_page_rule resource:
# Module is placed to module\main.tf
terraform {
experiments = [module_variable_optional_attrs]
}
terraform {
required_providers {
cloudflare = {
source = "cloudflare/cloudflare"
version = ">= 3.10.0"
}
}
}
variable "zone" {
type = string
description = "The DNS zone name which will be added, e.g. example.com."
}
variable "page_rules" {
type = list(object({
page_rule_name = string
target = string
actions = object({
forwarding_url = optional(object({
url = string
status_code = number
}))
})
priority = optional(number)
status = optional(string)
depends_on = optional(string)
}))
description = "Zone's page rules."
default = []
}
//noinspection HILUnresolvedReference
locals {
page_rule_dependencies = { for p in var.page_rules : p.page_rule_name => p.depends_on if p.depends_on != null }
}
# https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/resources/zone
resource "cloudflare_zone" "this" {
zone = var.zone
}
# https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/resources/page_rule
//noinspection HILUnresolvedReference
resource "cloudflare_page_rule" "this" {
for_each = var.page_rules != null ? { for p in var.page_rules : p.page_rule_name => p } : {}
zone_id = cloudflare_zone.this.id
target = each.value.target
actions {
//noinspection HILUnresolvedReference
forwarding_url {
status_code = each.value.actions.forwarding_url.status_code
url = each.value.actions.forwarding_url.url
}
}
priority = each.value.depends_on != null ? cloudflare_page_rule.this[local.page_rule_dependencies[each.key]].priority + 1 : each.value.priority
status = each.value.status
}
output "page_rule_dependencies" {
value = local.page_rule_dependencies
}
And a configuration that is used to create resources:
terraform {
required_version = ">= 0.15.0"
required_providers {
cloudflare = {
source = "cloudflare/cloudflare"
version = ">= 3.10.1"
}
}
}
variable "cloudflare_api_token" {
type = string
sensitive = true
}
provider "cloudflare" {
api_token = var.cloudflare_api_token
}
module "acme_com" {
source = "./module"
zone = "acme.com"
page_rules = [
{
page_rule_name = "page_rule_1"
target = "acme.com/url1"
actions = {
forwarding_url = {
status_code = 301
url = "https://www.example.com/url1"
}
}
priority = 1
},
{
page_rule_name = "page_rule_2"
target = "acme.com/url2"
actions = {
forwarding_url = {
status_code = 301
url = "https://www.example.com/url2"
}
}
priority = 2
depends_on = "page_rule_1"
},
{
page_rule_name = "page_rule_3"
target = "acme.com/url3"
actions = {
forwarding_url = {
status_code = 301
url = "https://www.example.com/url3"
}
}
priority = 3
depends_on = "page_rule_2"
}
]
}
output "page_rule_dependencies" {
value = module.acme_com.page_rule_dependencies
}
In this particular example, I've added the depends_on argument to the page_rules variable (don't confuse this argument with the depends_on meta argument). For the value of the depends_on argument, I specified the name of a page_fule on which another page_fule depends.
Next, I created a local variable page_rule_dependencies, the value of which, after calculations, is the following (you can check this yourself by replacing the priority = each.value.depends_on != null ? cloudflare_page_rule.this[local.page_rule_dependencies[each.key]].priority + 1 : each.value.priority construct with priority = each.value.priority and executing terraform apply):
page_rule_dependencies = {
"page_rule_2" = "page_rule_1"
"page_rule_3" = "page_rule_2"
}
Further, in the priority = each.value.depends_on != null ? cloudflare_page_rule.this[local.page_rule_dependencies[each.key]].priority + 1 : each.value.priority construct, I refer to the values of the local variable, thereby forming a "reference" to the page_fule instance, on which the current instance depends:
When creating page_rule_1, the value of its argument priority = 1.
When creating page_rule_2, the value of its argument priority = cloudflare_page_rule.this["page_rule_1"].priority + 1.
When creating page_rule_3, the value of its argument priority = cloudflare_page_rule.this["page_rule_2"].priority + 1.
However, I get an Error: Cycle: module.acme_com.cloudflare_page_rule.this["page_rule_3"], module.acme_com.cloudflare_page_rule.this["page_rule_2"], module.acme_com.cloudflare_page_rule.this["page_rule_1"] error.
Either I'm doing something wrong, or it's some kind of Terraform limitation/bug. Is there a way to get rid of this error?
P.S. Resulting graph after terraform graph -draw-cycles | dot -Tsvg > graph.svg or terraform graph -draw-cycles -type=plan | dot -Tsvg > graph-plan.svg (the same result):
P.P.S. I use Terraform v1.1.7.
I am attempting to only assign a log_config to a subnet, if the subnet name is in the "file_one.tf sample" below. To accomplish the conditional logic I am using the lookup function to only return a "log_config" block if the name matches a name in the variable referenced in "file_one.tf".
The error I am getting is:
An argument named "log_config" is not expected here. Did you mean to define a block type of "log_config"
Would someone please help out with this? Or present a more elegant approach to assigning blocks to resources with conditional logic?
Thank you
# file_one.tf sample
locals {
subnets = {
"one" = {
"name" = "one"
local_config = {
aggregation_interval = "INTERVAL_10_SEC"
metadata = "INCLUDE_ALL_METADATA"
}
}
}
# file_two.tf
resource "google_compute_subnetwork" "network-with-private-secondary-ip-ranges" {
foreach = local.subnets
name = each_value.name
ip_cidr_range = "10.2.0.0/16"
region = "us-central1"
network = google_compute_network.custom-test.id
secondary_ip_range {
range_name = "tf-test-secondary-range-update1"
ip_cidr_range = "192.168.10.0/24"
}
log_config = lookup(each.value, log_config, null)
}
You are attempting to pass a map type as a value for an argument, and the resource schema is expecting a block type and not an argument with a map. You would need a dynamic block for this situation:
dynamic "log_config" {
for_each = lookup(each.value, log_config, [])
content {
aggregation_interval = log_config.value["aggregation_interval"]
metadata = log_config.value["metadata"]
}
}
The key that you want in said map is called local_config, no? When lookup() returns its default value is null, which is also not a block. But here I believe you want to pass in a string key to the lookup function:
# file_two.tf
{
# ...
name = each.value.name
# ...
log_config = lookup(each.value, "local_config", null)
}
Since the title is not descriptive enough let me introduce my problem.
I'm creating DRY module code for CDN that contains profile/endpoint/custom_domain.
Variable cdn_config would hold all necessary/optional parameters and these are created based on the for_each loop.
Variable looks like this:
variable "cdn_config" {
profiles = {
"profile_1" = {}
}
endpoints = {
"endpoint_1" = {
custom_domain = {
}
}
}
}
Core of this module is working - in the means that it would create cdn_profile "profile_1" then cdn_endpoint "endpoint_1" will be created and assigned to this profile then cdn_custom_domain will be created and assigned to "endpoint_1" since it's the part of "endpoint_1" map.
Then I realize, what in case I want to create "cdn_custom_domain" only and specify resource ID manually?
I was thinking that adding the optional parameter "standalone" could help, so it would look like this:
variable "cdn_config" {
profiles = {
"profile_1" = {}
}
endpoints = {
"endpoint_1" = {
custom_domain = {
}
}
"endpoint_standalone" = {
custom_domain = {
standalone = true
cdn_endpoint_id = "xxxxx"
}
}
}
}
Having this "standalone" parameter eq true "endpoint_standalone" map should be totally ignored from looping in the azurerm_cdn_endpoint resource creation.
So far this direction is my only guess, clearly, it's not working - if I add "endpoint_standalone" it complains that not all required parameters are specified so it's surely finding it.
resource "azurerm_cdn_endpoint" "this" {
for_each = {for k in keys(var.cdn_config.endpoints) : k => var.cdn_config.endpoints[k] if lookup(var.cdn_config.endpoints[k],"standalone",null) != "true"}
I would be grateful if you have a solution for this problem.
You are comparing a bool type to a string type, so the logical comparison will always return false:
for_each = {for k in keys(var.cdn_config.endpoints) : k => var.cdn_config.endpoints[k] if lookup(var.cdn_config.endpoints[k],"standalone",null) != true }
While we are here, we can also improve this for expression:
for_each = { for endpoint, params in var.cdn_config.endpoints : endpoint => params if lookup(params.custom_domain, "standalone", null) != true }
I'm trying out the new for_each function on a module, which itself outputs some values that I need to pass into another resource.
module "vnets" {
source = "../caf-virtual-network"
for_each = var.vnet_list
ARM_ENVIRONMENT = var.ARM_ENVIRONMENT
ARM_LOCATION = var.ARM_LOCATION
ARM_SUBSCRIPTION_ID = var.ARM_SUBSCRIPTION_ID
diagnostics_map = local.diagnostics_map
location = var.ARM_LOCATION
netwatcher = local.netwatcher
networking_object = each.value
tags = var.global_settings.tags
virtual_network_rg = "${module.names.standard["resource-group"]}-${each.value.vnet.resource_group_name}"
depends_on = [
module.resource_groups_networking
]
}
I can grab the output of the module for one or more of those objects by specifying something like this
output "subnets" { value = module.vnets["vnet_shared_services_object"].vnet_subnets } , which in turn looks like this:
"vnet_shared_services_object" = {
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
}
Here I'm specifying the output of ONE object, but I want to dynamically specify the output of both objects in one hit.
So I want this;
"vnet_shared_services_object" = {
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
}
"vnet_transit_object" = {
"AzureFirewallSubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/AzureFirewallSubnet"
"GatewaySubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/GatewaySubnet"
"sn-dev-uks-asdf-bind-dns" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/sn-dev-uks-asdf-bind-dns"
}
...output to look like this:
subnets = {
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
"AzureFirewallSubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/AzureFirewallSubnet"
"GatewaySubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/GatewaySubnet"
"sn-dev-uks-asdf-bind-dns" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/sn-dev-uks-asdf-bind-dns"
}
So i know doing the following will work, but the point i'm trying to make is that i don't know how many vnet modules i'm going to produce, and thus i need to make this dynamic:
output merge{
value = merge({
for key, value in module.vnets["vnet_shared_services_object"].vnet_subnets:
key => value
},
{
for key, value in module.vnets["vnet_transit_object"].vnet_subnets:
key => value
})
}
Using the guide on Terraform to flatten (https://www.terraform.io/docs/configuration/functions/flatten.html) the output object works, but it's not how i wish for it to be:
output stuff {
value = flatten([
for key, value in module.vnets: [
for subnet, id in value.vnet_subnets: {
"${subnet}" = id
}
]
])
}
...which equats to:
stuff = [
{
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/rg-dev-uks-asdf-vnet-shared-services/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
},
{
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/rg-dev-uks-asdf-vnet-shared-services/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
},
...and so on
]
an FYI, this does not help me :(
output {
value = merge(
for key, value in module.vnets:
key => value.vnets_subnets
)
}
Any help on this would be greatly appreciated!
I'm not sure if I correctly understand the input maps, but I tried to replicate the issue creating some mock variables.
For that I created the following variables:
variable "vnets" {
default = {
"vnet_shared_services_object" = {
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
}
}
}
variable "vnet_subnets" {
default = {
"vnet_transit_object" = {
"AzureFirewallSubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/AzureFirewallSubnet"
"GatewaySubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/GatewaySubnet"
"sn-dev-uks-asdf-bind-dns" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/sn-dev-uks-asdf-bind-dns"
}
}
}
Then the output was defiend as:
output stuff {
value = {for k,v in flatten([
for key, value in merge(var.vnets, var.vnet_subnets):
[for subkey1, subval1 in value: {"${subkey1}" = subval1}]
]): keys(v)[0] => values(v)[0]}
}
which resulted in:
stuff = {
"AzureFirewallSubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/AzureFirewallSubnet"
"GatewaySubnet" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/GatewaySubnet"
"sn-dev-uks-asdf-app-dynamic" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-app-dynamic"
"sn-dev-uks-asdf-artifactory" = "/subscriptions/asdf/resourceGroups/asdf/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-shared-services/subnets/sn-dev-uks-asdf-artifactory"
"sn-dev-uks-asdf-bind-dns" = "/subscriptions/asdf/resourceGroups/qwer/providers/Microsoft.Network/virtualNetworks/vnet-dev-uks-asdf-transit/subnets/sn-dev-uks-asdf-bind-dns"
}
A colleague was able to answer this question with the following code:
locals {
subnet_list = {
for key, value in module.vnets:
key => value.vnet_subnets
}
subnet_map = merge(values(local.subnet_list)...)
}
it is the ... operator which is the key takeaway from this. you can look it up here; https://www.terraform.io/docs/configuration/expressions.html#expanding-function-arguments
... will expand a list of items to function parameters, hence you can call merge to merge a list of map