Conditional dynamic block - terraform

Trying to set an optional block called "sensitive_labels" and i'm trying to set it as an optional one, however, doesn't work.
My code:
variables.tf:
variable "notification_channels" {
type = any
}
variable "project_id" {
type = string
}
main.tf:
project = var.project_id
for_each = { for k, v in var.notification_channels : k => v }
type = each.value.type
display_name = each.value.display_name
description = each.value.description
labels = each.value.labels
enabled = each.value.enabled
dynamic "sensitive_labels" {
for_each = each.value.sensitive_labels != {} ?[each.value.sensitive_labels] : []
content {
auth_token = lookup(sensitive_labels.value, "auth_token", null)
}
}
}
dev.tfvars:
notification_channels = [
{
type = "email"
display_name = "a channel to send emails"
description = "a nice channel"
labels = {
email_address = "HeyThere#something.com"
}
enabled = true
sensitive_labels = {} // this one doesn't give any errors.
},
{
type = "email"
display_name = "HeyThere Email"
description = "a channel to send emails"
labels = {
email_address = "HeyThere2#something.com"
}
enabled = true
}
]
Getting:
Error: Unsupported attribute
on notification_channels.tf line 11, in resource "google_monitoring_notification_channel" "channels":
11: for_each = each.value.sensitive_labels != {} ? [each.value.sensitive_labels] : []
│ ├────────────────
each.value is object with 5 attributes
This object does not have an attribute named "sensitive_labels".
How can I make setting sensitive_labels an optional attribute here?
EDIT:
This seems to work but feels a bit off:
project = var.project_id
for_each = { for k, v in var.notification_channels : k => v }
type = each.value.type
display_name = each.value.display_name
description = each.value.description
labels = each.value.labels
enabled = each.value.enabled
dynamic "sensitive_labels" {
for_each = lookup(each.value, "sensitive_labels", {})
content {
auth_token = lookup(sensitive_labels.value, "auth_token", null)
}
}
}
Is there a better way that doesn't feel hacky?

A good place to start is to properly define a type constraint for your input variable, so that Terraform can understand better what data structure is expected and help ensure that the given value matches that data structure.
type = any is not there so you can skip defining a type constraint, but instead for the very rare situation where a module is just passing a data structure verbatim to a provider without interpreting it at all. Since your module clearly expects that input variable to be a map of objects (based on how you've used it), you should tell Terraform what object type you are expecting to recieve:
variable "notification_channels" {
type = map(object({
type = string
display_name = string
labels = map(string)
enabled = bool
sensitive_labels = object({
auth_token = string
password = string
service_key = string
})
}))
}
From your example it seems like you want sensitive_labels to be optional, so that the caller of the module can omit it. In that case you can use the optional modifier when you declare that particular attribute, and also the three attributes inside it:
sensitive_labels = optional(object({
auth_token = optional(string)
password = optional(string)
service_key = optional(string)
}))
An attribute that's marked as optional can be omitted by the caller, and in that case Terraform will automatically set it to null inside your module to represent that it wasn't set.
Now you can use this variable elsewhere in your module and safely assume that it will always have exactly the type defined in the variable block:
resource "google_monitoring_notification_channel" "channels" {
for_each = var.notification_channels
project = var.project_id
type = each.value.type
display_name = each.value.display_name
description = each.value.description
labels = each.value.labels
enabled = each.value.enabled
dynamic "sensitive_labels" {
for_each = each.value.sensitive_labels[*]
content {
auth_token = sensitive_labels.value.auth_token
password = sensitive_labels.value.password
service_key = sensitive_labels.value.service_key
}
}
}
The each.value.sensitive_labels[*] expression is a splat expression using the single values as lists feature, which concisely transforms the given value into either a one-element list or a zero-element list depending on whether the value is null. That effectively means that there will be one sensitive_labels block if each.value.sensitive_labels is set, and zero blocks of that type if that attribute is unset (null).
The attributes inside those blocks can also just be assigned directly without any special logic, because Terraform will have automatically set them to null if not specified by the caller and setting a resource argument to null is always the same as not setting it at all.
If you take the time to actually describe the types of variables you expect then it tends to make logic elsewhere in the module much simpler, because you no longer need to deal with all of the ways in which the caller might pass you an incorrect value: Terraform will either convert the value automatically to the expected type if possible, or will report an error to the caller explaining why the value they provided isn't acceptable.

Related

how does terraform pass value of variable when it is not being referenced in main.tf

I don't understand the logic of the following terraform code, and not sure, but I guess it might be me, but would appreciate some help with this.
So there's the following module https://github.com/gettek/terraform-azurerm-policy-as-code/blob/main/modules/definition/main.tf
resource azurerm_policy_definition def {
name = local.policy_name
display_name = local.display_name
description = local.description
policy_type = "Custom"
mode = var.policy_mode
management_group_id = var.management_group_id
metadata = jsonencode(local.metadata)
parameters = length(local.parameters) > 0 ? jsonencode(local.parameters) : null
policy_rule = jsonencode(local.policy_rule)
lifecycle {
create_before_destroy = true
}
timeouts {
read = "10m"
}
}
and https://github.com/gettek/terraform-azurerm-policy-as-code/blob/main/modules/definition/variables.tf
variable management_group_id {
type = string
description = "The management group scope at which the policy will be defined. Defaults to current Subscription if omitted. Changing this forces a new resource to be created."
default = null
}
variable policy_name {
type = string
description = "Name to be used for this policy, when using the module library this should correspond to the correct category folder under /policies/policy_category/policy_name. Changing this forces a new resource to be created."
default = ""
validation {
condition = length(var.policy_name) <= 64
error_message = "Definition names have a maximum 64 character limit, ensure this matches the filename within the local policies library."
}
}
variable display_name {
type = string
description = "Display Name to be used for this policy"
default = ""
validation {
condition = length(var.display_name) <= 128
error_message = "Definition display names have a maximum 128 character limit."
}
}
variable policy_description {
type = string
description = "Policy definition description"
default = ""
validation {
condition = length(var.policy_description) <= 512
error_message = "Definition descriptions have a maximum 512 character limit."
}
}
variable policy_mode {
type = string
description = "The policy mode that allows you to specify which resource types will be evaluated, defaults to All. Possible values are All and Indexed"
default = "All"
validation {
condition = var.policy_mode == "All" || var.policy_mode == "Indexed" || var.policy_mode == "Microsoft.Kubernetes.Data"
error_message = "Policy mode possible values are: All, Indexed or Microsoft.Kubernetes.Data (In Preview). Other modes are only allowed in built-in policy definitions, these include Microsoft.ContainerService.Data, Microsoft.CustomerLockbox.Data, Microsoft.DataCatalog.Data, Microsoft.KeyVault.Data, Microsoft.MachineLearningServices.Data, Microsoft.Network.Data and Microsoft.Synapse.Data"
}
}
variable policy_category {
type = string
description = "The category of the policy, when using the module library this should correspond to the correct category folder under /policies/var.policy_category"
default = null
}
variable policy_version {
type = string
description = "The version for this policy, if different from the one stored in the definition metadata, defaults to 1.0.0"
default = null
}
variable policy_rule {
type = any
description = "The policy rule for the policy definition. This is a JSON object representing the rule that contains an if and a then block. Omitting this assumes the rules are located in /policies/var.policy_category/var.policy_name.json"
default = null
}
variable policy_parameters {
type = any
description = "Parameters for the policy definition. This field is a JSON object that allows you to parameterise your policy definition. Omitting this assumes the parameters are located in /policies/var.policy_category/var.policy_name.json"
default = null
}
variable policy_metadata {
type = any
description = "The metadata for the policy definition. This is a JSON object representing additional metadata that should be stored with the policy definition. Omitting this will fallback to meta in the definition or merge var.policy_category and var.policy_version"
default = null
}
variable file_path {
type = any
description = "The filepath to the custom policy. Omitting this assumes the policy is located in the module library"
default = null
}
locals {
# import the custom policy object from a library or specified file path
policy_object = jsondecode(coalesce(try(
file(var.file_path),
file("${path.cwd}/policies/${title(var.policy_category)}/${var.policy_name}.json"),
file("${path.root}/policies/${title(var.policy_category)}/${var.policy_name}.json"),
file("${path.root}/../policies/${title(var.policy_category)}/${var.policy_name}.json"),
file("${path.module}/../../policies/${title(var.policy_category)}/${var.policy_name}.json")
)))
# fallbacks
title = title(replace(local.policy_name, "/-|_|\\s/", " "))
category = coalesce(var.policy_category, try((local.policy_object).properties.metadata.category, "General"))
version = coalesce(var.policy_version, try((local.policy_object).properties.metadata.version, "1.0.0"))
# use local library attributes if runtime inputs are omitted
policy_name = coalesce(var.policy_name, try((local.policy_object).name, null))
display_name = coalesce(var.display_name, try((local.policy_object).properties.displayName, local.title))
description = coalesce(var.policy_description, try((local.policy_object).properties.description, local.title))
metadata = coalesce(var.policy_metadata, try((local.policy_object).properties.metadata, merge({ category = local.category },{ version = local.version })))
parameters = coalesce(var.policy_parameters, try((local.policy_object).properties.parameters, null))
policy_rule = coalesce(var.policy_rule, try((local.policy_object).properties.policyRule, null))
# manually generate the definition Id to prevent "Invalid for_each argument" on set_assignment plan/apply
definition_id = var.management_group_id != null ? "${var.management_group_id}/providers/Microsoft.Authorization/policyDefinitions/${local.policy_name}" : azurerm_policy_definition.def.id
}
and an example how to use the module https://github.com/gettek/terraform-azurerm-policy-as-code/blob/main/examples/definitions.tf
module "deny_resource_types" {
source = "..//modules/definition"
policy_name = "deny_resource_types"
display_name = "Deny Azure Resource types"
policy_category = "General"
management_group_id = data.azurerm_management_group.org.id
}
From how I see it (might be wrrong) a variable can be used as a default value to the local in a Terraform script. So how is the value for the variable policy_name used when main.tf references local.policy_name instead of var.policy_name. The variable policy_name is also not having a default value.
What am I missing ?
Thank you !
EDIT:
Thank you, very clear explanation. I think I understand it better now. So basically, when I’m calling the definition module Terraform essentially load and process that module’s configuration files (including variables.tf). I was confused to see name = local.policy_name instead of for example mode = var.policy_mode. But the way I understand it now, is that when calling this module, I can set the value for the variable policy_name, which is then further processed inside the locals section, and result is what is actual provided to the name argument required by azurerm_policy_definition https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/policy_definition. Could you please confirm that my understanding is correct?
Thank you !
policy_name does have default value, but the value is empty.
default = ""
Empty value can be default value. Terraform expects module inputs only when there is no default attribute set in the input field.
OK, so there are two scenarios at play here.
When the policy_name isn't provided to the module, it takes on its default behaviour of empty string
When a value is provided to the policy_name, the locals section transforms that value and then uses it in the code as local.policy_name for the resources. L103
policy_name = coalesce(var.policy_name, try((local.policy_object).name, null))
If you look for the coalesce function, its purpose is to return the first element that is not null/empty.
Although, I don't see the point of that logic since if both cases are null it is supposed to return null. Could've used a simple condition instead.
I hope this clarifies things more.
Ps: an empty string (""), Terraform consider as a value.

Terraform: referencing each.key or each.value in module when calling variables

I'm trying to achieve (maybe by wrong means) something like that. I'd like to be able to create few types of endpoints in Azure (KV, SA for example).
module "endpoints" {
source = "./modules/private_endpoint"
for_each = toset(var.endpoint_type)
private_connection_resource_id = "var.${each.value}.private_connection_resource_id"
Where:
Endpoint_type is a list of endpoints (its value is "storage_account"),
private_connection_resource_id is in map(any) which looks like (there are other values, but I don't think they're important at this point):
storage_account = {
private_connection_resource_id = #VALUE
...
private_connection_resource_id = "var.${each.value}.private_connection_resource_id" --- this gets translated to literal string (var.storage_account.private_connection_resource_id), where I'd like it to get translated to exact value - the id of storage account (it's hardcoded in tfvars).
Thank you in advance for any tips!
Edit: It appears that Im as dumb as they come. Should've changed the map a bit:
endpoint_type = {
storage_account = {
private_connection_resource_id = #VALUE
...
And ref in module calling to: each.value.private_connection_resource_id
You cannot construct an expression inside a string and then evaluate it. Terraform always parses the full configuration first and then executes it once already parsed.
If you want to look up values dynamically based on a key then a map is the most appropriate data structure to use for that purpose. For example, you could define a input variables endpoint_types and endpoints like this:
variable "endpoint_types" {
type = map(object({
private_connection_resource_id = string
}})
}
variable "endpoints" {
type = map(object({
type = string
}))
}
My intent with the above example is that the type attribute inside the endpoints objects is a lookup key for the other map in endpoint_types.
When you then define your module block with for_each you will first refer to var.endpoints and then look up an appropriate endpoint type in var.endpoint_types based on its selected key:
module "endpoints" {
source = "./modules/private_endpoint"
for_each = var.endpoints
private_connection_resource_id = var.endpoint_types[each.value.type].private_connection_resource_id
}
The user of the outer module would need to provide both a map of endpoints and a map describing all of the possible types those endpoints might have, like this:
endpoints = {
storage_1 = {
type = "storage"
}
storage_2 = {
type = "storage"
}
other_1 = {
type = "other"
}
}
endpoint_types = {
storage = {
private_connection_resource_id = "something"
}
other = {
private_connection_resource_id = "something_else"
}
}

Passing data as variable to Terraform module

I am looking for a way to pass data template_cloudinit_config to another module. I am clear about how to pass variables to module for various data types including object, but I am not sure how to do this with data.
In this setup, I have a vm-basic module that will define all the virtual hardware configuration, and postgres Terraform script that will define service related information including cloud init scripts. The intention is to have vm virtual hardware configuration to be highly reusable as module, to allow me focusing only on service related info i.e. postgres, nginx etc.
This is my vm-basic vars.tf file that will accept parameters that will be used in virtual hardware configuration.
variable "prefix" {}
variable "rg" { type = object({
name = string
location = string
}) }
variable "vm_size" {}
variable "private_ip_address" {}
variable "subnet" { type = object({ id = string }) }
variable "data_disk_size_gb" { type = number }
variable "service_name" { type = string }
variable "admin_username" { type = string }
variable "admin_public_key_path" { type = string }
variable "nsg_allow_tcp_ports" { type = list(string) }
locals {
nsg_allow_tcp_ports = {for p in var.nsg_allow_tcp_ports: index(var.nsg_allow_tcp_ports, p) => p}
}
#### DOES NOT WORK ######
#### Expected an equals sign ("=") to mark the beginning of the attribute value. ######
variable "custom_data" { type = object({ data }) }
How custom data will be used in vm-basic module
resource "azurerm_linux_virtual_machine" "vm" {
name = "${var.prefix}-${var.service_name}-vm"
location = var.rg.location
resource_group_name = var.rg.name
...
...
custom_data = var.custom_data.rendered
...
...
}
How the other script will pass parameter to vm-basic module
module "vm-basic" {
source = "../../base/vm"
service_name = var.service_name
prefix = var.prefix
rg = var.rg
vm_size = var.vm_size
private_ip_address = var.private_ip_address
subnet = var.subnet
data_disk_size_gb = var.data_disk_size_gb
admin_username = var.admin_username
admin_public_key_path = var.admin_public_key_path
nsg_allow_tcp_ports = var.nsg_allow_tcp_ports
}
data "template_cloudinit_config" "config" {
gzip = true
base64_encode = true
part {
filename = "init-cloud-config"
content_type = "text/cloud-config"
content = file("init.yaml")
}
part {
filename = "init-shellscript"
content_type = "text/x-shellscript"
content = templatefile("init.sh",
{ hostname = "${var.prefix}-${var.service_name}" }
)
}
}
How can I pass data object to another Terraform module?
In the variable vars.tf file, it's enough to just do
variable "custom_data" {}
In the vm-basic module, refer to the variable through var, similar to others.
custom_data = var.custom_data.rendered
The meaning of the error you saw is that Terraform is expecting the argument to the object type constraint to be name = type pairs, but you only wrote data and so Terraform is reporting that there's a missing =.
To make this work, you'll need to write a valid type constraint. It's not clear to me from your question exactly what custom_data represents, but I do see your later example includes var.custom_data.rendered and so from that I can tell that the type constraint should at least include a rendered attribute in order to make that valid, and the custom_data argument to azurerm_linux_virtual_machine expects a string so I'll match that:
variable "custom_data" {
type = object({
rendered = string
})
}
This means that Terraform will accept any object value which has a rendered attribute that can convert to string, and thus your later reference to var.custom_data.rendered is guaranteed to work and always produce a string value.

Is there any method for looping over all variables in Terraform?

I'd like to have a resource block that can loop over every variable that I've defined with variable blocks. Is this possible?
For Example: Assuming I set myfirstvar and mysecondvar in a tfvar file. I'm searching for a solution that would take the following template and deploy all the names, values and descriptions to Terraform Cloud.
variable "myfirstvar" {
type = string
description = "a var to upload"
}
variable "mysecondvar" {
type = string
description = "another var to upload"
}
resource "tfe_variable" "test" {
for_each = var
key = currentvar.key
value = currentvar.value
category = "terraform"
workspace_id = tfe_workspace.test.id
description = currentvar.description
}
So far the only solutions I can think of are to put all tfvars in a single list type variable, but that wouldn't have the additional info from the variable blocks. Or I could do some further parsing in another program on the variable blocks and tfvars files gather all the info necessary. Neither is ideal.
The the answer to your direct question is no, because var is not actually an object in Terraform. Instead, it's just a prefix that all variable symbols have. This distinction is important in Terraform because it works by creating a dependency graph based on your references between objects, and there is no node in the dependency graph representing "all variables" and so there's no way for Terraform to represent a reference to it.
With that said, the typical way to achieve a similar result is to write a single variable that has a map type, like this:
variable "tfe_variables" {
type = map(object({
value = string
description = string
}))
}
resource "tfe_variable" "test" {
for_each = var.tfe_variables
key = each.key
value = each.value.value
category = "terraform"
workspace_id = tfe_workspace.test.id
description = each.value.description
}
Then when you define this variable you can set it to a map of objects, like this:
tfe_variables = {
myfirstvar = {
value = "example"
description = "a var to upload"
}
mysecondvar = {
value = "example"
description = "another var to upload"
}
}
It seems like you'd prefer to have the variables of this configuration match the variables defined on the workspace you're managing with it. There's no direct way to do that, because the variable definitions of the current configuration are metadata for outside consumption and not directly accessible by the module itself. The closest you could get to it is to construct the necessary map explicitly, so that Terraform can see all of the necessary dependency edges in order to execute the program:
variable "myfirstvar" {
type = string
}
variable "mysecondvar" {
type = string
}
locals {
variables = tomap({
myfirstvar = {
value = var.myfirstvar
description = "a var to upload"
}
mysecondvar = {
value = var.mysecondvar
description = "another var to upload"
}
})
}
resource "tfe_variable" "test" {
for_each = local.variables
key = each.key
value = each.value.value
category = "terraform"
workspace_id = tfe_workspace.test.id
description = each.value.description
}

How to handle optional dynamic blocks in terraform

I am trying to iterate in resource "launchdarkly_feature_flag" variables with dynamic block that might have optional nested objects - "variations" (could be 0 or 2+):
variable "feature_flags" {
default = {
flag_1 = {
project_key = "project"
key = "number example"
name = "number example flag"
description = "this is a multivariate flag with number variations."
variation_type = "number"
variations = {
value = 100
}
variations = {
value = 300
}
tags = ["example"]
},
flag_2 = {
project_key = "project"
key = "boolean example"
name = "boolean example flag"
description = "this is a boolean flag"
variation_type = "boolean"
tags = ["example2"]
}
}
}
Ive tried various scenarios of how to get all flag and always face different problems. Piece of code:
resource "launchdarkly_feature_flag" "ffs" {
for_each = var.feature_flag_map
project_key = each.value.project_key
key = each.value.key
name = each.value.name
description = each.value.description
variation_type = each.value.variation_type
# main problem here
dynamic variations {
for_each = lookup(each.value, "variations", {}) == {} ? {} : {
content {
value = each.value.variations.value
}
}
}
tags = each.value.tags
}
Could you please help with that? I am using 0.14v of Terraform
The first step would be to tell Terraform what type of value this variable expects. While it's often okay to omit an explicit type for a simple value and let Terraform infer it automatically from the default, when the data structure is this complex it's better to tell Terraform what you intended, because then you can avoid it "guessing" incorrectly and giving you confusing error messages as a result.
The following looks like a suitable type constraint for the default value you showed:
variable "feature_flags" {
type = map(object({
project_key = string
key = string
name = string
description = string
variation_type = string
tags = set(string)
variations = list(object({
value = string
}))
}))
}
With the type written out, Terraform will guarantee that any var.feature_flags value conforms to that type constraint, which means that you can then make your dynamic decisions based on whether the values are null or not:
resource "launchdarkly_feature_flag" "ffs" {
for_each = var.feature_flags
project_key = each.value.project_key
key = each.value.key
name = each.value.name
description = each.value.description
variation_type = each.value.variation_type
tags = each.value.tags
dynamic "variations" {
for_each = each.value.variations != null ? each.value.variations : []
content {
variations.value.value
}
}
}
As written above, Terraform will require that all values in feature_flags have all of the attributes defined, although the caller can set them to null to indicate that they are unset.
At the time of writing, in Terraform v0.14, there is an experimental feature for marking attributes as optional which seems like it would, once stabilized, be suitable for this use-case. Marking some or all of the attributes as optional would allow callers to omit them and thus cause Terraform to automatically set them to null, rather than the caller having to explicitly write out the null value themselves.
Hopefully that feature is stabilized in v0.15, at which point you could return to this and add the optional annotations to some attributes without changing anything else about the module.

Resources