I am looking for a way to pass data template_cloudinit_config to another module. I am clear about how to pass variables to module for various data types including object, but I am not sure how to do this with data.
In this setup, I have a vm-basic module that will define all the virtual hardware configuration, and postgres Terraform script that will define service related information including cloud init scripts. The intention is to have vm virtual hardware configuration to be highly reusable as module, to allow me focusing only on service related info i.e. postgres, nginx etc.
This is my vm-basic vars.tf file that will accept parameters that will be used in virtual hardware configuration.
variable "prefix" {}
variable "rg" { type = object({
name = string
location = string
}) }
variable "vm_size" {}
variable "private_ip_address" {}
variable "subnet" { type = object({ id = string }) }
variable "data_disk_size_gb" { type = number }
variable "service_name" { type = string }
variable "admin_username" { type = string }
variable "admin_public_key_path" { type = string }
variable "nsg_allow_tcp_ports" { type = list(string) }
locals {
nsg_allow_tcp_ports = {for p in var.nsg_allow_tcp_ports: index(var.nsg_allow_tcp_ports, p) => p}
}
#### DOES NOT WORK ######
#### Expected an equals sign ("=") to mark the beginning of the attribute value. ######
variable "custom_data" { type = object({ data }) }
How custom data will be used in vm-basic module
resource "azurerm_linux_virtual_machine" "vm" {
name = "${var.prefix}-${var.service_name}-vm"
location = var.rg.location
resource_group_name = var.rg.name
...
...
custom_data = var.custom_data.rendered
...
...
}
How the other script will pass parameter to vm-basic module
module "vm-basic" {
source = "../../base/vm"
service_name = var.service_name
prefix = var.prefix
rg = var.rg
vm_size = var.vm_size
private_ip_address = var.private_ip_address
subnet = var.subnet
data_disk_size_gb = var.data_disk_size_gb
admin_username = var.admin_username
admin_public_key_path = var.admin_public_key_path
nsg_allow_tcp_ports = var.nsg_allow_tcp_ports
}
data "template_cloudinit_config" "config" {
gzip = true
base64_encode = true
part {
filename = "init-cloud-config"
content_type = "text/cloud-config"
content = file("init.yaml")
}
part {
filename = "init-shellscript"
content_type = "text/x-shellscript"
content = templatefile("init.sh",
{ hostname = "${var.prefix}-${var.service_name}" }
)
}
}
How can I pass data object to another Terraform module?
In the variable vars.tf file, it's enough to just do
variable "custom_data" {}
In the vm-basic module, refer to the variable through var, similar to others.
custom_data = var.custom_data.rendered
The meaning of the error you saw is that Terraform is expecting the argument to the object type constraint to be name = type pairs, but you only wrote data and so Terraform is reporting that there's a missing =.
To make this work, you'll need to write a valid type constraint. It's not clear to me from your question exactly what custom_data represents, but I do see your later example includes var.custom_data.rendered and so from that I can tell that the type constraint should at least include a rendered attribute in order to make that valid, and the custom_data argument to azurerm_linux_virtual_machine expects a string so I'll match that:
variable "custom_data" {
type = object({
rendered = string
})
}
This means that Terraform will accept any object value which has a rendered attribute that can convert to string, and thus your later reference to var.custom_data.rendered is guaranteed to work and always produce a string value.
Related
Trying to set an optional block called "sensitive_labels" and i'm trying to set it as an optional one, however, doesn't work.
My code:
variables.tf:
variable "notification_channels" {
type = any
}
variable "project_id" {
type = string
}
main.tf:
project = var.project_id
for_each = { for k, v in var.notification_channels : k => v }
type = each.value.type
display_name = each.value.display_name
description = each.value.description
labels = each.value.labels
enabled = each.value.enabled
dynamic "sensitive_labels" {
for_each = each.value.sensitive_labels != {} ?[each.value.sensitive_labels] : []
content {
auth_token = lookup(sensitive_labels.value, "auth_token", null)
}
}
}
dev.tfvars:
notification_channels = [
{
type = "email"
display_name = "a channel to send emails"
description = "a nice channel"
labels = {
email_address = "HeyThere#something.com"
}
enabled = true
sensitive_labels = {} // this one doesn't give any errors.
},
{
type = "email"
display_name = "HeyThere Email"
description = "a channel to send emails"
labels = {
email_address = "HeyThere2#something.com"
}
enabled = true
}
]
Getting:
Error: Unsupported attribute
on notification_channels.tf line 11, in resource "google_monitoring_notification_channel" "channels":
11: for_each = each.value.sensitive_labels != {} ? [each.value.sensitive_labels] : []
│ ├────────────────
each.value is object with 5 attributes
This object does not have an attribute named "sensitive_labels".
How can I make setting sensitive_labels an optional attribute here?
EDIT:
This seems to work but feels a bit off:
project = var.project_id
for_each = { for k, v in var.notification_channels : k => v }
type = each.value.type
display_name = each.value.display_name
description = each.value.description
labels = each.value.labels
enabled = each.value.enabled
dynamic "sensitive_labels" {
for_each = lookup(each.value, "sensitive_labels", {})
content {
auth_token = lookup(sensitive_labels.value, "auth_token", null)
}
}
}
Is there a better way that doesn't feel hacky?
A good place to start is to properly define a type constraint for your input variable, so that Terraform can understand better what data structure is expected and help ensure that the given value matches that data structure.
type = any is not there so you can skip defining a type constraint, but instead for the very rare situation where a module is just passing a data structure verbatim to a provider without interpreting it at all. Since your module clearly expects that input variable to be a map of objects (based on how you've used it), you should tell Terraform what object type you are expecting to recieve:
variable "notification_channels" {
type = map(object({
type = string
display_name = string
labels = map(string)
enabled = bool
sensitive_labels = object({
auth_token = string
password = string
service_key = string
})
}))
}
From your example it seems like you want sensitive_labels to be optional, so that the caller of the module can omit it. In that case you can use the optional modifier when you declare that particular attribute, and also the three attributes inside it:
sensitive_labels = optional(object({
auth_token = optional(string)
password = optional(string)
service_key = optional(string)
}))
An attribute that's marked as optional can be omitted by the caller, and in that case Terraform will automatically set it to null inside your module to represent that it wasn't set.
Now you can use this variable elsewhere in your module and safely assume that it will always have exactly the type defined in the variable block:
resource "google_monitoring_notification_channel" "channels" {
for_each = var.notification_channels
project = var.project_id
type = each.value.type
display_name = each.value.display_name
description = each.value.description
labels = each.value.labels
enabled = each.value.enabled
dynamic "sensitive_labels" {
for_each = each.value.sensitive_labels[*]
content {
auth_token = sensitive_labels.value.auth_token
password = sensitive_labels.value.password
service_key = sensitive_labels.value.service_key
}
}
}
The each.value.sensitive_labels[*] expression is a splat expression using the single values as lists feature, which concisely transforms the given value into either a one-element list or a zero-element list depending on whether the value is null. That effectively means that there will be one sensitive_labels block if each.value.sensitive_labels is set, and zero blocks of that type if that attribute is unset (null).
The attributes inside those blocks can also just be assigned directly without any special logic, because Terraform will have automatically set them to null if not specified by the caller and setting a resource argument to null is always the same as not setting it at all.
If you take the time to actually describe the types of variables you expect then it tends to make logic elsewhere in the module much simpler, because you no longer need to deal with all of the ways in which the caller might pass you an incorrect value: Terraform will either convert the value automatically to the expected type if possible, or will report an error to the caller explaining why the value they provided isn't acceptable.
I'm trying to achieve (maybe by wrong means) something like that. I'd like to be able to create few types of endpoints in Azure (KV, SA for example).
module "endpoints" {
source = "./modules/private_endpoint"
for_each = toset(var.endpoint_type)
private_connection_resource_id = "var.${each.value}.private_connection_resource_id"
Where:
Endpoint_type is a list of endpoints (its value is "storage_account"),
private_connection_resource_id is in map(any) which looks like (there are other values, but I don't think they're important at this point):
storage_account = {
private_connection_resource_id = #VALUE
...
private_connection_resource_id = "var.${each.value}.private_connection_resource_id" --- this gets translated to literal string (var.storage_account.private_connection_resource_id), where I'd like it to get translated to exact value - the id of storage account (it's hardcoded in tfvars).
Thank you in advance for any tips!
Edit: It appears that Im as dumb as they come. Should've changed the map a bit:
endpoint_type = {
storage_account = {
private_connection_resource_id = #VALUE
...
And ref in module calling to: each.value.private_connection_resource_id
You cannot construct an expression inside a string and then evaluate it. Terraform always parses the full configuration first and then executes it once already parsed.
If you want to look up values dynamically based on a key then a map is the most appropriate data structure to use for that purpose. For example, you could define a input variables endpoint_types and endpoints like this:
variable "endpoint_types" {
type = map(object({
private_connection_resource_id = string
}})
}
variable "endpoints" {
type = map(object({
type = string
}))
}
My intent with the above example is that the type attribute inside the endpoints objects is a lookup key for the other map in endpoint_types.
When you then define your module block with for_each you will first refer to var.endpoints and then look up an appropriate endpoint type in var.endpoint_types based on its selected key:
module "endpoints" {
source = "./modules/private_endpoint"
for_each = var.endpoints
private_connection_resource_id = var.endpoint_types[each.value.type].private_connection_resource_id
}
The user of the outer module would need to provide both a map of endpoints and a map describing all of the possible types those endpoints might have, like this:
endpoints = {
storage_1 = {
type = "storage"
}
storage_2 = {
type = "storage"
}
other_1 = {
type = "other"
}
}
endpoint_types = {
storage = {
private_connection_resource_id = "something"
}
other = {
private_connection_resource_id = "something_else"
}
}
I use a lot of resource blocks with for_each that call variables of type map(object({})).
I need to both mark some values sensitive = true (i.e. password) AND also regex-validate other values. I've referenced Terraform - Adding Validation for type = map(object()) in variables.tf without recourse.
If I use these two functions in my variable declarations, I get "Error: Invalid for_each argument" with verbose explanation:
Sensitive values, or values derived from sensitive values, cannot be used as for_each arguments. If used, the sensitive value could be exposed as a resource instance key.
My code builds Cisco ACI [network] configuration (if that's relevant). What are some other ways I could accomplish these requirements, since for_each doesn't allow it?
main.tf
resource "aci_vmm_credential" "vmmcredLocalName" {
for_each = var.vmmacc_map
vmm_domain_dn = aci_vmm_domain.vmmdomainLocalName[each.key].id
name = each.value.name
annotation = "orchestrator:terraform"
pwd = each.value.password
usr = each.value.username
}
Variables.tf
variable "vmmacc_map" {
type = map(object( {
name = string
password = string
username = string
}))
sensitive = true
validation {
condition = (
can(regexall("
}
terraform.tfvars
vmmacc_map = {
"ACI_Non_Prod_VDSwitch-mktest" = {
name = "Non_Prod_Creds-mktest"
username = "svc_aci_vmm-mktest"
password = "password"
}
}
I'd like to have a resource block that can loop over every variable that I've defined with variable blocks. Is this possible?
For Example: Assuming I set myfirstvar and mysecondvar in a tfvar file. I'm searching for a solution that would take the following template and deploy all the names, values and descriptions to Terraform Cloud.
variable "myfirstvar" {
type = string
description = "a var to upload"
}
variable "mysecondvar" {
type = string
description = "another var to upload"
}
resource "tfe_variable" "test" {
for_each = var
key = currentvar.key
value = currentvar.value
category = "terraform"
workspace_id = tfe_workspace.test.id
description = currentvar.description
}
So far the only solutions I can think of are to put all tfvars in a single list type variable, but that wouldn't have the additional info from the variable blocks. Or I could do some further parsing in another program on the variable blocks and tfvars files gather all the info necessary. Neither is ideal.
The the answer to your direct question is no, because var is not actually an object in Terraform. Instead, it's just a prefix that all variable symbols have. This distinction is important in Terraform because it works by creating a dependency graph based on your references between objects, and there is no node in the dependency graph representing "all variables" and so there's no way for Terraform to represent a reference to it.
With that said, the typical way to achieve a similar result is to write a single variable that has a map type, like this:
variable "tfe_variables" {
type = map(object({
value = string
description = string
}))
}
resource "tfe_variable" "test" {
for_each = var.tfe_variables
key = each.key
value = each.value.value
category = "terraform"
workspace_id = tfe_workspace.test.id
description = each.value.description
}
Then when you define this variable you can set it to a map of objects, like this:
tfe_variables = {
myfirstvar = {
value = "example"
description = "a var to upload"
}
mysecondvar = {
value = "example"
description = "another var to upload"
}
}
It seems like you'd prefer to have the variables of this configuration match the variables defined on the workspace you're managing with it. There's no direct way to do that, because the variable definitions of the current configuration are metadata for outside consumption and not directly accessible by the module itself. The closest you could get to it is to construct the necessary map explicitly, so that Terraform can see all of the necessary dependency edges in order to execute the program:
variable "myfirstvar" {
type = string
}
variable "mysecondvar" {
type = string
}
locals {
variables = tomap({
myfirstvar = {
value = var.myfirstvar
description = "a var to upload"
}
mysecondvar = {
value = var.mysecondvar
description = "another var to upload"
}
})
}
resource "tfe_variable" "test" {
for_each = local.variables
key = each.key
value = each.value.value
category = "terraform"
workspace_id = tfe_workspace.test.id
description = each.value.description
}
I try to create kubernetes_config_map using for_each loop, when all of the data is inside a variable:
variables.tf:
variable "extra_configmaps" {
type = list(object({
name = string
namespace = string
data = object({})
}))
default = []
}
terraform.tfvars:
extra_configmaps = [
{
name = "common"
namespace = "production"
data = {
"somekey" = "somevalue"
}
}
]
main.tf:
resource "kubernetes_config_map" "extra_configmaps" {
for_each = { for record in var.extra_configmaps : record.name => record }
metadata {
name = each.key
namespace = each.value.namespace
}
data = tomap(each.value.data)
}
However, the configmap is created with no data!
# kubernetes_config_map.extra_configmaps["common"] will be created
+ resource "kubernetes_config_map" "extra_configmaps" {
+ id = (known after apply)
+ metadata {
+ generation = (known after apply)
+ name = "common"
+ namespace = "production"
+ resource_version = (known after apply)
+ self_link = (known after apply)
+ uid = (known after apply)
}
}
Plan: 1 to add, 0 to change, 0 to destroy.
I know data expects a map, but the plan doesn't show it even thoughI have provided it.
How do I create dynamic configmaps for Kubernetes without empty data? How do I create dynamic configmaps for Kubernetes without empty data?
You've specified that the type constraint of the data attribute is object({}), which is an object type with no attributes at all. Object type constraints require that at least the given attributes are present, but since you didn't specify any attributes this constraint will match any object value but it will also discard all its attributes during type conversion, because none are specified in the type constraint.
This behavior for a totally-empty object type is, of course, a bit weird. It's a natural but perhaps awkward consequence of the rule that allows declaring an object type constraint that is a subset of an object type defined by a provider's resource type, to allow callers the convenience of passing the whole resource in without having to write out all of the attributes that the module doesn't actually need or care about:
variable "aws_vpc" {
type = object({
id = string
cidr_block = string
# aws_vpc's object type has many other attributes,
# but this module only needs the two above so it
# will accept and silently discard all of the
# others. Caller can therefore do this:
# aws_vpc = aws_vpc.example
})
}
The data attribute of kubernetes_config_map is defined with the type constraint map(string), so I think the best way to get the result you were looking for here is to echo that same type constraint in your variable, like this:
variable "extra_configmaps" {
type = list(object({
name = string
namespace = string
data = map(string)
}))
default = []
}
There's more information about Terraform's type conversion rules in Conversion of Complex Types, which is part of the documentation on Type Constraints.