How do I create dynamic configmaps for Kubernetes without empty data? - terraform

I try to create kubernetes_config_map using for_each loop, when all of the data is inside a variable:
variables.tf:
variable "extra_configmaps" {
type = list(object({
name = string
namespace = string
data = object({})
}))
default = []
}
terraform.tfvars:
extra_configmaps = [
{
name = "common"
namespace = "production"
data = {
"somekey" = "somevalue"
}
}
]
main.tf:
resource "kubernetes_config_map" "extra_configmaps" {
for_each = { for record in var.extra_configmaps : record.name => record }
metadata {
name = each.key
namespace = each.value.namespace
}
data = tomap(each.value.data)
}
However, the configmap is created with no data!
# kubernetes_config_map.extra_configmaps["common"] will be created
+ resource "kubernetes_config_map" "extra_configmaps" {
+ id = (known after apply)
+ metadata {
+ generation = (known after apply)
+ name = "common"
+ namespace = "production"
+ resource_version = (known after apply)
+ self_link = (known after apply)
+ uid = (known after apply)
}
}
Plan: 1 to add, 0 to change, 0 to destroy.
I know data expects a map, but the plan doesn't show it even thoughI have provided it.
How do I create dynamic configmaps for Kubernetes without empty data? How do I create dynamic configmaps for Kubernetes without empty data?

You've specified that the type constraint of the data attribute is object({}), which is an object type with no attributes at all. Object type constraints require that at least the given attributes are present, but since you didn't specify any attributes this constraint will match any object value but it will also discard all its attributes during type conversion, because none are specified in the type constraint.
This behavior for a totally-empty object type is, of course, a bit weird. It's a natural but perhaps awkward consequence of the rule that allows declaring an object type constraint that is a subset of an object type defined by a provider's resource type, to allow callers the convenience of passing the whole resource in without having to write out all of the attributes that the module doesn't actually need or care about:
variable "aws_vpc" {
type = object({
id = string
cidr_block = string
# aws_vpc's object type has many other attributes,
# but this module only needs the two above so it
# will accept and silently discard all of the
# others. Caller can therefore do this:
# aws_vpc = aws_vpc.example
})
}
The data attribute of kubernetes_config_map is defined with the type constraint map(string), so I think the best way to get the result you were looking for here is to echo that same type constraint in your variable, like this:
variable "extra_configmaps" {
type = list(object({
name = string
namespace = string
data = map(string)
}))
default = []
}
There's more information about Terraform's type conversion rules in Conversion of Complex Types, which is part of the documentation on Type Constraints.

Related

Conditional dynamic block

Trying to set an optional block called "sensitive_labels" and i'm trying to set it as an optional one, however, doesn't work.
My code:
variables.tf:
variable "notification_channels" {
type = any
}
variable "project_id" {
type = string
}
main.tf:
project = var.project_id
for_each = { for k, v in var.notification_channels : k => v }
type = each.value.type
display_name = each.value.display_name
description = each.value.description
labels = each.value.labels
enabled = each.value.enabled
dynamic "sensitive_labels" {
for_each = each.value.sensitive_labels != {} ?[each.value.sensitive_labels] : []
content {
auth_token = lookup(sensitive_labels.value, "auth_token", null)
}
}
}
dev.tfvars:
notification_channels = [
{
type = "email"
display_name = "a channel to send emails"
description = "a nice channel"
labels = {
email_address = "HeyThere#something.com"
}
enabled = true
sensitive_labels = {} // this one doesn't give any errors.
},
{
type = "email"
display_name = "HeyThere Email"
description = "a channel to send emails"
labels = {
email_address = "HeyThere2#something.com"
}
enabled = true
}
]
Getting:
Error: Unsupported attribute
on notification_channels.tf line 11, in resource "google_monitoring_notification_channel" "channels":
11: for_each = each.value.sensitive_labels != {} ? [each.value.sensitive_labels] : []
│ ├────────────────
each.value is object with 5 attributes
This object does not have an attribute named "sensitive_labels".
How can I make setting sensitive_labels an optional attribute here?
EDIT:
This seems to work but feels a bit off:
project = var.project_id
for_each = { for k, v in var.notification_channels : k => v }
type = each.value.type
display_name = each.value.display_name
description = each.value.description
labels = each.value.labels
enabled = each.value.enabled
dynamic "sensitive_labels" {
for_each = lookup(each.value, "sensitive_labels", {})
content {
auth_token = lookup(sensitive_labels.value, "auth_token", null)
}
}
}
Is there a better way that doesn't feel hacky?
A good place to start is to properly define a type constraint for your input variable, so that Terraform can understand better what data structure is expected and help ensure that the given value matches that data structure.
type = any is not there so you can skip defining a type constraint, but instead for the very rare situation where a module is just passing a data structure verbatim to a provider without interpreting it at all. Since your module clearly expects that input variable to be a map of objects (based on how you've used it), you should tell Terraform what object type you are expecting to recieve:
variable "notification_channels" {
type = map(object({
type = string
display_name = string
labels = map(string)
enabled = bool
sensitive_labels = object({
auth_token = string
password = string
service_key = string
})
}))
}
From your example it seems like you want sensitive_labels to be optional, so that the caller of the module can omit it. In that case you can use the optional modifier when you declare that particular attribute, and also the three attributes inside it:
sensitive_labels = optional(object({
auth_token = optional(string)
password = optional(string)
service_key = optional(string)
}))
An attribute that's marked as optional can be omitted by the caller, and in that case Terraform will automatically set it to null inside your module to represent that it wasn't set.
Now you can use this variable elsewhere in your module and safely assume that it will always have exactly the type defined in the variable block:
resource "google_monitoring_notification_channel" "channels" {
for_each = var.notification_channels
project = var.project_id
type = each.value.type
display_name = each.value.display_name
description = each.value.description
labels = each.value.labels
enabled = each.value.enabled
dynamic "sensitive_labels" {
for_each = each.value.sensitive_labels[*]
content {
auth_token = sensitive_labels.value.auth_token
password = sensitive_labels.value.password
service_key = sensitive_labels.value.service_key
}
}
}
The each.value.sensitive_labels[*] expression is a splat expression using the single values as lists feature, which concisely transforms the given value into either a one-element list or a zero-element list depending on whether the value is null. That effectively means that there will be one sensitive_labels block if each.value.sensitive_labels is set, and zero blocks of that type if that attribute is unset (null).
The attributes inside those blocks can also just be assigned directly without any special logic, because Terraform will have automatically set them to null if not specified by the caller and setting a resource argument to null is always the same as not setting it at all.
If you take the time to actually describe the types of variables you expect then it tends to make logic elsewhere in the module much simpler, because you no longer need to deal with all of the ways in which the caller might pass you an incorrect value: Terraform will either convert the value automatically to the expected type if possible, or will report an error to the caller explaining why the value they provided isn't acceptable.

Configure Terraform map(object{}) variables where some key: values are sensitive and others have regex validation. For_each can't handle it

I use a lot of resource blocks with for_each that call variables of type map(object({})).
I need to both mark some values sensitive = true (i.e. password) AND also regex-validate other values. I've referenced Terraform - Adding Validation for type = map(object()) in variables.tf without recourse.
If I use these two functions in my variable declarations, I get "Error: Invalid for_each argument" with verbose explanation:
Sensitive values, or values derived from sensitive values, cannot be used as for_each arguments. If used, the sensitive value could be exposed as a resource instance key.
My code builds Cisco ACI [network] configuration (if that's relevant). What are some other ways I could accomplish these requirements, since for_each doesn't allow it?
main.tf
resource "aci_vmm_credential" "vmmcredLocalName" {
for_each = var.vmmacc_map
vmm_domain_dn = aci_vmm_domain.vmmdomainLocalName[each.key].id
name = each.value.name
annotation = "orchestrator:terraform"
pwd = each.value.password
usr = each.value.username
}
Variables.tf
variable "vmmacc_map" {
type = map(object( {
name = string
password = string
username = string
}))
sensitive = true
validation {
condition = (
can(regexall("
}
terraform.tfvars
vmmacc_map = {
"ACI_Non_Prod_VDSwitch-mktest" = {
name = "Non_Prod_Creds-mktest"
username = "svc_aci_vmm-mktest"
password = "password"
}
}

Passing data as variable to Terraform module

I am looking for a way to pass data template_cloudinit_config to another module. I am clear about how to pass variables to module for various data types including object, but I am not sure how to do this with data.
In this setup, I have a vm-basic module that will define all the virtual hardware configuration, and postgres Terraform script that will define service related information including cloud init scripts. The intention is to have vm virtual hardware configuration to be highly reusable as module, to allow me focusing only on service related info i.e. postgres, nginx etc.
This is my vm-basic vars.tf file that will accept parameters that will be used in virtual hardware configuration.
variable "prefix" {}
variable "rg" { type = object({
name = string
location = string
}) }
variable "vm_size" {}
variable "private_ip_address" {}
variable "subnet" { type = object({ id = string }) }
variable "data_disk_size_gb" { type = number }
variable "service_name" { type = string }
variable "admin_username" { type = string }
variable "admin_public_key_path" { type = string }
variable "nsg_allow_tcp_ports" { type = list(string) }
locals {
nsg_allow_tcp_ports = {for p in var.nsg_allow_tcp_ports: index(var.nsg_allow_tcp_ports, p) => p}
}
#### DOES NOT WORK ######
#### Expected an equals sign ("=") to mark the beginning of the attribute value. ######
variable "custom_data" { type = object({ data }) }
How custom data will be used in vm-basic module
resource "azurerm_linux_virtual_machine" "vm" {
name = "${var.prefix}-${var.service_name}-vm"
location = var.rg.location
resource_group_name = var.rg.name
...
...
custom_data = var.custom_data.rendered
...
...
}
How the other script will pass parameter to vm-basic module
module "vm-basic" {
source = "../../base/vm"
service_name = var.service_name
prefix = var.prefix
rg = var.rg
vm_size = var.vm_size
private_ip_address = var.private_ip_address
subnet = var.subnet
data_disk_size_gb = var.data_disk_size_gb
admin_username = var.admin_username
admin_public_key_path = var.admin_public_key_path
nsg_allow_tcp_ports = var.nsg_allow_tcp_ports
}
data "template_cloudinit_config" "config" {
gzip = true
base64_encode = true
part {
filename = "init-cloud-config"
content_type = "text/cloud-config"
content = file("init.yaml")
}
part {
filename = "init-shellscript"
content_type = "text/x-shellscript"
content = templatefile("init.sh",
{ hostname = "${var.prefix}-${var.service_name}" }
)
}
}
How can I pass data object to another Terraform module?
In the variable vars.tf file, it's enough to just do
variable "custom_data" {}
In the vm-basic module, refer to the variable through var, similar to others.
custom_data = var.custom_data.rendered
The meaning of the error you saw is that Terraform is expecting the argument to the object type constraint to be name = type pairs, but you only wrote data and so Terraform is reporting that there's a missing =.
To make this work, you'll need to write a valid type constraint. It's not clear to me from your question exactly what custom_data represents, but I do see your later example includes var.custom_data.rendered and so from that I can tell that the type constraint should at least include a rendered attribute in order to make that valid, and the custom_data argument to azurerm_linux_virtual_machine expects a string so I'll match that:
variable "custom_data" {
type = object({
rendered = string
})
}
This means that Terraform will accept any object value which has a rendered attribute that can convert to string, and thus your later reference to var.custom_data.rendered is guaranteed to work and always produce a string value.

How to handle optional dynamic blocks in terraform

I am trying to iterate in resource "launchdarkly_feature_flag" variables with dynamic block that might have optional nested objects - "variations" (could be 0 or 2+):
variable "feature_flags" {
default = {
flag_1 = {
project_key = "project"
key = "number example"
name = "number example flag"
description = "this is a multivariate flag with number variations."
variation_type = "number"
variations = {
value = 100
}
variations = {
value = 300
}
tags = ["example"]
},
flag_2 = {
project_key = "project"
key = "boolean example"
name = "boolean example flag"
description = "this is a boolean flag"
variation_type = "boolean"
tags = ["example2"]
}
}
}
Ive tried various scenarios of how to get all flag and always face different problems. Piece of code:
resource "launchdarkly_feature_flag" "ffs" {
for_each = var.feature_flag_map
project_key = each.value.project_key
key = each.value.key
name = each.value.name
description = each.value.description
variation_type = each.value.variation_type
# main problem here
dynamic variations {
for_each = lookup(each.value, "variations", {}) == {} ? {} : {
content {
value = each.value.variations.value
}
}
}
tags = each.value.tags
}
Could you please help with that? I am using 0.14v of Terraform
The first step would be to tell Terraform what type of value this variable expects. While it's often okay to omit an explicit type for a simple value and let Terraform infer it automatically from the default, when the data structure is this complex it's better to tell Terraform what you intended, because then you can avoid it "guessing" incorrectly and giving you confusing error messages as a result.
The following looks like a suitable type constraint for the default value you showed:
variable "feature_flags" {
type = map(object({
project_key = string
key = string
name = string
description = string
variation_type = string
tags = set(string)
variations = list(object({
value = string
}))
}))
}
With the type written out, Terraform will guarantee that any var.feature_flags value conforms to that type constraint, which means that you can then make your dynamic decisions based on whether the values are null or not:
resource "launchdarkly_feature_flag" "ffs" {
for_each = var.feature_flags
project_key = each.value.project_key
key = each.value.key
name = each.value.name
description = each.value.description
variation_type = each.value.variation_type
tags = each.value.tags
dynamic "variations" {
for_each = each.value.variations != null ? each.value.variations : []
content {
variations.value.value
}
}
}
As written above, Terraform will require that all values in feature_flags have all of the attributes defined, although the caller can set them to null to indicate that they are unset.
At the time of writing, in Terraform v0.14, there is an experimental feature for marking attributes as optional which seems like it would, once stabilized, be suitable for this use-case. Marking some or all of the attributes as optional would allow callers to omit them and thus cause Terraform to automatically set them to null, rather than the caller having to explicitly write out the null value themselves.
Hopefully that feature is stabilized in v0.15, at which point you could return to this and add the optional annotations to some attributes without changing anything else about the module.

How to have conditional resources inside a module with 0.12 for_each

I'm passing my modules a list and it's going to create EC2 instances and eips and attach.
I'm using for_each so users can reorder the list and Terraform won't try to destroy anything.
But how do I use conditional resources now? Do I still use count? If so how, because you can't use count with for_each?
This is my module now:
variable "mylist" {
type = set(string)
description = "Name used for tagging, AD, and chef"
}
variable "createip" {
type = bool
default = true
}
resource "aws_instance" "sdfsdfsdfsdf" {
for_each = var.mylist
user_data = data.template_file.user_data[each.key].rendered
tags = each.value
...
#conditional for EIP
resource "aws_eip" "public-ip" {
for_each = var.mylist
// I can't use this anymore!
// how can I say if true create else don't create
#count = var.createip ? 0 : length(tolist(var.mylist))
instance = aws_instance.aws-vm[each.key].id
vpc = true
tags = each.value
}
I also need to get the value of the mylist item for eip too because I use that to tag the eip. So I think I need to index into the foreach loop somehow and also be able to use count or another list to determine if it's created or not - is that correct?
I think I got it but I don't want to accept until it's confirmed this is not the wrong way (not as a matter of opinion but improper usage that will cause actual problems).
variable "mylist" {
type = set(string)
description = "Name used for tagging, AD, and chef"
}
variable "createip" {
type = bool
default = true
}
locals {
// set will-create-public-ip to empty array if false
// otherwise use same mylist which module uses for creating instances
will-create-public-ip = var.createip ? var.mylist : []
}
resource "aws_instance" "sdfsdfsdfsdf" {
for_each = var.mylist
user_data = data.template_file.user_data[each.key].rendered
tags = each.value
...
resource "aws_eip" "public-ip" {
// will-create-public-ip set to mylist or empty to skip this resource creatation
for_each = will-create-public-ip
instance = aws_instance.aws-vm[each.key].id
vpc = true
tags = each.value
}

Resources