I am attempting to loop over a module equal to the number of times a map appears inside a nested list of maps as follows:
vars.tf
variable "http_tcp_listeners" {
description = "A list of maps describing the HTTP listeners or TCP ports for this NLB"
type = any
default = [
{
"http_tcp_listener" = [
{
port = "80"
protocol = "TCP"
},
{
port = "7364"
protocol = "TCP"
}
]
},
{
"http_tcp_listener" = [
{
port = "8080"
protocol = "TCP"
},
{
port = "7365"
protocol = "TCP"
}
]
}
]
}
main.tf
module "create_network_lb" {
count = length(var."http_tcp_listeners")
source = "../../modules/lb"
subnets = tolist(data.aws_subnet_ids.private_compute[0].ids)
vpc_id = sort(data.aws_vpcs.platform_private_vpc.ids)[0]
target_groups = lookup(var.target_groups[count.index], "target_group", null)
http_tcp_listeners = lookup(var.http_tcp_listeners[count.index], "http_tcp_listener", null)
module
resource "aws_lb_listener" "frontend_http_tcp" {
count = var.create_lb ? length(var.http_tcp_listeners) : 0
load_balancer_arn = aws_lb.default[0].arn
port = var.http_tcp_listeners[count.index]["port"]
protocol = var.http_tcp_listeners[count.index]["protocol"]
dynamic "default_action" {
for_each = length(keys(var.http_tcp_listeners[count.index])) == 0 ? [] : [var.http_tcp_listeners[count.index]]
content {
type = lookup(default_action.value, "action_type", "forward")
target_group_arn = contains([null, "", "forward"], lookup(default_action.value, "action_type", "")) ? aws_lb_target_group.main[lookup(default_action.value, "target_group_index", count.index)].id : null
dynamic "redirect" {
for_each = length(keys(lookup(default_action.value, "redirect", {}))) == 0 ? [] : [lookup(default_action.value, "redirect", {})]
content {
path = lookup(redirect.value, "path", null)
host = lookup(redirect.value, "host", null)
port = lookup(redirect.value, "port", null)
protocol = lookup(redirect.value, "protocol", null)
query = lookup(redirect.value, "query", null)
status_code = redirect.value["status_code"]
}
}
dynamic "fixed_response" {
for_each = length(keys(lookup(default_action.value, "fixed_response", {}))) == 0 ? [] : [lookup(default_action.value, "fixed_response", {})]
content {
content_type = fixed_response.value["content_type"]
message_body = lookup(fixed_response.value, "message_body", null)
status_code = lookup(fixed_response.value, "status_code", null)
}
}
}
}
}
When performing a "terraform plan", it displays only the last "http_tcp_listener" value. The variable for the module must be in format "[{port=80, protocol="TCP"},{port=7364, protocol="TCP"}]" hence, everything after each iteration of "http_tcp_listener".
During troubleshooting, Terraform seems to think that the variable is a tuple with one element per the error:
Error: Invalid index
on main.tf line 86, in module "create_network_lb":
86: http_tcp_listeners = [lookup(var.http_tcp_listeners[1], "http_tcp_listener")]
|----------------
| var.http_tcp_listeners is tuple with 1 element
The given key does not identify an element in this collection value.
If I manually change one of the keys from "http_tcp_listener" to "http_tcp_listener1", and reflect this in the main.tf lookup value, it will display that value. i.e, if I rename the first key and reference it, terraform plan will display ports 80 and 7364 instead of 8080 and 7365.
Any help would be greatly appreciated.
Solved by re-creating the data structure, and using for_each to call the module. Details here.
Related
I have written code that works well in order to create Azure Firewall Network Rules via Terraform. Weeks later i'm asked to add an attribute "expiration" in order to enable users to create temporary network rules.
If the expiration field is 0, means the rule is permanent, if not, i compare the date with the sysdate.
I have tried nearly every possible "for_each" with "for" and "if" combinations, but nothing seems to work. That "expiration" attribute seems inaccessible from outside the loop.
Here are the files of my code:
main.tf
module.tf (edits must be done here, in the dynamic "rule" bloçk of the network_rule_collection block, all other files have been simplified and are here only for better understanding of the code logic)
variables.tf
net_rules.yaml
fw_collections.yaml
Main.tf
locals {
collections = yamldecode(file("../environments/fw_collections.yaml"))
netrules = yamldecode(file("../environments/net_rules.yaml"))
apprules = yamldecode(file("../environments/app_rules.yaml"))
}
module "tdf_az_firewall_policy_rule_collection_group" {
source = "../../modules/tdf_az_firewall_policy_rule_collection_group"
fw_policy_id = module.tdf_az_firewall_policy.firewall_policy_id
fw_c = local.collections.FW_Collections
fw_nr = local.netrules.FW_NET_Rules
fw_ar = local.apprules.FW_APP_Rules
rgname = data.azurerm_resource_group.ipgrg.name
sub = element(split("/", module.tdf_az_virtual_hub_module.virtual_hub_id), 2)
depends_on = [module.tdf_az_firewall_policy, module.tdf_az_firewall]
}
Module.tf
resource "azurerm_firewall_policy_rule_collection_group" "fw-prcg" {
for_each = var.fw_c
name = each.key
firewall_policy_id = var.fw_policy_id
priority = each.value[0]
dynamic "network_rule_collection" {
for_each = { for nr, name in var.fw_nr : nr => name if nr == each.key }
content {
name = each.key
priority = each.value[0]
action = each.value[1]
dynamic "rule" {
for_each = { for x, y in var.fw_nr[each.key] : x => y.expiration if timecmp(y.expiration, timestamp()) == 1 }
content {
name = rule.value.name
protocols = rule.value.protocols
source_ip_groups = [for s in rule.value.source_ip_groups : join("/", ["/subscriptions", var.sub, "resourceGroups", var.rgname, "providers/Microsoft.Network/ipGroups", s])]
destination_ip_groups = [for d in rule.value.destination_ip_groups : join("/", ["/subscriptions", var.sub, "resourceGroups", var.rgname, "providers/Microsoft.Network/ipGroups", d])]
destination_ports = rule.value.destination_ports
}
}
}
}
dynamic "application_rule_collection" {
for_each = { for ar, name in var.fw_ar : ar => name if ar == each.key }
content {
name = each.key
priority = each.value[0]
action = each.value[1]
dynamic "rule" {
for_each = var.fw_ar[each.key]
content {
name = rule.value.name
dynamic "protocols" {
for_each = rule.value.protocols
content {
type = protocols.value.type
port = protocols.value.port
}
}
source_ip_groups = [for s in rule.value.source_ip_groups : join("/", ["/subscriptions", var.sub, "resourceGroups", var.rgname, "providers/Microsoft.Network/ipGroups", s])]
destination_fqdns = rule.value.destination_fqdns
destination_urls = rule.value.destination_urls
terminate_tls = rule.value.terminate_tls
web_categories = rule.value.web_categories
destination_fqdn_tags = rule.value.destination_fqdn_tags
}
}
}
}}
variables.tf
variable "fw_policy_id" {}
variable "fw_c" {}
variable "fw_nr" {}
variable "fw_ar" {}
variable "sub" {}
variable "rgname" {}
netrules.yaml
FW_NET_Rules:
Blacklist-Net:
- name: blacklisted_inbound_ips_rule_inbound
source_ip_groups: ["blacklisted_inbound_ips"]
destination_ip_groups: ["Any-ip"]
estination_ports: ["*"]
protocols: ["Any"]
expiration: 0
- name: k8saas_backlisted_idps_rule_inbound
source_ip_groups: ["k8saas_blacklist_ip_inbound"]
destination_ip_groups: ["Any-ip"]
destination_ports: ["*"]
protocols: ["Any"]
expiration: 0
Mobility:
- name: Mobility-LAN-VNET
source_ip_groups: ["Mobility-LAN"]
destination_ip_groups: ["Mobility-VNET"]
destination_ports: ["443"]
protocols: ["TCP"]
expiration: "2022-12-12T00:00:00Z"
- name: Mobility-LAN-To-Data
source_ip_groups: ["Mobility-LAN"]
destination_ip_groups: ["Data-VNET"]
destination_ports: ["443"]
protocols: ["TCP"]
expiration: "2022-12-12T00:00:00Z"
fw_collections.yaml
FW_Collections:
Blacklist-Net: [100, "Deny"]
Mobility: [105, "Allow"]
I create ALBs based on a JSON file where I have a variable public which can be yes or no.
As below:
[
{
"service-name": "test1",
"public" : "yes"
},
{
"service-name": "test2",
"public" : "no"
}
]
I then use this JSON to create ALBs, with the snippet below, which works fine.
resource "aws_lb_listener" "lb_listener" {
count = length(var.services)
load_balancer_arn = aws_lb.some_alb[count.index].arn
port = 80
protocol = "HTTP"
default_action {
target_group_arn = aws_lb_target_group.some_target[count.index].arn
type = "forward"
}
}
What I am after is to have a dynamic default_action based on the public variable. I want it to redirect to HTTPS when public==yes and forward to the target_group when public==no.
I tried this:
resource "aws_lb_listener" "lb_listener" {
count = length(var.services)
load_balancer_arn = aws_lb.some-alb[count.index].arn
port = 80
protocol = "HTTP"
//Public ALB redirects to port 443
dynamic "default_action" {
for_each = [
for i in var.services : i
if var.services[i].public == "yes"
]
type = "redirect"
redirect {
port = "443"
protocol = "HTTPS"
status_code = "HTTP_301"
}
}
//Private just forwards to the target group
dynamic "default_action" {
for_each = [
for i in var.services : i
if var.services[i].public == "no"
]
default_action {
target_group_arn = aws_lb_target_group.some_target[count.index].arn
type = "forward"
}
}
}
But I got the error:
At least 1 "default_action" blocks are required.
Blocks of type "redirect" are not expected here.
Will appreciate your help. Thanks!
Based on the comments:
Instead of default_action, it should be content.
I'm tring to create a zabbix template with applications defined and trigger.
I can create the template, import my hosts and associate to it.
Now when I try to add the trigger to the template, I receive the error in the object.
this is my
data.tf
data "zabbix_hostgroup" "group" {
name = "Templates"
}
data "zabbix_template" "template" {
for_each = {
common_simple = { name = "Common Simple" }
common_snmp = { name = "Common SNMP" }
class_template = { name = var.class_names[var.class_id] }
}
name = each.value.name
}
data "zabbix_proxy" "proxy" {
for_each = {
for inst in var.instances :
"${inst.instance}.${inst.site}" => inst.site
}
#host = "zabpxy01.${each.value}.mysite.local"
host = "mon-proxy1.${each.value}.mtsite.local"
}
and this is my hosts.tf:
# create host group for specific to service
resource "zabbix_hostgroup" "hostgroup" {
name = var.class_names[var.class_id]
}
# create template
resource "zabbix_template" "template" {
host = var.class_id
name = var.class_names[var.class_id]
description = var.class_names[var.class_id]
groups = [
data.zabbix_hostgroup.group.id
]
}
# create application
resource "zabbix_application" "application" {
hostid = data.zabbix_template.template.id
name = var.class_names[var.class_id]
}
# create snmp disk_total item
resource "zabbix_item_snmp" "disk_total_item" {
hostid = data.zabbix_template.template.id
key = "snmp_disk_root_total"
name = "Disk / total"
valuetype = "unsigned"
delay = "1m"
snmp_oid="HOST-RESOURCES-MIB::hrStorageSize[\"index\", \"HOST-RESOURCES-MIB::hrStorageDescr\", \"/\"]"
depends_on = [
data.zabbix_template.template
]
}
# create snmp disk_used item
resource "zabbix_item_snmp" "disk_used_item" {
hostid = data.zabbix_template.template.id
key = "snmp_disk_root_used"
name = "Disk / used"
valuetype = "unsigned"
delay = "1m"
snmp_oid="HOST-RESOURCES-MIB::hrStorageUsed[\"index\", \"HOST-RESOURCES-MIB::hrStorageDescr\", \"/\"]"
depends_on = [
data.zabbix_template.template
]
}
# create trigger > 75%
resource "zabbix_trigger" "trigger" {
name = "Disk Usage 75%"
expression = "({${data.zabbix_template.template.host}:${zabbix_item_snmp.disk_used_item.key}.last()} / {${data.zabbix_template.template.host}:${zabbix_item_snmp.disk_total_item.key}.last()}) * 100 >= 75"
priority = "warn"
enabled = true
multiple = false
recovery_none = false
manual_close = false
}
# create hosts
resource "zabbix_host" "host" {
for_each = {
for inst in var.instances : "${var.class_id}${format("%02d", inst.instance)}.${inst.site}" => inst
}
host = var.ip_addresses[var.class_id][each.value.site][each.value.instance]["hostname"]
name = var.ip_addresses[var.class_id][each.value.site][each.value.instance]["hostname"]
enabled = false
proxyid = data.zabbix_proxy.proxy["${each.value.instance}.${each.value.site}"].id
groups = [
zabbix_hostgroup.hostgroup.id
]
templates = concat ([
data.zabbix_template.template["common_simple"].id,
data.zabbix_template.template["common_snmp"].id,
zabbix_template.template.id
])
# add SNMP interface
interface {
type = "snmp"
ip = var.ip_addresses[var.class_id][each.value.site][each.value.instance]["mgmt0"]
main = true
port = 161
}
# Add Zabbix Agent interface
interface {
type = "agent"
ip = var.ip_addresses[var.class_id][each.value.site][each.value.instance]["mgmt0"]
main = true
port = 10050
}
macro {
name = "{$INTERFACE_MONITOR}"
value = var.ip_addresses[var.class_id][each.value.site][each.value.instance]["mgmt0"]
}
macro {
name = "{$SNMP_COMMUNITY}"
value = var.ip_addresses[var.class_id][each.value.site][each.value.instance]["snmp"]
}
depends_on = [
zabbix_hostgroup.hostgroup,
data.zabbix_template.template,
data.zabbix_proxy.proxy,
]
}
output "class_template_id" {
value = zabbix_template.template.id
description = "Template ID of created class template for items"
}
When I run "Terraform plan" I receive the error:
Error: Missing resource instance key │ │ on hosts/hosts.tf line 26,
in resource "zabbix_application" "application": │ 26: hostid =
data.zabbix_template.template.id │ │ Because
data.zabbix_template.template has "for_each" set, its attributes must
be accessed on specific instances. │ │ For example, to correlate with
indices of a referring resource, use: │
data.zabbix_template.template[each.key]
Where is my error?
Thanks for the support
UPDATE
I tried to use
output "data_zabbix_template" {
value = data.zabbix_template.template
}
but I don't see any output when I run terraform plan
I tried to modify in:
hostid = data.zabbix_template.template.class_template.id
but I continue to receive the same error:
Error: Missing resource instance key on hosts/hosts.tf line 27, in
resource "zabbix_application" "application": 27: hostid =
data.zabbix_template.template.class_template.id Because
data.zabbix_template.template has "for_each" set, its attributes must
be accessed on specific instances.
For example, to correlate with indices of a referring resource, use:
data.zabbix_template.template[each.key]
Error: Unsupported attribute on hosts/hosts.tf line 27, in resource
"zabbix_application" "application": 27: hostid =
data.zabbix_template.template.class_template.id This object has no
argument, nested block, or exported attribute named "class_template".
UPDATE:
My script for each host taht I'll add, set two existing template ("Common Simple" and "Common SNMP") and create a new template as below:
# module.mytemplate-servers_host.zabbix_template.template will be created
+ resource "zabbix_template" "template" {
+ description = "mytemplate-servers"
+ groups = [
+ "1",
]
+ host = "mytemplate-servers"
+ id = (known after apply)
+ name = "mytemplate-servers"
}
Now my scope is to add on this template an application and set two items and one trigger
When you use for_each in a data source or resource, the output of that data source or resource is a map, where the keys in the map are the same as the keys in the for_each and the values are the regular output of that data/resource for the given input value with that key.
Try using:
output "data_zabbix_template" {
value = data.zabbix_template.template
}
And you'll see what I mean. The output will look something like:
data_zabbix_template = {
common_simple = {...}
common_snmp = {...}
class_template = {...}
}
So in order to use this data source (on the line where the error is being thrown), you need to do:
hostid = data.zabbix_template.template.common_simple.id
And replace common_simple in that line with whichever key in the for_each you want to use. You'll need to do this everywhere that you use data.zabbix_template.template.
I'm trying to deploy a list of Azure Front Doors and their custom https configuration resources. I have a list of Azure Front Door resources which are deployed like this pseudo code. They work correctly (although without custom https configuration)
resource "azurerm_frontdoor" "front_door" {
count = length(local.frontdoors)
... config
}
I then try and add some terraform to create the custom https configuration as described here and useterraform azure frontdoor custom https config docs the following fragment:
resource "azurerm_frontdoor_custom_https_configuration" "custom_https_configuration" {
count = length(local.frontdoors)
for_each = { for frontend in azurerm_frontdoor.front_door[count.index].frontend_endpoint : frontend.id => frontend_id }
frontend_endpoint_id = each.value.frontend_id
custom_https_provisioning_enabled = each.key != "front_door" ? local.frontend_https_configurations[each.key].custom_https_provisioning_enabled : false
dynamic "custom_https_configuration" {
for_each = (each.key != "front_door" ? local.frontend_https_configurations[each.key].custom_https_provisioning_enabled : false) ? [1] : []
content {
certificate_source = "AzureKeyVault"
azure_key_vault_certificate_secret_name = XXXX
azure_key_vault_certificate_secret_version = XXXX
azure_key_vault_certificate_vault_id = XXXX
}
}
}
I'm getting this syntax error:
Error: Invalid combination of "count" and "for_each"
if i try and remove the count, and use the for_each structure instead:
resource "azurerm_frontdoor_custom_https_configuration" "custom_https_configuration" {
for_each = {
for frontdoor in azurerm_frontdoor.front_door :
[
for key, value in frontdoor.frontend_endpoint: value.frontend.id => frontend_id
]
}
frontend_endpoint_id = each.value.frontend_id
custom_https_provisioning_enabled = each.key != "front_door" ? local.frontend_https_configurations[each.key].custom_https_provisioning_enabled : false
dynamic "custom_https_configuration" {
for_each = (each.key != "front_door" ? local.frontend_https_configurations[each.key].custom_https_provisioning_enabled : false) ? [1] : []
content {
certificate_source = "AzureKeyVault"
azure_key_vault_certificate_secret_name = XXXX
azure_key_vault_certificate_secret_version = XXXX
azure_key_vault_certificate_vault_id = XXXX
}
}
}
I get this error instead:
Error: Invalid 'for' expression
on main.tf line 25, in resource "azurerm_frontdoor_custom_https_configuration" "custom_https_configuration":
173: for_each = {
174: for frontdoor in azurerm_frontdoor.front_door :
175: [
176: for key, value in frontdoor.frontend_endpoint: value.frontend.id => frontend_id
177: ]
178: }
Key expression is required when building an object.
How can i have a nested loop so that i can successfully deploy a f
When you need to nest for loops, you need to use the flatten function (for lists) or the merge function in combination with the ... list expansion operator (for maps).
Basically, like so:
// To make a list
for_each = flatten([
for idx1, val1 in var.list1:
[
for idx2, val2 in val2.list_field:
// Here is where you construct whatever value/object for each element
]
])
// To make a list
for_each = merge([
for key1, val1 in var.map1:
{
for key2, val2 in val1.map_field:
// Some key/value pair, such as:
"${key1}-${key2}" => val2
}
]...)
You also have your map comprehension key/value reversed. Try this:
for_each = merge([
for idx, frontdoor in azurerm_frontdoor.front_door :
{
for key, value in frontdoor.frontend_endpoints:
"${idx}-${key}" => {
endpoint_key = key
endpoint_id = value
}
}
]...)
And now within your resource, you can use each.value.endpoint_key and each.value.endpoint_id.
What I want
In terraform, I have a map service_map:
variable "service_map" {
type = map
description = "Map of some services and their ports."
default = {
"dns" = "53"
"web" = "443"
"ssh" = "22"
"proxy" = ""
}
}
To create LB listeners on AWS, I want to call the resource aws_lb_listener, looping over the map service_map, skipping all items without value (in this case, only proxy):
resource "aws_lb_listener" "listeners" {
for_each = var.service_map
load_balancer_arn = aws_lb.all_lbs[each.key].arn
port = each.value
protocol = each.key != "dns" ? "TCP" : "TCP_UDP"
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.service_map-tg[each.key].arn
}
}
What I tried
Create a second, local map with all key=value pairs where value is not empty:
locals {
service_map_temp = [ for service, port in var.service_map : service, port if port != "" ]
}
Which does not work: Extra characters after the end of the 'for' expression.. And I guess there are smarter solutions than that approach.
Idea: Skipping empty each.values:
resource "aws_lb_listener" "listeners" {
for_each = var.service_map
load_balancer_arn = aws_lb.all_lbs[each.key].arn
port = each.value != "" # Skipping part
protocol = each.key != "dns" ? "TCP" : "TCP_UDP"
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.service_map-tg[each.key].arn
}
}
But I doubt that's going to work because I am still calling the resource but with an empty port, which will fail. Since I am just starting out with terraform, I am sure there is a solution I did not think/read about yet.
Your first solution failed because you have used list brackets [ ... ] but you intend to produce a map. To produce a map from a for expression, use map brackets { ... }:
locals {
service_map_temp = {
for service, port in var.service_map :
service => port if port != ""
}
}
The key difference is that a map for expression expects two expressions after the colon (the key and the value), while the list for expression expects only one.
If you like, you can inline that expression directly in the for_each argument, to keep everything together in one block:
resource "aws_lb_listener" "listeners" {
for_each = {
for service, port in var.service_map :
service => port if port != ""
}
load_balancer_arn = aws_lb.all_lbs[each.key].arn
port = each.value
protocol = each.key != "dns" ? "TCP" : "TCP_UDP"
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.service_map-tg[each.key].arn
}
}
I realized that if you set the parameter to null instead of an empty string "", it will automatically remove the key from the map without having to create a for loop.
variable "service_map" {
type = map
description = "Map of some services and their ports."
default = {
"dns" = "53"
"web" = "443"
"ssh" = "22"
"proxy" = null
}
}
output "service_map" {
value = var.service_map
}
$ terraform apply
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
service_map = {
"dns" = "53"
"ssh" = "22"
"web" = "443"
}