I have written code that works well in order to create Azure Firewall Network Rules via Terraform. Weeks later i'm asked to add an attribute "expiration" in order to enable users to create temporary network rules.
If the expiration field is 0, means the rule is permanent, if not, i compare the date with the sysdate.
I have tried nearly every possible "for_each" with "for" and "if" combinations, but nothing seems to work. That "expiration" attribute seems inaccessible from outside the loop.
Here are the files of my code:
main.tf
module.tf (edits must be done here, in the dynamic "rule" bloçk of the network_rule_collection block, all other files have been simplified and are here only for better understanding of the code logic)
variables.tf
net_rules.yaml
fw_collections.yaml
Main.tf
locals {
collections = yamldecode(file("../environments/fw_collections.yaml"))
netrules = yamldecode(file("../environments/net_rules.yaml"))
apprules = yamldecode(file("../environments/app_rules.yaml"))
}
module "tdf_az_firewall_policy_rule_collection_group" {
source = "../../modules/tdf_az_firewall_policy_rule_collection_group"
fw_policy_id = module.tdf_az_firewall_policy.firewall_policy_id
fw_c = local.collections.FW_Collections
fw_nr = local.netrules.FW_NET_Rules
fw_ar = local.apprules.FW_APP_Rules
rgname = data.azurerm_resource_group.ipgrg.name
sub = element(split("/", module.tdf_az_virtual_hub_module.virtual_hub_id), 2)
depends_on = [module.tdf_az_firewall_policy, module.tdf_az_firewall]
}
Module.tf
resource "azurerm_firewall_policy_rule_collection_group" "fw-prcg" {
for_each = var.fw_c
name = each.key
firewall_policy_id = var.fw_policy_id
priority = each.value[0]
dynamic "network_rule_collection" {
for_each = { for nr, name in var.fw_nr : nr => name if nr == each.key }
content {
name = each.key
priority = each.value[0]
action = each.value[1]
dynamic "rule" {
for_each = { for x, y in var.fw_nr[each.key] : x => y.expiration if timecmp(y.expiration, timestamp()) == 1 }
content {
name = rule.value.name
protocols = rule.value.protocols
source_ip_groups = [for s in rule.value.source_ip_groups : join("/", ["/subscriptions", var.sub, "resourceGroups", var.rgname, "providers/Microsoft.Network/ipGroups", s])]
destination_ip_groups = [for d in rule.value.destination_ip_groups : join("/", ["/subscriptions", var.sub, "resourceGroups", var.rgname, "providers/Microsoft.Network/ipGroups", d])]
destination_ports = rule.value.destination_ports
}
}
}
}
dynamic "application_rule_collection" {
for_each = { for ar, name in var.fw_ar : ar => name if ar == each.key }
content {
name = each.key
priority = each.value[0]
action = each.value[1]
dynamic "rule" {
for_each = var.fw_ar[each.key]
content {
name = rule.value.name
dynamic "protocols" {
for_each = rule.value.protocols
content {
type = protocols.value.type
port = protocols.value.port
}
}
source_ip_groups = [for s in rule.value.source_ip_groups : join("/", ["/subscriptions", var.sub, "resourceGroups", var.rgname, "providers/Microsoft.Network/ipGroups", s])]
destination_fqdns = rule.value.destination_fqdns
destination_urls = rule.value.destination_urls
terminate_tls = rule.value.terminate_tls
web_categories = rule.value.web_categories
destination_fqdn_tags = rule.value.destination_fqdn_tags
}
}
}
}}
variables.tf
variable "fw_policy_id" {}
variable "fw_c" {}
variable "fw_nr" {}
variable "fw_ar" {}
variable "sub" {}
variable "rgname" {}
netrules.yaml
FW_NET_Rules:
Blacklist-Net:
- name: blacklisted_inbound_ips_rule_inbound
source_ip_groups: ["blacklisted_inbound_ips"]
destination_ip_groups: ["Any-ip"]
estination_ports: ["*"]
protocols: ["Any"]
expiration: 0
- name: k8saas_backlisted_idps_rule_inbound
source_ip_groups: ["k8saas_blacklist_ip_inbound"]
destination_ip_groups: ["Any-ip"]
destination_ports: ["*"]
protocols: ["Any"]
expiration: 0
Mobility:
- name: Mobility-LAN-VNET
source_ip_groups: ["Mobility-LAN"]
destination_ip_groups: ["Mobility-VNET"]
destination_ports: ["443"]
protocols: ["TCP"]
expiration: "2022-12-12T00:00:00Z"
- name: Mobility-LAN-To-Data
source_ip_groups: ["Mobility-LAN"]
destination_ip_groups: ["Data-VNET"]
destination_ports: ["443"]
protocols: ["TCP"]
expiration: "2022-12-12T00:00:00Z"
fw_collections.yaml
FW_Collections:
Blacklist-Net: [100, "Deny"]
Mobility: [105, "Allow"]
Related
So i have a terraform variable type list(string) that is called zones and contains
zones = [
"example.com",
"example2.com",
"example3.com",
...
]
and i m using data cloudflare_zones resource to fetch all zones info
data "cloudflare_zones" "zones" {
for_each = toset(var.zones)
filter {
name = each.value
}
}
Output for each of the zones
data.cloudflare_zones.zones["example.com"]
{
"filter" = tolist([
{
"account_id" = ""
"lookup_type" = "exact"
"match" = ""
"name" = "example.com"
"paused" = false
"status" = ""
},
])
"id" = "9f7xxx3xxxx"
"zones" = tolist([
{
"id" = "e13xxxx"
"name" = "example.com"
},
])
}
To fetch the zone id you need to parse data.cloudflare_zones as below:
data.cloudflare_zones.zones["example.com"].zones[0].id
What i want to create then is a variable that will be an object with all the zones names as keys and zone ids ad values, so i can use them in other resources.
For Example:
zones_ids =
{
"example.com" = "xxxzone_idxxx",
"example2.com" = "xxxzone_id2xxx",
"example3.com" = "xxxzone_id3xxx",
...
}
I would like to achieve this inside locals block
locals {
...
}
That should be easy:
locals {
zones_ids = { for k,v in data.cloudflare_zones.zones: k => v.zones[0].id }
}
Or alternatively:
locals {
zones_ids = { for k,v in data.cloudflare_zones.zones: v.zones[0].name => v.zones[0].id }
}
The above answers helped me here but did not give me the final answer. For anyone looking to update A records for cloudflare with a list of domain names that gets the zone_ids for you. Here is how I did it:
locals {
domains = ["example1.com", "example2.com"]
}
data "cloudflare_zones" "zones" {
count = "${length(local.domains)}"
filter {
name = "${element(local.domains, count.index)}"
}
}
locals {
zones_ids = { for k,v in data.cloudflare_zones.zones: k => v.zones[0].id }
}
resource "cloudflare_record" "redir-A-record" {
for_each = local.zones_ids
zone_id = each.value
name = "#"
value = "24.1.1.1"
type = "A"
proxied = false
}
resource "cloudflare_record" "redir-A-record-www" {
for_each = local.zones_ids
zone_id = each.value
name = "www"
value = "24.1.1.1"
type = "A"
proxied = false
}
Getting output for these values did not seem to work based on the above answer. This could of just been my confusion but I wanted to print out the zone_id for each domain. I found since it is a tuple it requires the use of a number instead of a name so I was required to do the following to get the proper output:
# Get information for Domain 1
output "Domain_Information" {
value = data.cloudflare_zones.zones[0].zones[0].id
}
# Get information for Domain 2
output "Domain_Information2" {
value = data.cloudflare_zones.zones[1].zones[0].id
}
There is a way to loop this in output with Terraform but in my case I only had 2 domains and did not need to spend additional time on this.
Now when I want to spin up a server in AWS and have multiple domains point to 1 IP address this code works.
This line here posted by #Marko E was the solution to my issues for looping and saving the data that could be used later.:
locals {
zones_ids = { for k,v in data.cloudflare_zones.zones: k => v.zones[0].id }
}
I am fairly new to terraform and trying to create a google_compute_backend_service using terraform and there are multiple backend blocks inside the resource as shown below:
resource "google_compute_backend_service" "app-backend" {
log_config {
enable = "true"
sample_rate = "1"
}
name = "app-backend"
port_name = "http-34070"
project = "my-project"
protocol = "HTTP"
session_affinity = "NONE"
timeout_sec = "30"
backend {
group = "insatnce-group1"
}
backend {
group = "instance-group2"
}
backend {
group = "instance-group3"
}
health_checks = [google_compute_http_health_check.app-http-l7.name]
}
As seen in the code block above the backend block repeats multiple times. I want to make it dynamic so I do not have to write multiple blocks manually.
I tried the following:
Created a variable in the variables.tf file that contains all the instance groups:
variable "groups" {
type = list(object({
name = string
}))
default = [{ name = "instance-group1"},
{ name = "instance-group2"},
{ name = "instance-group3"}]
}
And modified my resource block to this:
resource "google_compute_backend_service" "app-backend" {
log_config {
enable = "true"
sample_rate = "1"
}
name = "app-backend"
port_name = "http-34070"
project = "my-project"
protocol = "HTTP"
session_affinity = "NONE"
timeout_sec = "30"
dynamic "backend" {
for_each = var.groups
iterator = item
group = item.value.name
}
health_checks = [google_compute_http_health_check.app-http-l7.name]
}
However, when I execute terraform plan I get the following error:
Error: Unsupported argument
│
│ on backend_service.tf line 15, in resource "google_compute_backend_service" "app-backend":
│ 15: group = item.value.name
│
│ An argument named "group" is not expected here.
Where am I going wrong? Is there a better way to achieve this?
You can check the dynamic blocks documentation for the syntax. Otherwise, you had the right idea.
dynamic "backend" {
for_each = var.groups
content {
group = backend.value.name
}
}
You can also simplify the variable structure to make this even easier.
variable "groups" {
type = set(string)
default = ["instance-group1", "instance-group2", "instance-group3"]
}
dynamic "backend" {
for_each = var.groups
content {
group = backend.value
}
}
I have a terraform file with the following contents:
resource "aws_iam_group" "developers" {
name = each.value
for_each = toset(var.groups)
}
resource "aws_iam_group_membership" "developers_team" {
name = "Developers Team"
users = [each.value]
for_each = toset(var.group_users)
group = aws_iam_group.developers.name
}
I would like to reference aws_iam_group from aws_iam_group_membership. How would I do that? The current terraform file is not working.
I tried this:
group = aws_iam_group.developers[each.value] //This will not work since it uses the for_each of
its own code block
The variable file is as below:
variable "groups" {
type = list(string)
default = [
"terraform_group1",
"terraform_group2",
"terraform_group3",
]
}
variable "group_users" {
type = list(string)
default = [
"terraform_test_user1",
"terraform_test_user2"
]
}
Edit:
I tried the below, but it is not working
resource "aws_iam_group_membership" "developers_team" {
name = "Developers Team"
users = [for group_user in var.group_users : group_user]
for_each = toset(var.groups)
group = aws_iam_group.developers[each.key]
}
Apparently, this is working:
resource "aws_iam_group" "developer" {
name = "truedeveloper"
}
resource "aws_iam_group_membership" "developers_team" {
name = "Developers_Team"
users = [for group_user in var.group_users : group_user]
for_each = toset(var.groups)
group = aws_iam_group.developer.name
}
How can I iterate over the JSON rendered data.aws_iam_policy_document documents within an aws_iam_policy?
data "aws_iam_policy_document" "role_1" {
statement {
sid = "CloudFront1"
actions = [
"cloudfront:ListDistributions",
"cloudfront:ListStreamingDistributions"
]
resources = ["*"]
}
}
data "aws_iam_policy_document" "role_2" {
statement {
sid = "CloudFront2"
actions = [
"cloudfront:CreateInvalidation",
"cloudfront:GetDistribution",
"cloudfront:GetInvalidation",
"cloudfront:ListInvalidations"
]
resources = ["*"]
}
}
variable "role_policy_docs" {
type = list(string)
description = "Policies associated with Role"
default = [
"data.aws_iam_policy_document.role_1.json",
"data.aws_iam_policy_document.role_2.json",
]
}
locals {
role_policy_docs = { for s in var.role_policy_docs: index(var.role_policy_docs, s) => s}
}
resource "aws_iam_policy" "role" {
for_each = local.role_policy_docs
name = format("RolePolicy-%02d", each.key)
description = "Custom Policies for Role"
policy = each.value
}
resource "aws_iam_role_policy_attachment" "role" {
for_each = { for p in aws_iam_policy.role : p.name => p.arn }
role = aws_iam_role.role.name
policy_arn = each.value
}
This example has been reduced down to the very basics. The policy documents are dynamically generated with the source_json and override_json conventions. I cannot simply combine the statements into a single policy document.
Terraform Error:
Error: "policy" contains an invalid JSON policy
on role.tf line 35, in resource "aws_iam_policy" "role":
35: policy = each.value
This:
variable "role_policy_docs" {
type = list(string)
description = "Policies associated with Role"
default = [
"data.aws_iam_policy_document.role_1.json",
"data.aws_iam_policy_document.role_2.json",
]
}
Is literally defining those default values as strings, so what you're getting is this:
+ role_policy_docs = {
+ 0 = "data.aws_iam_policy_document.role_1.json"
+ 1 = "data.aws_iam_policy_document.role_2.json"
}
If you tried removing the quotations around the data blocks, it will not be valid because you cannot use variables in default definitions. Instead, assign your policy documents to a new local, and use that local in your for loop instead:
locals {
role_policies = [
data.aws_iam_policy_document.role_1.json,
data.aws_iam_policy_document.role_2.json,
]
role_policy_docs = {
for s in local.role_policies :
index(local.role_policies, s) => s
}
}
I'd like to create every type of DNS record for a domain using a generic module, and so be able to call it with something like:
module "example_com_dns" {
source = "[PATH_TO_MODULES]/modules/dns"
domain = "example.com"
a_records = {
"#" = [SOME IP]
"www" = [SOME IP]
"home" = [SOME IP]
}
txt_records = {
"#" = "txt-foobar1"
"#" = "txt-foobar2"
"mail._domainkey.self" = "foobar"
}
mx_entries = {
"10" = "mail.someprovider.com"
"20" = "mail2.someprovider.com"
}
cname_records {
"cname-foo" = "cname-bar
}
}
I have something that works fine for A, CNAME , and MX records, but TXT has an edge case which I need to work around. My module has resource blocks for each type of record, which run through loops. I'll just paste the TXT one, but they're all the same:
resource "digitalocean_record" "this_txt_record" {
for_each = var.txt_records
domain = var.domain
type = "TXT"
name = each.key
value = each.value
}
This all works fine, except for the fact that since there are 2 records with "#" for their key, it results in only the last one being created (in my example above, this being "txt-foobar2"):
...
# module.example_com.digitalocean_record.this_txt_record["#"] will be created
+ resource "digitalocean_record" "this_txt_record" {
+ domain = "example.com"
+ fqdn = (known after apply)
+ id = (known after apply)
+ name = "#"
+ ttl = (known after apply)
+ type = "TXT"
+ value = "txt-foobar2"
}
I'd like for it to create both "txt-foobar1" and "txt-foobar2", even given non-unique keys in the map.
Perhaps this is the wrong way and I just need to figure out a clever loop for for parsing this structure instead?:
txt_records = [
{ "#" = "foo" },
{ "#" = "bar"},
{ "mail._domainkey.self" = "foobar"}
]
If so, I'm currently failing there too :)
Resources cannot be created by for_each'ing a list since there must be a unique key that will become part of the terraform resource name. List indexes cannot be a reliable key since your TF plan will be all messed up if you reorder items in the list.
Maps on the other hand do have unique keys by definition.
You can generate a map from a list though! I found this little trick here. Note you additionally need to manually compute the unique map key (${txt_record[0]}=${txt_record[1]} in the example below).
Your resources with updates in place:
module "example_com_dns" {
...
txt_records = [
["#", "txt-foobar1"],
["#", "txt-foobar2"],
["mail._domainkey.self", "foobar"],
]
}
resource "digitalocean_record" "this_txt_record" {
for_each = {for txt_record in var.txt_records: "${txt_record[0]}=${txt_record[1]}" => txt_record}
domain = var.domain
type = "TXT"
name = each.value[0]
value = each.value[1]
}
or slightly more verbose if you prefer:
module "example_com_dns" {
...
txt_records = [
{name: "#", value: "txt-foobar1"},
{name: "#", value: "txt-foobar2"},
{name: "mail._domainkey.self", value: "foobar"},
]
}
resource "digitalocean_record" "this_txt_record" {
for_each = {for txt_record in var.txt_records: "${txt_record.name}=${txt_record.value}" => txt_record}
domain = var.domain
type = "TXT"
name = each.value.name
value = each.value.value
}
Alternative way to already given one is to use the following:
variable "txt_records" {
default = {
"#" = ["foo", "bar"],
"mail._domainkey.self" = ["foobar"]
}
}
Then you can flatten the txt_records using:
locals {
txt_records_flat = merge([
for key, values in var.txt_records:
{for value in values:
"${key}-${value}" => {"record_name" = key, "record_value" = value}
}
]...)
}
which results in local.txt_records_flat of:
{
"#-bar" = {
"record_name" = "#"
"record_value" = "bar"
}
"#-foo" = {
"record_name" = "#"
"record_value" = "foo"
}
"mail._domainkey.self-foobar" = {
"record_name" = "mail._domainkey.self"
"record_value" = "foobar"
}
}
Then you use it:
resource "digitalocean_record" "this_txt_record" {
for_each = local.txt_records_flat
domain = var.domain
type = "TXT"
name = each.value.record_name
value = each.value.record_value
}