Multiple zone ids from cloudflare_zones in terraform - terraform

So i have a terraform variable type list(string) that is called zones and contains
zones = [
"example.com",
"example2.com",
"example3.com",
...
]
and i m using data cloudflare_zones resource to fetch all zones info
data "cloudflare_zones" "zones" {
for_each = toset(var.zones)
filter {
name = each.value
}
}
Output for each of the zones
data.cloudflare_zones.zones["example.com"]
{
"filter" = tolist([
{
"account_id" = ""
"lookup_type" = "exact"
"match" = ""
"name" = "example.com"
"paused" = false
"status" = ""
},
])
"id" = "9f7xxx3xxxx"
"zones" = tolist([
{
"id" = "e13xxxx"
"name" = "example.com"
},
])
}
To fetch the zone id you need to parse data.cloudflare_zones as below:
data.cloudflare_zones.zones["example.com"].zones[0].id
What i want to create then is a variable that will be an object with all the zones names as keys and zone ids ad values, so i can use them in other resources.
For Example:
zones_ids =
{
"example.com" = "xxxzone_idxxx",
"example2.com" = "xxxzone_id2xxx",
"example3.com" = "xxxzone_id3xxx",
...
}
I would like to achieve this inside locals block
locals {
...
}

That should be easy:
locals {
zones_ids = { for k,v in data.cloudflare_zones.zones: k => v.zones[0].id }
}
Or alternatively:
locals {
zones_ids = { for k,v in data.cloudflare_zones.zones: v.zones[0].name => v.zones[0].id }
}

The above answers helped me here but did not give me the final answer. For anyone looking to update A records for cloudflare with a list of domain names that gets the zone_ids for you. Here is how I did it:
locals {
domains = ["example1.com", "example2.com"]
}
data "cloudflare_zones" "zones" {
count = "${length(local.domains)}"
filter {
name = "${element(local.domains, count.index)}"
}
}
locals {
zones_ids = { for k,v in data.cloudflare_zones.zones: k => v.zones[0].id }
}
resource "cloudflare_record" "redir-A-record" {
for_each = local.zones_ids
zone_id = each.value
name = "#"
value = "24.1.1.1"
type = "A"
proxied = false
}
resource "cloudflare_record" "redir-A-record-www" {
for_each = local.zones_ids
zone_id = each.value
name = "www"
value = "24.1.1.1"
type = "A"
proxied = false
}
Getting output for these values did not seem to work based on the above answer. This could of just been my confusion but I wanted to print out the zone_id for each domain. I found since it is a tuple it requires the use of a number instead of a name so I was required to do the following to get the proper output:
# Get information for Domain 1
output "Domain_Information" {
value = data.cloudflare_zones.zones[0].zones[0].id
}
# Get information for Domain 2
output "Domain_Information2" {
value = data.cloudflare_zones.zones[1].zones[0].id
}
There is a way to loop this in output with Terraform but in my case I only had 2 domains and did not need to spend additional time on this.
Now when I want to spin up a server in AWS and have multiple domains point to 1 IP address this code works.
This line here posted by #Marko E was the solution to my issues for looping and saving the data that could be used later.:
locals {
zones_ids = { for k,v in data.cloudflare_zones.zones: k => v.zones[0].id }
}

Related

How to iterate over list(string) and append to same records Terraform

I want to iterate over list of string and apply individual vale to same resource route53
I have list of ip in variable
variable "app_list" {
description = "List of ESAs to be allowed. (For instance, \[192.168.1.123, 10.1.1.11\] etc.)"
type = list(string)
default = \["192.168.1.123","10.1.1.11"\]
}
Creating route53 TXT record where I have to append this variable and create single record
resource "aws_route53_record" "spf_txt" {
zone_id = data.aws_route53_zone.public.zone_id
name = ""
type = "TXT"
ttl = 300
records = \["v=spf1 ip4:192.168.1.123 ip4:10.1.1.11 \~all"\]
}
Here i used for_each and count. it is trying to create two seperate TXT record. How can I iterate the list and pass it to record.
Please someone help me
Tried :
resource "aws_route53_record" "spf_txt" {
zone_id = data.aws_route53_zone.public.zone_id
name = ""
type = "TXT"
ttl = 300
count = length(var.app_list)
for_each = var.app_list
records = \["v=spf1 ip4:value \~all"\]
}
It errored as two elements with tuples
tried this as well
locals {
spf_record = "${formatlist("ip4:", var.app_list)}"
}
resource "aws_route53_record" "spf_txt" {
zone_id = data.aws_route53_zone.public.zone_id
name = ""
type = "TXT"
ttl = 300
records = \["v=spf1 ${local.spf_record} ip4:${data.aws_nat_gateway.nat_ip.public_ip} \~all"\]
}
It failed with this error
spf_record = "${formatlist("ip4:", var.app_list)}"
while calling formatlist(format, args...)
var.app_esas is list of string with 2 elements
Call to function "formatlist" failed: error on format
iteration 0: too many arguments; no verbs in format
string.
Even if you don't use count or for_each you would accomplish the purpose, I think.
resource "aws_route53_record" "spf_txt" {
...
records = ["v=spf1 ${join(" ", [for i in var.app_list : "ip4:${i}"])} ~all"]
}
Test:
variable "app_list" {
description = "List of ESAs to be allowed. (For instance, [192.168.1.123, 10.1.1.11] etc.)"
type = list(string)
default = ["192.168.1.123","10.1.1.11"]
}
output "spf_txt" {
value = ["v=spf1 ${join(" ", [for i in var.app_list : "ip4:${i}"])} ~all"]
}
$ terraform plan
Changes to Outputs:
+ spf_txt = [
+ "v=spf1 ip4:192.168.1.123 ip4:10.1.1.11 ~all",
]
If using formatlist:
resource "aws_route53_record" "spf_txt" {
...
records = ["v=spf1 ${join(" ", formatlist("ip4:%s", var.app_list))} ~all"]
}

How to Use a Non-unique Key in Terraform Resource Loop to Create Unique Resources

I'd like to create every type of DNS record for a domain using a generic module, and so be able to call it with something like:
module "example_com_dns" {
source = "[PATH_TO_MODULES]/modules/dns"
domain = "example.com"
a_records = {
"#" = [SOME IP]
"www" = [SOME IP]
"home" = [SOME IP]
}
txt_records = {
"#" = "txt-foobar1"
"#" = "txt-foobar2"
"mail._domainkey.self" = "foobar"
}
mx_entries = {
"10" = "mail.someprovider.com"
"20" = "mail2.someprovider.com"
}
cname_records {
"cname-foo" = "cname-bar
}
}
I have something that works fine for A, CNAME , and MX records, but TXT has an edge case which I need to work around. My module has resource blocks for each type of record, which run through loops. I'll just paste the TXT one, but they're all the same:
resource "digitalocean_record" "this_txt_record" {
for_each = var.txt_records
domain = var.domain
type = "TXT"
name = each.key
value = each.value
}
This all works fine, except for the fact that since there are 2 records with "#" for their key, it results in only the last one being created (in my example above, this being "txt-foobar2"):
...
# module.example_com.digitalocean_record.this_txt_record["#"] will be created
+ resource "digitalocean_record" "this_txt_record" {
+ domain = "example.com"
+ fqdn = (known after apply)
+ id = (known after apply)
+ name = "#"
+ ttl = (known after apply)
+ type = "TXT"
+ value = "txt-foobar2"
}
I'd like for it to create both "txt-foobar1" and "txt-foobar2", even given non-unique keys in the map.
Perhaps this is the wrong way and I just need to figure out a clever loop for for parsing this structure instead?:
txt_records = [
{ "#" = "foo" },
{ "#" = "bar"},
{ "mail._domainkey.self" = "foobar"}
]
If so, I'm currently failing there too :)
Resources cannot be created by for_each'ing a list since there must be a unique key that will become part of the terraform resource name. List indexes cannot be a reliable key since your TF plan will be all messed up if you reorder items in the list.
Maps on the other hand do have unique keys by definition.
You can generate a map from a list though! I found this little trick here. Note you additionally need to manually compute the unique map key (${txt_record[0]}=${txt_record[1]} in the example below).
Your resources with updates in place:
module "example_com_dns" {
...
txt_records = [
["#", "txt-foobar1"],
["#", "txt-foobar2"],
["mail._domainkey.self", "foobar"],
]
}
resource "digitalocean_record" "this_txt_record" {
for_each = {for txt_record in var.txt_records: "${txt_record[0]}=${txt_record[1]}" => txt_record}
domain = var.domain
type = "TXT"
name = each.value[0]
value = each.value[1]
}
or slightly more verbose if you prefer:
module "example_com_dns" {
...
txt_records = [
{name: "#", value: "txt-foobar1"},
{name: "#", value: "txt-foobar2"},
{name: "mail._domainkey.self", value: "foobar"},
]
}
resource "digitalocean_record" "this_txt_record" {
for_each = {for txt_record in var.txt_records: "${txt_record.name}=${txt_record.value}" => txt_record}
domain = var.domain
type = "TXT"
name = each.value.name
value = each.value.value
}
Alternative way to already given one is to use the following:
variable "txt_records" {
default = {
"#" = ["foo", "bar"],
"mail._domainkey.self" = ["foobar"]
}
}
Then you can flatten the txt_records using:
locals {
txt_records_flat = merge([
for key, values in var.txt_records:
{for value in values:
"${key}-${value}" => {"record_name" = key, "record_value" = value}
}
]...)
}
which results in local.txt_records_flat of:
{
"#-bar" = {
"record_name" = "#"
"record_value" = "bar"
}
"#-foo" = {
"record_name" = "#"
"record_value" = "foo"
}
"mail._domainkey.self-foobar" = {
"record_name" = "mail._domainkey.self"
"record_value" = "foobar"
}
}
Then you use it:
resource "digitalocean_record" "this_txt_record" {
for_each = local.txt_records_flat
domain = var.domain
type = "TXT"
name = each.value.record_name
value = each.value.record_value
}

Default DNS records in every zone managed via terraform (eg. MX records)

I'm looking for a way to manage cloudflare zones and records with terraform and create some default records (eg. MX) in every zone that is managed via terraform, something like this:
resource "cloudflare_zone" "example_net" {
type = "full"
zone = "example.net"
}
resource "cloudflare_zone" "example_com" {
type = "full"
zone = "example.com"
}
resource "cloudflare_record" "mxrecord"{
for_each=cloudflare_zone.*
name = "${each.value.zone}"
priority = "1"
proxied = "false"
ttl = "1"
type = "MX"
value = "mail.foo.bar"
zone_id = each.value.id
}
Does anyone have a clue for me how to achieve this (and if this is even possible...)?
Thanks a lot!
You could create a module responsible for the zone resource, e.g.:
# modules/cf_zone/main.tf
resource "cloudflare_zone" "cf_zone" {
type = "full"
zone = var.zone_name
}
resource "cloudflare_record" "mxrecord"{
name = "${cloudflare_zone.cf_zone.name}"
priority = "1"
proxied = "false"
ttl = "1"
type = "MX"
value = "mail.foo.bar"
zone_id = "${cloudflare_zone.cf_zone.id}"
}
# main.tf
module "example_net" {
source = "./modules/cf_zone"
zone_name = "example_net"
}
module "example_com" {
source = "./modules/cf_zone"
zone_name = "example_com"
}
This would give you an advantage on creation of default resources and settings per zone (DNS entries, security settings, page rules, etc.). It is also a good way to keep all the default values in a single place for review.
You can ready more about terraform modules here.
This is easy to do if you use a module, as was correctly noted in the other answer, but you don't have to create one, you can use this module.
Then your configuration will look like this:
terraform {
required_providers {
cloudflare = {
source = "cloudflare/cloudflare"
}
}
}
variable "cloudflare_api_token" {
type = string
sensitive = true
description = "The Cloudflare API token."
}
provider "cloudflare" {
api_token = var.cloudflare_api_token
}
locals {
domains = [
"example.com",
"example.net"
]
mx = "mail.foo.bar"
}
module "domains" {
source = "registry.terraform.io/alex-feel/zone/cloudflare"
version = "1.8.0"
for_each = toset(local.domains)
zone = each.value
records = [
{
record_name = "mx_1"
type = "MX"
value = local.mx
priority = 1
}
]
}
You can find an example of using this module that matches your question here.

How can I pass a comma separated array to a resource in terraform v0.12.0?

In the following code block I'm trying to pass an array of server names to the attributes_json block:
resource "aws_instance" "consul-server" {
ami = var.consul-server
instance_type = "t2.nano"
key_name = var.aws_key_name
iam_instance_profile = "dna_inst_mgmt"
vpc_security_group_ids = [
"${aws_security_group.yutani_consul.id}",
"${aws_security_group.yutani_ssh.id}"
]
subnet_id = "${aws_subnet.public_1_subnet_us_east_1c.id}"
associate_public_ip_address = true
tags = {
Name = "consul-server${count.index}"
}
root_block_device {
volume_size = "30"
delete_on_termination = "true"
}
connection {
type = "ssh"
user = "chef"
private_key = "${file("${var.aws_key_path}")}"
timeout = "2m"
agent = false
host = self.public_ip
}
count = var.consul-server_count
provisioner "chef" {
attributes_json = <<-EOF
{
"consul": {
"servers": ["${split(",",aws_instance.consul-server[count.index].id)}"]
}
}
EOF
use_policyfile = true
policy_name = "consul_server"
policy_group = "aws_stage_enc"
node_name = "consul-server${count.index}"
server_url = var.chef_server_url
recreate_client = true
skip_install = true
user_name = var.chef_username
user_key = "${file("${var.chef_user_key}")}"
version = "14"
}
}
Running this gives me an error:
Error: Cycle: aws_instance.consul-server[1], aws_instance.consul-server[0]
(This is after declaring a count of 2 in a variable for var.consul-server_count)
Can anyone tell me what the proper way is to do this?
There are two issues here: (1) How to interpolate a comma-separated list in a JSON string ; and (2) What is causing the cyclic dependency error.
How to interpolate a list to make a valid JSON array
Use jsonencode
The cleanest method is to not use a heredoc at all and just use the jsonencode function.
You could do this:
locals {
arr = ["host1", "host2", "host3"]
}
output "test" {
value = jsonencode(
{
"consul" = {
"servers" = local.arr
}
})
}
And this yields as output:
Outputs:
test = {"consul":{"servers":["host1","host2","host3"]}}
Use the join function and a heredoc
The Chef provisioner's docs suggest to use a heredoc for the JSON string, so you can also do this:
locals {
arr = ["host1", "host2", "host3"]
sep = "\", \""
}
output "test" {
value = <<-EOF
{
"consul": {
"servers": ["${join(local.sep, local.arr)}"]
}
}
EOF
}
If I apply that:
Outputs:
test = {
"consul": {
"servers": ["host1", "host2", "host3"]
}
}
Some things to pay attention to here:
You are trying to join your hosts so that they become valid JSON in the context of a JSON array. You need to join them with ",", not just a comma. That's why I've defined a local variable sep = "\", \"".
You seem to be trying to split there when you apparently need join.
Cyclic dependency issue
The cause of the error message:
Error: Cycle: aws_instance.consul-server[1], aws_instance.consul-server[0]
Is that you have a cyclic dependency. Consider this simplified example:
resource "aws_instance" "example" {
count = 3
ami = "ami-08589eca6dcc9b39c"
instance_type = "t2.micro"
user_data = <<-EOF
hosts="${join(",", aws_instance.example[count.index].id)}"
EOF
}
Or you could use splat notation there too for the same result i.e. aws_instance.example.*.id.
Terraform plan then yields:
▶ terraform012 plan
...
Error: Cycle: aws_instance.example[2], aws_instance.example[1], aws_instance.example[0]
So you get a cycle error there because aws_instance.example.*.id depends on the aws_instance.example being created, so the resource depends on itself. In other words, you can't use a resources exported values inside the resource itself.
What to do
I don't know much about Consul, but all the same, I'm a bit confused tbh why you want the EC2 instance IDs in the servers field. Wouldn't the Consul config be expecting IP addresses or hostnames there?
In any case, you probably need to calculate the host names yourself outside of this resource, either as a static input parameter or something that you can calculate somehow. And I imagine you'll end up with something like:
variable "host_names" {
type = list
default = ["myhost1"]
}
resource "aws_instance" "consul_server" {
...
provisioner "chef" {
attributes_json = jsonencode(
{
"consul" = {
"servers" = var.host_names
}
})
}
}

map list of maps to a list of selected field values in terraform

If resources use a count parameter to specify multi resources in terraform there is a simple syntax for providing a list/array of dedicated fields for the resource instances.
for example
aws_subnet.foo.*.id
Since quite a number of versions it is possible to declare variables with a complex structure, for example lists of maps.
variable "data" {
type = "list"
default = [
{
id = "1"
...
},
{
id = "10"
...
}
]
}
I'm looking for a possibility to do the same for varaibles I can do for multi resources: a projection of an array to an array of field values of the array elements.
Unfortunately
var.data.*.id
does not work as for resources. Is there any possibility to do this?
UPDATE
Massive fancy features have been added into terraform since Terraform 0.12 was released, e.g., list comprehension, with which the solution is super easy.
locals {
ids = [for d in var.data: d.id]
#ids = [for d in var.data: d["id"]] #same
}
# Then you could get the elements this way,
# local.ids[0]
Solution before terraform 0.12
template_file can help you out.
data "template_file" "data_id" {
count = "${length(var.data)}"
template = "${lookup(var.data[count.index], "id")}"
}
Then you get a list "${data.template_file.data_id.*.rendered}", whose elements are value of "id".
You can get its element by index like this
"${data.template_file.data_id.*.rendered[0]}"
or through function element()
"${element(data.template_file.data_id.*.rendered, 0)}"
NOTE: This answer and its associated question are very old at this point, and this answer is now totally stale. I'm leaving it here for historical reference, but nothing here is true of modern Terraform.
At the time of writing, Terraform doesn't have a generalized projection feature in its interpolation language. The "splat syntax" is implemented as a special case for resources.
While deep structure is possible, it is not yet convenient to use, so it's recommended to still keep things relatively flat. In future it is likely that new language features will be added to make this sort of thing more usable.
If have found a working solution using template rendering to by-pass the list of map's issue:
resource "aws_instance" "k8s_master" {
count = "${var.master_count}"
ami = "${var.ami}"
instance_type = "${var.instance_type}"
vpc_security_group_ids = ["${aws_security_group.k8s_sg.id}"]
associate_public_ip_address = false
subnet_id = "${element(var.subnet_ids,count.index % length(var.subnet_ids))}"
user_data = "${file("${path.root}/files/user_data.sh")}"
iam_instance_profile = "${aws_iam_instance_profile.master_profile.name}"
tags = "${merge(
local.k8s_tags,
map(
"Name", "k8s-master-${count.index}",
"Environment", "${var.environment}"
)
)}"
}
data "template_file" "k8s_master_names" {
count = "${var.master_count}"
template = "${lookup(aws_instance.k8s_master.*.tags[count.index], "Name")}"
}
output "k8s_master_name" {
value = [
"${data.template_file.k8s_master_names.*.rendered}",
]
}
This will result in the following output:
k8s_master_name = [
k8s-master-0,
k8s-master-1,
k8s-master-2
]
A potentially simpler answer is to use the zipmap function.
Starting with an environment variable map compatible with ECS template definitions:
locals {
shared_env = [
{
name = "DB_CHECK_NAME"
value = "postgres"
},
{
name = "DB_CONNECT_TIMEOUT"
value = "5"
},
{
name = "DB_DOCKER_HOST_PORT"
value = "35432"
},
{
name = "DB_DOCKER_HOST"
value = "localhost"
},
{
name = "DB_HOST"
value = "my-db-host"
},
{
name = "DB_NAME"
value = "my-db-name"
},
{
name = "DB_PASSWORD"
value = "XXXXXXXX"
},
{
name = "DB_PORT"
value = "5432"
},
{
name = "DB_QUERY_TIMEOUT"
value = "30"
},
{
name = "DB_UPGRADE_TIMEOUT"
value = "300"
},
{
name = "DB_USER"
value = "root"
},
{
name = "REDIS_DOCKER_HOST_PORT"
value = "6380"
},
{
name = "REDIS_HOST"
value = "my-redis"
},
{
name = "REDIS_PORT"
value = "6379"
},
{
name = "SCHEMA_SCRIPTS_PATH"
value = "db-scripts"
},
{
name = "USE_LOCAL"
value = "false"
}
]
}
In the same folder launch terraform console for testing built-in functions. You may need to terraform init if you haven't already.
terraform console
Inside the console type:
zipmap([for m in local.shared_env: m.name], [for m in local.shared_env: m.value])
Observe the output of each list-item-map being a name-value-pair of a single map:
{
"DB_CHECK_NAME" = "postgres"
"DB_CONNECT_TIMEOUT" = "5"
"DB_DOCKER_HOST" = "localhost"
"DB_DOCKER_HOST_PORT" = "35432"
"DB_HOST" = "my-db-host"
"DB_NAME" = "my-db-name"
"DB_PASSWORD" = "XXXXXXXX"
"DB_PORT" = "5432"
"DB_QUERY_TIMEOUT" = "30"
"DB_UPGRADE_TIMEOUT" = "300"
"DB_USER" = "root"
"REDIS_DOCKER_HOST_PORT" = "6380"
"REDIS_HOST" = "my-redis"
"REDIS_PORT" = "6379"
"SCHEMA_SCRIPTS_PATH" = "db-scripts"
"USE_LOCAL" = "false"
}

Resources