variable "static_policies" {
type = list(tuple([string, string]))
description = "Map, where key is the auth method type and the value is the path to static policy file"
default = [
["app", "ROOT_PATH_STATIC/global/demo.hcl"]
]
}
locals {
root_path_static = "${path.root}/${var.static_policy_folder}/"
root_path_template = "${path.root}/${var.templated_policies_folder}/"
static_policies_replaced = [for policy in var.static_policies :
[
element(policy, 0), replace(element(policy, 1), "ROOT_PATH_STATIC", "${local.root_path_static}")
[
]
}
My Terraform module has a list of tuples with some strings. I am trying to iterate through each of them and do a find and replace of a substring, with a path based on where the module is executed. This is because Terraform can’t use interpolation in a variable. I want to produce the same list but with a substring replaced with a path. Any help appreciated
Related
On terraform v0.14.4
My variable looks like this:
variable "my_config" {
type = object({
instances = set(string)
locations = set(string)
})
default = {
locations = [
"us",
"asia"
]
instances = [
"instance1",
"instance2"
]
}
I want to loop over this var in a resource and create an instance of the resource for each location + instance. The "name" field of the resource will be "<LOCATION>_<INSTANCE>" as well.
I could create a new var in locals that reads the my_config var and generates a new var that looks like this:
[
"us_instance1",
"us_instance2",
"asia_instance1",
"asia_instance2",
]
I would prefer to not generate a new terraform var from this existing var though. Is it possible in a foreach loop to aggregate these two lists directly in a resource definition? Or is the only way to create a new data structure in locals?
EDIT
I cannot get the flatten example in answer provided to work inside a resource definition. I get this error: The given "for_each" argument value is unsuitable: the "for_each" argument must be a map, or set of strings, and you have provided a value of type tuple. This error happens if the type is set(string) or list(string).
# This works
output "test" {
value = flatten(
[
for location in var.my_config.locations : [
for instance in var.my_config.instances : "${location}_${instance}"
]
]
)
}
# This throws the error
resource "null_resource" "test" {
for_each = flatten(
[
for location in var.my_config.locations : [
for instance in var.my_config.instances : "${location}_${instance}"
]
]
)
provisioner "local-exec" {
command = "echo test"
}
}
To achieve the return value of:
[
"us_instance1",
"us_instance2",
"asia_instance1",
"asia_instance2",
]
with the input of the variable my_config, you could:
flatten([for location in var.my_config.locations : [
for instance in var.my_config.instances : "${location}_${instance}"
]])
Whether or not you define this in a locals block is up to you. If you plan on re-using this value multiple times, then it would be more efficient to define it as a local. If you plan on on only using it once, then it would certainly make more sense to not define it in locals.
Note this also assumes my_config type is object(list(string)). The type was not given in the question, but if the type were otherwise then the code becomes much more obfuscated.
For the additional question about using this value as a for_each meta-argument value at the resource scope, it would need to be converted to type set(string). This can be done easily with the toset function:
resource "resource" "this" {
for_each = toset(<expression above or variable with return value of above assigned to it>)
}
I need to set additional variables in my value.yaml (link to jaeger https://github.com/jaegertracing/helm-charts/blob/main/charts/jaeger/values.yaml#L495) helm chart via terraform + terragrunt. In values.yaml, the code looks like this:
spark:
extraEnv: []
It is necessary that it be like this:
spark:
extraEnv:
- name: JAVA_OPTS
value: "-Xms4g -Xmx4g"
Terraform uses this dynamic block:
dynamic "set" {
for_each = var.extraEnv
content {
name = "spark.extraEnv [${set.key}]"
value = set.value
}
}
The variable is defined like this:
variable "extraEnv" {
type = map
}
From terragrunt I pass the value of the variable:
extraEnv = {
"JAVA_OPTS" = "-Xms4g -Xmx4g"
}
And I get this error:
Error: failed parsing key "spark.extraEnv [JAVA_OPTS]" with value -Xms4g -Xmx4g, error parsing index: strconv.Atoi: parsing "JAVA_OPTS": invalid syntax
on main.tf line 16, in resource "helm_release" "jaeger":
16: resource "helm_release" "jaeger" {
Tell me how to use the dynamic block correctly in this case. I suppose that in this case you need to use a list of maps, but I do not understand how to use this in a dynamic block.
UPD:
I solved my problem in a different way.
In values, defined the list 'spark.extraEnv' using yamlencode.
values = [
"${file("${path.module}/values.yaml")}",
yamlencode({
spark = {
extraEnv = var.spark_extraEnv
}
})
]
in variables.tf
variable "spark_extraEnv" {
type = list(object({
name = string
value = string
}))
}
And in terragrunt passed the following variable value:
spark_extraEnv = [
{
name = "JAVA_OPTS"
value = "-Xms4g -Xmx4g"
}
]
I landed here while I was looking for setting extraEnv for a different chart. Finally figured answer for the above question as well:
set {
name = "extraEnv[0].name"
value = "JAVA_OPTS"
}
set {
name = "extraEnv[0].value"
value = "-Xms4g -Xmx4g"
}
I am trying to flatten a tuple to a string as follows
network_acls {
default_action = "Deny"
bypass = "AzureServices"
virtual_network_subnet_ids = ["${data.azurerm_subnet.blah_snet.id}", "${join("\",\"", azurerm_subnet.subnets.*.id)}"]
}
This is very similar to How to flatten a tuple of server ids to a string?, however it's not working for me.
The result is: "*subnetid1*\",\"*subnetid2*" - where \",\" should be properly escaped and result as ","
I can't figure out why this isn't working. I've tried many variations of escaping this to no benefit
This is an expression which has varying Terraform 0.11.x and Terraform 0.12.x outcomes:
["${data.azurerm_subnet.blah_snet.id}", "${join("\",\"", azurerm_subnet.subnets.*.id)}"]
In Terraform 0.11.x, it will result in a concatenation as you shown with "," separating the subnet IDs. In Terraform 0.12.x, it will be a type-error and fail to apply. FWIW, Terraform 0.12.x is correct here.
If you are using Terraform 0.12.x, the following should work:
virtual_network_subnet_ids = join(",", concat(
[data.azurerm_subnet.blah_snet.id], azurerm_subnet.subnets[*].id
)
If you are using Terraform 0.11.x, the following should work:
virtual_network_subnet_ids = "${join(",", concat(
[data.azurerm_subnet.blah_snet.id], azurerm_subnet.subnets[*].id
)}"
This is a potential solution using Terraform 12 syntax (it looks like the splat syntax wasn't supported for non-count resources until TF12 so your mileage may vary depending on your version).
The local item approximates the attribute structure of a data.azurerm_subnet.
Note the use of both the \\ to escape the single \ literal or the simple "," since I couldn't quite tell what the desired output string was meant to be.
locals {
subnets = [
{
id = 12345,
name = "item1"
},
{
id = 9354,
name = "item2"
}
]
}
output "comma_and_escape_char_for_literal" {
value = "${join("\\,", local.subnets.*.id)}"
}
output "comma_only" {
value = "${join(",", local.subnets.*.id)}"
}
I'd love some help with Terraform's count/for_each functions.
The goal is to read multiple json files(current two) into a list of maps and create specific amount of aws_instances with specific naming convention.
Configurations
cat test_service_1.json
{
"instance_name": "front",
"instance_count": "3",
"instance_type": "t2.micro",
"subnet_type": "private",
"elb": "yes",
"data_volume": ["no", "0"]
}
cat test_service_2.json
{
"instance_name": "back",
"instance_count": "3",
"instance_type": "t2.micro",
"subnet_type": "private",
"elb": "yes",
"data_volume": ["no", "0"]
}
cat main.tf
locals {
services = [jsondecode(file("${path.module}/test_service_1.json")),
jsondecode(file("${path.module}/test_service_2.json"))]
}
resource "aws_instance" "test_instance" {
ami = "amzn-ami-hvm-2018.03.0.20200206.0-x86_64-gp2"
instance_type = "t2.micro"
tags = merge(
map("Name", "prod-app-?"),
map("env", "prod")
)
}
Eventually I want the code to go over both json files and create:
prod-front-1
prod-front-2
prod-front-3
prod-back-1
prod-back-2
prod-back-3
I can do that with [count.index +1] but I don't know how to loop through more than one map.
When using resource for_each our task is always to write an expression that produces a map where there is one element per instance we want to create. In this case, that seems to be an expression that can expand a single object containing a count to instead be multiple objects of the number given in the count.
The building blocks we can use to do this in Terraform are:
for expressions to project one collection value into another.
The range function to generate sequences of integers given a count.
The flatten function to turn multiple nested lists into a single flat list.
Let's take this step by step. We'll start with your existing expression to load the data from the JSON files:
locals {
services = [
jsondecode(file("${path.module}/test_service_1.json")),
jsondecode(file("${path.module}/test_service_2.json")),
]
}
The result of this expression is a list of objects, one per file. Next, we'll expand each of those objects into a list of objects whose length is given in instance_count:
locals {
service_instance_groups = [
for svc in local.services : [
for i in range(1, svc.instance_count+1) : {
instance_name = "${svc.instance_name}-${i}"
instance_type = svc.instance_type
subnet_type = svc.subnet_type
elb = svc.elb
data_volume = svc.data_volume
}
]
]
}
The result of this one is a list of lists of objects, each of which will have a unique instance_name value due to concatenating the value i to the end.
To use for_each though we we will need a flat collection with one element per instance, so we'll use the flatten function to achieve that:
locals {
service_instances = flatten(local.service_instance_groups)
}
Now we have a list of objects again, but with six elements (three from each of the two input objects) instead of two.
Finally, we need to project that list to be a map whose keys are the unique identifiers Terraform will use to track the instances. I usually prefer to do this final step directly inside the for_each argument because this result is specific to that use-case and unlikely to be used anywhere else in the module:
resource "aws_instance" "test_instance" {
for_each = {
for inst in local.service_instances : inst.instance_name => inst
}
ami = "amzn-ami-hvm-2018.03.0.20200206.0-x86_64-gp2"
instance_type = each.value.instance_type
tags = {
Name = "prod-app-${each.key}"
Env = "prod"
}
}
This should result in Terraform planning to create instances with addresses like aws_instance.test_instance["front-2"].
I wrote each of the above steps out separately to explain what each one was achieving, but in practice I'd usually do the service_instance_groups and service_instances steps together in a single expression, because that intermediate service_instance_groups result isn't likely to be reused elsewhere. Bringing that all together into a single example, then:
locals {
services = [
jsondecode(file("${path.module}/test_service_1.json")),
jsondecode(file("${path.module}/test_service_2.json")),
]
service_instances = flatten([
for svc in local.services : [
for i in range(1, svc.instance_count+1) : {
instance_name = "${svc.instance_name}-${i}"
instance_type = svc.instance_type
subnet_type = svc.subnet_type
elb = svc.elb
data_volume = svc.data_volume
}
]
])
}
resource "aws_instance" "test_instance" {
for_each = {
for inst in local.service_instances : inst.instance_name => inst
}
ami = "amzn-ami-hvm-2018.03.0.20200206.0-x86_64-gp2"
instance_type = each.value.instance_type
tags = {
Name = "prod-app-${each.key}"
Env = "prod"
}
}
As a bonus, beyond what you were asking about here, if you give those JSON files systematic names and group them together into a subdirectory then you could use Terraform's fileset function to to automatically pick up any new files added in that directory later, without changing the Terraform configuration. For example:
locals {
services = [
for fn in fileset("${path.module}", "services/*.json") :
jsondecode(file("${path.module}/${fn}"))
]
}
The above will produce a list containing an object for each of the files in the services subdirectory that have names ending in .json.
Using terraform I wish to refer to the content of a list of files (ultimately I want to zip them up using the archive_file provider, but in the context of this post that isn't important). These files all live within the same directory so I have two variables:
variable "source_root_dir" {
type = "string"
description = "Directory containing all the files"
}
variable "source_files" {
type = "list"
description = "List of files to be added to the cloud function. Locations are relative to source_root_dir"
}
I want to use the template data provider to refer to the content of the files. Given the number of files in source_files can vary I need to use a count to carry out the same operation on all of them.
Thanks to the information provided at https://stackoverflow.com/a/43195932/201657 I know that I can refer to the content of a single file like so:
provider "template" {
version = "1.0.0"
}
variable "source_root_dir" {
type = "string"
}
variable "source_file" {
type = "string"
}
data "template_file" "t_file" {
template = "${file("${var.source_root_dir}/${var.source_file}")}"
}
output "myoutput" {
value = "${data.template_file.t_file.rendered}"
}
Notice that that contains nested string interpolations. If I run:
terraform init && terraform apply -var source_file="foo" -var source_root_dir="./mydir"
after creating file mydir/foo of course then this is the output:
Success!
Now I want to combine that nested string interpolation syntax with my count. Hence my terraform project now looks like this:
provider "template" {
version = "1.0.0"
}
variable "source_root_dir" {
type = "string"
description = "Directory containing all the files"
}
variable "source_files" {
type = "list"
description = "List of files to be added to the cloud function. Locations are relative to source_root_dir"
}
data "template_file" "t_file" {
count = "${length(var.source_files)}"
template = "${file("${"${var.source_root_dir}"/"${element("${var.source_files}", count.index)}"}")}"
}
output "myoutput" {
value = "${data.template_file.t_file.*.rendered}"
}
yes it looks complicated but syntactically, its correct (at least I think it is). However, if I run init and apply:
terraform init && terraform apply -var source_files='["foo", "bar"]' -var source_root_dir='mydir'
I get errors:
Error: data.template_file.t_file: 2 error(s) occurred:
* data.template_file.t_file[0]: __builtin_StringToInt: strconv.ParseInt: parsing "mydir": invalid syntax in:
${file("${"${var.source_root_dir}"/"${element("${var.source_files}", count.index)}"}")}
* data.template_file.t_file1: __builtin_StringToInt: strconv.ParseInt: parsing "mydir": invalid syntax in:
${file("${"${var.source_root_dir}"/"${element("${var.source_files}", count.index)}"}")}
My best guess is that its interpreting the / as a division operation hence its attempting to parse the value mydir in source_root_dir as an int.
I've played around with this for ages now and can't figure it out. Can someone figure out how to use nested string interpolations together with a count in order to refer to the content of multiple files using the template provider?
OK, I think I figured it out. formatlist to the rescue
provider "template" {
version = "1.0.0"
}
variable "source_root_dir" {
type = "string"
description = "Directory containing all the files"
}
variable "source_files" {
type = "list"
description = "List of files to be added to the cloud function. Locations are relative to source_root_dir"
}
locals {
fully_qualified_source_files = "${formatlist("%s/%s", var.source_root_dir, var.source_files)}"
}
data "template_file" "t_file" {
count = "${length(var.source_files)}"
template = "${file(element("${local.fully_qualified_source_files}", count.index))}"
}
output "myoutput" {
value = "${data.template_file.t_file.*.rendered}"
}
when applied:
terraform init && terraform apply -var source_files='["foo", "bar"]' -var source_root_dir='mydir'
outputs:
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
myoutput = [
This is the content of foo
,
This is the content of bar
]