I’m using mapped variables in order to create local vars based on longer variable names, I'm using them where we would have abbreviations or where the resource wants a sanitized or shortened version of a value used elsewhere.
Eg
variable "env-short" {
description = "create a shortened version of the name of use in resource naming"
type = "map"
default = {
"Proof Of Concept" = "poc"
"User Acceptance Testing" = "uat"
"Production" = "prd"
}
}
variable "org-short" {
description = "create a shortened version of the name of use in resource naming"
type = map(string)
default = {
"My Big Company" = "MBC"
"My Little Company" = "MLC"
}
}
variable "loc-short" {
description = "create a shortened version of the name of use in resource naming"
type = map(string)
default = {
"UK South" = "UKS"
"UK West" = "UKW"
"North Europe" = "NEU"
"West Europe" = "WEU"
}
}
And use corresponding variables for their full length mapping equiverlants.
Now I could use as is within a resource block by something like
Name = “${lower(“${var.loc-short[$var.location]}”)-${lower(“${var.org-short[$var.organisation]}”)-${lower(“${var.env-short[$var.environment]}”)-myresource”
But like all good coders I like to keep things neat and readable by declaring local variables that I can then refer to.
locals {
org-short = "${lower("${var.org-short["${var.organisation}"]}")}"
loc-short = "${lower("${var.loc-short["${var.location}"]}")}"
env-short = "${lower("${var.env-short["${var.environment}"]}")}"
# I also create additional for commonly used configurations of them
name-prefix = "${lower("${var.org-short["${var.organisation}"]}")}-${lower("${var.loc-short["${var.location}"]}")}"
name-prefix-storage = "${lower("${var.org-short["${var.organisation}"]}")}${lower("${var.loc-short["${var.location}"]}")}"
}
This works really great and keeps things neat tidy and readable.
resource "provisioner_example" "test" {
location = var.location
name = “${local.loc-short}-${local.env-short}-my resource”
I would like however to be able to use this format when I start creating multiple resources using the count functionality.
resource "provisioner_example" "test" {
count = length(var.location)
location = var.location[count.index]
name = “${local.loc-short[count.index]}-${local.env-short}-my resource”
Terraform then complains that the index is invalid in the locals lookup, varlocation is tuple with 2 elements,| var.loc-short is map of string with 4 elements. The given key does not identify an element in this collection value: string required.
Now I know I can work around this by getting rid of the locals variables andincluding the variable calculation directly
name =”${lower("${var.loc-short["${var.locations[count.index]}"]}")}-${local.env-short}-my resource"
But to me it then makes the code seem more messy and less structured.
Any ideas on how I can pass the count index value to the map lookup?
Related
I would like to set the disk_size doing something like the following:
resource "google_compute_disk" "terraform-additional-persistent-disk" {
name = "terraform-additional-persistent-disk"
zone = local.zone
type = local.default_disk_type
size = exists(local.default_disk_size) ? local.default_disk_size : 50
image = local.default_ubuntu_image
labels = {
created_by = "terraform"
}
}
However, I have not been able to find a exists function in Terraform. The main aim is to take the value of a variable/local if it exists and, if it has not been declared anywhere, then take the value I pass as second argument.
I have been checking other questions like 1 but neither can nor try function are helping me achieve my goal. I will always get either A local value with the name "default_disk_size" has not been declared or An input variable with the name "default_disk_size" has not been declared (depending on whether I use a non-existing local or var).
I have even tried to run the following, but it will always raise an exception if the variable/local has not been set. Is there a way of achieving this without explicitly declaring the variable with a default value of null/""?
Thanks!
resource "google_compute_disk" "terraform-additional-persistent-disk" {
name = "terraform-additional-persistent-disk"
zone = local.zone
type = local.default_disk_type
size = merge({sizee=50}, {sizee = local.default_disk_sizee})["sizee"]
image = local.default_ubuntu_image
labels = {
created_by = "terraform"
}
}
I think you're coming at this with the idea that input variables and locals may or may not exist at the time this resource is created, like they are system environment variables or something. However in Terraform, those things have to be explicitly declared in one of the .tf files in the same folder as the file your google_compute_disk.terraform-additional-persistent-disk is declared.
There would be no way in Terraform's syntax to have either local or input variables appear dynamically at runtime, they have to be declared ahead of time in your code. They will always exist.
If you want to allow someone using your Terraform code the option of passing in a variable or not, you have to explicitly declare the variable, and give it a default value. Then the person using your Terraform code can optionally override that default value. Like this:
variable "disk_size" {
type = number
default = 50
description = "The size of the additional persistent disk"
}
resource "google_compute_disk" "terraform-additional-persistent-disk" {
name = "terraform-additional-persistent-disk"
zone = local.zone
type = local.default_disk_type
size = var.disk_size
image = local.default_ubuntu_image
labels = {
created_by = "terraform"
}
}
Then when someone uses your Terraform code, if they don't specify a value for the disk_size input variable, the default of 50 will be used, but if they do specify something, then the value they specified will be used.
I have multiple aws_glue_catalog_table resources and I want to create a single output that loops over all resources to show the S3 bucket location of each one. The purpose of this is to test if I am using the correct location (because it is a concatenation of variables) for each resource in Terratest. I cannot use aws_glue_catalog_table.* or aws_glue_catalog_table.[] because Terraform does not allow to reference a resource without specifying its name.
So I created a variable "table_names" with r1, r2, rx. Then, I can loop over the names. I want to create the string aws_glue_catalog_table.r1.storage_descriptor[0].location dynamically, so I can check if the location is correct.
resource "aws_glue_catalog_table" "r1" {
name = "r1"
database_name = var.db_name
storage_descriptor {
location = "s3://${var.bucket_name}/${var.environment}-config/r1"
}
...
}
resource "aws_glue_catalog_table" "rX" {
name = "rX"
database_name = var.db_name
storage_descriptor {
location = "s3://${var.bucket_name}/${var.environment}-config/rX"
}
}
variable "table_names" {
description = "The list of Athena table names"
type = list(string)
default = ["r1", "r2", "r3", "rx"]
}
output "athena_tables" {
description = "Athena tables"
value = [for n in var.table_names : n]
}
First attempt: I tried to create an output "athena_tables_location" with the syntax aws_glue_catalog_table.${table} but does does.
output "athena_tables_location" {
// HOW DO I ITERATE OVER ALL TABLES?
value = [for t in var.table_names : aws_glue_catalog_table.${t}.storage_descriptor[0].location"]
}
Second attempt: I tried to create a variable "table_name_locations" but IntelliJ already shows an error ${t} in the for loop [for t in var.table_names : "aws_glue_catalog_table.${t}.storage_descriptor[0].location"].
variable "table_name_locations" {
description = "The list of Athena table locations"
type = list(string)
// THIS ALSO DOES NOT WORK
default = [for t in var.table_names : "aws_glue_catalog_table.${t}.storage_descriptor[0].location"]
}
How can I list all table locations in the output and then test it with Terratest?
Once I can iterate over the tables and collect the S3 location I can do the following test using Terratest:
athenaTablesLocation := terraform.Output(t, terraformOpts, "athena_tables_location")
assert.Contains(t, athenaTablesLocation, "s3://rX/test-config/rX",)
It seems like you have an unusual mix of static and dynamic here: you've statically defined a fixed number of aws_glue_catalog_table resources but you want to use them dynamically based on the value of an input variable.
Terraform doesn't allow dynamic references to resources because its execution model requires building a dependency graph between all of the objects, and so it needs to know which exact resources are involved in a particular expression. However, you can in principle build your own single value that includes all of these objects and then dynamically choose from it:
locals {
tables = {
r1 = aws_glue_catalog_table.r1
r2 = aws_glue_catalog_table.r2
r3 = aws_glue_catalog_table.r3
# etc
}
}
output "table_locations" {
value = {
for t in var.table_names : t => local.tables[t].storage_descriptor[0].location
}
}
With this structure Terraform can see that output "table_locations" depends on local.tables and local.tables depends on all of the relevant resources, and so the evaluation order will be correct.
However, it also seems like your table definitions are systematic based on var.table_names and so could potentially benefit from being dynamic themselves. You could achieve that using the resource for_each feature to declare multiple instances of a single resource:
variable "table_names" {
description = "Athena table names to create"
type = set(string)
default = ["r1", "r2", "r3", "rx"]
}
resource "aws_glue_catalog_table" "all" {
for_each = var.table_names
name = each.key
database_name = var.db_name
storage_descriptor {
location = "s3://${var.bucket_name}/${var.environment}-config/${each.key}"
}
...
}
output "table_locations" {
value = {
for k, t in aws_glue_catalog_table.all : k => t.storage_descriptor[0].location
}
}
In this case aws_glue_catalog_table.all represents all of the tables together as a single resource with multiple instances, each one identified by the table name. for_each resources appear in expressions as maps, so this will declare resource instances with addresses like this:
aws_glue_catalog_table.all["r1"]
aws_glue_catalog_table.all["r2"]
aws_glue_catalog_table.all["r3"]
...
Because this is already a map, this time we don't need the extra step of constructing the map in a local value, and can instead just access this map directly to build the output value, which will be a map from table name to storage location:
{
r1 = "s3://BUCKETNAME/ENVNAME-config/r1"
r2 = "s3://BUCKETNAME/ENVNAME-config/r2"
r3 = "s3://BUCKETNAME/ENVNAME-config/r3"
# ...
}
In this example I've assumed that all of the tables are identical aside from their names, which I expect isn't true in practice but I was going only by what you included in the question. If the tables do need to have different settings then you can change var.table_names to instead be a variable "tables" whose type is a map of object type where the values describe the differences between the tables, but that's a different topic kinda beyond the scope of this question, so I won't get into the details of that here.
Is there any way I can feed the Terraform data source output to another Terraform file as input
The scenario is, I have a terraform code to fetch the private IP addresses (here 3 IPs 10.1.1.1,10.1.1.2,10.1.1.3) for particular tags(here jenkins) using data source
data "aws_instances" "mytag" {
filter {
name = "tag:Application"
values = ["jenkins"]
}
}
output "output from aws" {
value = data.aws_instances.mytag_private_ips
}
Whenever, I do the terraform apply, the on the pattern section in the
below metric-filter code should be able to fetch the resultant IPs from the above code and make them available in the live state ( aws console )
resource "aws_cloudwatch_log_metric_filter" "test" {
name = "test-metric-filter"
pattern = "[w1,w2,w3,w4!=\"*<IP1>*\"&&w4!=\"*<IP2>*\"&&w4!=\"*<IP3>*\",w5=\"*admin*\"]"
log_group_name = var.test_log_group_name
metric_transformation {
name ="test-metric-filter"
namespace = "General"
}
}
So, the final result of metric pattern in the aws console should be like below
[w1,w2,w3,w4!="*10.1.1.1*"&&w4!="*10.1.1.2*"&&w4!="*10.1.1.3*",w5="*admin*"]
The end goal is whenever if the new IPs are generated, it will get populated to pattern (in aws-console) without changing the metric-filter code.
Any help is appreciated, as I could not find any precise document on terraform allows us to dynamically generate strings using data sources
Not sure why you need two files for something this simple...
Here is what I would do:
provider "aws" {
region = "us-east-1"
}
data "aws_instances" "test" {
filter {
name = "architecture"
values = ["x86_64"]
}
}
resource "aws_cloudwatch_log_metric_filter" "test" {
name = "test-metric-filter"
pattern = "[w1,w2,w3,w4!=\"*${data.aws_instances.test.private_ips[0]}*\",w5=\"*admin*\"]"
log_group_name = "test_log_group_name"
metric_transformation {
name = "test-metric-filter"
namespace = "General"
value = 1
}
}
And a terraform plan will show
Terraform will perform the following actions:
# aws_cloudwatch_log_metric_filter.test will be created
+ resource "aws_cloudwatch_log_metric_filter" "test" {
+ id = (known after apply)
+ log_group_name = "test_log_group_name"
+ name = "test-metric-filter"
+ pattern = "[w1,w2,w3,w4!=\"*172.31.70.170*\",w5=\"*admin*\"]"
+ metric_transformation {
+ name = "test-metric-filter"
+ namespace = "General"
+ unit = "None"
+ value = "1"
}
}
Plan: 1 to add, 0 to change, 0 to destroy.
Concatenating strings is easy: "foo ${var.bar} 123"
and on this case our private_ips is an array so we need the [x]
For more complex concatenations look into the format function:
https://www.terraform.io/docs/language/functions/format.html
I did changed the filter to be able to test on my environment and also used a shorter pattern than yours, but that is the basis for what you need, just add more of make changes to suit your needs.
What you are looking for is string interpolation in Terraform.
I believe you would want to do the following:
pattern = "[w1,w2,w3,w4!=\"*${data.aws_instances.mytag.private_ips[0]}*\"&&w4!=\"*${data.aws_instances.mytag.private_ips[1]}*\"&&w4!=\"*${data.aws_instances.mytag.private_ips[2]}*\",w5=\"*admin*\"]"
I suggest being careful with this statement, because it will fail if you don't have at least 3 instances. You would want to have something dynamic instead.
we have a standard naming convention within azure, but in order to sometimes be able to make an exception, it must be possible to provide a name yourself when calling the module
How can be indicated within count which variable Var.Log Name or Local.ComponetName should be used and how can we pass this to the name of the resource
resource "azurerm_log_analytics_workspace" "LOG" {
count = length(var.LOG_Name) == "" ? length(local.ComponentNames) : null
name = var.LOG_Name[count.index] == "" ? local.ComponentNames[count.index] : null
resource_group_name = element(var.resourcegroup_name[*], count.index)
location = var.location
sku = var.LOG_Sku
retention_in_days = var.LOG_RetentionPeriod
}
What you are actually looking for are loops. Within loops you can reference the name of the resource and in case there is no such resource available it won't create them, which seems to be what you tried to indicate when mentioning null.
Here is a great link regarding loops in terraform that thoroughly explains the different types of loops and how to use them: https://blog.gruntwork.io/terraform-tips-tricks-loops-if-statements-and-gotchas-f739bbae55f9
strange question. Can I, instead of using a variable, also use the resource property of an resource + a string to construct a name:
For example:
resource "azurerm_network_security_group" "nsgvmss" {
name = **"NSG - azurerm_resource_virtual_machine_scale_set.vmss.name"**
location = azurerm_resource_group.rgapp.location
resource_group_name = azurerm_resource_group.rgapp.name
}
this works of course with variables like "NSG, ${var.vssname}" but again,
was wondering if i can use the resource name of the object in TF as well
Thanks
This is called string interpolation. Also see expressions (which isn't as pre-0.12-centric)
...
name = "NSG - ${azurerm_resource_virtual_machine_scale_set.vmss.name}"
...