how to dynamically pass parameter to terraform module and run a module based on a condition - terraform

I have a requirement to run modules on conditional basis and also to build the parameter list for the called module dynamically from map variable.
My main.tf file looks like below
provider "aws" {
region = var.region
}
module "CreateResource1" {
source = "./modules/CreateResource1"
ProductName = "Test1"
ProductColour = "Red"
ProductShape = "Hexagone"
}
module "CreateResource2" {
source = "./modules/CreateResource2"
ProductName = "Test2"
ProductType = "xyz"
ProductModel = "abc"
ProductPrice = ""
}
the requirement is a conditional variable module_name which user will pass and based on that i need to execute any one of the modules as per condition.
Also instead of passing the parameter like ProductName, ProductColour, ProductShape as a separate variables the user will be passing them as a dict variable and i would like to build the variable dynamically with both key and value for the module.
Input from user will be like below.
module "Resource" {
module_name = "CreateResource1"
parameters = {
ProductName = "Test1"
ProductColour = "Red"
ProductShape = "Hexagone"
}
}
based on above inputs i need to select the module to run and build parameters for the module.
as i am new to terraform any leads will be appreciated.
Terraform Version used is 1.0.0
Regards

If possible, I would just change the modules to accept a map like the calling module:
(Disclaimer: partial, untested example)
modules/resource/main.tf
variable "module_name" {
type = string
}
variable "parameters" {
type = map
}
module "CreateResource1" {
source = "../CreateResource1"
count = var.module_name == "CreateResource1" ? 1 : 0
parameters = var.parameters
}
module "CreateResource2" {
source = "../CreateResource2"
count = var.module_name == "CreateResource2" ? 1 : 0
parameters = var.parameters
}
main.tf
module "Resource" {
source = "./modules/resource"
module_name = "CreateResource1"
parameters = {
ProductName = "Test1"
ProductColour = "Red"
ProductShape = "Hexagone"
}
}
The modules that are called would need a small modification to define a parameter input variable of type map and then the values could be accessed in the code using a lookup function (e.g. lookup(var.parameters, "ProductName", "") - note the 3rd param allows you to specify a default if the element does not exist in the map. There is no reason why that value cannot be taken from an input variable, so if you don't want to hard code it here, you could for example, pass it in from a default set in the calling module, and passed into all/some of the children). So for example:
modules/CreateResource1/main.tf (Partial example)
variable "parameters" {
type = map
}
resource "some_resource_type" "some_resource_name" {
ProductName = lookup(var.parameters, "ProductName", "Product1")
ProductColour = lookup(var.parameters, "ProductColour", "Red")
ProductShape = lookup(var.parameters, "ProductShape", "Circle")
}
So in this example, the resource created by CreateResource1 requires 3 params. If they exist in the map that is passed by module Resource then they will be used, and for any that are not present in the map that is passed in, the defaults will be used (in this case "Product1", "Red" & "Circle").

Related

Conditionally skipping a variable assignment in a terraform module

I'm currently using the terraform-aws-eks module and wanted to setup a managed node group in an existing cluster. However, I only want this node group to appear in our dev environment (but still want the cluster to remain unchanged). Is there a way to skip a variable assignment conditionally for a module? I tried the below approach but get an error if var.deploy_managed_node_group = false. Terraform version 0.14.11.
module "eks" {
source = "./modules/eks-17.24.0"
cluster_enabled_log_types = var.cluster_enabled_log_types
cluster_name = local.eks_cluster_name
cluster_version = local.eks_version
iam_path = "/eks/"
manage_aws_auth = true
map_users = local.eks_users
map_roles = local.eks_roles
subnets = module.eks_vpc.private_subnets
vpc_id = module.eks_vpc.vpc_id
worker_groups = local.worker_groups
node_groups = var.deploy_managed_node_group ? local.node_groups : null
}
Error: Iteration over null value
node_groups variable from module:
variable "node_groups" {
description = "Map of map of node groups to create. See `node_groups` module's documentation for more details"
type = any
default = {}
}
When using types in Terraform such as set, list, or map, the omitted value should be "empty" instead of null if the value is utilized for iteration instead of an argument. Therefore:
node_groups = var.deploy_managed_node_group ? local.node_groups : {}
would be the ideal ternary here as the falsey value returned by the conditional is an empty map constructor.

How to concatenate strings in Terraform output with for loop?

I have multiple aws_glue_catalog_table resources and I want to create a single output that loops over all resources to show the S3 bucket location of each one. The purpose of this is to test if I am using the correct location (because it is a concatenation of variables) for each resource in Terratest. I cannot use aws_glue_catalog_table.* or aws_glue_catalog_table.[] because Terraform does not allow to reference a resource without specifying its name.
So I created a variable "table_names" with r1, r2, rx. Then, I can loop over the names. I want to create the string aws_glue_catalog_table.r1.storage_descriptor[0].location dynamically, so I can check if the location is correct.
resource "aws_glue_catalog_table" "r1" {
name = "r1"
database_name = var.db_name
storage_descriptor {
location = "s3://${var.bucket_name}/${var.environment}-config/r1"
}
...
}
resource "aws_glue_catalog_table" "rX" {
name = "rX"
database_name = var.db_name
storage_descriptor {
location = "s3://${var.bucket_name}/${var.environment}-config/rX"
}
}
variable "table_names" {
description = "The list of Athena table names"
type = list(string)
default = ["r1", "r2", "r3", "rx"]
}
output "athena_tables" {
description = "Athena tables"
value = [for n in var.table_names : n]
}
First attempt: I tried to create an output "athena_tables_location" with the syntax aws_glue_catalog_table.${table} but does does.
output "athena_tables_location" {
// HOW DO I ITERATE OVER ALL TABLES?
value = [for t in var.table_names : aws_glue_catalog_table.${t}.storage_descriptor[0].location"]
}
Second attempt: I tried to create a variable "table_name_locations" but IntelliJ already shows an error ${t} in the for loop [for t in var.table_names : "aws_glue_catalog_table.${t}.storage_descriptor[0].location"].
variable "table_name_locations" {
description = "The list of Athena table locations"
type = list(string)
// THIS ALSO DOES NOT WORK
default = [for t in var.table_names : "aws_glue_catalog_table.${t}.storage_descriptor[0].location"]
}
How can I list all table locations in the output and then test it with Terratest?
Once I can iterate over the tables and collect the S3 location I can do the following test using Terratest:
athenaTablesLocation := terraform.Output(t, terraformOpts, "athena_tables_location")
assert.Contains(t, athenaTablesLocation, "s3://rX/test-config/rX",)
It seems like you have an unusual mix of static and dynamic here: you've statically defined a fixed number of aws_glue_catalog_table resources but you want to use them dynamically based on the value of an input variable.
Terraform doesn't allow dynamic references to resources because its execution model requires building a dependency graph between all of the objects, and so it needs to know which exact resources are involved in a particular expression. However, you can in principle build your own single value that includes all of these objects and then dynamically choose from it:
locals {
tables = {
r1 = aws_glue_catalog_table.r1
r2 = aws_glue_catalog_table.r2
r3 = aws_glue_catalog_table.r3
# etc
}
}
output "table_locations" {
value = {
for t in var.table_names : t => local.tables[t].storage_descriptor[0].location
}
}
With this structure Terraform can see that output "table_locations" depends on local.tables and local.tables depends on all of the relevant resources, and so the evaluation order will be correct.
However, it also seems like your table definitions are systematic based on var.table_names and so could potentially benefit from being dynamic themselves. You could achieve that using the resource for_each feature to declare multiple instances of a single resource:
variable "table_names" {
description = "Athena table names to create"
type = set(string)
default = ["r1", "r2", "r3", "rx"]
}
resource "aws_glue_catalog_table" "all" {
for_each = var.table_names
name = each.key
database_name = var.db_name
storage_descriptor {
location = "s3://${var.bucket_name}/${var.environment}-config/${each.key}"
}
...
}
output "table_locations" {
value = {
for k, t in aws_glue_catalog_table.all : k => t.storage_descriptor[0].location
}
}
In this case aws_glue_catalog_table.all represents all of the tables together as a single resource with multiple instances, each one identified by the table name. for_each resources appear in expressions as maps, so this will declare resource instances with addresses like this:
aws_glue_catalog_table.all["r1"]
aws_glue_catalog_table.all["r2"]
aws_glue_catalog_table.all["r3"]
...
Because this is already a map, this time we don't need the extra step of constructing the map in a local value, and can instead just access this map directly to build the output value, which will be a map from table name to storage location:
{
r1 = "s3://BUCKETNAME/ENVNAME-config/r1"
r2 = "s3://BUCKETNAME/ENVNAME-config/r2"
r3 = "s3://BUCKETNAME/ENVNAME-config/r3"
# ...
}
In this example I've assumed that all of the tables are identical aside from their names, which I expect isn't true in practice but I was going only by what you included in the question. If the tables do need to have different settings then you can change var.table_names to instead be a variable "tables" whose type is a map of object type where the values describe the differences between the tables, but that's a different topic kinda beyond the scope of this question, so I won't get into the details of that here.

Terraform dynamically generate strings using data sources output

Is there any way I can feed the Terraform data source output to another Terraform file as input
The scenario is, I have a terraform code to fetch the private IP addresses (here 3 IPs 10.1.1.1,10.1.1.2,10.1.1.3) for particular tags(here jenkins) using data source
data "aws_instances" "mytag" {
filter {
name = "tag:Application"
values = ["jenkins"]
}
}
output "output from aws" {
value = data.aws_instances.mytag_private_ips
}
Whenever, I do the terraform apply, the on the pattern section in the
below metric-filter code should be able to fetch the resultant IPs from the above code and make them available in the live state ( aws console )
resource "aws_cloudwatch_log_metric_filter" "test" {
name = "test-metric-filter"
pattern = "[w1,w2,w3,w4!=\"*<IP1>*\"&&w4!=\"*<IP2>*\"&&w4!=\"*<IP3>*\",w5=\"*admin*\"]"
log_group_name = var.test_log_group_name
metric_transformation {
name ="test-metric-filter"
namespace = "General"
}
}
So, the final result of metric pattern in the aws console should be like below
[w1,w2,w3,w4!="*10.1.1.1*"&&w4!="*10.1.1.2*"&&w4!="*10.1.1.3*",w5="*admin*"]
The end goal is whenever if the new IPs are generated, it will get populated to pattern (in aws-console) without changing the metric-filter code.
Any help is appreciated, as I could not find any precise document on terraform allows us to dynamically generate strings using data sources
Not sure why you need two files for something this simple...
Here is what I would do:
provider "aws" {
region = "us-east-1"
}
data "aws_instances" "test" {
filter {
name = "architecture"
values = ["x86_64"]
}
}
resource "aws_cloudwatch_log_metric_filter" "test" {
name = "test-metric-filter"
pattern = "[w1,w2,w3,w4!=\"*${data.aws_instances.test.private_ips[0]}*\",w5=\"*admin*\"]"
log_group_name = "test_log_group_name"
metric_transformation {
name = "test-metric-filter"
namespace = "General"
value = 1
}
}
And a terraform plan will show
Terraform will perform the following actions:
# aws_cloudwatch_log_metric_filter.test will be created
+ resource "aws_cloudwatch_log_metric_filter" "test" {
+ id = (known after apply)
+ log_group_name = "test_log_group_name"
+ name = "test-metric-filter"
+ pattern = "[w1,w2,w3,w4!=\"*172.31.70.170*\",w5=\"*admin*\"]"
+ metric_transformation {
+ name = "test-metric-filter"
+ namespace = "General"
+ unit = "None"
+ value = "1"
}
}
Plan: 1 to add, 0 to change, 0 to destroy.
Concatenating strings is easy: "foo ${var.bar} 123"
and on this case our private_ips is an array so we need the [x]
For more complex concatenations look into the format function:
https://www.terraform.io/docs/language/functions/format.html
I did changed the filter to be able to test on my environment and also used a shorter pattern than yours, but that is the basis for what you need, just add more of make changes to suit your needs.
What you are looking for is string interpolation in Terraform.
I believe you would want to do the following:
pattern = "[w1,w2,w3,w4!=\"*${data.aws_instances.mytag.private_ips[0]}*\"&&w4!=\"*${data.aws_instances.mytag.private_ips[1]}*\"&&w4!=\"*${data.aws_instances.mytag.private_ips[2]}*\",w5=\"*admin*\"]"
I suggest being careful with this statement, because it will fail if you don't have at least 3 instances. You would want to have something dynamic instead.

Terraform: Create block only if variable matches certain values

I'm trying to create a module that creates interconnect-attachments, but some parts are only defined if the attachment is using ipsec encryption and if it's not, that block must not exist in the resource else it causes an error (even if it only contains a value set to null.)
I've tried using a dynamic, but I can't quite get the layout right to have it work:
resource "google_compute_interconnect_attachment" "interconnect-attachment" {
project = var.project
region = var.region
name = var.name
edge_availability_domain = var.availability_domain
type = var.type
router = google_compute_router.router.name
encryption = var.encryption
dynamic "ipsec_internal_addresses" {
for_each = var.encryption != "IPSEC" ? [] : [1]
content {
var.address
}
}
}
Essentially, if var.encryption is set to IPSEC then i want the following block included:
ipsec_internal_addresses = [
var.address,
]
The other issue is it appears a dynamic block expects some kind of assignment to happen, but the terraform examples just have the value inside the ipsec_internal_addresses so I'm unsure how to to achieve this.
ipsec_internal_addresses is not a block in the google_compute_interconnect_attachment resource. It is an argument. Therefore, you can use the normal pattern for specifying optional arguments where the conditional returns a null type if you do not want to specify a value. Using your conditional and variables:
ipsec_internal_addresses = var.encryption == "IPSEC" ? [var.address] : null
which will return and assign your [var.address] to ipsec_internal_addresses when var.encryption equals the string IPSEC. Otherwise, it will return null and the ipsec_internal_addresses argument will be ignored.

how would be possbile to assign a value from a rout call to a terraform variable?

is it possible to get a value from a data call and assign it to a variable ?
i m trying to get some values with the calls like :
data "oci_core_vcns" "test_vcns" {
#Required
compartment_id = "${var.compartment_ocid}"
}
output "vcn_state" {
value = ${data.oci_core_vcns.test_vcns.virtual_networks[0].state}"
}
i dont want to reference to my scripts the return value
${data.oci_core_vcns.test_vcns.virtual_networks[0].state}
but i wanted to do something like
var.vcn_state = {data.oci_core_vcns.test_vcns.virtual_networks[0].state}
where vcn_state is declared as a variable in my variables.tf file
but this code is not working
Would be possible to create a variable like :
variable "vcn_state" {
type = "string"
default = ""
}
and then assign to it values from data calls ?
thanks
locals is what you are looking for
Input Variables in Terraform are most analogous to the parameters on a function: they are assigned by the caller and used by the module configuration.
What you are looking for here is an analog to local variables, which in Terraform are called Local Values, reflecting the fact that they have a fixed expression at declaration time and cannot vary during an operation or between operations.
variable "compartment_ocid" {
type = string
}
data "oci_core_vcns" "test_vcns" {
compartment_id = var.compartment_ocid
}
locals {
vcn_state = data.oci_core_vcns.test_vcns.virtual_networks[0].state
}
output "vcn_state" {
value = local.vcn_state
}
To complete the analogy to functions, note that output values (the output block type) serve as similar role as a function's return values.
The above example uses Terraform 0.12 syntax. If you are using Terraform 0.11 then the above patterns should work if you wrap each of the reference expressions in the "${ ... }" interpolation syntax.

Resources