Terraform - dynamic block within resource with "count" doesn't work - terraform

I'm using the AWS provider for creating CloudWatch Metric Alarms. I created a module which takes in a variable that is a list of instance IDs, and the resource it has uses the "count" functionality to create an alarm per an Instance ID from that variable.
The "aws_cloudwatch_metric_alarm" resource can take in multiple "metric_query" blocks, and my plan was to do this as dynamic block to be able to define as many as needed in the root module.
Issue I'm experiencing is with accessing the "for_each" iterator values.
The high-level end solution should be something among these lines: Use 3 metric blocks, two are available metrics and a third one for an expression on top of the other two, and create this alarm for every instance that is provided in the instance list.
Resource definition, module code:
resource "aws_cloudwatch_metric_alarm" "alarm" {
count = length(var.dimension_values)
alarm_name = "${var.alarm_name}_${var.dimension_values[count.index]}"
comparison_operator = var.comparison_operator
evaluation_periods = var.evaluation_periods
threshold = var.threshold
actions_enabled = var.actions_enabled
alarm_actions = var.alarm_actions
dynamic "metric_query" {
for_each = var.metric_queries
content {
id = metric_queries.value.id
return_data = metric_queries.value.return_data
expression = metric_queries.value.expression
label = metric_queries.value.label
metric {
namespace = metric_queries.value.namespace
metric_name = metric_queries.value.metric_name
period = metric_queries.value.period
stat = metric_queries.value.stat
dimensions = {
"${metric_queries.value.dimension_name}" = var.dimension_values[count.index]
}
}
}
}
tags = merge(
var.common_tags,
{
Name = "${var.alarm_name}_${var.dimension_values[count.index]}"
}
)
}
Module variables (only metric_queries pasted):
variable "metric_queries" {
type = list(object({
id = string
return_data = bool
expression = string
label = string
namespace = string
metric_name = string
period = number
stat = string
dimension_name = string
}))
description = "Metric query for the CloudWatch alarm"
default = []
}
And finally, the root module:
module "cpu_alarms" {
source = "../../Modules/cloudwatch_metric_alarm/"
common_tags = local.common_tags
# Metrics
alarm_name = "EC2_CPU_80_PERCENT"
comparison_operator = "GreaterThanOrEqualToThreshold"
evaluation_periods = 3
threshold = 80
actions_enabled = true
alarm_actions = ["redacted"]
dimension_values = local.all_ec2_instance_ids
metric_queries = [
{
id = "m1"
return_data = true
expression = null
label = "CPU utilization"
namespace = "AWS/EC2"
metric_name = "CPUUtilization"
period = 60
stat = "Average"
dimension_name = "InstanceId"
}
]
}
I'm getting two separate errors with this approach depending on how I'm referring to the "for_each" iterator object.
When using "each" as reference to the iterator the error is:
A reference to "each.value" has been used in a context in which it unavailable, such as when the configuration no longer contains the value in its "for_each" expression. Remove this reference to each.value in your configuration to work │ around this error.
When using "metric_queries" as reference to the iterator the error is:
A managed resource "metric_queries" "value" has not been declared in module.cpu_alarms.
What could be the root cause of this?

Please see the documentation on dynamic blocks. You are trying to use the syntax for the resource level for_each meta-argument, not the syntax for dynamic blocks. It's confusing that they have different syntax, but since a dynamic block could exist inside a resource with for_each, the syntax has to be different to prevent name clashes.
For dynamic blocks the name of the variable is what you put after the dynamic key word, in your case "metric_query". So your code should look like this:
dynamic "metric_query" {
for_each = var.metric_queries
content {
id = metric_query.value.id
return_data = metric_query.value.return_data
expression = metric_query.value.expression
label = metric_query.value.label
metric {
namespace = metric_query.value.namespace
metric_name = metric_query.value.metric_name
period = metric_query.value.period
stat = metric_query.value.stat
dimensions = {
"${metric_query.value.dimension_name}" = var.dimension_values[count.index]
}
}
}
}

Related

In terraform making string variable optional in action block

Hi i am creating alert rules. For some of the alerts i dont need to send any notification using action group.For that i need to make action group variable (string) as optional.
I tried using condition like var.action_group_id != null action_group_id : "". But i am getting following error.
Error: Can not parse "action.0.action_group_id" as a resource id: Cannot parse Azure ID: parse "null": invalid URI for request
And i tried making action block as dynamic block but it is not iterable. for_each is not supporting dynamic block with single variable with one values.
resource "azurerm_monitor_metric_alert" "keyvault_alert" {
for_each = var.keyvault_alert_rules
name = "${var.kv_name} - ${each.value.severity}"
resource_group_name = var.resource_group_name
description = each.value.description
scopes = var.alert_scope
severity = each.value.severity
frequency = each.value.frequency
`window_size = each.value.windowsize`
# criteria block
criteria {
metric_namespace = "Microsoft.KeyVault/vaults"
threshold = each.value.threshold
metric_name = each.value.metric_name
aggregation = each.value.aggregation
operator = each.value.operator
# dimension block
dynamic "dimension" {
for_each = each.value.dimension != null ? each.value.dimension : []
content {
name = dimension.value.dimensionname
operator = dimension.value.dimensionoperator
values = dimension.value.dimensionvalues
}
}
}
action {
action_group_id = var.action_group_id
}
variable.tf
variable "action_group_id" {
type = string
description = "ID of the action group"
}
variable "resource_group_name" {
type = string
description = "name of the resource group"
}
/* in the variables i am passing warning as n input.is there any way i can append warning to alert name in the main.tf based on the severity value which is given down below*/
variable "kv_alert_rules" {
type = map(object({
display_name = string
# display_name = "(severity numeric equalent ex:warning)-(metric name)"
#------details for the alert criteria
metric_name = string
operator = string
threshold = number
aggregation = string
#------ dimension vaules----------
dimension = list(object({
dimensionname = string
dimensionoperator = string
dimensionvalues = list(string)
}))
#-----------------------------------
severity = number
frequency = string
windowsize = string
# window size must be gretar than Frequency values be PT1M, PT5M, PT15M, PT30M, PT1H, PT6H, PT12H and P1D. Defaults to PT5M
description = string
}))
description = "This variable for alert criteria for key vault"
default = {
"Alert_1" = {
# display_name = "(severity numeric equalent ex:warning)-(generic word for metric name)"
display_name = "warning-used capacity"
severity = 2
dimension = null
metric_name = "SaturationShoebox"
aggregation = "Average"
frequency = "PT30M"
description = "Alert fires When Used vault capacity is GreaterThan 85"
windowsize = "PT1H"
operator = "GreaterThan"
threshold = 85
}
}
}
variable "kv_name" {
description = "key vault name "
type = string
}
module calling
module "keyvault" {
source = "../testing/key-vault-alert"
alert_scope = [data.azurerm_key_vault.examplekeyvault.id]
action_group_id = module.action-group.AGidout
resource_group_name = var.resource_group_name
kv_name = data.azurerm_key_vault.examplekeyvault.name
}
I would like to make action_group_id as optional variable.I dont wanna reference action group ID in some instances
If anyone knows a approach how to do that please guide me
Thanks
action is a dynamic block, so to make it optional you can do as follows:
dynamic "action" {
for_each = var.action_group_id != null ? [1] : []
content {
action_group_id = var.action_group_id
}
}

In terraform how to convert numeric value to a string of word based on variable

Hi i am trying to create the alert name based on the severity in the alert.In the input i have given numeric value for severity but i am trying to append equalent string(word) for that severity to the alert name
Critical = 0 Error= 1 Warning= 2 Informational = 3 Verbose = 4
I am getting alert name like keyvault - 0
I would like to get keyvault - critical
resource "azurerm_monitor_metric_alert" "keyvault_alert" {
for_each = var.keyvault_alert_rules
name = "${var.kv_name} - ${each.value.severity}"
resource_group_name = var.resource_group_name
description = each.value.description
scopes = var.alert_scope
severity = each.value.severity
frequency = each.value.frequency
`window_size = each.value.windowsize`
# criteria block
criteria {
metric_namespace = "Microsoft.KeyVault/vaults"
threshold = each.value.threshold
metric_name = each.value.metric_name
aggregation = each.value.aggregation
operator = each.value.operator
# dimension block
dynamic "dimension" {
for_each = each.value.dimension != null ? each.value.dimension : []
content {
name = dimension.value.dimensionname
operator = dimension.value.dimensionoperator
values = dimension.value.dimensionvalues
}
}
}
action {
action_group_id = var.action_group_id
}
variable.tf
variable "action_group_id" {
type = any
description = "ID of the action group"
}
variable "resource_group_name" {
type = string
description = "name of the resource group"
}
/* in the variables i am passing warning as n input.is there any way i can append warning to alert name in the main.tf based on the severity value which is given down below*/
variable "kv_alert_rules" {
type = map(object({
display_name = string
# display_name = "(severity numeric equalent ex:warning)-(metric name)"
#------details for the alert criteria
metric_name = string
operator = string
threshold = number
aggregation = string
#------ dimension vaules----------
dimension = list(object({
dimensionname = string
dimensionoperator = string
dimensionvalues = list(string)
}))
#-----------------------------------
severity = number
frequency = string
windowsize = string
# window size must be gretar than Frequency values be PT1M, PT5M, PT15M, PT30M, PT1H, PT6H, PT12H and P1D. Defaults to PT5M
description = string
}))
description = "This variable for alert criteria for key vault"
default = {
"Alert_1" = {
# display_name = "(severity numeric equalent ex:warning)-(generic word for metric name)"
display_name = "warning-used capacity"
severity = 2
dimension = null
metric_name = "SaturationShoebox"
aggregation = "Average"
frequency = "PT30M"
description = "Alert fires When Used vault capacity is GreaterThan 85"
windowsize = "PT1H"
operator = "GreaterThan"
threshold = 85
}
}
}
variable "kv_name" {
description = "key vault name "
type = string
}
module calling
module "keyvault" {
source = "../testing/key-vault-alert"
alert_scope = [data.azurerm_key_vault.examplekeyvault.id]
action_group_id = module.action-group.AGidout
resource_group_name = var.resource_group_name
kv_name = data.azurerm_key_vault.examplekeyvault.name
}
If anyone knows a approach how to do that please guide me
Thanks
Just create a new map in the locals section:
locals {
severity_alerts = {
0 = "Critical",
1 = "Error",
2 = "Warning"
}
}
Then update you're resource azurerm_monitor_metric_alert name attribute to:
format("%s - %s", var.kv_name, lookup(local.severity_alerts, each.value.severity))
By using the lookup function you can retrieve a value from a map by providing the key.

if condition in terraform in count

I am adding autoscale settings in the Azure cosmosdb database, My problem is not all our db requires autoscale only a selection of database require autoscalse rest are manual. I will not be able to specify the autoscalse block also the throughout in the same resource as there are conflicts between those two. so I thought of using the count but I will be not be able to run the resouece block for only one of the DB. for the below example
variable
variable "databases" {
description = "The list of Cosmos DB SQL Databases."
type = list(object({
name = string
throughput = number
autoscale = bool
max_throughput = number
}))
default = [
{
name = "testcoll1"
throughput = 400
autoscale = false
max_throughput = 0
},
{
name = "testcoll2"
throughput = 400
autoscale = true
max_throughput = 1000
}
]
}
For the first I dont need autoscale and next one I need. My main.tf code
resource "azurerm_cosmosdb_mongo_database" "database_manual" {
count = length(var.databases)
name = var.databases[count.index].name
resource_group_name = azurerm_cosmosdb_account.cosmosdb.resource_group_name
account_name = local.account_name
throughput = var.databases[count.index].throughput
}
resource "azurerm_cosmosdb_mongo_database" "database_autoscale" {
count = length(var.databases)
name = var.databases[count.index].name
resource_group_name = azurerm_cosmosdb_account.cosmosdb.resource_group_name
account_name = local.account_name
autoscale_settings {
max_throughput = var.databases[count.index].max_throughput
}
}
First I thought of running two blocks one with scale and on without, but I will not be able to proceed because it requires the count numbers
count = var.autoscale_required == true ? len(databases) : 0
at the start but in my case I will only know at the time of iteration. I have tried to use dynamic within the block but errored out.
*Update
I have switched to foreach and able to run the condition but still it requires 2 blocks
resource "azurerm_cosmosdb_mongo_database" "database_autoscale"
resource "azurerm_cosmosdb_mongo_database" "database_manual"
resource "azurerm_cosmosdb_mongo_database" "database_autoscale" {
for_each = {
for key, value in var.databases : key => value
if value.autoscale_required == true }
name = each.value.name
resource_group_name = azurerm_cosmosdb_account.cosmosdb.resource_group_name
account_name = local.account_name
autoscale_settings {
max_throughput = each.value.max_throughput
}
}
If I understand correctly, I think you could do what you want using the following:
resource "azurerm_cosmosdb_mongo_database" "database_autoscale" {
count = length(var.databases)
name = var.databases[count.index].name
resource_group_name = azurerm_cosmosdb_account.cosmosdb.resource_group_name
account_name = local.account_name
throughput = var.databases[count.index].autoscale == false ? var.databases[count.index].throughput : null
dynamic "autoscale_settings" {
for_each = var.databases[count.index].autoscale == false ? [] : [1]
content {
max_throughput = var.databases[count.index].max_throughput
}
}
}

Loop over a Map of Objects

How do I do a for_each loop for the following?
I want to create a tfe_variable node_count & vm_size.
I need both these tfe_variables in both wksp1 and wksp2
variable "custom_variables" {
type = map(object({
node_count = number
vm_size = string
}))
default = {
wksp1 = {
node_count = 2
vm_size = "Standard_D2_v3"
},
wksp2 = {
node_count = 5
vm_size = "Standard_D2_v5"
}
}
}
resource "tfe_variable" "custom" {
for_each = {
# for each workspace & variable in var.custom_variables create a tfe_variable
}
key = each.value.name
value = each.value.value
category = "terraform"
workspace_id = each.value.workspace_id
}
You're really close! Here are a couple of things to consider:
Option 1: Multiple tfe_variable resources
Create a tfe_variable resource for each variable you want to create
Make sure the key in you custom_variables map is the workspace ID.
variable "custom_variables" {
type = map(object({
node_count = number
vm_size = string
}))
default = {
wksp1_id = {
node_count = 2
vm_size = "Standard_D2_v3"
},
wksp2_id = {
node_count = 5
vm_size = "Standard_D2_v5"
}
}
}
resource "tfe_variable" "node_count" {
for_each = var.custom_variables
key = "node_count"
value = each.value.node_count
category = "terraform"
workspace_id = each.key
}
resource "tfe_variable" "vm_size" {
for_each = var.custom_variables
key = "vm_size"
value = each.value.vm_size
category = "terraform"
workspace_id = each.key
}
The drawback to this option is that you'll need an additional resource for each variable.
Option 2: A list of variable objects
Define a list of the keys, values, and workspace IDs of each variable
Use count to iterate the list
variable "custom_variables" {
type = list(object({
key = string
value = string
workspace_id = string
}))
default = [
{
key = "node_count"
value = "2"
workspace_id = "wksp1_id"
},
{
key = "node_count"
value = "5"
workspace_id = "wksp2_id"
},
{
key = "vm_size"
value = "Standard_D2_v3"
workspace_id = "wksp1_id"
},
{
key = "vm_size"
value = "Standard_D2_v5"
workspace_id = "wksp2_id"
}
]
}
resource "tfe_variable" "custom" {
count = length(var.custom_variables)
key = var.custom_variables[count.index].key
value = var.custom_variables[count.index].value
workspace_id = var.custom_variables[count.index].workspace_id
category = "terraform"
}
There are a couple of drawbacks to this approach as well:
There is a fair amount of duplicated code in the variable definition
The value must always be of the same type
If you're struggling with loop concepts in Terraform, this blog post might help you.
The main requirement to keep in mind for for_each is that we always need to create a map that has one element for each instance of the resource we want to create. In this case, that means you need a map with one element per workspace per variable, because tfe_variable describes a single variable on a single workspace.
Our job then is to write an expression to project the map-of-objects value coming in via the variable to a collection that has a separate element per variable. Here's one way to get that done, using the flatten function in a way similar to an example in its documentation:
locals {
workspace_variables = flatten([
for ws_name, ws in var.custom_variables : [
for var_name, value in ws : {
workspace = ws_name
variable = var_name
value = value
}
]
])
}
The above should produce a local.workspace_variables that looks like this:
[
{ workspace = "wksp1", variable = "node_count", value = 2 },
{ workspace = "wksp1", variable = "vm_size", value = "Standard_D2_v3" },
{ workspace = "wksp2", variable = "node_count", value = 5 },
{ workspace = "wksp2", variable = "vm_size", value = "Standard_D2_v5" },
]
This now meets the requirement of having one element per desired tfe_variable instance, so our only remaining job is to project this into a map to provide unique identifiers for each element and describe how to populate the tfe_variable arguments based on these objects:
resource "tfe_variable" "custom" {
for_each = {
for wsv in local.workspace_variables : "${wsv.workspace}.${wsv.variable}" => wsv
}
key = each.value.variable
value = each.value.value
category = "terraform"
workspace_id = each.value.workspace
}
One thing I didn't contend with above, because it wasn't directly you question, is the value of workspace_id in tfe_variable. If I recall correctly, that argument is expecting a workspace id rather than a workspace name, in which case you might need a slightly more complicated expression for the workspace_id argument. If you already have a tfe_workspace resource using the workspace names as keys then something like this might work, for example:
workspace_id = tfe_workspace.example[each.value.workspace].id
If your workspaces are created in a different way then you may have to do something more complicated here, but that's getting far off the topic of your original question so I won't try to dig into that here. I'm happy to try to help with it in a separate question on this site though, if you like.

Terraform: How to set variables in a module based on a conditional?

I would like to pass a variable that will allow me to specify the list of VPC and subnet settings for an AWS instance. There are fixed VPC and subnet settings that make sense so I just want to allow a user to pick one using a single variable, i.e. use A or B.
For instance, let's say I have two available VPCs, and these are specified in a variables.tf file for a module my_instance:
variable "a_vpc_cidr_block" { default = "105.191.44.0/22" }
variable "a_vpc_id" { default = "id_a"}
variable "a_vpc_name" { default = "vpc_a" }
variable "a_subnet_availability_zone" { default = "us-east-1a" }
variable "a_subnet_cidr_block" { default = "105.191.25.0/25" }
variable "a_subnet_name" { default = "instance_A" }
variable "b_vpc_cidr_block" { default = "105.191.45.0/22" }
variable "b_vpc_id" { default = "id_b"}
variable "b_vpc_name" { default = "vpc_b" }
variable "b_subnet_availability_zone" { default = "us-east-1a" }
variable "b_subnet_cidr_block" { default = "105.191.35.0/25" }
variable "b_subnet_name" { default = "instance_B" }
The my_instance module will take a single input variable that an environment will specify, with a value of either 'A' or 'B' (is there a way to limit options for a variable to a list of values such as options=['A', 'B']?), and will be called like so in the terraform.tf for a Terraform configuration with a single instance:
module "my_instance" {
source = "../../modules/my_instance"
option = "A"
}
I want to now implement some logic within the module's main file (modules/my_instance/my_instance.tf) where it decides on which of the two collections of VPC and subnet settings it should use from the ones in modules/my_instance/variables.tf. I want to something like this (pseudocode):
if var.option == 'A'
vpc_cidr_block = var.a_vpc_cidr_block
vpc_id = var.a_vpc_id
vpc_name = var.a_vpc_name
subnet_availability_zone = var.a_subnet_availability_zone
subnet_cidr_block = var.a_subnet_cidr_block
subnet_name = var.a_subnet_name
else if var.option == 'B'
vpc_cidr_block = var.b_vpc_cidr_block
vpc_id = var.b_vpc_id
vpc_name = var.b_vpc_name
subnet_availability_zone = var.b_subnet_availability_zone
subnet_cidr_block = var.b_subnet_cidr_block
subnet_name = var.b_subnet_name
else
raise an error
# get a data resource identified by the VPC variables
data "aws_vpc" "instance_vpc" {
cidr_block = var.vpc_cidr_block
tags = {
Name = var.vpc_name
}
}
# get a data resource identified by the VPC variables
data "aws_subnet" "instance_subnet" {
vpc_id = var.vpc_id
cidr_block = var.subnet_cidr_block
availability_zone = var.subnet_availability_zone
tags = {
Name = var.subnet_name
}
}
# create an AWS key pair resource
resource "aws_key_pair" "instance_aws_key_pair" {
key_name = "component_key_${terraform.workspace}"
public_key = file("~/.ssh/terraform.pub")
}
# create the AWS EC2 instance
resource "aws_instance" "my_aws_instance" {
key_name = aws_key_pair.instance_aws_key_pair.key_name
ami = "ami-b12345"
instance_type = "t2.micro"
subnet_id = data.aws_subnet.instance_subnet.id
connection {
type = "ssh"
user = "terraform"
private_key = file("~/.ssh/terraform")
host = self.public_ip
}
tags = {
"Name" : "my_instance_name"
"Terraform" : "true"
}
}
Is this a matter of somehow using a count, something like this:
count = var.option == 'A'? 1 : 0
Is there a way to do this, or is there a better approach? I am very new to Terraform so I may be missing something obvious.
You have a couple of questions here.
Firstly, you should be able to use the newer, experimental custom validation rules to assert that a value is in a specific list of values.
Secondly, for determining which set of variables to use, I'd recommend going with a good old map in a local value.
For example,
locals {
vpc_info = {
"A" = {
vpc_cidr_block = var.a_vpc_cidr_block
vpc_id = var.a_vpc_id
vpc_name = var.a_vpc_name
subnet_availability_zone = var.a_subnet_availability_zone
subnet_cidr_block = var.a_subnet_cidr_block
subnet_name = var.a_subnet_name
}
"B" = {
vpc_cidr_block = var.b_vpc_cidr_block
vpc_id = var.b_vpc_id
vpc_name = var.b_vpc_name
subnet_availability_zone = var.b_subnet_availability_zone
subnet_cidr_block = var.b_subnet_cidr_block
subnet_name = var.b_subnet_name
}
}
}
Then you should be able to reference a specific field, within the chose option like the following
local.vpc_info[var.option].vpc_name
Let me know if this hits all your questions.

Resources