how to fix terraform unpredict instance creation issue? - terraform

Question

I'm getting the below error while running terraform plan and apply
on main.tf line 517, in resource "aws_lb_target_group_attachment" "ecom-tga":
│ 517: for_each = local.service_instance_map
│ ├────────────────
│ │ local.service_instance_map will be known only after apply
│
│ The "for_each" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will
│ be created. To work around this, use the -target argument to first apply only the resources that the for_each depends on.
My configuration file is as below
variable "instance_count" {
type = string
default = 3
}
variable "service-names" {
type = list
default = ["valid","jsc","test"]
}
locals {
helper_map = {for idx, val in setproduct(var.service-names, range(var.instance_count)):
idx => {service_name = val[0]}
}
}
resource "aws_instance" "ecom-validation-service" {
for_each = local.helper_map
ami = data.aws_ami.ecom.id
instance_type = "t3.micro"
tags = {
Name = "${each.value.service_name}-service"
}
vpc_security_group_ids = [data.aws_security_group.ecom-sg[each.value.service_name].id]
subnet_id = data.aws_subnet.ecom-subnet[each.value.service_name].id
}
data "aws_instances" "ecom-instances" {
for_each = toset(var.service-names)
instance_tags = {
Name = "${each.value}-service"
}
instance_state_names = ["running", "stopped"]
depends_on = [
aws_instance.ecom-validation-service
]
}
locals {
service_instance_map = merge([for env, value in data.aws_instances.ecom-instances:
{
for id in value.ids:
"${env}-${id}" => {
"service-name" = env
"id" = id
}
}
]...)
}
resource "aws_lb_target_group_attachment" "ecom-tga" {
for_each = local.service_instance_map
target_group_arn = aws_lb_target_group.ecom-nlb-tgp[each.value.service-name].arn
port = 80
target_id = each.value.id
depends_on = [aws_lb_target_group.ecom-nlb-tgp]
}
Since i'm passing count as var and its value is 3,i thought terraform will predict as it needs to create 9 instances.But it didn't it seems and throwing error as unable to predict.
Do we have anyway to by pass this by giving some default values for instances count prediction or for that local service_instance_map?
Tried try function but still no luck
Error: Invalid for_each argument
│
│ on main.tf line 527, in resource "aws_lb_target_group_attachment" "ecom-tga":
│ 527: for_each = try(local.service_instance_map,[])
│ ├────────────────
│ │ local.service_instance_map will be known only after apply
│
│ The "for_each" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will
│ be created. To work around this, use the -target argument to first apply only the resources that the for_each depends on.
My requirement got changed and now i have to create 3 instances in 3 subnets available in that region.I changed the locals as like below But same prediction issue
locals {
merged_subnet_svc = try(flatten([
for service in var.service-names : [
for subnet in aws_subnet.ecom-private.*.id : {
service = service
subnet = subnet
}
]
]), {})
variable "azs" {
type = list(any)
default = ["ap-south-1a", "ap-south-1b", "ap-south-1c"]
}
variable "private-subnets" {
type = list(any)
default = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
}
resource "aws_instance" "ecom-instances" {
for_each = {
for svc in local.merged_subnet_svc : "${svc.service}-${svc.subnet}" => svc
}
ami = data.aws_ami.ecom.id
instance_type = "t3.micro"
tags = {
Name = "ecom-${each.value.service}-service"
}
vpc_security_group_ids = [aws_security_group.ecom-sg[each.value.service].id]
subnet_id = each.value.subnet
}
}

Answer 1

In your configuration you've declared that data "aws_instances" "ecom-instances" depends on aws_instance.ecom-validation-service. Since that other object won't exist yet on your first run, Terraform must therefore wait until the apply step to read data.aws_instances.ecom-instances because otherwise it would fail to honor the dependency you've declared, because aws_instance.ecom-validation-service wouldn't exist yet.
To avoid the error message you saw here, you need to make sure that for_each only refers to values that Terraform will know before any objects are actually created. Because EC2 assigns instance ids only once the instance is created, it's not correct to use an EC2 instance id as part of a for_each instance key.
Furthermore, there's no need for a data "aws_instances" block to retrieve instance information here because you already have the relevant instance information as a result of the resource "aws_instance" "ecom-validation-service" block.
With all of that said, let's start from your input variables and build things up again while making sure that we only build instance keys only from values we'll know during planning. The variables you have stay essentially the same; I've just tweaked the type constraints a little to match how we're using each one:
variable "instance_count" {
type = string
default = 3
}
variable "service_names" {
type = set(string)
default = ["valid", "jsc", "test"]
}
I understand from the rest of your example that you are intending to create var.instance_count instances for each distinct element of var.service_names. Your setproduct to produce all of the combinations of those is also good, but I'm going to tweak it to assign the instances unique keys that include the service name:
locals {
instance_configs = tomap({
for pair in setproduct(var.service_names, range(var.instance_count)) :
"${pair[0]}${pair[1]}" => {
service_name = pair[0]
}
})
}
This will produce a data structure like the following:
{
valid0 = { service_name = "valid" }
valid1 = { service_name = "valid" }
valid2 = { service_name = "valid" }
jsc0 = { service_name = "jsc" }
jsc1 = { service_name = "jsc" }
jsc2 = { service_name = "jsc" }
test0 = { service_name = "test" }
test1 = { service_name = "test" }
test2 = { service_name = "test" }
}
This matches the shape that for_each expects, so we can use it directly to declare nine aws_instance instances:
resource "aws_instance" "ecom-validation-service" {
for_each = local.instance_configs
instance_type = "t3.micro"
ami = data.aws_ami.ecom.id
subnet_id = data.aws_subnet.ecom-subnet[each.value.service_name].id
vpc_security_group_ids = [
data.aws_security_group.ecom-sg[each.value.service_name].id,
]
tags = {
Name = "${each.value.service_name}-service"
Service = each.value_service_name
}
}
So far this has been mostly the same as what you shared. But this is the point where I'm going to go in a totally different direction: rather than now trying to read back the instances this declared using a separate data resource, I'll just gather the same data directly from the aws_instance.ecom-validation-service resource. It's generally best for a Terraform configuration to either manage a particular object or read it, not both at the same time, because this way the necessary dependency ordering is revealed automatically be the references.
Notice that I included an extra tag Service on each of the instances to give a more convenient way to get the service name back. If you can't do that then you could get the same information by trimming the -service suffix from the Name tag, but I prefer to keep things direct where possible.
It seemed like your goal then was to have a aws_lb_target_group_attachment instance per instance, with each one connected to the appropriate target group based on the service name. Because that aws_instance resource has for_each set, aws_instance.ecom-validation-service in expressions elsewhere is a map of objects where the keys are the same as the keys in var.instance_configs. That means that value is also compatible with the requirements for for_each and so we can use it directly to declare the target group attachments:
resource "aws_lb_target_group_attachment" "ecom-tga" {
for_each = aws_instance.ecom-validation-service
target_group_arn = aws_lb_target_group.ecom-nlb-tgp[each.value.tags.Service].arn
port = 80
target_id = each.value.id
}
I relied on the extra Service tag from earlier to easily determine which service each instance belongs to in order to look up the appropriate target group ARN. each.value.id works here because each.value is always an aws_instance object, which exports that id attribute.
The result of this is two sets of instances that each have keys matching those in local.instance_configs:
aws_instance.ecom-validation-service["valid0"]
aws_instance.ecom-validation-service["valid1"]
aws_instance.ecom-validation-service["valid2"]
aws_instance.ecom-validation-service["jsc0"]
aws_instance.ecom-validation-service["jsc1"]
aws_instance.ecom-validation-service["jsc2"]
...
aws_lb_target_group_attachment.ecom-tga["valid0"]
aws_lb_target_group_attachment.ecom-tga["valid1"]
aws_lb_target_group_attachment.ecom-tga["valid2"]
aws_lb_target_group_attachment.ecom-tga["jsc0"]
aws_lb_target_group_attachment.ecom-tga["jsc1"]
aws_lb_target_group_attachment.ecom-tga["jsc2"]
...
Notice that all of these keys contain only information specified directly in the configuration, and not any information decided by the remote system. That means we avoid the "Invalid for_each argument" error even though each instance still has an appropriate unique key. If you were to add a new element to var.service_names or increase var.instance_count later then Terraform will also see from the shape of these instance keys that it should just add new instances of each resource, rather than renaming/renumbering any existing instances.