Obtain value from a local list - terraform

I am trying to tag the docker swarm instances using terraform
I defined variable and locals as
variables.tf
variable "instance_count" {
default = "3"
}
variable "instance_type" {
default = "t2.micro"
}
variable "aws_region" {
default = "us-east-1"
}
variable "ami" {
default = "ami-09e67e426f25ce0d7"
}
variable "host_name" {
type = map(number)
default = {
"Manager" = 1
"Worker" = 2
}
}
When i refer to this list's each value to assign it as a tag to ec2 instance like this
ec2instance.tf
resource "aws_instance" "swarm_instance" {
count = var.instance_count
ami = var.ami
instance_type = var.instance_type
key_name = aws_key_pair.dockerswarm.key_name
tags = {
Name = "Swarm_Instance-${count.index + 1}"
}
tags = {
Name = "${local.expanded_names}"
}
locals {
expanded_names = {
for name, count in var.host_name : name => [
for i in range(count) : format("%s-%02d", name, i+1)
]
}
}
Terraform complains
local.expanded_names is object with 2 attributes
I tried with ${local.expanded_names.value}, but then it complained object does not have an attribute named "value".
So how to retrieve the value from the list when value attribute is not available in terraform.

The tags should be strings, in your case I would use jsonencode to get a string out of that object you are building, see my sample code below
variable "host_name" {
type = map(number)
default = {
"Manager" = 1
"Worker" = 2
}
}
locals {
expanded_names = jsonencode({
for name, count in var.host_name : name => [
for i in range(count) : format("%s-%02d", name, i+1)
]
})
}
provider "aws" {
region = "us-east-1"
}
resource "aws_instance" "instance" {
ami = "ami-1c761163"
instance_type = "r5.large"
tags = {
Terraformed = "true"
Name = local.expanded_names
}
}
if we run a terraform plan on that, here is what we get:
Terraform will perform the following actions:
# aws_instance.instance will be created
+ resource "aws_instance" "instance" {
+ ami = "ami-1c761163"
...
+ instance_state = (known after apply)
+ instance_type = "r5.large"
...
+ subnet_id = (known after apply)
+ tags = {
+ "Name" = jsonencode(
{
+ Manager = [
+ "Manager-01",
]
+ Worker = [
+ "Worker-01",
+ "Worker-02",
]
}
)
+ "Terraformed" = "true"
}
Or maybe what you meant to do is create an array of names:
Manager-01
Worker-01
Worker-02
Then use that as the instance names... if that is the case your expanded_names should not be an object {} but an array [], then we use that instead of your count, see code sample below:
variable "host_name" {
type = map(number)
default = {
"Manager" = 1
"Worker" = 2
}
}
locals {
expanded_names = flatten([
for name, count in var.host_name : [
for i in range(count) : format("%s-%02d", name, i+1)
]
])
}
provider "aws" {
region = "us-east-1"
}
resource "aws_instance" "instance" {
for_each = toset(local.expanded_names)
ami = "ami-1c761163"
instance_type = "r5.large"
tags = {
Terraformed = "true"
Name = each.value
}
}
and a terraform plan on that outputs:
Terraform will perform the following actions:
# aws_instance.instance["Manager-01"] will be created
+ resource "aws_instance" "instance" {
+ ami = "ami-1c761163"
...
+ tags = {
+ "Name" = "Manager-01"
+ "Terraformed" = "true"
}
...
}
# aws_instance.instance["Worker-01"] will be created
+ resource "aws_instance" "instance" {
+ ami = "ami-1c761163"
...
+ tags = {
+ "Name" = "Worker-01"
+ "Terraformed" = "true"
}
...
}
# aws_instance.instance["Worker-02"] will be created
+ resource "aws_instance" "instance" {
+ ami = "ami-1c761163"
...
+ tags = {
+ "Name" = "Worker-02"
+ "Terraformed" = "true"
}
...
}
Plan: 3 to add, 0 to change, 0 to destroy.

Related

This map does not have an element with the key in terraform

I am doing the following in my terraform module:
data "vault_policy_document" "this" {
dynamic "rule" {
for_each = {
for p in var.policy.policy_content : format("%s-%s-%s", p.path, join(",", p.capabilities)) => p
}
content {
path = rule.value.path
capabilities = rule.value.capabilities
}
}
}
The variable is declared as
variable "policy" {
description = "The policy to be created"
type = map(any)
}
and with a certain confidence, it is of the form
{
+ "policy-test-1" = {
+ policy_content = [
+ {
+ capabilities = [
+ "read",
+ "create",
]
+ path = "/foo/lala"
},
+ {
+ capabilities = [
+ "read",
+ "create",
]
+ path = "/bar/lala"
},
]
}
},
Why is the code failing to access the policy_content element?
Your initial attempt isn't iterating over the keys of the policy map, which you can fix by looping the policy document datasource itself over the map:
data "vault_policy_document" "this" {
for_each = var.policy
dynamic "rule" {
for_each = {
for p in each.value.policy_content : format("%s-%s", p.path, join(",", p.capabilities)) => p
}
content {
path = rule.value.path
capabilities = rule.value.capabilities
}
}
}
resource "vault_policy" "this" {
for_each = var.policy
name = each.key
policy = data.vault_policy_document.this[each.key].hcl
}
This can probably be cleaned up further (like by iterating the policy resource over the data source elements rather than the same policy var again).
But it does produce what looks like the right plan, to me:
# vault_policy.this["policy-test-1"] will be created
+ resource "vault_policy" "this" {
+ id = (known after apply)
+ name = "policy-test-1"
+ policy = <<-EOT
path "/bar/lala" {
capabilities = ["read", "create"]
}
path "/foo/lala" {
capabilities = ["read", "create"]
}
EOT
}

Terraform: How would I reference a variable in For_Each that is not included in a map, such as file_system_id?

Maybe this is possible, maybe it's not. I'm attempting to mount an EFS target using some of the values stored in a var.ec2_server map which includes subnets, EBS volumes, etc.
The issue I've run into is that I created the EFS File System using a for_each statement; since the efs_file_system was created with a for_each, I must reference the attributes within specified instances when referring to the resource in other variables.
The file_system_id is only known after creation so how would I reference it within a map or other variable inside other for_each statements, such as the aws_efs_mount_target resource defined below? Will what I'm doing even work?
I'm using the antiquated resource.tf > variable.tf > terraform.tfvars (config) style code :
...the ec2.tf file:
###############################################################################
# EC2 Instance
resource "aws_instance" "ec2" {
for_each = var.ec2_servers
ami = data.aws_ami.ec2[each.key].id
disable_api_termination = var.disable_api_termination
iam_instance_profile = aws_iam_instance_profile.ec2[each.key].id
instance_type = each.value.instance_type
monitoring = true
vpc_security_group_ids = [aws_security_group.ec2[each.key].id]
subnet_id = each.value.subnet_name != null ? aws_subnet.private["${each.value.vpc_name}.${each.value.subnet_name}.${each.value.availability_zone}"].id : null
key_name = aws_key_pair.ec2.key_name
user_data = each.value.user_data == "" ? null : templatefile("./${each.value.user_data}", { region = data.aws_region.current.name })
private_ip = each.value.private_ip
metadata_options {
http_endpoint = "enabled"
http_tokens = "required"
}
root_block_device {
delete_on_termination = true
encrypted = true
volume_size = each.value.root_volume_size
volume_type = "gp2"
tags = {
Name = replace("${var.project_name}-${each.value.vpc_name}-${each.key}-EBS", " ", "")
}
}
dynamic "ebs_block_device" {
for_each = each.value.ebs_volumes
content {
volume_type = ebs_block_device.value.volume_type
volume_size = ebs_block_device.value.volume_size
device_name = ebs_block_device.value.device_name
tags = {
Name = replace("${var.project_name}-${each.value.vpc_name}-${each.key}-EBS", " ", "") }
}
}
tags = {
Name = replace("${var.project_name}-${each.value.vpc_name}-${each.key}-EC2", " ", "")
Backup = "true"
}
}
...the efs.tf file:
###############################################################################
# Create EFS File System
resource "aws_efs_file_system" "efs" {
for_each = {
for object, property in var.efs_config : object => property if var.efs_config.efs_enabled
}
creation_token = var.efs_config.efs_creation_token
encrypted = var.efs_config.efs_encrypt
kms_key_id = aws_kms_key.efs_kms.arn
tags = {
Name = replace("${var.project_name}-${var.efs_config.efs_vpc}-EFS", " ", "")
}
}
resource "aws_efs_backup_policy" "efs_backup_policy" {
file_system_id = "NEEDS TO BE DETERMINED"
backup_policy {
status = "ENABLED"
}
}
resource "aws_efs_mount_target" "efs_mount_target" {
for_each = var.ec2_servers
file_system_id = "NEEDS TO BE DETERMINED"
subnet_id = each.value.subnet_name == "app" ? aws_subnet.private["${each.value.vpc_name}.${each.value.subnet_name}.${each.value.availability_zone}"].id : null
ip_address = lookup(var.efs_config, "efs_private_ip")
security_groups = [aws_security_group.ec2[each.key].id]
}
...the variables.tf file:
variable "ec2_servers" {
description = "A configurable map of EC2 settings."
type = map(any)
}
...the terraform.tfvars file:
###############################################################################
# EFS Configurations
efs_config = {
efs_enabled = true
efs_creation_token = "Prod_EFS"
efs_encrypt = true
efs_vpc = "Prod"
efs_private_ip = "10.200.0.5"
}
# Server Configurations
ec2_servers = {
EC201 = {
ami_owner = "XXXXXXXXXXXX"
ami_name = "xxxxx-xxxxxx"
instance_type = "t2.micro"
root_volume_size = "10"
ebs_volumes = [
{
volume_size = "20"
volume_type = "gp3"
device_name = "/dev/xvdba"
},
{
volume_size = "20"
volume_type = "gp3"
device_name = "/dev/xvdbb"
}
]
vpc_name = "Prod"
subnet_name = "web"
set_ec2_hostname = false
ec2_hostname = "xxxxxxxxx"
availability_zone = "a"
public_dns = false
private_dns = true
policy_names = []
s3_storage = false
transfer_files = false
user_data = "setup_ssm_linux.tftpl"
private_ip = "10.200.0.132"
ingress = {
ssh = {
description = "Internal address"
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = [
"10.200.0.0/22"
]
}
}
}
}
I've tried a number of things such as creating a data resource for aws_efs_mount_target and nothing I do seems to work. If anyone could provide a little insight, both my project leads and myself would be greatly appreciated!
If I missed anything here, please let me know and I will update the question with the relevant information.
Your aws_efs_backup_policy needs a for_each also, since you need to create one for each EFS volume:
resource "aws_efs_backup_policy" "efs_backup_policy" {
for_each = aws_efs_file_system.efs
file_system_id = each.id
backup_policy {
status = "ENABLED"
}
}
For your EFS mount target, I would use the same for_each you use for the EFS volumes:
resource "aws_efs_mount_target" "efs_mount_target" {
for_each = {
for object, property in var.efs_config : object => property if var.efs_config.efs_enabled
}
file_system_id = aws_efs_file_system.efs[each.key].id
...
}
I think you need to clean up those other lookups in aws_efs_mount_target by moving those values into the efs_config var.

How to create multiple instances with multiple subnet ids using terraform?

I have 2 services test1,test2 and for each service i have to create 6 vm's.This 6 vm's should be placed in 3 subnet id's which created in 3 different zones in a same region
In this services,test1 will be in private subnets and test2 will be in public subnets.So i have to pass that correct subnet id when creating ec2 instances
root module:
provider "aws" {
region = var.region
}
module "ecom-vpc" {
source = "./modules/vpc"
}
module "ecom-public-subnet" {
source = "./modules/subnets/public"
vpc-id = module.ecom-vpc.vpc-id
}
module "ecom-private-subnet" {
source = "./modules/subnets/private"
vpc-id = module.ecom-vpc.vpc-id
}
module "ecom-instances-sg" {
source = "./modules/sg"
vpc-id = module.ecom-vpc.vpc-id
}
module "ecom-vm-instances" {
source = "./modules/vm"
priv-subnet-ids = module.ecom-private-subnet.ecom_private_subnets
pub-subnet-ids = module.ecom-public-subnet.ecom_public_subnets
instances-sg = module.ecom-instances-sg.ecom-inst-sg
}
From child modules - vpc,subnets,ec2
variable "service-names" {
type = list(any)
default = ["ecom-app-TEST1","ecom-app-TEST2"]
}
variable "availability_zones" {
type = map
default = {
ap-south-1a = {
private_subnet = "10.0.1.0/24"
public_subnet = "10.0.4.0/24"
}
ap-south-1b = {
private_subnet = "10.0.2.0/24"
public_subnet = "10.0.5.0/24"
}
ap-south-1c = {
private_subnet = "10.0.3.0/24"
public_subnet = "10.0.6.0/24"
}
}
}
resource "aws_vpc" "ecom-vpc" {
cidr_block = var.ecom-cidr
}
output "vpc-id" {
value = aws_vpc.ecom-vpc.id
}
resource "aws_subnet" "ecom-private" {
for_each = var.availability_zones
vpc_id = var.vpc-id
cidr_block = each.value.private_subnet
availability_zone = each.key
map_public_ip_on_launch = false
tags = {
Name = "${split("-", each.key)[2]}"
Subnet_Type = "private"
}
}
output "ecom_private_subnets" {
value = aws_subnet.ecom-private
}
resource "aws_subnet" "ecom-public" {
for_each = var.availability_zones
vpc_id = var.vpc-id
cidr_block = each.value.public_subnet
availability_zone = each.key
map_public_ip_on_launch = true
tags = {
Name = "${split("-", each.key)[2]}"
Subnet_Type = "public"
}
depends_on = [aws_internet_gateway.igw
]
}
output "ecom_public_subnets" {
value = aws_subnet.ecom-public
}
I'm trying to achieve the same by creating a locals which combines service names,priv,public subnet id's,instance count(2).But the problem is i'm not able to make it because i'm not able to create a unique combination of keys
locals {
service_subnets = {
for pair in setproduct(var.service-names, values(var.priv-subnet-ids),values(var.pub-subnet-ids),range(var.instance_count)) :
"${pair[0]}:${pair[1].availability_zone}-${pair[3]}" => {
service_name = pair[0]
priv-subnet = pair[1]
pub-subnet = pair[2]
}
}
}
resource "aws_instance" "ecom-instances" {
for_each = local.service_subnets
ami = data.aws_ami.ecom.id
instance_type = "t3.micro"
tags = {
Name = each.value.service_name
Service = each.value.service_name
}
vpc_security_group_ids = [var.instances-sg[each.value.service_name].id]
subnet_id = "${split("-", each.value.service_name)[2] == "TEST1" ? each.value.pub-subnet.id : each.value.priv-subnet.id }"
}
I'm getting the below error.
Two different items produced the key "ecom-app-TEST1:ap-south-1c-1" in this 'for' expression. If duplicates are expected, use the ellipsis (...)
│ after the value expression to enable grouping by key.
If i add ... and change it as below then it is converted as tuple and i'm not sure how to get and pass the value from each.value in the aws_instance resource
locals {
service_subnets = {
for pair in setproduct(var.service-names, values(var.priv-subnet-ids),values(var.pub-subnet-ids),range(var.instance_count)) :
"${pair[0]}:${pair[1].availability_zone}-${pair[3]}" => {
service_name = pair[0]
priv-subnet = pair[1]
pub-subnet = pair[2]
}
... }
}
on modules/vm/main.tf line 60, in resource "aws_instance" "ecom-instances":
│ 60: subnet_id = "${split("-", each.value.service_name)[2] == "TEST1" ? each.value.pub-subnet.id : each.value.priv-subnet.id }"
│ ├────────────────
│ │ each.value is tuple with 3 elements
│
│ This value does not have any attributes.
Please Guide me
You could try adding the index to your for loop and making it part of your name, might help you avoid elipsis/tuple conversion.
locals {
service_subnets = {
for index, pair in setproduct(var.service-names, values(var.priv-subnet-ids),values(var.pub-subnet-ids),range(var.instance_count)) :
"${pair[0]}:${pair[1].availability_zone}-${pair[3]}=${index}" => {
service_name = pair[0]
priv-subnet = pair[1]
pub-subnet = pair[2]
}
}
}

Is it possible to reference the resource name inside the resource itself

I'd like to use a resource name inside the resource itself to avoid string duplication and copy/paste errors.
resource "aws_instance" "bastion-euw3-infra-01" {
ami = "ami-078db6d55a16afc82"
instance_type = "t2.micro"
key_name = "sylvain"
user_data = templatefile("./scripts/cloudinit.yaml", {
hostname = "bastion-euw3-infra-01"
tailscale_authkey = var.tailscale_authkey
})
network_interface {
device_index = 0
network_interface_id = aws_network_interface.bastion-euw3-infra-01.id
}
tags = {
Name = "bastion-euw3-infra-01"
Environment = "infra"
}
lifecycle {
ignore_changes = [user_data]
}
}
Basically I'd like to replace "bastion-euw3-infra-01" inside the resource with some kind of var, e.g.:
resource "aws_instance" "bastion-euw3-infra-01" {
...
user_data = templatefile("./scripts/cloudinit.yaml", {
hostname = ___name___
tailscale_authkey = var.tailscale_authkey
})
...
tags = {
Name = ___name___
Environment = "infra"
}
...
}
Does terraform provide a way to do this ?

terraform How to use conditional if in for_each into map object

I have maps of variables like this:
users.tfvars
users = {
"testterform" = {
path = "/"
force_destroy = true
email_address = "testterform#example.com"
group_memberships = [ "test1" ]
tags = { department : "test" }
ssh_public_key = "ssh-rsa AAAAB3NzaC1yc2EAAA4l7"
}
"testterform2" = {
path = "/"
force_destroy = true
email_address = "testterform2#example.com"
group_memberships = [ "test1" ]
tags = { department : "test" }
ssh_public_key = ""
}
I would like to upload ssh key only if ssh_public_key not empty for the user. But don't understand how to check this
#main.tf
resource "aws_iam_user" "this" {
for_each = var.users
name = each.key
path = each.value["path"]
force_destroy = each.value["force_destroy"]
tags = merge(each.value["tags"], { Provisioner : var.provisioner, EmailAddress : each.value["email_address"] })
}
resource "aws_iam_user_group_membership" "this" {
for_each = var.users
user = each.key
groups = each.value["group_memberships"]
depends_on = [ aws_iam_user.this ]
}
resource "aws_iam_user_ssh_key" "this" {
for_each = var.users
username = each.key
encoding = "SSH"
public_key = each.value["ssh_public_key"]
depends_on = [ aws_iam_user.this ]
}
It sounds like what you need here is a derived "users that have non-empty SSH keys" map. You can use the if clause of a for expression to derive a new collection from an existing one while filtering out some of the elements:
resource "aws_iam_user_ssh_key" "this" {
for_each = {
for name, user in var.users : name => user
if user.ssh_public_key != ""
}
username = each.key
encoding = "SSH"
public_key = each.value.ssh_public_key
depends_on = [aws_iam_user.this]
}
The derived map here uses the same keys and values as the original var.users, but is just missing some of them. That means that the each.key results will correlate and so you'll still get the same username value you were expecting, and your instances will have addresses like aws_iam_user_ssh_key.this["testterform"].
You can use a for loop to exclude those blanks.
For example, you can do it on local:
variable "users" {
default = {
"testterform" = {
path = "/"
force_destroy = true
tags = { department : "test" }
ssh_public_key = "ssh-rsa AAAAB3NzaC1yc2EAAA4l7"
}
"testterform2" = {
path = "/"
force_destroy = true
tags = { department : "test" }
ssh_public_key = ""
}
}
}
locals {
public_key = flatten([
for key, value in var.users :
value.ssh_public_key if ! contains([""], value.ssh_public_key)
])
}
output "myout" {
value = local.public_key
}
that will output:
myout = [
"ssh-rsa AAAAB3NzaC1yc2EAAA4l7",
]
As you can see the empty ones have been removed, and you can add other stuff you want to exclude on that contains array.
Then you can use that local.public_key in the for_each for your ssh keys

Resources