Benefit of having Concat function in terraform - terraform

I am trying to understand the benefit of having a Concat function in the output section.
as I have experiment output section with and without Concat. But I cannot see any difference.
example
resource "aws_security_group" "sg_22" {
name = "sgx_22"
vpc_id = var.vpc
ingress {
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
resource "aws_security_group" "sg_8080" {
name = "sgx_8080"
vpc_id = var.vpc
ingress {
from_port = 8080
to_port = 8080
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
output "web_security_group_ids" {
value = concat([aws_security_group.sg_22.id, aws_security_group.sg_8080.id])
}
VS
resource "aws_security_group" "sg_22" {
name = "sgx_22"
vpc_id = var.vpc
ingress {
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
resource "aws_security_group" "sg_8080" {
name = "sgx_8080"
vpc_id = var.vpc
ingress {
from_port = 8080
to_port = 8080
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
output "web_security_group_ids" {
value = [aws_security_group.sg_22.id, aws_security_group.sg_8080.id]
}
In both case, Terraform give the same output in the same format.
Any idea?

That is a mistake in the learning guide. The concat function "concatenates" two different argument lists into a single list containing all the elements of each. In this case, you have a single list of two lists as an argument to the concat function. I believe that the actual function the guide should have mentioned is flatten. This function will "flatten" redundant nested lists into a single list:
output "web_security_group_ids" {
value = flatten([aws_security_group.sg_22.id, aws_security_group.sg_8080.id])
}
However, this is still rather pointless as the only reason it is a nested list is because the two lists are arguments to a list constructor. Alternatively, the guide may actually have meant to concatenate the two lists directly:
output "web_security_group_ids" {
value = concat(aws_security_group.sg_22.id, aws_security_group.sg_8080.id)
}
The actual best way to define this output would be to simply use the resource attribute related to the resource argument. This will then be a simple definition in the output, a single list, and guaranteed to update dynamically based on changes to your arguments.
output "web_security_group_ids" {
value = aws_instance.web.vpc_security_group_ids
}

Related

Terraform: Creating multiple instances with Terraform with for_each and mapping security_group_rule

I have a scenario where I need to create multiple EC2 instances which are part of a cluster.These hosts have to be accessible on specific ports from one another and need to have two ebs_volumes attached to them of size 16GB and 700GB.
snip of my variable.tf looks like this:-
variable "instances" {
default = {
instance_name = "a"
tcp_ports = ["53","22","2022","80","443"]
udp_ports = ["53","67","68","123","161","162","500"]
"xvdf" = "16"
"xvdg" = "700"
}
}
I am struggling to get this mapping to work with my TF script:-
resource "aws_security_group_rule" "tcp_ingress" {
for_each = {
for inst in local.instances : inst.tcp_ports => {
for i in inst: i.tcp_ports => i
}
}
type = "ingress"
from_port = each.value.tcp_ports
to_port = each.value.tcp_ports
protocol = "tcp"
cidr_blocks = [ for i in aws_instance.instance: format("%s/32", i.private_ip ) ]
security_group_id = aws_security_group.ha-sg.id
}
Is there a way I can iterate through the ports and form the security group rules.
Is there a way I can iterate through the ports and form the security group rules.
Since your variable "instances" is a single map, to access the tcp_ports you can simply do the following:
var.instances.tcp_ports
Then to use it in for_each:
resource "aws_security_group_rule" "tcp_ingress" {
for_each = toset(var.instances.tcp_ports)
type = "ingress"
from_port = each.value
to_port = each.value
protocol = "tcp"
cidr_blocks = [ for i in aws_instance.instance: format("%s/32", i.private_ip ) ]
security_group_id = aws_security_group.ha-sg.id
}

Getting the CIDR block of a set of subnets

I am trying to get the CIDR block of a set of subnets IDs provided a parameter.
data "aws_subnet" "target" {
for_each = "${toset(var.subnet_ids)}"
id = "${each.value}"
}
resource "aws_security_group" "registry" {
vpc_id = "${var.vpc_id}"
ingress {
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["${data.aws_subnet.target.*.cidr_block}"]
}
tags = {
Name = "${var.name}"
}
}
The error I am getting is:
cidr_blocks = "${data.aws_subnet.target.*.cidr_block}"
This object does not have an attribute named "cidr_block".
Terraform configuration:
Terraform v0.12.24
+ provider.aws v2.55.0
+ provider.template v2.1.2
Thanks to whoever can help!
You should use a splat expression to the values of the map.
resource "aws_security_group" "registry" {
vpc_id = "${var.vpc_id}"
ingress {
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = values(data.aws_subnet.target).*.cidr_block
}
tags = {
Name = "${var.name}"
}
}
I think the problem here is that the for_each argument makes data.aws_subnet.target appear as a map value, and the splat operator .* is defined to behave as a no-op if it's applied to anything other than a list or set. Therefore expression evaluation here is taking the following steps:
First evaluate data.aws_subnet.target, producing a map of objects whose keys are the strings from var.subnet_ids, which presumably look like subnet-12345.
Then apply .* to that map. Because it's a map, that produces the same result again.
Then apply .cidr_block to that map, which fails because its keys are the subnet ids, not the attributes of the subnet objects.
To get the desired result here, we can use values to disregard the keys and take the values from the map as a list:
cidr_blocks = values(data.aws_subnet.target).*.cidr_block
Because the result of values(...) is a list, the .* operator will behave as expected and try to find the cidr_block attribute in each of the elements of the list.
Another option is to use a for expression, which is a generalization of the splat syntax that works for any collection-typed value:
cidr_blocks = [for s in data.aws_subnet.target : s.cidr_block]

How do I send block of the resource module externally

I want to send block of security rule mentioned below to security group as an input,Is it possible in terraform?
ingress {
from_port = 5985
to_port = 5986
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
If we consider variables,we have a provision to read the variable value externally
instance_type = ${var.instance_type}
In variable.tf, we declare instance_type.
Likewise,is there any option to send whole of the below mentioned block to resource "aws_security_group" "allow_al"
ingress {
from_port = 5985
to_port = 5986
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
It isn't possible to send the whole block, but you can define a variable whose type is an object with a similar set of attributes and then use that object to populate the block.
variable "ingress_rule" {
type = object({
from_port = number
to_port = number
protocol = string
cidr_blocks = list(string)
})
}
resource "aws_security_group" "example" {
# ...
ingress {
from_port = var.ingress_rule.from_port
to_port = var.ingress_rule.to_port
protocol = var.ingress_rule.protocol
cidr_blocks = var.ingress_rule.cidr_blocks
}
}
The above assumes you want exactly one ingress block and just want to let the caller customize its contents. If you instead want to let the caller specify zero or more ingress blocks then the answer is slightly more complicated, but follows the same principle:
variable "ingress_rules" {
type = list(object({
from_port = number
to_port = number
protocol = string
cidr_blocks = list(string)
}))
}
resource "aws_security_group" "example" {
# ...
dynamic "ingress" {
for_each = var.ingress_rules
content {
from_port = ingress.value.from_port
to_port = ingress.value.to_port
protocol = ingress.value.protocol
cidr_blocks = ingress.value.cidr_blocks
}
}
}
The first notable difference here is that the variable is now defined as being a list of object instead of just one object. The second difference is that it's dynamically generating one ingress block for each element of that list using dynamic blocks.
It's always required to fully define your object type and define how it maps to the arguments within the resource block, because this then allows Terraform to validate separately your own module code (whether you spelled the argument names correctly, whether they have the right types) from the calling module's object or list of objects value, and thus report any errors at the correct source location.

Unable to create a nested dynamic block

I have a set of 9 security group rules that I need to apply to 4 different sources. I wanted to build it as a module so instead of copy/pasting the same block multiple times, I just need to pass the ports and source as variables.
I have tried to create a module that gets the ports as for_each in a dynamic block and also passes the sources with count since I failed to provide an additional dynamic block with for_each also for the sources.
modules/sg/main.tf
resource "aws_security_group" "test" {
name = "test2"
count = length(var.groups)
vpc_id = var.vpc_id
dynamic "ingress_tcp" {
for_each = var.tcp_ports
content {
from_port = ingress_tcp.value
to_port = ingress_tcp.value
protocol = "tcp"
security_groups = [var.groups[*].id]
}
}
dynamic "ingress_udp" {
for_each = var.udp_ports
content {
from_port = ingress_udp.value
to_port = ingress_udp.value
protocol = "udp"
security_groups = [var.groups[*].id]
}
}
}
main.tf
module "rules" {
source = "./module/sg"
vpc_id = var.vpc_id
name = "tomer-test"
tcp_ports = var.tcp_ports
udp_ports = var.udp_ports
groups = [var.groups[*].id]
}
variables.tf
variable "groups" {
description = "source groups"
type = "list"
default = [{
name = "Enforcement-STG",
id = "sg-c9db2183abcd"
},
{
name = "Managment-STG",
id = "sg-b0e71dfa123"
}]
}
variable "name" {
type = string
}
variable "vpc_id" {
type = string
default = ""
}
variable "tcp_ports" {
description = "tcp ports to open"
default = [514,1514, 11514, 12514, 6514]
}
variable "udp_ports" {
description = "tcp ports to open"
default = [514,1514, 11514, 12514]
}
I accept the output to build a set of rules per source groups, but the root module fails to invoke the module.
The error that I'm currently getting is
terraform plan
Error: Unsupported block type
on module/sg/main.tf line 7, in resource "aws_security_group" "test":
7: dynamic "ingress_tcp" {
Blocks of type "ingress_tcp" are not expected here.
Error: Unsupported block type
on module/sg/main.tf line 16, in resource "aws_security_group" "test":
16: dynamic "ingress_udp" {
Blocks of type "ingress_udp" are not expected here.
As the error message suggests, what you tried here is not valid because ingress_tcp is not a block type expected inside an aws_security_group resource. The correct name for this nested block type is ingress:
resource "aws_security_group" "test" {
count = length(var.groups)
name = "test2"
vpc_id = var.vpc_id
dynamic "ingress" {
for_each = var.tcp_ports
content {
from_port = ingress.value
to_port = ingress.value
protocol = "tcp"
security_groups = var.groups[*].id
}
}
dynamic "ingress" {
for_each = var.udp_ports
content {
from_port = ingress.value
to_port = ingress.value
protocol = "udp"
security_groups = var.groups[*].id
}
}
}
If you are using Terraform 0.12.6 or later, you may prefer to write this using resource for_each instead of count, like this:
resource "aws_security_group" "test" {
for_each = { for g in var.groups : g.name => g }
name = "test2-${each.key}"
vpc_id = var.vpc_id
dynamic "ingress" {
for_each = var.tcp_ports
content {
from_port = ingress.value
to_port = ingress.value
protocol = "tcp"
security_groups = var.groups[*].id
}
}
dynamic "ingress" {
for_each = var.udp_ports
content {
from_port = ingress.value
to_port = ingress.value
protocol = "udp"
security_groups = var.groups[*].id
}
}
}
This will have a similar result to your count example, but will produce instances with addresses like aws_security_group.test["Enforcement-STG"] instead of aws_security_group.test[0], which means that when you add and remove elements from var.groups in future Terraform will be able to determine which instance corresponds with each element and only add/remove the individual instances needed.
This map-based resource is likely to be easier to use elsewhere in the configuration too, since you'll be able to easily find the specific security group for each of your symbolic group names.

How to Keep Usage of Terraform aws_security_group DRY

I've written a simple module to provision a variable AZ numbered AWS VPC. It creates the route tables, gateways, routes, etc., but I'm having trouble keeping the security groups part DRY, i.e. keeping the module re-usable when specifying security groups.
This is as close as I can get:
varibles.tf:
variable "staging_security_groups" {
type = "list"
default = [ {
"name" = "staging_ssh"
"from port" = "22"
"to port" = "22"
"protocol" = "tcp"
"cidrs" = "10.0.0.5/32,10.0.0.50/32,10.0.0.200/32"
"description" = "Port 22"
} ]
}
main.tf:
resource "aws_security_group" "this_security_group" {
count = "${length(var.security_groups)}"
name = "${lookup(var.security_groups[count.index], "name")}"
description = "${lookup(var.security_groups[count.index], "description")}"
vpc_id = "${aws_vpc.this_vpc.id}"
ingress {
from_port = "${lookup(var.security_groups[count.index], "from port")}"
to_port = "${lookup(var.security_groups[count.index], "to port")}"
protocol = "${lookup(var.security_groups[count.index], "protocol")}"
cidr_blocks = ["${split(",", lookup(var.security_groups[count.index], "cidrs"))}"]
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
tags {
Name = "${lookup(var.security_groups[count.index], "name")}"
environment = "${var.name}"
terraform = "true"
}
}
Now this is fine, as long as what you want is to create a security group per port :) What I really need, is some way to call ingress the number of times that there are values in the variable staging_security_groups[THE SECURITY GROUP].from_port (please excuse the made-up notation).
You could look at using aws_security_group_rule instead of having your rules inline. You can then create a module like this:
module/sg/sg.tf
resource "aws_security_group" "default" {
name = "${var.security_group_name}"
description = "${var.security_group_name} group managed by Terraform"
vpc_id = "${var.vpc_id}"
}
resource "aws_security_group_rule" "egress" {
type = "egress"
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
description = "All egress traffic"
security_group_id = "${aws_security_group.default.id}"
}
resource "aws_security_group_rule" "tcp" {
count = "${var.tcp_ports == "default_null" ? 0 : length(split(",", var.tcp_ports))}"
type = "ingress"
from_port = "${element(split(",", var.tcp_ports), count.index)}"
to_port = "${element(split(",", var.tcp_ports), count.index)}"
protocol = "tcp"
cidr_blocks = ["${var.cidrs}"]
description = ""
security_group_id = "${aws_security_group.default.id}"
}
resource "aws_security_group_rule" "udp" {
count = "${var.udp_ports == "default_null" ? 0 : length(split(",", var.udp_ports))}"
type = "ingress"
from_port = "${element(split(",", var.udp_ports), count.index)}"
to_port = "${element(split(",", var.udp_ports), count.index)}"
protocol = "udp"
cidr_blocks = ["${var.cidrs}"]
description = ""
security_group_id = "${aws_security_group.default.id}"
}
modules/sg/variables.tf
variable "tcp_ports" {
default = "default_null"
}
variable "udp_ports" {
default = "default_null"
}
variable "cidrs" {
type = "list"
}
variable "security_group_name" {}
variable "vpc_id" {}
Use the module in your main.tf
module "sg1" {
source = "modules/sg"
tcp_ports = "22,80,443"
cidrs = ["10.0.0.5/32", "10.0.0.50/32", "10.0.0.200/32"]
security_group_name = "SomeGroup"
vpc_id = "${aws_vpc.this_vpc.id}"
}
module "sg2" {
source = "modules/sg"
tcp_ports = "22,80,443"
cidrs = ["10.0.0.5/32", "10.0.0.50/32", "10.0.0.200/32"]
security_group_name = "SomeOtherGroup"
vpc_id = "${aws_vpc.this_vpc.id}"
}
References:
For why optionally excluding a resource with count looks like this (source):
count = "${var.udp_ports == "default_null" ? 0 : length(split(",", var.udp_ports))}"
And the variable is set to:
variable "udp_ports" {
default = "default_null"
}
I managed to create really simple yet dynamic security group module that you can use. Idea here is to have ability to add any port you desire, and add to that port any range of ips you like. You can even remove egress from module as it will be created by default, or follow idea i used in ingress so you have granular egress rules (if you wish so).
module/sg/sg.tf
data "aws_subnet_ids" "selected" {
vpc_id = "${var.data_vpc_id}"
}
resource "aws_security_group" "main" {
name = "${var.sg_name}-sg"
vpc_id = "${var.data_vpc_id}"
description = "Managed by Terraform"
ingress = ["${var.ingress}"]
lifecycle {
create_before_destroy = true
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}
module/sg/vars.tf
variable "sg_name" {}
variable "data_vpc_id" {}
variable "ingress" {
type = "list"
default = []
}
ingress var needs to be type list. If you call vpc id manually you dont need data bit in module, im calling my vpc_id from terraform state that is stored in s3.
main.tf
module "aws_security_group" {
source = "module/sg/"
sg_name = "name_of_sg"
data_vpc_id = "${data.terraform_remote_state.vpc.vpc_id}"
ingress = [
{
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
description = "Managed by Terraform"
},
{
from_port = 0
to_port = 100
protocol = "tcp"
cidr_blocks = ["10.10.10.10/32"]
description = "Managed by Terraform"
},
{
from_port = 2222
to_port = 2222
protocol = "tcp"
cidr_blocks = ["100.100.100.0/24"]
description = "Managed by Terraform"
},
]
}
You can add as many ingress blocks you like, i have only 3 for test purposes. Hope this helps.
Note: You can follow this idea for many resources like RDS, where you need to specify parameters in parameter group or even tags. Cheers
Not sure if it was available at the time Brandon Miller's answer was written, but avoid count loops as they are ordered. So if you add or delete one port, it will cause all rules after it to be rebuilt as they rely on the count index, which changes. Far better to use a for_each loop. Make sure you use set not lists for this eg
variable "tcp_ports" {
default = [ ]
# or maybe default = [ "22", "443" ]
type = set(string)
}
resource "aws_security_group_rule" "tcp" {
for_each = var.tcp_ports
description = "Allow ${var.cdir} to connect to TCP port ${each.key}"
type = "ingress"
from_port = each.key
to_port = each.key
protocol = "tcp"
cidr_blocks = var.cdir
security_group_id = aws_security_group.default.id
}
Now you can add and delete ports without incurring unnecessary create and destroys
you you cant alter your data from lists to sets for any reason just wrap it eg
toset(var.tcp_ports)
or use a local to munge your data accordingly. You can also use maps as well

Resources