I see people using count to block resource creation in terraform. I want to create some resources if a condition is set to true. Is there such a thing same as in CloudFormation?
You answered yourself, the most similar thing is the count
You can use it combined with a conditional expression, like
resource "x" "y"{
count = var.tag == "to_deploy" ? 1 : 0
}
But this is just a stupid example, you can put everything, also use functions
count = max(var.array) >= 3 ? 1 : 0
And if you need to put a condition on something more complex, you can evaluate to use a locals block where do all elaboration you need, and just use some bool, or what you want, resultant from that in conditional expression.
I would like to help you more, but I should know your specific case, what are the conditions you would have.
In CloudFormation a "condition" is a top-level object type alongside resources, outputs, mappings, etc.
The Terraform language takes a slightly more general approach of just having values of various data types, combining and transforming them using expressions. Therefore there isn't a concept exactly equivalent to CloudFormation's "conditions", but you can achieve a similar effect in other ways using Terraform.
For example, if you want to encode the decision rule in only a single place and then refer to it many times then you can define a Local Value of boolean type and then refer to that from multiple resource blocks. A local value of boolean type is essentially equivalent to a condition object in CloudFormation. The CloudFormation documentation page you linked to has, at the time of writing, an example titled "Simple condition" and the following is a roughly-equivalent version of that example in the Terraform language:
variable "environment_type" {
type = string
validation {
condition = contains(["prod", "test"], var.environment_type)
error_message = "Must be either 'prod' or 'test'."
}
}
locals {
create_prod_resources = (var.environment_type == "prod")
}
resource "aws_instance" "example" {
ami = "ami-0ff8a91507f77f867"
instance_type = "..."
}
resource "aws_ebs_volume" "example" {
count = local.create_prod_resources ? 1 : 0
availability_zone = aws_instance.example.availability_zone
}
resource "aws_volume_attachment" "example" {
count = local.create_prod_resources ? 1 : 0
volume_id = aws_ebs_volume.example[count.index].id
instance_id = aws_instance.example.id
device = "/dev/sdh"
}
Two different resource blocks can both refer to local.create_prod_resources, in the same way that the two resources MountPoint and NewVolume can refer to the shared condition CreateProdResources in the CloudFormation example.
Related
I have a Terraform parent-resource that gets created conditionally, by using the count meta arg. This works fine. However, if the parent-resource doesn't get created because count is set to 0, and it has dependent child-resources, Terraform will fail. Is there a practical way to tell Terraform to ignore the children-resources, if the parent doesn't get created? The only way I can think to do it is to perform a count operation on each resource, and this seems cumbersome.
Something like this:
create_dev_compartment = 0
create_dev_subnet *skip creation*
create_dev_instance *skip creation*
create_mgt_compartment = 1
create_mgt_subnet *create resource*
create_mgt_instance *create resource*
The Terraform documentation has a section Chaining for_each between resources which describes declaring chains of resources that have the same (or derived) for_each expressions so that they can all repeat based on the same source information.
The documentation doesn't include an explicit example of the equivalent pattern for count, but it follows a similar principle: the count expression for the downstream resource will derive from the value representing the upstream resource.
Since you didn't include any Terraform code I can only show a contrived example, but here's the general idea:
variable "manage_network" {
type = bool
}
resource "compartment" "example" {
count = var.manage_network ? 1 : 0
}
resource "subnet" "example" {
count = length(compartment.example)
compartment_id = compartment.example[count.index].id
}
resource "instance" "example" {
count = length(subnet.example)
subnet_id = subnet.example[count.index].id
}
In the case of chained for_each, the full object representing the corresponding upstream resource is temporarily available as each.value inside the downstream resource block. count can't carry values along with it in the same way, so the equivalent is to refer to the upstream resource directly and then index it with count.index, which exploits the fact that these resources all have the same count value and will thus all have the same indices. Currently the only possible index will be zero, because you have a maximum count of 1, but if you change count in future to specify two or more instances then the downstream resources will all grow in the same way, creating several correlated instances all at once.
I have resources defined in .tf files that are generic to several applications. I populate many of the fields via a .tfvars file. I need to omit some of the resources entirely based on variables in the .tfvars.
For example if I have a resource like:
resource "cloudflare_record" "record" {
zone_id = "${data.cloudflare_zones.domain.zones[0].id}"
name = "${var.subdomain}"
value = "${var.origin_server}"
type = "CNAME"
ttl = 1
proxied = true
}
But then I declare something like cloudflare = false in my .tfvars file I'd like to be able to do something like this:
if var.cloudflare {
resource "cloudflare_record" "record" {
zone_id = "${data.cloudflare_zones.domain.zones[0].id}"
name = "${var.subdomain}"
value = "${var.origin_server}"
type = "CNAME"
ttl = 1
proxied = true
}
}
I've looked at dynamic blocks but that looks like you can only use those to edit fields and blocks within a resource. I need to be able to ignore an entire resource.
Add a count parameter with a ternary conditional using the variable declared in .tfvars like this:
resource "cloudflare_record" "record" {
count = var.cloudflare ? 1 : 0
zone_id = "${data.cloudflare_zones.domain.zones[0].id}"
name = "${var.subdomain}"
value = "${var.origin_server}"
type = "CNAME"
ttl = 1
proxied = true
}
In this example var.cloudflare is a boolean declared in the .tfvars file. If it is true a count of 1 record will be created. If it is false a count of 0 record will be created.
After the count apply the resource becomes a group, so later in the reference use 0-index of the group:
cloudflare_record.record[0].some_field
Expanding on #Joel Guerra's answer, after you use count to determine whether to deploy the resource or not, you can use the one() function to refer to the resource without an index (i.e. without having to use [0]).
For example, after defining the resource like below
resource "cloudflare_record" "record" {
count = var.cloudflare ? 1 : 0
}
Define a local variable like below
locals {
cloudflare_record_somefield = one(cloudflare_record.record[*].some_field)
}
Now instead of cloudflare_record.record[0].some_field, you can use
local.cloudflare_record_somefield
If the count is 0 (e.g. var.cloudflare is false and the resource wasn't created) then local.cloudflare_record_somefield would return null (instead of returning an error when indexing using [0]).
Reference: https://developer.hashicorp.com/terraform/language/functions/one
An issue i'm seeing this with is if the resource your trying to create is already using a for_each then you can't use both count and for_each in the resource. I'm still trying to find an answer on this will update if I find something better.
I am creating a series of resources in terraform (in this case, dynamo DB table). I want to apply IAM policies to subgroups of them. E.g.
resource "aws_dynamodb_table" "foo" {
count = "${length(var.tables)}"
name = "foo-${element(var.tables,count.index)}"
tags {
Name = "foo-${element(var.tables,count.index)}"
Environment = "<unsure how to get this>"
Source = "<unsure how to get this>"
}
}
All of these share some common element, e.g. var.sources is a list composed of the Cartesian product of var.environments and var.sources:
environments = ["dev","qa","prod"]
sources = ["a","b","c"]
So:
tables = ["a:dev","a:qa","a:prod","b:dev","b:qa","b:prod","c:dev","c:qa","c:prod"]
I want to get the arns of the created dynamo tables that have, e.g. c (i.e. those with the name ["c:dev","c:qa","c:prod"]) or prod(i.e. those with the name ["a:prod","b:prod","c:prod"]).
Is there any sane way to do this with terraform 0.11 (or even 0.12 for that matter)?
I am looking to:
group the dynamo db table resources by some of the inputs (environment or source) so I can apply some policy to each group
Extract the input for each created one so I can apply the correct tags
I was thinking of, potentially, instead of creating the cross-product list, to create maps for each input:
{
"a": ["dev","qa","prod"],
"b": ["dev","qa","prod"],
"c": ["dev","qa","prod"]
}
or
{
"dev": ["a","b","c"],
"qa": ["a","b","c"],
"prod": ["a","b","c"]
}
It would make it easy to find the target names for each one, since I can look up by the input, but that only gives me the names, but not make it easy to get the actual resources (and hence the arns).
Thanks!
A Terraform 0.12 solution would be to derive the cartesian product automatically (using setproduct) and use a for expression to shape it into a form that's convenient for what you need. For example:
locals {
environments = ["dev", "qa", "prod"]
sources = ["a", "b", "c"]
tables = [for pair in setproduct(local.environments, local.sources) : {
environment = pair[0]
source = pair[1]
name = "${pair[1]}:${pair[0]}"
})
}
resource "aws_dynamodb_table" "foo" {
count = length(local.tables)
name = "foo-${local.tables[count.index].name}"
tags {
Name = "foo-${local.tables[count.index].name}"
Environment = local.tables[count.index].environment
Source = local.tables[count.index].source
}
}
At the time I write this the resource for_each feature is still in development, but in a near-future Terraform v0.12 minor release it should be possible to improve this further by making these table instances each be identified by their names, rather than by their positions in the local.tables list:
# (with the same "locals" block as in the above example)
resource "aws_dynamodb_table" "foo" {
for_each = { for t in local.tables : t.name => t }
name = "foo-${each.key}"
tags {
Name = "foo-${each.key}"
Environment = each.value.environment
Source = each.value.source
}
}
As well as cleaning up some redundancy in the syntax, this new for_each form will cause Terraform to identify this instances with addresses like aws_dynamodb_table.foo["a:dev"] instead of aws_dynamodb_table.foo[0], which means that you'll be able to freely add and remove members of the two initial lists without causing churn and replacement of other instances because the list indices changed.
This sort of thing would be much harder to achieve in Terraform 0.11. There are some general patterns that can help translate certain 0.12-only constructs to 0.11-compatible features, which might work here:
A for expression returning a sequence (one with square brackets around it, rather than braces) can be simulated with a data "null_data_source" block with count set, if the result would've been a map of string values only.
A Terraform 0.12 object in a named local value can in principle be replaced with a separate simple map of local value for each object attribute, using a common set of keys in each map.
Terraform 0.11 does not have the setproduct function, but for sequences this small it's not a huge problem to just write out the cartesian product yourself as you did in the question here.
The result will certainly be very inelegant, but I expect it's possible to get something working on Terraform 0.11 if you apply the above ideas and make some compromises.
I need to create 6 subnets with below cidr value but it's order has been changed while creating it with terraform.
private_subnets = {
"10.1.80.0/27" = "x"
"10.1.80.32/27" = "x"
"10.1.80.64/28" = "y"
"10.1.80.80/28" = "y"
"10.1.80.96/27" = "z"
"10.1.80.128/27" = "z"
}
Terraform is creating with 10.1.80.0/27 , 10.1.80.128/27,10.1.80.32/27,10.1.80.64/28,10.1.80.80/28,10.1.80.96/27 order
Module of terraform:
resource "aws_subnet" "private" {
vpc_id = "${var.vpc_id}"
cidr_block = "${element(keys(var.private_subnets), count.index)}"
availability_zone = "${element(var.availability_zones, count.index)}"
count = "${length(var.private_subnets)}"
tags {
Name = "${lookup(var.private_subnets, element(keys(var.private_subnets), count.index))}
}
}
Updated Answer:
Thanks to the discussion in the comments, I revise my answer:
You are assuming an order within a dictionary. This is not intended behaviour. As from your example, one can see that terraform orders the keys alphabetically internally, i.e., you can "think" of your variable as
private_subnets = {
"10.1.80.0/27" = "x"
"10.1.80.128/27" = "z"
"10.1.80.32/27" = "x"
"10.1.80.64/28" = "y"
"10.1.80.80/28" = "y"
"10.1.80.96/27" = "z"
}
You are running into problems, because you are having mismatches with your other variable var.availability_zones where you assume the index to be sorted the same as for var.private_subnets.
Relying on the above ordering (alphabetically), is not a good solution, since it may change with any version of terraform (order of keys is not guaranteed).
Hence, I propose to use a list of maps:
private_subnets = [
{
"cidr" = "10.1.80.0/27"
"name" = "x"
"availability_zone" = 1
},
{
"cidr" = "10.1.80.32/27"
"name" = "x"
"availability_zone" = 2
},
…
]
I encoded the availability zone as index of your var.availability_zones list. However, you could also consider using the availability zone directly.
The adaption of your code is straightforward: Get (element(…)) the list element to get the map and then lookup(…) the desired key.
Old Answer (not applicable here):
Before Terraform creates any resources, it creates a graphstructure to represent all the objects it wants to track (create, update, delete) and the dependencies upon one another.
In your example, 6 different aws_subnet objects are created in the graph which do not depend on each other (there is no variable in one subnet dependent on another subnet).
When Terraform now tries to create the attributes, it does so concurrently in (potentially) multiple threads and creates resources potentially simultaniously, if they do not depend on each other.
This is why you might see very different orders of execution within multiple runs of terraform.
Note that this is a feature, since if you have many resources to be created that have no dependency on each other, they all are created simultaneously saving a lot of time with long-running creation operations.
A solution to your problem is to explicitly model the dependencies you are thinking of. Why should one subnet be created before the other? And if so, how can you make them dependent (e.g. via depends_on parameter)?
Answering this questions should bring you into the right direction to model your code according to your required layout.
Say that I need to provision a large number of vpc subnets in terraform. Each subnet has a cidr, a name and a availability zone. So in other config management tools I'd do something like:
[
{
"name":"subnet1",
"cidr":"10.0.0.1/24",
"az":"us-west-1a"
},
{
"name":"subnet2",
"cidr":"10.0.0.2/24",
"az":"us-west-1b"
}
]
And then iterate over that array.
Terraform doesn't have a notion of arrays/objects as far as I can see. So, for arrays of single attributes I would just use a list item:
subnets: ["10.0.0.1/24","10.0.0.2/24"]
But that doesn't allow me to name or place the subnets where I want.
I know that I can also use multiple lists in Terraform, something like:
subnet_names: ["subnet1", "subnet2"]
subnets: ["10.0.0.1/24","10.0.0.2/24"]
subnet_az: ["us-west-1a", "us-west-1b"]
But that strikes me as messy and counter-intuitive. The last option I see is to mash everything togehter into an ugly list of strings, and then split them apart in Terraform:
things: ["subnet1__10.0.0.1/24__us-west-1a","subnet2__10.0.0.2/24__us-west-2a"]
But thats just ugly.
How can I deal with array/object-type of repeats in Terraform? For now I've just explicitly defined all my things, which caused a simple vpc definition to be 300 lines long :-(
As you've seen, at present Terraform doesn't support lists of structured data like you're trying to create here.
Having multiple flat lists of strings as you showed in your question is one common solution to this problem. It works, but as you've seen it's somewhat counter-intuitive to keep track of which values belong together that way.
An alternative approach that is likely to produce a more readable and maintainable result is to factor your aws_subnet resource out into a module that takes care of the elements that are always the same for all subnets. Then you can instantiate the module once per subnet, providing only the values that vary:
module "subnet1" {
source = "./subnet"
name = "subnet1"
cidr = "10.0.0.1/24"
az = "us-west-1a"
}
module "subnet2" {
source = "./subnet"
name = "subnet2"
cidr = "10.0.0.2/24"
az = "us-west-1b"
}
In many cases there's some sort of systematic relationship between AZs and CIDR blocks. If that's true for you then you can also use your module to encode these numbering rules. For example, in your subnet module:
variable "region_network_numbers" {
default = {
"us-west-1" = 0
"us-east-1" = 1
"us-west-2" = 2
}
}
variable "az_network_numbers" {
default = {
a = 1
b = 2
}
}
variable "base_cidr_block" {
default = "10.0.0.0/8"
}
variable "az" {
}
data "aws_availability_zone" "selected" {
name = "${var.az}"
}
resource "aws_subnet" "main" {
cidr_block = "${cidrsubnet(cidrsubnet(var.base_cidr_block, 8, var.region_network_numbers[data.aws_availability_zone.selected.region]), 4, var.az_network_numbers[data.aws_availability_zone.selected.name_suffix])}"
# ...
}
With this it's sufficient to provide just the az argument to the module, with the cidr and name produced systematically from the AZ name. This is the same general idea as shown in the example for the aws_availability_zone data source, and there's a more complete, elaborate example of this in the Terraform repository itself.