The problem I am trying to solve is I need to identify one of the Azure subnets in a virtual network by part of it's name. This is so I can then later retrieve it's CIDR. I only know part beforehand such as "mgmt-1" or "egress-1". The actual name of the subnet is much longer but will end in something like that. This was my process:
I have the vnet name so I pull all subnets:
data "azurerm_virtual_network" "this" {
name = local.vnet
resource_group_name = "myrg"
}
Now what I wish I could do is this:
locals {
mgmt_index = index(data.azurerm_virtual_network.this.subnets, "*mgmt-1")
mgmt_subnet = data.azurerm_virtual_network.this.subnets[local.mgmt_index]
}
However index wants an exact match, not a regex. Is this possible to do? Perhaps a better way?
Thank you,
It is not possible to directly look up a list item using a regex match, but you can use for expressions to apply arbitrary filters to a collection when constructing a new collection:
locals {
mgmt_subnets = toset([
for s in data.azurerm_virtual_network.this.subnets : s
if can(regex(".*?mgmt-1", s.name))
])
}
In principle an expression like the above could match more than one object, so I wrote this to produce a set of objects that match.
If you expect that there will never be more than one object whose name matches the pattern then you can use Terraform's one function to assert that and then Terraform will check to confirm that there's no more than one element (returning an error if not) and then return that one value.
locals {
mgmt_subnet = one([
for s in data.azurerm_virtual_network.this.subnets : s
if can(regex(".*?mgmt-1", s.name))
])
}
If the condition doesn't match any of the subnet objects then in the first case you'll have an empty set and in the second case you'll have the value null.
Related
I'm trying to tell data.github_ip_ranges to what name to use so I could create a list of CIDRs and my code look cleaner. I was trying to find answers, but no luck so far.
And I'm trying to see if there is a way of passing my variables to it...
variable "git_services" {
default = ["hooks_ipv4", "dependabot_ipv4", "dependabot_ipv6", "git_ipv4", "hooks_ipv6"]
}
locals {
github_ips = concat(data.github_ip_ranges.git.name) # name is my custom variable
}
Here is my original approach
locals {
github_ips = concat(data.github_ip_ranges.git.hooks_ipv4, data.github_ip_ranges.git.hooks_ipv6,
data.github_ip_ranges.git.dependabot_ipv4, data.github_ip_ranges.git.dependabot_ipv6)
}
Please help if you could. Thank you!
I think I understand what you're trying to accomplish. You would do it like so:
variable "git_services" {
default = ["hooks_ipv4", "dependabot_ipv4", "dependabot_ipv6", "git_ipv4", "hooks_ipv6"]
}
locals {
github_ips = distinct(flatten([
for service in var.git_services:
data.github_ip_ranges.git[service]
]))
}
What this is doing is creating a list of lists, where each element in the first level is for a service, and each element in the second level is a CIDR bock for that service. The flatten function turns this list of lists into a flat list of CIDR blocks. Since the same CIDR might be used for multiple services, and we probably don't want duplicates if we're using this for something like security group rules, we use the distinct function to remove any duplicate CIDR blocks.
I am trying to create a map of secondary ranges for the GCP VPC module here and have the following defined in my locals:
secondary_ranges = {
for name, config in var.subnet_config : config.subnet_name => [
{
range_name = local.ip_range_pods
ip_cidr_range = "10.${index(keys(var.subnet_config), name)}.0.0/17"
},
{
range_name = local.ip_range_services
ip_cidr_range = "10.${index(keys(var.subnet_config), name)}.128.0/17"
}
]
}
subnet_config is defined as follows:
subnet_config = {
cluster1 = {
region = "us-east1"
subnet_name = "default"
},
cluster2 = {
region = "us-west1"
subnet_name = "default"
}
}
This creates the secondary subnets just fine if the subnet names are unique but fails with the error below if the subnet names (which end up being the key values) are not unique:
Two different items produced the key "default" in this 'for' expression. If duplicates are expected, use the ellipsis (...) after the value expression to enable grouping by key.
I'm trying to figure out if I can use grouping mode if the value is a list and if so, how?
Any help would be greatly appreciated.
If you use the grouping mode in this case then it would be to group the outermost for expression, which is producing a map, because that's the one whose keys you'd be grouping by.
We can start by adding the grouping mode modifier to that and see what happens:
secondary_ranges_pairs = {
for name, config in var.subnet_config : config.subnet_name => [
{
range_name = local.ip_range_pods
ip_cidr_range = "10.${index(keys(var.subnet_config), name)}.0.0/17"
},
{
range_name = local.ip_range_services
ip_cidr_range = "10.${index(keys(var.subnet_config), name)}.128.0/17"
}
]...
}
The effect of the expression above would be to create a map of lists of lists of objects, where the deepest lists are each pairs of objects because of how your inner for expression is written.
To turn that into the map of lists of objects which I think you're hoping for, you can then use flatten in a separate step:
secondary_ranges = {
for k, pairs in local.secondary_ranges_pairs : k => flatten(pairs)
}
flatten recursively walks a data structure where there are lists of lists and concatenates all of the nested lists together into a single flat list.
A word of caution: you seem to be using a lexical sort of the subnet_config keys in order to derive network numbering. That means that if you add new elements to your var.subnet_config whose keys sort earlier than any existing ones (for example, if you were to add in a cluster0 into what you showed in your question) then you'll implicitly renumber all of the subsequent networks, which is likely to cause a lot of churn recreating objects, and the change might not even be possible if those networks contain other objects.
I'd typically recommend instead being explicit about what number you've assigned to each network, by including then as part of the var.subnet_config objects. You can then clearly see which numbers you've assigned and make sure that any new networks will always be assigned a later number without disturbing any existing assignments.
There's also an official Terraform module hashicorp/subnets/cidr which aims to encapsulate subnet numbering calculations. The design of that module means that it wouldn't be completely straightforward to adopt it for your use-case (since you're allocating two levels of subnet at once) but it might be useful to study to see whether any of the design tradeoffs made there are relevant to your module.
I am creating a series of resources in terraform (in this case, dynamo DB table). I want to apply IAM policies to subgroups of them. E.g.
resource "aws_dynamodb_table" "foo" {
count = "${length(var.tables)}"
name = "foo-${element(var.tables,count.index)}"
tags {
Name = "foo-${element(var.tables,count.index)}"
Environment = "<unsure how to get this>"
Source = "<unsure how to get this>"
}
}
All of these share some common element, e.g. var.sources is a list composed of the Cartesian product of var.environments and var.sources:
environments = ["dev","qa","prod"]
sources = ["a","b","c"]
So:
tables = ["a:dev","a:qa","a:prod","b:dev","b:qa","b:prod","c:dev","c:qa","c:prod"]
I want to get the arns of the created dynamo tables that have, e.g. c (i.e. those with the name ["c:dev","c:qa","c:prod"]) or prod(i.e. those with the name ["a:prod","b:prod","c:prod"]).
Is there any sane way to do this with terraform 0.11 (or even 0.12 for that matter)?
I am looking to:
group the dynamo db table resources by some of the inputs (environment or source) so I can apply some policy to each group
Extract the input for each created one so I can apply the correct tags
I was thinking of, potentially, instead of creating the cross-product list, to create maps for each input:
{
"a": ["dev","qa","prod"],
"b": ["dev","qa","prod"],
"c": ["dev","qa","prod"]
}
or
{
"dev": ["a","b","c"],
"qa": ["a","b","c"],
"prod": ["a","b","c"]
}
It would make it easy to find the target names for each one, since I can look up by the input, but that only gives me the names, but not make it easy to get the actual resources (and hence the arns).
Thanks!
A Terraform 0.12 solution would be to derive the cartesian product automatically (using setproduct) and use a for expression to shape it into a form that's convenient for what you need. For example:
locals {
environments = ["dev", "qa", "prod"]
sources = ["a", "b", "c"]
tables = [for pair in setproduct(local.environments, local.sources) : {
environment = pair[0]
source = pair[1]
name = "${pair[1]}:${pair[0]}"
})
}
resource "aws_dynamodb_table" "foo" {
count = length(local.tables)
name = "foo-${local.tables[count.index].name}"
tags {
Name = "foo-${local.tables[count.index].name}"
Environment = local.tables[count.index].environment
Source = local.tables[count.index].source
}
}
At the time I write this the resource for_each feature is still in development, but in a near-future Terraform v0.12 minor release it should be possible to improve this further by making these table instances each be identified by their names, rather than by their positions in the local.tables list:
# (with the same "locals" block as in the above example)
resource "aws_dynamodb_table" "foo" {
for_each = { for t in local.tables : t.name => t }
name = "foo-${each.key}"
tags {
Name = "foo-${each.key}"
Environment = each.value.environment
Source = each.value.source
}
}
As well as cleaning up some redundancy in the syntax, this new for_each form will cause Terraform to identify this instances with addresses like aws_dynamodb_table.foo["a:dev"] instead of aws_dynamodb_table.foo[0], which means that you'll be able to freely add and remove members of the two initial lists without causing churn and replacement of other instances because the list indices changed.
This sort of thing would be much harder to achieve in Terraform 0.11. There are some general patterns that can help translate certain 0.12-only constructs to 0.11-compatible features, which might work here:
A for expression returning a sequence (one with square brackets around it, rather than braces) can be simulated with a data "null_data_source" block with count set, if the result would've been a map of string values only.
A Terraform 0.12 object in a named local value can in principle be replaced with a separate simple map of local value for each object attribute, using a common set of keys in each map.
Terraform 0.11 does not have the setproduct function, but for sequences this small it's not a huge problem to just write out the cartesian product yourself as you did in the question here.
The result will certainly be very inelegant, but I expect it's possible to get something working on Terraform 0.11 if you apply the above ideas and make some compromises.
I have a list variable "test" in variables.tf. I am trying to use this list variable inside my zone.tf .
I do not want to use list indexes here infact I want to run a loop to get all the values of the list from list variable dynamically. How can I accomplish this ? Any help is much appreciated.
I have tried to use count in test.tf inside resource resource "aws_route53_record" but it creates multiple record sets which I do not want as I just need a single record set with multiple records
resource "aws_route53_record" "test" {
zone_id = "${data.aws_route53_zone.dns.zone_id}"
name = "${lower(var.environment)}xyz"
type = "CAA"
ttl = 300
count = "${length(var.test)}"
records = [
"0 issue \"${element(var.test, count.index)}\"",
]
}
variables.tf :-
variable "test" {
type = "list"
default = ["godaddy.com", "Namecheap.org"]
}
zone.tf :-
resource "aws_route53_record" "test" {
zone_id = "${data.aws_route53_zone.dns.zone_id}"
name = "${lower(var.environment)}xyz"
type = "CAA"
ttl = 300
records = [
"0 issue \"${var.test[0]}\"",
"0 issue \"${var.test[1]}\"",
]
}
Expecting to get the one record set with two records.
Actual :- getting Two record sets with two records.
so if I am understanding correctly you want to associate two records with your zone, but right now when using count you are getting two zones with one record.
This is because by specifying county terraform will create the resource that has the count attribute on it equal to the number of count.
The issue is fundamentally that right now you have a list variable and are trying to pass it to where a list is expected by extracting each individual element of the list to put back element by element into the list attribute.
Rather than go through that additional work an easier solution would be to just add the additional parts of the string, the "0 issue" part to the definition of the variable and then just pass the whole list object in as shown below
variable "test" {
type = "list"
default = ["0 issue godaddy.com", "0 issue Namecheap.org"]
}
zone.tf :-
resource "aws_route53_record" "test" {
zone_id = "${data.aws_route53_zone.dns.zone_id}"
name = "${lower(var.environment)}xyz"
type = "CAA"
ttl = 300
records = ["${var.test}"]
}
This will then pass the list in for that attribute and terraform will take care of the marshaling and unmarshaling and handling of the list. I hope this answers your question.
Is there a better way than the following to handle a Terraform data resource aws_ami_ids returning an empty list?
Always want the module to return the latest AMI's ID if found.
If the list was empty I was getting a "list "data.aws_ami_ids.full_unencrypted_ami.ids" does not have any elements so cannot determine type." error, so this was the workaround.
data "aws_ami_ids" "full_unencrypted_ami" {
name_regex = "${var.ami_unencrypted_regex}"
owners = ["123456789","self"]
}
locals {
notfound = "${list("AMI Not Found")}"
unencrypted_ami = "${concat(data.aws_ami_ids.full_unencrypted_ami.ids,local.notfound)}"
}
output "full_ami_unencrypted_id" {
description = "Full Unencrypted AMI ID"
value = "${local.full_unencrypted_ami[0]}"
}
1) Use aws_ami_id instead of aws_ami_ids so that terraform apply fails if the AMI is gone, forcing you to update your Terraform solution.
OR
2) Create two aws_ami_ids data sources (the second being a fallback), concat the results and take the first item. But, as ydaetskcoR hinted at, why would you want this implicit (possibly undetected) fallback?