Trying to put something together to get passed a limitation of the tfe plugin.
I have 200+ workspaces that I manage with a variable in Terraform Cloud that I need to update. All workspaces that I need to update start with "dev-workspace" in this case.
I have a data block with the following:
data "tfe_workspace_ids" "all" {
names = ["*"]
organization = "myorganization"
}
I can't do a wildcard search for these workspaces due to a limitation of the module. This data block returns a map of strings that include all of my workspaces:
aa = {
"dev-workspace-1" = "ws-anonymized"
"dev-workspace-2" = "ws-ws-anonymized"
"dev-workspace-3" = "ws-ws-anonymized"
"test-workspace-1" = "ws-ws-anonymized"
"prod-workspace-1" = "ws-ws-anonymized"
}
My problem is that I need to take this map of strings and filter it down to just return the ones that have "dev-workspace" in the key. I've tried something like the following:
resource "tfe_variable" "dev-workspace" {
for_each = contains(data.tfe_workspace_ids.all.ids, "dev-workspace")
key = "access_key"
value = "XXXX"
category = "terraform"
workspace_id = each.value
sensitive = true
description = "AWS IAM secret access key."
}
But it doesn't look like you can use contains in this manner with for_each:
Error: Error in function call
on main.tf line 16, in resource "tfe_variable" "dev-workspace":
16: for_each = contains(data.tfe_workspace_ids.all.ids, "dev-workspace")
|----------------
| data.tfe_workspace_ids.all.ids is map of string with 284 elements
Call to function "contains" failed: argument must be list, tuple, or set.
I'm not really sure what to do here, but have tried this several ways and can't figure it out. Thanks for any help.
If you want to filter, your resource could be (you have to change var.aa to the value of data.tfe_workspace_ids which produces the input map):
variable "aa" {
default = {
"dev-workspace-1" = "ws-anonymized"
"dev-workspace-2" = "ws-ws-anonymized"
"dev-workspace-3" = "ws-ws-anonymized"
"test-workspace-1" = "ws-ws-anonymized"
"prod-workspace-1" = "ws-ws-anonymized"
}
}
resource "tfe_variable" "dev-workspace" {
for_each = {for k, v in var.aa:
k => v if length(regexall("dev-workspace", k)) > 0}
key = "access_key"
value = "XXXX"
category = "terraform"
workspace_id = each.value
sensitive = true
description = "AWS IAM secret access key."
}
Related
When using azurecaf to generate multiple names like in the following code, how do I use the results output?
resource "azurecaf_name" "names" {
name = var.appname
resource_type = "azurerm_resource_group"
resource_types = ["azurerm_mssql_database"]
prefixes = [var.environment]
suffixes = [var.resource_group_location_short]
random_length = 5
clean_input = false
}
results - The generated name for the Azure resources based in the resource_types list
How to use this? Also, can I somehow debug / print out what results looks like? (I don't know if it is an array, a key-value structure etc)
You can view the results in two common ways. It is applicable to all attributes of the resource.
[1] Exporting the attribute required as a terraform output.
When you add any attribute as an output in your code by default terraform will show you the values with terraform apply.
In your used case.
output "caf_name_result" {
value = azurecaf_name.names.result
}
output "caf_name_results" {
value = azurecaf_name.names.results
}
Apply the config with the above outputs definitions you will have the below output on your terminal.
Changes to Outputs:
+ caf_name_result = (known after apply)
+ caf_name_results = (known after apply)
azurecaf_name.names: Creating...
azurecaf_name.names: Creation complete after 0s [id=YXp1cmVybV9yZXNvdXJjZV9ncm91cAlkZXYtcmctc3RhY2tvdmVyZmxvdy15b2RncC13ZXUKYXp1cmVybV9tc3NxbF9kYXRhYmFzZQlkZXYtc3FsZGItc3RhY2tvdmVyZmxvdy15b2RncC13ZXU=]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Outputs:
caf_name_result = "dev-rg-stackoverflow-yodgp-weu"
caf_name_results = tomap({
"azurerm_mssql_database" = "dev-sqldb-stackoverflow-yodgp-weu"
"azurerm_resource_group" = "dev-rg-stackoverflow-yodgp-weu"
})
After the first successful terraform apply you can view them anytime when you want by using the terraform output command.
$ terraform output
caf_name_result = "dev-rg-stackoverflow-yodgp-weu"
caf_name_results = tomap({
"azurerm_mssql_database" = "dev-sqldb-stackoverflow-yodgp-weu"
"azurerm_resource_group" = "dev-rg-stackoverflow-yodgp-weu"
})
You can be very specific also to check only particular output value.
$ terraform output caf_name_results
tomap({
"azurerm_mssql_database" = "dev-sqldb-stackoverflow-yodgp-weu"
"azurerm_resource_group" = "dev-rg-stackoverflow-yodgp-weu"
})
[2] View your applied resources via Terraform State Commands
This is only available after the resources are applied and only in cases when terraform execution was done from the same machine where this command is running. (in simple the identity doing terraform execution satisfies all the authentication, authorization and network connectivity conditions. )
It is not recommended, just to share another option available when requiring a quick look on the resources applied.
$ terraform state list
azurecaf_name.names
$ terraform state show azurecaf_name.names
# azurecaf_name.names:
resource "azurecaf_name" "names" {
clean_input = false
id = "YXp1cmVybV9yZXNvdXJjZV9ncm91cAlkZXYtcmctc3RhY2tvdmVyZmxvdy15b2RncC13ZXUKYXp1cmVybV9tc3NxbF9kYXRhYmFzZQlkZXYtc3FsZGItc3RhY2tvdmVyZmxvdy15b2RncC13ZXU="
name = "stackoverflow"
passthrough = false
prefixes = [
"dev",
]
random_length = 5
random_seed = 1676730686950185
random_string = "yodgp"
resource_type = "azurerm_resource_group"
resource_types = [
"azurerm_mssql_database",
]
result = "dev-rg-stackoverflow-yodgp-weu"
results = {
"azurerm_mssql_database" = "dev-sqldb-stackoverflow-yodgp-weu"
"azurerm_resource_group" = "dev-rg-stackoverflow-yodgp-weu"
}
separator = "-"
suffixes = [
"weu",
]
use_slug = true
}
[1]
[2]
I need to put a specific description of a csv file to each group I created.
On my csv, I have 2 columns: 1 for the group name and another for the description.
locals {
Right_Groups = [for x in csvdecode(file("${path.module}/_RightGroups.csv")) : x.droits_groups]
}
I create each group with :
resource "azuread_group" "Terra-Aad-Group-Right" {
for_each = toset(local.Right_Groups)
display_name = lower(each.value)
security_enabled = true
description = each.value
}
With this, the description is equal to the group name.
If I set "each.value.description" to "description", it doesn't work.
The csv is like this :
droits_groups,description
con-inf-dev01,dev01
con-axw-rec01,rec01
Anyone have an idea ?
Thank you.
I would suggest creating a map instead of list which you are converting to a set. This is because of the fact that sets will have the same keys and values [1]:
each.value — The map value corresponding to this instance. (If a set was provided, this is the same as each.key).
To convert the data to a map you could try:
Right_Groups = {for x in csvdecode(file("${path.module}/_RightGroups.csv")) : x.droits_groups => x.description}
This will create the following map:
> local.Right_Groups
{
"con-axw-rec01" = "rec01"
"con-inf-dev01" = "dev01"
}
Then, in the resource you would do:
resource "azuread_group" "Terra-Aad-Group-Right" {
for_each = local.Right_Groups
display_name = lower(each.key)
security_enabled = true
description = each.value
}
[1] https://www.terraform.io/language/meta-arguments/for_each#the-each-object
I am struggling with a few terraform concepts.
I am successfully using the aztfmod/azurecaf provider to name my resourcegroup, but this means I need to get that name as an output for the companynet.resource_group module, so that I can use that name again when calling the companynet.key_vault module.
# terraform.tfvars
resource_groups = {
rg1 = {
name = "resourcegroup1"
location = "eastus"
}
rg2 = {
name = "resourcegroup2"
location = "eastus"
}
}
# root main.tf
provider "azurerm" {
features {}
}
module "companynet" {
source = "./modules/companynet"
tenant_id = var.tenant_id
environment = var.environment
resource_groups = var.resource_groups
key_vaults = var.key_vaults
storage_accounts = var.storage_accounts
app_service_plans = var.app_service_plans
}
# modules/companynet/main.tf
module "resource_group" {
source = "../companynet.resource_group"
environment = var.environment
resource_groups = var.resource_groups
}
module "key_vault" {
source = "../companynet.key_vault"
tenant_id = var.tenant_id
environment = var.environment
resource_groups = "${module.resource_group.resource_groups.companynet}"
key_vaults = var.key_vaults
}
The module resource_group has the following main.tf:
# modules/companynet.resource_group/main.tf
resource "azurecaf_name" "resource_group" {
for_each = var.resource_groups
name = each.value.name
resource_type = "azurerm_resource_group"
suffixes = ["${var.environment}", "001"]
}
resource "azurerm_resource_group" "resource_group" {
for_each = var.resource_groups
name = azurecaf_name.resource_group[each.key].result
location = each.value.location
}
but I don't know how to get the output of that resource_group name.
I have tried a few different things that do not work
# modules/companynet.resource_group/outputs.tf
output "resource_groups" {
value = azurerm_resource_group.resource_group[*].name
}
value = azurerm_resource_group.resource_group.name
value = azurerm_resource_group.resource_group.companynet.name
value = azurerm_resource_group.resource_group[companynet].name
Each of these results in one error or another, all indicating a problem with modules/companynet.resource_group/outputs.tf
Ideally I would get an object that I can then iterate through in another module. I expect to be able to call something like to get access to those resource group names in other modules such as:
# modules/companynet.key_vault/main.tf
resource "azurerm_key_vault" "key_vault" {
for_each = var.key_vaults
name = azurecaf_name.key_vault[each.key].result
location = var.resource_groups.location
resource_groups = "${module.resource_group.resource_groups.[companynet]}"
sku_name = "standard"
tenant_id = var.tenant_id
}
azurerm_resource_group.resource_group is declared with for_each, and so that expression refers to a map of objects where the keys match the keys of the for_each expression and the values are the corresponding declared resource instances.
In References to Resource Attributes there are various examples of referring to resource attributes in different situations, including the following about resources using for_each:
When a resource has the for_each argument set, the resource itself becomes a map of instance objects rather than a single object, and attributes of instances must be specified by key, or can be accessed using a for expression.
aws_instance.example["a"].id returns the id of the "a"-keyed resource.
[for value in aws_instance.example: value.id] returns a list of all of the ids of each of the instances.
That second item shows how to use a for expression to produce a list of the ids of aws_instance.example, but it doesn't show exactly how to produce a map and instead expects you to refer to the linked documentation about for expressions to learn about that:
The type of brackets around the for expression decide what type of result it produces.
The above example uses [ and ], which produces a tuple. If you use { and } instead, the result is an object and you must provide two result expressions that are separated by the => symbol:
{for s in var.list : s => upper(s)}
This expression produces an object whose attributes are the original elements from var.list and their corresponding values are the uppercase versions. For example, the resulting value might be as follows:
{
foo = "FOO"
bar = "BAR"
baz = "BAZ"
}
A for expression alone can only produce either an object value or a tuple value, but Terraform's automatic type conversion rules mean that you can typically use the results in locations where lists, maps, and sets are expected.
This section describes how to produce an object and then notes that you can use the result in a location where a map is expected. In practice it's often possible to use object-typed values and mapped-type values interchangeably in Terraform, because they both have in common that they have elements identified by string keys. The difference is that an object type can have a separate type for each of its attributes, whereas a map must have the same type for all attributes.
Given all of this information, we can produce an object value describing the names for each resource group like this:
output "resource_groups" {
value = { for k, g in azurerm_resource_group.resource_group : k => g.name }
}
For most purposes it doesn't really matter that this is an object-typed result rather than specifically a map, but since we know that .name is always a string we can infer that all of the attributes of this object have string-typed values, and so it would also be valid to explicitly convert to a map of strings using the tomap function (which is a "location where [...] maps [...] are expected", per the above documentation):
output "resource_groups" {
value = tomap({
for k, g in azurerm_resource_group.resource_group : k => g.name
})
}
I have the following variable
variable "instance_types" {
default = {
instances : [
{
count = 1
name = "control-plane"
ami = "ami-xxxxx"
instance_type = "t2.large"
iam_instance_profile = "xxx-user"
subnet_id = "subnet-xxxxx"
},
{
count = 3
name = "worker"
ami = "ami-xxxxx"
instance_type = "t2.large"
iam_instance_profile = "xxx-user"
subnet_id = "subnet-xxxxx"
}
]
}
}
With the following instance declaration (that I'm attempting to iterate)
resource "aws_instance" "k8s-node" {
# Problem here : How to turn an array of 2 objects into 4 (1 control_plane, 3 workers)
for_each = {for x in var.instance_types.instances: x.count => x}
ami = lookup(each.value, "ami")
instance_type = lookup(each.value, "instance_type")
iam_instance_profile = lookup(each.value, "iam_instance_profile")
subnet_id = lookup(each.value, "subnet_id")
tags = {
Name = lookup(each.value, "name")
Type = each.key
}
}
Goal: Get the aws_instance to iterate 4 times (1 control_plane + 3 workers) and populate the values the index of instance_types.
Problem : Cannot iterate the over the object array correctly with desired result. In a typical programming language this would be achieved in a double for loop.
This can be solved easier with a data type of map(object)) for your input variable. The transformed data structure appears like:
variable "instance_types" {
...
default = {
"control-plane" = {
count = 1
ami = "ami-xxxxx"
instance_type = "t2.large"
iam_instance_profile = "xxx-user"
subnet_id = "subnet-xxxxx"
},
"worker" = {
count = 3
ami = "ami-xxxxx"
instance_type = "t2.large"
iam_instance_profile = "xxx-user"
subnet_id = "subnet-xxxxx"
}
}
}
Note the name key in the object is subsumed into the map key for efficiency and cleanliness.
If the resources are split between the control plane and worker nodes, then we are finished and can immediately leverage this variable's value in a for_each meta-argument. However, combining the resources now requires a data transformation:
locals {
instance_types = flatten([ # need this for final structure type
for instance_key, instance in var.instance_types : [ # iterate over variable input objects
for type_count in range(1, instance.count + 1) : { # sub-iterate over objects by "count" value specified; use range function and begin at 1 for human readability
new_key = "${instance_key} ${type_count}" # for resource uniqueness
type = instance_key # for easier tag value later
ami = instance.ami # this and below retained from variable inputs
instance_type = instance.instance_type
iam_instance_profile = instance.iam_instance_profile
subnet_id = instance.subnet_id
}
]
])
}
Now we can iterate within the resource with the for_each meta-argument, and utilize the for expression to reconstruct the input for suitable usage within the resource.
resource "aws_instance" "k8s-node" {
# local.instance_types is a list of objects, and we need a map of objects with unique resource keys
for_each = { for instance_type in local.instance_types : instance_type.new_key => instance_type }
ami = each.value.ami
instance_type = each.value.instance_type
iam_instance_profile = each.value.iam_instance_profile
subnet_id = each.value.subnet_id
tags = {
Name = each.key
Type = each.value.type
}
}
This will give you the behavior you desire, and you can modify it for style preferences or different uses as the need arises.
Note the lookup functions are removed since they are only useful when default values are specified as a third argument, and that is not possible in object types within variable declarations except as an experimental feature in 0.14.
The absolute namespace for these resources' exported resource attributes would be:
(module?.<declared_module_name>?.)<resource_type>.<resource_name>[<resource_key>].<attribute>
For example, given an intra-module resource, first worker node, and private ip address exported attribute:
aws_instance.k8s-node["worker 1"].private_ip
Note you can also access all resources' exported attributes by terminating the namespace at <resource_name> (retaining the map of all resources instead of accessing a singular resource value). Then you could also use a for expression in an output declaration to create a custom aggregate output for all of the similar resources and their identical exported attribute(s).
{ for node_key, node in aws_instance.k8s-node : node_key => node.private_ip }
I'm trying to reference a value within a list from a map but can't seem to get terraform to recognize that its a string.
Below is my module that I'm working on along with the variable defined.
resource "aws_transfer_user" "aws_transfer_users" {
for_each = var.transfer_users_and_keys
server_id = aws_transfer_server.aws_transfer_service.id
user_name = each.key
role = aws_iam_role.aws_transfer_role.arn
home_directory = format("/%s/%s",var.transfer_users_and_keys[each.value[1]],var.transfer_users_and_keys[each.key])
tags = {
Name = each.key
Project = var.product_name
Terraform = true
}
}
variable "transfer_users_and_keys" {
type = map(list(string))
}
For some reason when I call the value from the list it gives me the following error:
on main.tf line 38, in resource "aws_transfer_user" "aws_transfer_users":
38: home_directory = format("/%s/%s",var.transfer_users_and_keys[each.value[1]],var.tran
sfer_users_and_keys[each.key])
|----------------
| each.value[1] is "bucket-dev-client"
| var.transfer_users_and_keys is map of list of string with 2 elements
The given key does not identify an element in this collection value.
Here is my variable that I'm creating:
transfer_users_and_keys = {
format("key-%s",local.environment) = ["value.pub",tostring(local.sftp_bucket[0])]
format("key-%s02",local.environment) = ["value02.pub",local.sftp_bucket]
}
sftp_bucket = [format("bucket-%s-client",local.environment)]
The goal here is to build out the home_directory based on the 2nd value in the "transfer_users_and_keys" variable (tostring(local.sftp_bucket[0])).
When using for_each, you don't need to keep referencing the variable and indexing it. Change:
home_directory = format("/%s/%s",var.transfer_users_and_keys[each.value[1]],var.transfer_users_and_keys[each.key])
to simply
home_directory = format("/%s/%s", each.value[1], each.key)