Extract Values from terraform set variable - terraform

I am trying to provision aws service catalog product using terraform resource
resource "aws_servicecatalog_provisioned_product" "example" {}
Terraform resource output description
one of the export value of the resource is outputs which is in form of set and i am collecting that into an output variable using below
output "Provisioned_Product_Outputs" {
value = aws_servicecatalog_provisioned_product.example.outputs
}
Output Looks Like
Provisioned_Product_Outputs = toset([
{
"description" = "Backup plan"
"key" = "BackupPlan"
"value" = "light"
},
{
"description" = "Current user zone to run"
"key" = "CurrentAZ"
"value" = "primary"
},
{
"description" = "InstanceID of Vm"
"key" = "EC2InstanceID"
"value" = "i-04*******"
},
{
"description" = "InstanceHostName"
"key" = "InstanceHostName"
"value" = "{\"fqdn\":\"foo.domain.com\"}"
},
{
"description" = "The ARN of the launched Cloudformation Stack"
"key" = "CloudformationStackARN"
"value" = "arn:aws:cloudformation:{region}:{AccountID}:stack/SC-{AccountID}-pp-iy******"
},
])
i would like to have only selected outputs values rather than entire set like below.
output "EC2InstanceID" {
value = "i-04*******"
}
output "InstanceHostName" {
value = ""{\"fqdn\":\"foo.domain.com\"}""
}
output "CloudformationStackARN" {
value = "arn:aws:cloudformation:{region}:{AccountID}:stack/SC-{AccountID}-pp-iy******"
}
Is there a way to apply or have some condition which allows me to check for the right values using key value pair and apply the value in the outputs
regards

Since you know that your output is set, you can create a filter on the objects inside the set using contains:
output "outputs" {
value = {
for output in aws_servicecatalog_provisioned_product.example.outputs : output.key =>
output.value if contains(["EC2InstanceID", "InstanceHostName", "CloudformationStackARN"], output.key)
}
}
The output will be similar to this:
outputs = {
"CloudformationStackARN" = "arn:aws:cloudformation:{region}:{AccountID}:stack/SC-{AccountID}-pp-iy******"
"EC2InstanceID" = "i-04*******"
"InstanceHostName" = "{\"fqdn\":\"foo.domain.com\"}"
}
If you want to have separate outputs, you have to type out each output manually:
output "EC2InstanceID" {
value = [for output in aws_servicecatalog_provisioned_product.example.outputs : output.value if output.key == "EC2InstanceID"][0]
}
output "InstanceHostName" {
value = [for output in aws_servicecatalog_provisioned_product.example.outputs : output.value if output.key == "InstanceHostName"][0]
}
output "CloudformationStackARN" {
value = [for output in aws_servicecatalog_provisioned_product.example.outputs : output.value if output.key == "CloudformationStackARN"][0]
}
You can not have a for_each attribute for outputs. Currently resource and module blocks support for_each attributes.

Related

terraform - use count and for_each together

I'm creating multiple Google Cloud Projects using a module which creates a custom service account for each one and provides that as an output eg
module "app_projects" {
count = var.number_application_projects
etc
}
I then have a list of IAM roles I want to assign to each Project Service account like this:
sa_roles = ["roles/run.developer","roles/appengine.deployer"]
resource "google_project_iam_member" "proj_sa_iam_roles" {
count = var.number_application_projects
for_each = toset(var.sa_roles)
project = module.app_projects[count.index].project_id
role = each.value
member = "serviceAccount:${module.app_projects[count.index].service_account_email}"
}
This runs into the error about "count" and "for_each" being mutually-exclusive, and I can't for the life of me figure out what to use instead, any help would be appreciated!
You can probably use setproduct, but the standard method is using flatten.
They even have a section tailored for similar use cases. Flattening nested structures for for_each.
Here is an example, that doesn't use your exact resource, but should be instructive and you can actually run it to test it out.
modules/app-project/variables.tf
variable "name" {}
modules/app-project/outputs.tf
output "name" {
value = var.name
}
modules/member/variables.tf
variable "project" {}
variable "role" {}
modules/member/outputs.tf
output "project_role" {
value = "${var.project}-${var.role}"
}
main.tf
locals {
roles = ["rad", "tubular"]
}
module "app_project" {
source = "./modules/app-project"
count = 2
name = "app-project-${count.index}"
}
module "project_role" {
source = "./modules/member"
for_each = { for pr in flatten([for p in module.app_project[*] : [
for r in local.roles : {
app_project_name = p.name
role = r
}]
]) : "${pr.app_project_name}-${pr.role}" => pr }
project = each.value.app_project_name
role = each.value.role
}
output "project_roles" {
value = values(module.project_role)[*].project_role
}
terraform plan output
Changes to Outputs:
+ project_roles = [
+ "app-project-0-rad",
+ "app-project-0-tubular",
+ "app-project-1-rad",
+ "app-project-1-tubular",
]
In your case specifically, I think something like this would work:
resource "google_project_iam_member" "proj_sa_iam_roles" {
for_each = { for i, pr in flatten([for p in module.app_project[*] : [
for r in var.sa_roles : {
app_project = p
role = r
}]
]) : "${i}-${pr.role}" => pr }
project = each.value.app_project.project_id
role = each.value.role
member = "serviceAccount:${each.value.app_project.service_account_email}"
}

Dynamically retrieve key's value from a terraform yaml file

I am trying to get a value from a key in a yaml file after decoding it in locals:
document.yaml
name: RandomName
emailContact: email#domain.com
tags:
- key: "BusinessUnit"
value: "BUnit"
- key: "Criticality"
value: "Criticality-Eng"
- key: "OpsCommitment"
value: "OpsCommitment-Eng"
- key: "OpsTeam"
value: "OpsTeam-Eng"
- key: "BudgetAmount"
value: "100"
Then I have locals in main.tf:
locals {
file = yamldecode(file(document.yaml))
}
And a have a budget.tf file where I need to retrieve the BudgetAmount of 100 dollars based on the tag key: BudgetAmount
resource "azurerm_consumption_budget_subscription" "budget" {
name = format("%s_%s", lower(var.subscription_name), "budget")
subscription_id = data.azurerm_subscription.current.id
amount = local.landing_zone.tags[5].value
time_grain = "Monthly"
time_period {
start_date = formatdate("YYYY-MM-01'T'00:00:00'Z'", timestamp())
end_date = local.future_10_years
}
notification {
enabled = true
threshold = 80.0
operator = "EqualTo"
contact_emails = [
]
contact_roles = [
"Owner"
]
}
}
This local.landing_zone.tags[5].value works, but it's not a good idea if I have multiple yaml files and the position changes
Q: how do I get the BudgetAmount value of 100 from the yaml file without specifying its location inside the file, but referring to the tag's name?
I did try this:
matchkeys([local.file .tags[*].key], [local.file .tags[*].value], ["BudgetAmount"])
but it keeps telling me the value needs to be a number (obviously is getting a value, but it's a text, from one of the many key/value pairs I have in the yaml file)
I managed to get the budget by converting the list of maps into a single map with each tag being a key value.
The way you were doing it would result in the following data structure under local.file.tags:
[
{
"key" = "BusinessUnit"
"value" = "BUnit"
},
{
"key" = "Criticality"
"value" = "Criticality-Eng"
},
{
"key" = "OpsCommitment"
"value" = "OpsCommitment-Eng"
},
{
"key" = "OpsTeam"
"value" = "OpsTeam-Eng"
},
{
"key" = "BudgetAmount"
"value" = "100"
},
]
That was hard to work with and I couldn't think of any functions to help at the time so I went with changing it via the following locals:
locals {
file = yamldecode(file("document.yaml"))
tags = {
for tag in local.file.tags :
tag.key => tag.value
}
}
which got the tags to a structure of:
> local.tags
{
"BudgetAmount" = "100"
"BusinessUnit" = "BUnit"
"Criticality" = "Criticality-Eng"
"OpsCommitment" = "OpsCommitment-Eng"
"OpsTeam" = "OpsTeam-Eng"
}
You can reference each of the tags in this state by using something like:
budget = local.tags["BudgetAmount"]
This was tested on Terraform v1.0.10 via terraform console

Terraform issues when using for_each local variable created based on another local

I am trying to create azure keyvault secrets using locals which reference data resources. I am iterating over an array containing my environments and creating a list of maps where
each item is the set of secrets for a given environment.
Using another local, I then proceed to merge these maps into a single one by creating two lists, one with keys and another with values and then zipping them.
I finally use for_each on the second local to create the resource.
If I run my root module without creating the actual secret resources ("azurerm_key_vault_secret) and a second time with it, it all works fine.
If I try to do it all in one go, as I want to implement on my CI/CD I get the error message:
|Error: Invalid for_each argument
|on variables.tf line 239, in resource “azurerm_key_vault_secret” “example”:
│239: for_each = nonsensitive(local.example_map)
│ local.example_map will be known only after apply
|The “for_each” value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work around this, use the -target argument to first apply only the resources that the for_each depends on.
If anybody has any idea how i could make work. It seems to me that this data transformation within locals doesn't quite work.
Maybe I am going about the whole thing wrong. Any pointers would be highly appreciated.
Here is the code I am trying to make work:
variable "environment" {
default = [ "dev", "prod"]
}
locals {
example = distinct(flatten([
for namespace in var.environment : {
"${environment}-password1" = "${environment}-password",
"${environment}-password2" = "{\"connection_string\" : \"${data.azurerm_storage_account.storage_account_example["${environment}"].primary_connection_string}\"}",
"${environment}-password3" = "{\"client_id\" : \"${jsondecode("${data.azurerm_key_vault_secret.other_credentials["${environment}"].value}").clients["example"].client_id}\"}",
"${environment}-password4" = "{\"password\" : \"${data.azurerm_key_vault_secret.k_password.value}\"}",
"${environment}-password5" = "{\"azurestorageaccountname\" : \"${data.azurerm_storage_account.example.name}\", \"azurestorageaccountkey\" : \"${data.azurerm_storage_account.example.primary_access_key}\"}",
"${environment}-password6" = "{\"connection_string\" : \"${module.some_module.connection_string}\"}",
}]))
example_map = zipmap(
flatten(
[for item in local.example : keys(item)]
),
flatten(
[for item in local.example : values(item)]
)
)
}
resource "azurerm_key_vault_secret" "example" {
for_each = nonsensitive(local.example_map)
name = each.key
value = each.value
key_vault_id = module.keyvault.id
content_type = "password"
}
Here is the data structures created by local.example and local.example_map
"example": {
"value": [
{
"dev-password1" = "dev-password",
"dev-password2" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=mystorage;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}",
"dev-password3" = "{\"client_id\" : \"myclientID\"}",
"dev-password4" = "{\"password\" : \"password123\"}",
"dev-password5" = "{\"azurestorageaccountname\" : \"somestorageaccount\", \"azurestorageaccountkey\" : \"XXXxxxxXXXXxxxxXXXxxxxxxe++++++NNNNNNNNNCCCccccccccccccccccc==}\"}",
"dev-password6" = "{\"connection_string\" : \"${module.some_module.connection_string}\"}"
},
{
"prod-password1" = "prod-password",
"prod-password2" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=mystorage;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}",
"prod-password3" = "{\"client_id\" : \"myclientID\"}",
"prod-password4" = "{\"password\" : \"password123\"}",
"prod-password5" = "{\"azurestorageaccountname\" : \"somestorageaccount\", \"azurestorageaccountkey\" : \"XXXxxxxXXXXxxxxXXXxxxxxxe++++++NNNNNNNNNCCCccccccccccccccccc==}\"}",
"prod-password6" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=yetanotherone;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}"
}
]
}
"example_map": {
"value": {
"dev-password1" = "dev-password",
"dev-password2" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=mystorage;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}",
"dev-password3" = "{\"client_id\" : \"myclientID\"}",
"dev-password4" = "{\"password\" : \"password123\"}",
"dev-password5" = "{\"azurestorageaccountname\" : \"somestorageaccount\", \"azurestorageaccountkey\" : \"XXXxxxxXXXXxxxxXXXxxxxxxe++++++NNNNNNNNNCCCccccccccccccccccc==}\"}",
"dev-password6" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=yetanotherone;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}"
"prod-password1" = "prod-password",
"prod-password2" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=mystorage;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}",
"prod-password3" = "{\"client_id\" : \"myclientID\"}",
"prod-password4" = "{\"password\" : \"password123\"}",
"prod-password5" = "{\"azurestorageaccountname\" : \"somestorageaccount\", \"azurestorageaccountkey\" : \"XXXxxxxXXXXxxxxXXXxxxxxxe++++++NNNNNNNNNCCCccccccccccccccccc==}\"}",
"prod-password6" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=yetanotherone;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}"
},
"type": [
"object",
{
"dev-password1": "string",
"dev-password2": "string",
"dev-password3": "string",
"dev-password4": "string",
"dev-password5": "string",
"dev-password6": "string",
"prod-password1": "string",
"prod-password2": "string",
"prod-password3": "string",
"prod-password4": "string",
"prod-password5": "string",
"prod-password6": "string",
}
]
}
Also what confuses me the most is that if I work with the following data structure, which is hard coding instead of doing the first transformation based on namespaces. The entry getting information from another module doesn't cause any problems and it all works wonderfully.
locals {
hardcoding_namespaces = {
"dev-password1" = "dev-password"
"dev-password2" = "{\"connection_string\" : \"${data.azurerm_storage_account.storage_account_example["dev"].primary_connection_string}\"}"
"dev-password3" = "{\"connection_string\" : \"${module.some_module.connection_string}\"}"
"prod-password1" = "prod-password"
"prod-password2" = "{\"connection_string\" : \"${data.azurerm_storage_account.storage_account_example["prod"].primary_connection_string}\"}"
"prod-password3" = "{\"connection_string\" : \"${module.some_module.connection_string}\"}"
}
}
resource "azurerm_key_vault_secret" "another_example" {
for_each = local.hardcoding_namespaces
name = each.key
value = each.value
key_vault_id = module.keyvault.id
content_type = "password"
}
if the resulting data structure is the same, why for_each works for one and not for the other?
[1]: https://i.stack.imgur.com/cTq5f.png
from the doc
Sensitive values, such as sensitive input variables, sensitive outputs, or sensitive resource attributes, cannot be used as arguments to for_each. The value used in for_each is used to identify the resource instance and will always be disclosed in UI output, which is why sensitive values are not allowed. Attempts to use sensitive values as for_each arguments will result in an error.(visite
https://www.terraform.io/language/meta-arguments/for_each#limitations-on-values-used-in-for_each)
Keys () will always return a sensitive value if the input is sensitive, so instead try the following :
example_map = zipmap(
flatten(
[for item,value in local.example : item]
),
flatten(
[for item, value in local.example : value]
)
)
This is probably because of module.some_module.connection_string. You can't use dynamic values in for_each. As the error message says, you have to use target to first create those dynamic resources, and then your for_each will work.

How to pass json inside the terraform variable?

I am trying to create an AWS parameter store via terraform that can also pass default values with the JSON format. Here is a sample of the code.
resource "aws_ssm_parameter" "secret" {
name = "/something/env"
description = "This is a something values"
type = "SecureString"
value = "test"
tags = {
environment = "production"
}
}
Instead of passing out a single value as a "test" from value, how can I pass the json one inside value one.
So that AWS parameter store value will be like
{
"key": "value"
}
I think you are looking for jsonencode, which could be used as such:
resource "aws_ssm_parameter" "secret" {
name = "/something/env"
description = "This is a something values"
type = "SecureString"
value = jsonencode({
"key" : "value"
})
tags = {
environment = "production"
}
}

How do I output an attribute of multiple instances of a resource created with for_each?

Let's say I have a map of environments to supply to for_each
environments = {
"0" = "dev"
"1" = "test"
"2" = "stage"
}
Then for whatever reason I want to create an Azure Resource Group for each environment.
resource "azurerm_resource_group" "resource_group" {
for_each = var.environments
name = "${var.resource-group-name}-${each.value}-rg"
location = var.location
}
How do I get the outputs? I've tried the new splat to no avail.
output "name" {
value = "${azurerm_resource_group.resource_group[*].name}"
}
output "id" {
value = "${azurerm_resource_group.resource_group[*].id}"
}
output "location" {
value = "${azurerm_resource_group.resource_group[*].location}"
}
Error: Unsupported attribute
in output "id":
6: value = "${azurerm_resource_group.resource_group[*].id}"
This object does not have an attribute named "id".
How do I output an attribute of multiple instances of a resource created with for_each?
The [*] is a shorthand for extracting attributes from a list of objects. for_each makes a resource appear as a map of objects instead, so the [*] operator is not appropriate.
However, for expressions are a more general mechanism that can turn either a list or a map into another list or map by evaluating an arbitrary expression for each element of the source collection.
Therefore we can simplify a map of azurerm_resource_group objects into a map of names of those objects like this:
output "name" {
value = { for k, group in azurerm_resource_group.resource_group: k => group.name }
}
Your input map uses numeric indexes as keys, which is unusual but allowed. Because of that, the resulting value for the output would be something like this:
{
"0" = "something-dev-rg"
"1" = "something-test-rg"
"2" = "something-stage-rg"
}
It's more common for a map in for_each to include a meaningful name as the key, so that the resulting instances are identified by that meaningful name rather than by incrementing integers. If you changed your configuration to use the environment name as the key instead, the map of names would look like this instead:
{
"dev" = "something-dev-rg"
"test" = "something-test-rg"
"stage" = "something-stage-rg"
}
EDIT: for_each doesn't work with output
output "name"{
value = { for k, v in var.environments : v => azurerm_resource_group.resource_group[k].name }
}
output "id"{
value = { for k, v in var.environments : v => azurerm_resource_group.resource_group[k].id }
}
output "location"{
value = { for k, v in var.environments : v => azurerm_resource_group.resource_group[k].location }
}
Example output,
id = {
"dev" = "xxx"
"stage" = "yyy"
"test" = "zzz"
}

Resources