how to access keys and values from list of map dictionary using Terraform for loop - terraform

I tried this so much but could not figure out how to do this. I have a following YAML file below.
---
project_team_mapping:
- Terraform-Project-1: ## this is project name
- team_name: example-team-1
roles:
- GROUP_READ_ONLY
- GROUP_DATA_ACCESS_READ_WRITE
- team_name: example-team-2
roles:
- GROUP_READ_ONLY
- GROUP_DATA_ACCESS_READ_WRITE
- Terraform-Project-2: ## this is 2nd project name
- team_name: example-team-3
roles:
- GROUP_READ_ONLY
- GROUP_DATA_ACCESS_READ_WRITE
- team_name: example-team-4
roles:
- GROUP_READ_ONLY
- GROUP_DATA_ACCESS_READ_WRITE
I'm using some TF module to decode the YAML files and here is the module and the YAML decoded output JSON.
module "yaml_json_multidecoder" {
source = "levmel/yaml_json/multidecoder"
version = "0.2.1"
filepaths = ["./examples/*.yaml"]
}
output "project_team_mapping" {
value = module.yaml_json_multidecoder.files.project.project_team_mapping
}
below is the output when using above decoder module.
project_team_mapping = [
{
"Terraform-Project-1" = [
{
"roles" = [
"GROUP_READ_ONLY",
"GROUP_DATA_ACCESS_READ_WRITE",
]
"team_name" = "dx-example-team-3"
},
{
"roles" = [
"GROUP_READ_ONLY",
"GROUP_DATA_ACCESS_READ_WRITE",
]
"team_name" = "dx-example-team-4"
},
]
}
]
i would like to create resources using Terraform for/for_each function and dynamic functions for a particular resource block below, i'm strugging not able to understand the outcome.
Below is the resource block going to use
resource "mongodbatlas_project" "test" {
name = "project-name"
org_id = "<ORG_ID>"
project_owner_id = "<OWNER_ACCOUNT_ID>"
# want to use dynamic function here to create multiple teams
teams {
team_id = "5e0fa8c99ccf641c722fe645"
role_names = ["GROUP_OWNER"]
}
}
can someone help please?
resource "mongodbatlas_project" "test" {
name = "project-name"
org_id = "<ORG_ID>"
project_owner_id = "<OWNER_ACCOUNT_ID>"
# want to use dynamic function here to create multiple teams
teams {
team_id = "5e0fa8c99ccf641c722fe645"
role_names = ["GROUP_OWNER"]
}
}
I might use wrong YAML structuring, please suggest good structure to fit my needs. i can change the YAML.
above question i'm using team_name in YAML but the resource block takes team_id. i'm planning to use data function to get ID but if you happen to have a easy solution please suggest

Many unknowns you have left. But a best guess would be the example below.
Assuming you have decoded yaml into a local.decodedyaml. Also you possibly would need team_id instead of team_name, maybe you can load them via data source, or will have to retrieve and add to yaml manually.
resource "mongodbatlas_project" "test" {
for_each = local.decodedyaml.project_team_mapping
name = each.key
org_id = "<ORG_ID>"
project_owner_id = "<OWNER_ACCOUNT_ID>"
dynamic "teams" {
for_each = {
for team in each.value: team.team_name => team.roles
}
content {
team_id = each.key
role_names = each.value
}
}
}
This will work if you change the data design to "unwrap" the initial lists:
Instead of
project_team_mapping:
- Terraform-Project Name-1: ## this is project name
- team_name: example-team-1
roles:
- GROUP_READ_ONLY
- GROUP_DATA_ACCESS_READ_WRITE
...
- Terraform-Project Name-2: ## this is 2nd project name
- team_name: example-team-3
roles:
- GROUP_READ_ONLY
- GROUP_DATA_ACCESS_READ_WRITE
...
Use
project_team_mapping:
Terraform-Project Name-1: ## this is project name
- team_name: example-team-1
roles:
- GROUP_READ_ONLY
- GROUP_DATA_ACCESS_READ_WRITE
...
Terraform-Project Name-2: ## this is 2nd project name
- team_name: example-team-3
roles:
- GROUP_READ_ONLY
- GROUP_DATA_ACCESS_READ_WRITE
...
This eases the data structure and greatly eases the hcl parsing.
Lastly, I am not sure yaml keys with spaces are a risk-free endevour.

Related

how to get ONLY values form all KEYS members and form a list - Terraform

below is a teams.yaml file in the Terraform directory.
---
TEAMS:
- name: dx-example-team2
roles:
- GROUP_DATA_ACCESS_READ_WRITE:
members:
- ab#ad.com
- cd#ad.com
- GROUP_OWNER:
members:
- pr#ad.com
Hello all,
can someone help me with gettting only values from nested members KEY from above YAML file and how to form a list merge of all the values.
TRIED Terraform Code , but below is just sample i have:
locals {
teams_file = yamldecode(file("${path.cwd}/teams.yaml"))["TEAMS"]
all_members = flatten([for team in local.teams_file : [
for role in team.roles : {
"name" = role
}
]
])
}
output "sample" {
value = local.all_members
}
EXPECTED OUTPUT:
all_members = [ "ab#ad.com" , "cd#ad.com", "pr#ad.com"]
In your case you can just use spat expression:
all_members = flatten(local.teams_file[*].roles[*].members)

yamldecode in terraform for multiple list

I am using the following yaml to get tag information for my outputs.tf.
yaml file:
---
common_tags:
TAG:type: lab
TAG:bu: crm
account_list:
friendly_name:
TAG:name: salesforce
TAG:aws_regions: eu-central-1
workload_accounts:
TAG:account_name: some-sf-acc
TAG:acc-no: 1234567890
TAG:environment: development
resource_tags:
- vpc_name: mgmt
TAG:name: mgmt-vpc
TAG:owner: AutoScaling
TAG:role: pipeline
TAG:product: sf1
TAG:app-id: app-id
- vpc_name: app
TAG:name: conftest
TAG:owner: devops-cloud#test.com
TAG:role: development
TAG:product: sf2
TAG:app-id: vcob
- vpc_name: app
TAG:name: app-vpc
TAG:owner: devops-cloud#test.com
TAG:role: automation
TAG:product: sf3
TAG:app-id: ser2
- vpc_name: app
TAG:name: conftest
TAG:owner: devops-cloud#test.com
TAG:role: build
TAG:product: sf4
TAG:app-id: entdb
My outputs.tf is as below:
output "product" {
value = [yamldecode(file("resource/rnd/amc.yaml"))["account_list"]["resource_tags"][0]["TAG:product"]
}
output "bu" {
value = yamldecode(file("resource/rnd/amc.yaml"))["common_tags"]["TAG:bu"]
}
output "acc-no" {
value = yamldecode(file("resource/rnd/amc.yaml"))["account_list"]["workload_accounts"]["TAG:acc-no"]
}
output "role" {
value = yamldecode(file("resource/rnd/amc.yaml"))["account_list"]["resource_tags"][0]["TAG:role"]
}
output "environment" {
value = yamldecode(file("resource/rnd/amc.yaml"))["account_list"]["workload_accounts"]["TAG:environment"]
}
output "type" {
value = yamldecode(file("resource/rnd/amc.yaml"))["common_tags"]["TAG:type"]
}
The plan gives this output:
Plan: 1 to add, 0 to change, 0 to destroy.
Changes to Outputs:
acc-no = 1234567890
bu = "crm"
environment = "development"
product = "sf1"
role = "pipeline"
type = "lab"
I want to display all the products and roles in my output as below.
Expected output:
Changes to Outputs:
acc-no = 1234567890
bu = "crm"
environment = "development"
product = "sf1","sf2","sf3"
role = "pipeline","development","automation","build"
type = "lab"
Please tell me how can I do it using yamldecode.
You have to use splat (i.e. *):
output "product" {
value = yamldecode(file("resource/rnd/amc.yaml"))["account_list"]["resource_tags"][*]["TAG:product"]
}
output "role" {
value = yamldecode(file("resource/rnd/amc.yaml"))["account_list"]["resource_tags"][*]["TAG:role"]
}
With my multidecoder for YAML and JSON you are able to access multiple YAML and/or JSON files with their relative paths in one step.
Documentations can be found here:
Terraform Registry -
https://registry.terraform.io/modules/levmel/yaml_json/multidecoder/latest?tab=inputs
GitHub:
https://github.com/levmel/terraform-multidecoder-yaml_json
Usage
Place this module in the location where you need to access multiple different YAML and/or JSON files (different paths possible) and pass
your path/-s in the parameter filepaths which takes a set of strings of the relative paths of YAML and/or JSON files as an argument. You can change the module name if you want!
module "yaml_json_decoder" {
source = "levmel/yaml_json/multidecoder"
version = "0.2.1"
filepaths = ["routes/nsg_rules.yml", "failover/cosmosdb.json", "network/private_endpoints/*.yaml", "network/private_links/config_file.yml", "network/private_endpoints/*.yml", "pipeline/config/*.json"]
}
Patterns to access YAML and/or JSON files from relative paths:
To be able to access all YAML and/or JSON files in a folder entern your path as follows "folder/rest_of_folders/*.yaml", "folder/rest_of_folders/*.yml" or "folder/rest_of_folders/*.json".
To be able to access a specific YAML and/or a JSON file in a folder structure use this "folder/rest_of_folders/name_of_yaml.yaml", "folder/rest_of_folders/name_of_yaml.yml" or "folder/rest_of_folders/name_of_yaml.json"
If you like to select all YAML and/or JSON files within a folder, then you should use "*.yml", "*.yaml", "*.json" format notation. (see above in the USAGE section)
YAML delimiter support is available from version 0.1.0!
WARNING: Only the relative path must be specified. The path.root (it is included in the module by default) should not be passed, but everything after it.
Access YAML and JSON entries
Now you can access all entries within all the YAML and/or JSON files you've selected like that: "module.yaml_json_decoder.files.[name of your YAML or JSON file].entry". If the name of your YAML or JSON file is "name_of_your_config_file" then access it as follows "module.yaml_json_decoder.files.name_of_your_config_file.entry".
Example of multi YAML and JSON file accesses from different paths (directories)
first YAML file:
routes/nsg_rules.yml
rdp:
name: rdp
priority: 80
direction: Inbound
access: Allow
protocol: Tcp
source_port_range: "*"
destination_port_range: 3399
source_address_prefix: VirtualNetwork
destination_address_prefix: "*"
---
ssh:
name: ssh
priority: 70
direction: Inbound
access: Allow
protocol: Tcp
source_port_range: "*"
destination_port_range: 24
source_address_prefix: VirtualNetwork
destination_address_prefix: "*"
second YAML file:
services/logging/monitoring.yml
application_insights:
application_type: other
retention_in_days: 30
daily_data_cap_in_gb: 20
daily_data_cap_notifications_disabled: true
logs:
# Optional fields
- "AppMetrics"
- "AppAvailabilityResults"
- "AppEvents"
- "AppDependencies"
- "AppBrowserTimings"
- "AppExceptions"
- "AppExceptions"
- "AppPerformanceCounters"
- "AppRequests"
- "AppSystemEvents"
- "AppTraces"
first JSON file:
test/config/json_history.json
{
"glossary": {
"title": "example glossary",
"GlossDiv": {
"title": "S",
"GlossList": {
"GlossEntry": {
"ID": "SGML",
"SortAs": "SGML",
"GlossTerm": "Standard Generalized Markup Language",
"Acronym": "SGML",
"Abbrev": "ISO 8879:1986",
"GlossDef": {
"para": "A meta-markup language, used to create markup languages such as DocBook.",
"GlossSeeAlso": ["GML", "XML"]
},
"GlossSee": "markup"
}
}
}
}
}
main.tf
module "yaml_json_multidecoder" {
source = "levmel/yaml_json/multidecoder"
version = "0.2.1"
filepaths = ["routes/nsg_rules.yml", "services/logging/monitoring.yml", test/config/*.json]
}
output "nsg_rules_entry" {
value = module.yaml_json_multidecoder.files.nsg_rules.aks.ssh.source_address_prefix
}
output "application_insights_entry" {
value = module.yaml_json_multidecoder.files.monitoring.application_insights.daily_data_cap_in_gb
}
output "json_history" {
value = module.yaml_json_multidecoder.files.json_history.glossary.title
}
Changes to Outputs:
nsg_rules_entry = "VirtualNetwork"
application_insights_entry = 20
json_history = "example glossary"

How to use tags from yaml file - terraform

I am trying to extract certain tags from a YAML file with Terraform, but i just don't know how.
Yaml file looks like this:
---
name: subscriptionName
emailContact: someemail#domain.com
tags:
- key: "tagKey1"
value: "tagValue1"
- key: "tagKey2"
value: "tagValue2"
- key: "tagKey3"
value: "tagValue3"
- key: "tagKey4"
value: "tagValue4"
What i am interested in is getting 2 (let's say key1 and key3) key-value pairs as tags and tag resouces. I know that 'locals' plays a role here, but i am kinda new to terraform and cannot get any resource tagged.
The resources are Azure (if it matters).
The resource i am trying to tag is:
resource "azurerm_network_security_group" "nsg" {
name = "randomname"
location = "westeurope"
resource_group_name = "random_rg"
tags { }
}
If you really want two random tags, you can use random_shuffle:
locals {
loaded_yaml = yamldecode(file("./your_file.yaml"))
}
resource "random_shuffle" "indices" {
input = range(0, length(local.loaded_yaml.tags))
result_count = 2
seed = timestamp()
}
output "random_tags" {
value = [for idx in random_shuffle.indices.result:
local.loaded_yaml.tags[idx]]
}
update
For example:
tags = {
(local.loaded_yaml.tags[0].key) = local.loaded_yaml.tags[0].value
(local.loaded_yaml.tags[3].key) = local.loaded_yaml.tags[3].value
}

Terraform Combine tuple elements as list

Can someone suggest how to combine terraform tuple with elements to list as shown below.
Scenario:
I have bunch of yaml files structure as below, I flatten them with local values and assign to the resource attributes.
On my resource i need to assign roles values as a list , example below
roles = local.acs_tup #["READER,OWNERSHIP","OWNERSHIP1","READER1,DEVELOPER1"]
yaml files content format
schemagrants :
- schema: BDV
privilege: USAGE
roles:
- READER
- OWNERSHIP
environment: DEV
- schema: BDV
privilege: USAGE
roles:
- OWNERSHIP1
- DEVELOPER1
environment: DEV
- schema: CDV
privilege: USAGE
roles:
- DEVELOPER2
environment: DEV
locals {
acs_files = fileset("./objects/schema-accessgrant", "*.yml")
acs_data = [for f in local.acs_files : yamldecode(file("./objects/schema-accessgrant/${f}"))]
acs_flatten = flatten([for access in local.acs_data :
[for sgr in access.schemagrants : {
id = "${sch.schema}.${sch.privilege}.${sch.environment}"
roles = sgr.roles
environment = sgr.environment
}
]]
)
acs_tup = [for roles in local.acs_flatten :
{
role = join(",", roles.roles)
}
]
}
resource "snowflake_database_grant" "database-access" {
database_name = "database"
privilege = "USAGE"
roles = local.acs_tup #["READER,OWNERSHIP","OWNERSHIP1","DEVELOPER1","DEVELOPER2"]
shares = []
with_grant_option = false
}
# Current Output
[ + {
+ role = "READER,OWNERSHIP"
},
+ {
+ role = "ONWERSHIP1,DEVELOPER1"
},
+ {
+ role = "DEVELOPER2"
},
]
# Expected Output
["READER,OWNERSHIP","OWNERSHIP1","DEVELOPER1","DEVELOPER2"]
Assuming that your expected output should be ["READER","OWNERSHIP","OWNERSHIP1","DEVELOPER1","DEVELOPER2"], your acs_tup must be:
acs_tup = flatten([for roles in local.acs_flatten: roles.roles])

This object does not have an attribute named yaml structures

I want to construct my organisation structure in one yaml file for github provider to manage teams, members, some teams contain subelements like subteams, some don't, for loop errors with "This object does not have an attribute named" when structure in yaml is not the same, I tried playing with lookup(), contains(), if() but I am just starting using terraform so maybe you could help me how I could achieve this?
# cat locals.tf
locals {
teams = yamldecode(file("teams.yaml"))["teams"]
teams_flatten = flatten([
for team in local.teams :
{
team_name = team.name
description = lookup(team, "description", "")
privacy = lookup(team, "privacy", "closed")
}
])
subteams_flatten = flatten([
for team in local.teams : [
for subteam in team.subteams : {
team_name = team.name
subteam_name = subteam.name
description = lookup(subteam, "description", "")
privacy = lookup(subteam, "privacy", "closed")
}
]
])
}
│ on locals.tf line 16, in locals:
│ 16: for subteam in team.subteams : {
│
│ This object does not have an attribute named "subteams".
# cat teams.yaml
---
teams:
# Parent team
- name: Engineering
# This team has sub-teams
subteams:
- name: access
members:
- member: alex
role: maintainer
- member: bob
role: maintainer
- name: payments
members:
- member: alice
role: maintainer
# Parent team
- name: Marketing
# This team does not have any sub-teams
members:
- member: bob
role: maintainer
- member: alex
role: maintainer
# cat teams.tf
resource "github_team" "teams" {
for_each = {
for team in local.teams_flatten : team.team_name => team
}
name = each.value.team_name
description = each.value.description
privacy = each.value.privacy
}
resource "github_team" "subteams" {
depends_on = [github_team.teams]
for_each = {
for subteam in local.subteams_flatten : subteam.subteam_name => subteam
}
name = each.value.subteam_name
parent_team_id = lookup(github_team.teams, each.value.team_name)["id"]
description = each.value.description
privacy = each.value.privacy
}
When working with raw human-edited files like this where the structure is not necessarily consistent, it can sometimes help to explicitly normalize the data structure first as a separate step, and then to use the more consistent data structure elsewhere. That then keeps the dense conditional code isolated in one place and so hopefully makes the rest of the module easier to read.
For example:
locals {
teams_raw = yamldecode(file("${path.module}/teams.yaml"))["teams"]
teams = {
for team_raw in local.teams_raw : tostring(team_raw.name) => {
name = tostring(team_raw.name)
subteams = tomap({
for subteam_raw in try(team_raw.subteams, []) :
tostring(subteam_raw.name) => {
name = tostring(subteam_raw.name)
members = tomap({
for member_raw in subteam_raw.members :
tostring(member_raw.member) => {
name = tostring(member_raw.member)
role = tostring(member_raw.role)
}
})
}
})
members = tomap({
for member_raw in try(team_raw.members, []) :
tostring(member_raw.member) => {
name = tostring(member_raw.member)
role = tostring(member_raw.role)
}
})
}
}
}
The two main relevant characteristics of the local.teams definition above are:
It uses try in a couple places to provide a fallback value for when a particular key isn't available. That means that the resulting data structure will always have an attribute of that type but it might refer to an empty map, which is easier to deal with than an attribute that may not exist at all.
It uses tomap and tostring liberally to assert the expected types for each of the attributes. This will therefore allow Terraform to raise an error early if the given value isn't suitable for one of those constraints, and ensures that Terraform can infer a suitable static type for this data structure which you can rely on elsewhere.
With this new normalized data structure you can refer to members and subteams on all of the elements, and know that it'll always be set but might be set to an empty map.

Resources