I am looking for the operator logic to check values in an array (terraform) - has anyone tackled a similar problem and has a solution?
the resource is like this
resource "google_project_iam_binding" "my_project_iam_bigquery_dataviewer" {
provider = google.my-project
project = "my-project"
role = "roles/bigquery.admin"
members = [
"group:my-first-group#my-domain.com",
"group:my-second-group#my-domain.com"
]
}
I have tried adding a * (like with lists) to the attribute but - without success
- cond_type: "attribute"
resource_types:
- "google_project_iam_member"
- "google_project_iam_binding"
attribute: "members.*"
operator: "starting_with"
value: "group"
otherwise, my thoughts of an operator that knows to iterate over the array
- cond_type: "attribute"
resource_types:
- "google_project_iam_member"
- "google_project_iam_binding"
attribute: "members"
operator: "iterate_array.starting_with"
value: "group"
edit: this is how the python custom policy checks each value of the members array: https://github.com/bridgecrewio/checkov/blob/HEAD/checkov/terraform/checks/resource/gcp/ArtifactRegistryPrivateRepo.py#L34-L42
for context. If I was to check the value of an attribute that isn't an array i.e. member in a resource:
resource "google_project_iam_binding" "my_project_iam_bigquery_dataviewer" {
provider = google.my-project
project = "my-project"
role = "roles/bigquery.admin"
member = "group:my-group#mydomain.com"
}
I can (and do) use this yaml
- cond_type: "attribute"
resource_types:
- "google_project_iam_member"
- "google_project_iam_binding"
attribute: "member"
operator: "starting_with"
value: "group"
I cannot find a way to do the same check for members
Related
I tried this so much but could not figure out how to do this. I have a following YAML file below.
---
project_team_mapping:
- Terraform-Project-1: ## this is project name
- team_name: example-team-1
roles:
- GROUP_READ_ONLY
- GROUP_DATA_ACCESS_READ_WRITE
- team_name: example-team-2
roles:
- GROUP_READ_ONLY
- GROUP_DATA_ACCESS_READ_WRITE
- Terraform-Project-2: ## this is 2nd project name
- team_name: example-team-3
roles:
- GROUP_READ_ONLY
- GROUP_DATA_ACCESS_READ_WRITE
- team_name: example-team-4
roles:
- GROUP_READ_ONLY
- GROUP_DATA_ACCESS_READ_WRITE
I'm using some TF module to decode the YAML files and here is the module and the YAML decoded output JSON.
module "yaml_json_multidecoder" {
source = "levmel/yaml_json/multidecoder"
version = "0.2.1"
filepaths = ["./examples/*.yaml"]
}
output "project_team_mapping" {
value = module.yaml_json_multidecoder.files.project.project_team_mapping
}
below is the output when using above decoder module.
project_team_mapping = [
{
"Terraform-Project-1" = [
{
"roles" = [
"GROUP_READ_ONLY",
"GROUP_DATA_ACCESS_READ_WRITE",
]
"team_name" = "dx-example-team-3"
},
{
"roles" = [
"GROUP_READ_ONLY",
"GROUP_DATA_ACCESS_READ_WRITE",
]
"team_name" = "dx-example-team-4"
},
]
}
]
i would like to create resources using Terraform for/for_each function and dynamic functions for a particular resource block below, i'm strugging not able to understand the outcome.
Below is the resource block going to use
resource "mongodbatlas_project" "test" {
name = "project-name"
org_id = "<ORG_ID>"
project_owner_id = "<OWNER_ACCOUNT_ID>"
# want to use dynamic function here to create multiple teams
teams {
team_id = "5e0fa8c99ccf641c722fe645"
role_names = ["GROUP_OWNER"]
}
}
can someone help please?
resource "mongodbatlas_project" "test" {
name = "project-name"
org_id = "<ORG_ID>"
project_owner_id = "<OWNER_ACCOUNT_ID>"
# want to use dynamic function here to create multiple teams
teams {
team_id = "5e0fa8c99ccf641c722fe645"
role_names = ["GROUP_OWNER"]
}
}
I might use wrong YAML structuring, please suggest good structure to fit my needs. i can change the YAML.
above question i'm using team_name in YAML but the resource block takes team_id. i'm planning to use data function to get ID but if you happen to have a easy solution please suggest
Many unknowns you have left. But a best guess would be the example below.
Assuming you have decoded yaml into a local.decodedyaml. Also you possibly would need team_id instead of team_name, maybe you can load them via data source, or will have to retrieve and add to yaml manually.
resource "mongodbatlas_project" "test" {
for_each = local.decodedyaml.project_team_mapping
name = each.key
org_id = "<ORG_ID>"
project_owner_id = "<OWNER_ACCOUNT_ID>"
dynamic "teams" {
for_each = {
for team in each.value: team.team_name => team.roles
}
content {
team_id = each.key
role_names = each.value
}
}
}
This will work if you change the data design to "unwrap" the initial lists:
Instead of
project_team_mapping:
- Terraform-Project Name-1: ## this is project name
- team_name: example-team-1
roles:
- GROUP_READ_ONLY
- GROUP_DATA_ACCESS_READ_WRITE
...
- Terraform-Project Name-2: ## this is 2nd project name
- team_name: example-team-3
roles:
- GROUP_READ_ONLY
- GROUP_DATA_ACCESS_READ_WRITE
...
Use
project_team_mapping:
Terraform-Project Name-1: ## this is project name
- team_name: example-team-1
roles:
- GROUP_READ_ONLY
- GROUP_DATA_ACCESS_READ_WRITE
...
Terraform-Project Name-2: ## this is 2nd project name
- team_name: example-team-3
roles:
- GROUP_READ_ONLY
- GROUP_DATA_ACCESS_READ_WRITE
...
This eases the data structure and greatly eases the hcl parsing.
Lastly, I am not sure yaml keys with spaces are a risk-free endevour.
I have the following config
config:
groups:
group1:
capabilities:
- create
- read
- update
members:
- robert#gmail.com
- paul#gmail.com
group2:
capabilities:
- create
- list
members:
- peter#gmail.com
group3:
capabilities:
- read
- list
members:
- john#gmail.com
So I want to create a set of vault identities in terraform
resource "vault_identity_entity" "this" {
for_each = ?1
name = each.key
}
How should I iterate (or should I say parse the yaml config) so that the vault_identity_entity is created for everything under config.groups.*.members , i.e. for all the email entries?
for_each accepts a set of strings, so you can use:
resource "vault_identity_entity" "this" {
for_each = toset(flatten(values(local.config.config.groups)[*].members))
name = each.key
}
We are using Terraform to store secrets inside AWS secrets manager. We would like to expand our Terraform to add resource access policy to each secret to only allow certain IAM roles or user access the secret and get it is value. We are defining each secret and it is metadata using YAML. Terraform will then decode the yaml and store all the contents in a map. We then have a for_each to iterate through each map and create the secrets. Below is the yaml definition for a secret
nonprod:
- name: my-super-secret
metadata:
description: my-super-secret
value: somesecret
policy: true #This is the feature we trying to add. It will tell TF to add resource access policy
iam_roles:
- "arn:aws:iam::account-id:role/sagemaker"
- "arn:aws:iam::account-id:user/jon.doe"
- "arn:aws:iam::account-id:role/test"
tags:
purpose: sagemaker
The YAML is decoded and then stored in a var.secrets map. Using TF console this is what TF store in var.secrets after decoding YAML.
{
"metadata" = {
"description" = "my-super-secret"
}
"name" = "my-super-secret"
"policy" = true
"iam_roles" = [
"arn:aws:iam::account-id:role/sagemaker",
"arn:aws:iam::account-id:user/jane.doe",
"arn:aws:iam::account-id:role/test",
]
"tags" = {
"purpose" = "sagemkaer"
}
"value" = "somesecret"
}
on the main.tf file, I added the following code for the IAM policy document:
data "aws_iam_policy_document" "example" {
for_each = { for item in var.secrets : item.name => item }
statement {
sid = "EnableAccessFor${each.value.name}"
principals {
type = "AWS"
identifiers = [lookup(each.value, "iam_roles")]
}
actions = [
"secretsmanager:GetSecretValue",
]
resources = [
"*",
]
}
}
then I am passing the policy document to aws_secretsmanager_secret_policy resource to attach the policy to the secret that has policy set as true
resource "aws_secretsmanager_secret_policy" "policy" {
depends_on = [aws_secretsmanager_secret.secret]
for_each = { for item in var.secrets : item.name => item }
secret_arn = each.key
policy = data.aws_iam_policy_document.example.json
}
no matter what way I use, I always get errors when I run a plan. I have used the following functions with no luck:
join, concat, toset, splat, jsonencode and for expressions
I get the following errors:
using for expression identifiers = [for r in lookup(each.value, "iam_roles") : r] produces this error: Invalid value for "inputMap" parameter: the given object has no attribute "iam_roles"
using splat identifiers = "${each.value[*].iam_roles}" produces this error: Inappropriate value for attribute "identifiers": element 0: string required.
using toset with lookup identifiers = "${toset(lookup(each.value, "iam_roles", ""))}" produces this error Invalid value for "v" parameter: cannot convert string to set of any single type.
using join with lookup identifiers = ["${join(", ", lookup(each.value, "iam_roles", ""))}"] produces this error Invalid value for "lists" parameter: list of string required.
using jsonencode identifiers = [jsonencode(each.value.iam_roles)] produces this error This object does not have an attribute named "iam_roles".
using just each.value without lookup identifiers = [each.value.iam_roles] produces this error Inappropriate value for attribute "identifiers": element 0: string required.
Any idea?
Update
I got rid of the iam_policy_document and instead opt-in to use the aws_secretsmanager_secret_policy resource with a json-policy. See example below:
resource "aws_secretsmanager_secret_policy" "policy" {
depends_on = [aws_secretsmanager_secret.secret]
for_each = { for item in var.secrets : item.name => item if try(item.policy, false)}
secret_arn = aws_secretsmanager_secret.secret[each.key].arn
policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EnableAccessFor${each.value.name}",
"Effect": "Allow",
"Principal": {
"AWS": [
"${join(",", formatlist("\"%s\"", each.value.iam_roles))}"
]
},
"Action": "secretsmanager:GetSecretValue",
"Resource": "*"
}
]
}
POLICY
}
The part I am having issue is with the each.value.iam_roles as that is a tuple vs. a string. I tried multiple ways to convert that into string but it is not working. Perhaps someone can help me with that.
issue was resolved with encoding the entire policy to JSON using the `jsonencode function in Terraform.
below is a teams.yaml file in the Terraform directory.
---
TEAMS:
- name: dx-example-team2
roles:
- GROUP_DATA_ACCESS_READ_WRITE:
members:
- ab#ad.com
- cd#ad.com
- GROUP_OWNER:
members:
- pr#ad.com
Hello all,
can someone help me with gettting only values from nested members KEY from above YAML file and how to form a list merge of all the values.
TRIED Terraform Code , but below is just sample i have:
locals {
teams_file = yamldecode(file("${path.cwd}/teams.yaml"))["TEAMS"]
all_members = flatten([for team in local.teams_file : [
for role in team.roles : {
"name" = role
}
]
])
}
output "sample" {
value = local.all_members
}
EXPECTED OUTPUT:
all_members = [ "ab#ad.com" , "cd#ad.com", "pr#ad.com"]
In your case you can just use spat expression:
all_members = flatten(local.teams_file[*].roles[*].members)
I have this root module which calls the child module to create a GCP project and create IAM role bindings.
module "test_project" {
source = "terraform.dev.mydomain.com/Dev/sbxprjmodule/google"
version = "1.0.3"
short_name = "looker-nwtest"
owner_bindings = ["group:npe-cloud-platformeng-contractors#c.mydomain.com", "group:npe-sbox-rw-tfetraining#c.mydomain.com"]
}
variable "owner_bindings" {
type = list(string)
default = null
}
This is the child module which does the assignments
resource "google_project_iam_binding" "g-sbox-iam-owner" {
count = var.owner_bindings == null ? 0 : length(var.owner_bindings)
project = "${var.project_id}-${var.short_name}"
role = "roles/owner"
members = [var.owner_bindings[count.index]]
}
variable "owner_bindings" {
type = list(string)
default = null
}
/*
When I do a terraform plan and apply, it creates both the bindings properly, looping through twice. Then when I run a terraform plan again and apply, it shows this change below.
# module.lookernwtest_project.google_project_iam_binding.g-sbox-iam-owner[0] will be updated in-place
~ resource "google_project_iam_binding" "g-sbox-iam-owner" {
id = "g-prj-npe-sbox-looker-nwtest/roles/owner"
~ members = [
+ "group:npe-cloud-platformeng-contractors#c.mydomain.com",
- "group:npe-sbox-rw-tfetraining#c.mydomain.com",
]
# (3 unchanged attributes hidden)
}
Next time I do a terraform plan and apply, it shows the below. It then alternates between the two of the groups on each subsequent plan and apply.
# module.lookernwtest_project.google_project_iam_binding.g-sbox-iam-owner[1] will be updated in-place
~ resource "google_project_iam_binding" "g-sbox-iam-owner" {
id = "g-prj-npe-sbox-looker-nwtest/roles/owner"
~ members = [
- "group:npe-cloud-platformeng-contractors#c.relayhealth.com",
+ "group:npe-sbox-rw-tfetraining#c.relayhealth.com",
]
# (3 unchanged attributes hidden)
}
Tried to change the data structure from list to set and had the same issue.
The groups are not inherited and are applied only at the project level too. So not sure what I'm doing wrong here.
Instead of count you can use a for_each the change is simple...
the resource in your child module will look something like this:
resource "google_project_iam_binding" "g-sbox-iam-owner" {
for_each = var.owner_bindings == null ? toset([]) : toset(var.owner_bindings)
project = "${var.project_id}-${var.short_name}"
role = "roles/owner"
members = [each.value]
}
The count changes for_each and in the members we use the each.value
With a for_each the state changes, you will no longer see the numeric array:
# module.lookernwtest_project.google_project_iam_binding.g-sbox-iam-owner[0]
...
# module.lookernwtest_project.google_project_iam_binding.g-sbox-iam-owner[1]
instead it will have the names, something like:
# module.lookernwtest_project.google_project_iam_binding.g-sbox-iam-owner["abc"]
...
# module.lookernwtest_project.google_project_iam_binding.g-sbox-iam-owner["def"]
To loop or not to loop
After looking at this for a while; I'm questioning why do we need individual iam_binding if they all will have the same role, if all members have the same "roles/owner" we could just do:
resource "google_project_iam_binding" "g-sbox-iam-owner" {
project = "${var.project_id}-${var.short_name}"
role = "roles/owner"
members = [var.owner_bindings]
}