Assuming the following yaml input file
admins:
members:
- john#gmail.com
- peter#gmail.com
viewers:
members:
- maria#gmail.com
- john#gmail.com
that is passed to var.memberships via yamldecode, is it possible to produce a list of objects that has the member as key and the groups it belongs to as values, i.e. something like
{
"maria#gmail.com" = {
groups = ["viewers"]
},
"john#gmail.com" = {
groups = ["admins", "viewers"]
},
"peter#gmail.com" = {
groups = ["admins"]
}
}
The general answer for projecting one data structure into another in the Terraform language is for expressions.
In this particular case the Grouping Results mechanism would be a part of the solution, because you want to map potentially many elements from the input to a single element in the output by grouping by a particular key (the email addresses).
I think I'd start by first flattening out this data structure so that it's just a single set of member+group pairs where each element is therefore describing the relationship between one member and one group alone.
locals {
memberships = setunion([
for group_name, group in var.memberships : toset([
for member_email in group.members : {
member_email = member_email
group_name = group_name
}
])
]...)
}
With this declaration and your example input I would expect the intermediate result to be:
memberships = toset([
{ member_email = "john#gmail.com", group_name = "admins" },
{ member_email = "peter#gmail.com", group_name = "admins" },
{ member_email = "maria#gmail.com", group_name = "viewers" },
{ member_email = "john#gmail.com", group_name = "viewers" },
])
We can then project this a second time to transform it into a map from email addresses to collections of groups:
locals {
member_groups = tomap({
for membership in local.memberships :
membership.member_email => membership.group_name...
})
}
member_groups = tomap({
"john#gmail.com" = ["admins", "viewers"]
"maria#gmail.com" = ["viewers"]
"peter#gmail.com" = ["admins"]
})
This now has the same information as the data structure you wanted but not quite the right shape, so one more transformation can achieve exactly the map of objects you described in your question:
locals {
users = tomap({
for email, groups in local.member_groups : email => {
groups = toset(groups)
}
})
}
users = tomap({
"john#gmail.com" = {
groups = toset(["admins", "viewers"])
}
"maria#gmail.com" = {
groups = toset(["viewers"])
}
"peter#gmail.com" = {
groups = toset(["admins"])
}
})
Related
I Want to add the users to the team and assigned them roles as per the yaml
but the error is value does not have any attributes.
i will have multiple users in single team with different roles
this is
variables.tf
variable "admin_role_id" {
description = "The id to give access admin role to the user"
type = string
default = "1111111111111"
}
variable "user_role_id" {
description = "The id to give access user role to the user"
type = string
default = "22222222222222222"
}
this is my yaml file
TEAM1:
- users:
- joe#gmail.com
- pa#gmail.com
roles:
- ${user_role}
- ${admin_role}
- users:
- test#gmail.com
roles:
- ${user_role}
TEAM2:
- users:
- joe#gmail.com
roles:
- ${test_user_role}
This is TERRAFORM CODE
i am using local variable and i flatten the values there
locals {
render_membership = templatefile("${path.module}/teammembers.yaml",
{
admin_role = var.admin_role_id
user_role = var.user_role_id
}
)
membership_nested = yamldecode(local.render_membership)
membership_flat = flatten(
[
for team_key, team in local.membership_nested : [
for user in team.users : {
team_name = team_key
user_name = user
roles = team.roles
}
]
]
)
}
resource "squadcast_team_member" "membership" {
for_each = { for i, v in local.membership_flat : i => v }
team_id = data.squadcast_team.teams[each.value.team_name].id
user_id = data.squadcast_user.users[each.value.user_name].id
role_ids = each.value.roles
}
data "squadcast_team" "teams" {
for_each = { for i, v in local.membership_flat : i => v }
name = each.value.team_name
}
data "squadcast_user" "users" {
for_each = { for i, v in local.membership_flat : i => v }
email = each.value.user_name
}
output "rendered_yaml" {
value = local.membership_nested
}
Error:
│ Error: Unsupported attribute
│
│ on teammembers.tf line 112, in locals:
│ 112: for user in team.users : {
│
│ This value does not have any attributes.
Decoding the YAML document you showed would produce the following Terraform value:
{
TEAM1 = [
{
users = [
"joe#gmail.com",
"pa#gmail.com",
]
roles = [
"...",
"...",
]
},
]
TEAM2 = [
{
users = [
"joe#gmail.com",
]
roles = [
"...",
],
},
]
}
Notice that the values assigned to TEAM1 and TEAM2 are lists -- or more accurately: tuples -- that each contain one object.
That means that in your membership_flat expression the value of team in the first for expression is not an object, and so it's not valid to write team.users: the . operator for accessing an attribute is only valid for object types or map types.
If the structure of this YAML is fixed and you need to change the Terraform configuration to work with it then one way would be to access the first element of the tuple and then take the users attribute of that object:
for user in team[0].users : {
This solution will only work if your team values are always single-element sequences. If you had a team with more than one object assigned to it then the above expression would ignore the other elements, and if you had a team with zero objects assigned to it then it would fail because there would be no index zero.
If you are able to change the YAML structure instead, then I would suggest removing the YAML sequences so that each "team" is represented by only a single mapping, which Terraform will then decode as a single object:
TEAM1:
users:
- joe#gmail.com
- pa#gmail.com
roles:
- ${user_role}
- ${admin_role}
TEAM2:
users:
- joe#gmail.com
roles:
- ${test_user_role}
Terraform's yamldecode would decode the above to the following value instead:
{
TEAM1 = {
users = [
"joe#gmail.com",
"pa#gmail.com",
]
roles = [
"...",
"...",
]
}
TEAM2 = {
users = [
"joe#gmail.com",
]
roles = [
"...",
],
}
}
Notice that now each team is just a single object rather than a tuple of objects, and so team.users would now be a valid way to access the users attribute of each single object.
I'm creating multiple Google Cloud Projects using a module which creates a custom service account for each one and provides that as an output eg
module "app_projects" {
count = var.number_application_projects
etc
}
I then have a list of IAM roles I want to assign to each Project Service account like this:
sa_roles = ["roles/run.developer","roles/appengine.deployer"]
resource "google_project_iam_member" "proj_sa_iam_roles" {
count = var.number_application_projects
for_each = toset(var.sa_roles)
project = module.app_projects[count.index].project_id
role = each.value
member = "serviceAccount:${module.app_projects[count.index].service_account_email}"
}
This runs into the error about "count" and "for_each" being mutually-exclusive, and I can't for the life of me figure out what to use instead, any help would be appreciated!
You can probably use setproduct, but the standard method is using flatten.
They even have a section tailored for similar use cases. Flattening nested structures for for_each.
Here is an example, that doesn't use your exact resource, but should be instructive and you can actually run it to test it out.
modules/app-project/variables.tf
variable "name" {}
modules/app-project/outputs.tf
output "name" {
value = var.name
}
modules/member/variables.tf
variable "project" {}
variable "role" {}
modules/member/outputs.tf
output "project_role" {
value = "${var.project}-${var.role}"
}
main.tf
locals {
roles = ["rad", "tubular"]
}
module "app_project" {
source = "./modules/app-project"
count = 2
name = "app-project-${count.index}"
}
module "project_role" {
source = "./modules/member"
for_each = { for pr in flatten([for p in module.app_project[*] : [
for r in local.roles : {
app_project_name = p.name
role = r
}]
]) : "${pr.app_project_name}-${pr.role}" => pr }
project = each.value.app_project_name
role = each.value.role
}
output "project_roles" {
value = values(module.project_role)[*].project_role
}
terraform plan output
Changes to Outputs:
+ project_roles = [
+ "app-project-0-rad",
+ "app-project-0-tubular",
+ "app-project-1-rad",
+ "app-project-1-tubular",
]
In your case specifically, I think something like this would work:
resource "google_project_iam_member" "proj_sa_iam_roles" {
for_each = { for i, pr in flatten([for p in module.app_project[*] : [
for r in var.sa_roles : {
app_project = p
role = r
}]
]) : "${i}-${pr.role}" => pr }
project = each.value.app_project.project_id
role = each.value.role
member = "serviceAccount:${each.value.app_project.service_account_email}"
}
With Terraform, is it possible to convert an object something like
locals {
data = {
"project1" = {
user_assigned = ["user1", "user2", "user3"]
}
"project2" = {
user_assigned = ["user2", "user3", "user4"]
}
}
to an output like
user1 = ["project1"]
user2 = ["project1","project2"]
user3 = ["project1","project2"]
user4 = ["project2"]
Note that data is an object with maps of keys(projects) and values(user_assigned)
Here's another way to do it, which I'm sharing just in case it's interesting -- the accepted answer is a fine approach too.
locals {
data = {
"project1" = {
user_assigned = ["user1", "user2", "user3"]
}
"project2" = {
user_assigned = ["user2", "user3", "user4"]
}
}
project_user = flatten([
for proj_name, proj in local.data : [
for username in proj.user_assigned : {
project_name = proj_name,
username = username
}
]
])
}
output "example" {
value = {
for pu in local.project_user :
pu.username => pu.project_name...
}
}
Outputs:
example = {
"user1" = [
"project1",
]
"user2" = [
"project1",
"project2",
]
"user3" = [
"project1",
"project2",
]
"user4" = [
"project2",
]
}
I typically use this sort of approach because a data structure like that intermediate local.project_user value -- which is a list with an element for each project/user pair -- often ends up being useful when declaring resources that represent those pairings.
There wasn't any context in the question about what these projects and users represent or which provider they might related to, so I'm going to use github_team and github_team_membership as an example to illustrate what I mean:
resource "github_team" "example" {
for_each = local.data
name = each.key
}
resource "github_team_membership" "example" {
for_each = {
for pu in local.project_user : "${pu.username}:${pu.project_name}" => pu
}
team_id = github_team.example[each.value.project_name].id
username = each.value.username
}
Lots of providers have resources that represent a relationship between two objects like this, and so having an intermediate data structure that contains an element for each pair is a useful building block for those cases, and then you can derive from that mappings in either direction as I did in the output "example" in my original snippet.
You can't dynamically create fully independent variables. Instead you can create a map in few ways. One way would be with the help of transpose and zipmap:
output "test1" {
value = transpose(zipmap(keys(local.data), values(local.data)[*].user_assigned))
}
Resulting in:
test1 = tomap({
"user1" = tolist([
"project1",
])
"user2" = tolist([
"project1",
"project2",
])
"user3" = tolist([
"project1",
"project2",
])
"user4" = tolist([
"project2",
])
})
I have the following tf file:
locals {
schemas = {
"ODS" = {
usage_roles = ["TRANSFORMER"]
}
"EXT" = {
usage_roles = []
}
"INT" = {
usage_roles = ["REPORTER"]
}
"DW" = {
usage_roles = ["LOADER"]
}
}
}
resource "snowflake_schema" "schema" {
for_each = local.schemas
name = each.key
database = ???????
usage_roles = each.value.usage_roles
}
I want to maintain the locals as it is (different usage_roles for each schema and hardcoded here) while having several values as database for each schema. In pseudo-code it would be:
for database in ['db_1', 'db_2', 'db_3']:
resource "snowflake_schema" "schema" {
for_each = local.schemas
name = each.key
database = database
usage_roles = each.value.usage_roles
}
So that we have the same schema resource in the three different databases. I have read some articles that point me to the belief that it is possible to make this loop but pre assigning all the values, meaning that I would have to put usage_roles in a list or something instead of hardcoding here in locals, which I think is less readable. For instance:
Terraform - how to use for_each loop on a list of objects to create resources
Is even possible what I am asking for? If so, how? Thank you very much in advance
The main requirement for for_each is that the map you provide must have one element per instance of the resource you want to create. In your case, I think that means you need a map with one element per every combination of database and schema.
The operation of finding every combination of values in two sets is formally known as the cartesian product, and Terraform has the setproduct function to perform that operation. In your case, the two sets to apply it to are the set of database names and the set of keys in your schemas map, like this:
locals {
databases = toset(["db_1", "db_2", "db_3"])
database_schemas = [
for pair in setproduct(local.databases, keys(local.schemas)) : {
database_name = pair[0]
schema_name = pair[1]
usage_roles = local.schemas[pair[1]].usage_roles
}
]
}
The local.database_schemas value would then contain an object for each combination, like this:
[
{
database_name = "db_1"
schema_name = "ODS"
usage_roles = ["TRANSFORMER"]
},
{
database_name = "db_1"
schema_name = "EXT"
usage_roles = []
},
# ...
{
database_name = "db_2"
schema_name = "ODS"
usage_roles = ["TRANSFORMER"]
},
{
database_name = "db_2"
schema_name = "EXT"
usage_roles = []
},
# ...
{
database_name = "db_3"
schema_name = "ODS"
usage_roles = ["TRANSFORMER"]
},
{
database_name = "db_3"
schema_name = "EXT"
usage_roles = []
},
# ...
]
This meets the requirement of having one element per instance you want to create, but we still need to convert it to a map with a unique key per element to give Terraform a unique tracking key for each instance, so we can do one more for projection in the for_each argument:
resource "snowflake_schema" "schema" {
for_each = {
for s in local.database_schemas :
"${s.database_name}:${s.schema_name}" => s
}
name = each.value.schema_name
database = each.value.database_name
usage_roles = each.value.usage_roles
}
Terraform will track these instances with addresses like this:
snowflake_schema.schema["db_1:ODS"]
snowflake_schema.schema["db_1:EXT"]
...
snowflake_schema.schema["db_2:ODS"]
snowflake_schema.schema["db_2:EXT"]
...
snowflake_schema.schema["db_3:ODS"]
snowflake_schema.schema["db_3:EXT"]
...
how can I parse this data type in terraform 0.12
variable "groups" {
type = list(object({
group_id = string
permissions = list(string)
}))
}
Example:
groups = [
{
group_id = "gcp-org-admin"
permissions = [ "roles/resourcemanager.organizationAdmin",
"roles/resourcemanager.folderViewer",
"roles/viewer",
"roles/iam.organizationRoleViewer",
"roles/orgpolicy.policyViewer"
]
},
{
group_id = "gcp-security-ops"
permissions = [ "roles/resourcemanager.folderViewer",
"roles/logging.viewer",
"roles/monitoring.editor",
"roles/iam.securityReviewer"
]
}]
for each of the groups, I would like to pair group_id and each permissions
that is like
{
group_id = "gcp-org-admin"
permissions = "roles/resourcemanager.organizationAdmin"
},
{
group_id = "gcp-org-admin"
permissions = "roles/resourcemanager.folderViewer"
},
{
group_id = "gcp-org-admin"
permissions = "roles/viewer"
}
Would like to create organization_iam_resource
for each of the permissions within each group_id, I have to create a resource.
Is there any way to do this
Take a look at this example as it shows you a possible answer and some problems associated with nested lists: https://github.com/hashicorp/terraform/issues/11036