I'm having an issue with the Vault Terraform. I am able to create Entities, Namespaces, Groups, and policies but linking them together is not happening for me. I can get the policy added to the group just fine, but adding members to that group I cannot.
Here's what I have so far:
# module.users returns vault_identity_entity.entity.id
data "vault_identity_entity" "user_lookup" {
for_each = toset([for user in local.groups : user.name])
entity_name = each.key
depends_on = [
module.users
]
}
# module.devops_namespace returns vault_namespace.namespace.path
resource "vault_identity_group" "devops" {
depends_on = [
vault_policy.policy
]
name = "devops_users"
namespace = module.devops_namespace.vault_namespace
member_entity_ids = [for user in data.vault_identity_entity.user_lookup : jsondecode(user.data_json).id]
}
resource "vault_identity_group_policies" "default" {
policies = [vault_policy.gitlab_policy.name]
exclusive = false
group_id = vault_identity_group.devops.id
}
What I need to do is create a namespace and add users and a policy to that namespace.
Any help would be appreciated, thanks!
resource "vault_policy" "namespace" {
depends_on = [module.namespace]
name = "namespace"
policy = file("policies/namespace.hcl")
namespace = "devops"
}
resource "vault_identity_group" "devops" {
depends_on = [
module.users
]
name = "devops_users"
namespace = module.devops_namespace.vault_namespace
policies = [vault_policy.gitlab_policy.name]
member_entity_ids = [for user in module.users : user.entity_id]
}
By referring the users the module created I was able to achieve the correct result.
Since the module created the users from locals and the data resource was trying to pull down the same users, the extra data resource section wasn't needed.
Thank you Marko E!
Related
I realised that terraform modules are recreating its resources per module declaration. So the way to reference a resource created in a module can only be done from the module, if it's defined as output. I'm looking for a way where I can reuse a module not in the way so it's recreating resources.
Imagine a scenario where I have three terraform modules.
One is creating an IAM policy (AWS), second is creating an IAM role, third is creating a different IAM role, and both roles share the same IAM policy.
In code:
# policy
resource "aws_iam_policy" "secrets_manager_read_policy" {
name = "SecretsManagerRead"
description = "Read only access to secrets manager"
policy = {} # just to shorten demonstration
}
output "policy" {
value = aws_iam_policy.secrets_manager_read_policy
}
# test-role-1
resource "aws_iam_role" "test_role_1" {
name = "test-role-1"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = "sts:AssumeRole"
Effect = "Allow"
Sid = ""
Principal = {
Service = "ecs-tasks.amazonaws.com"
}
},
]
})
}
module "policy" {
source = "../test-policy"
}
resource "aws_iam_role_policy_attachment" "attach_secrets_manager_read_to_role" {
role = aws_iam_role.test_role_1.name
policy_arn = module.policy.policy.arn
}
# test-role-2
resource "aws_iam_role" "test_role_2" {
name = "test-role-2"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = "sts:AssumeRole"
Effect = "Allow"
Sid = ""
Principal = {
Service = "ecs-tasks.amazonaws.com"
}
},
]
})
}
module "policy" {
source = "../test-policy"
}
resource "aws_iam_role_policy_attachment" "attach_secrets_manager_read_to_role" {
role = aws_iam_role.test_role_2.name
policy_arn = module.policy.policy.arn
}
# create-roles
module "role-1" {
source = "../../../modules/resources/test-role-1"
}
module "role-2" {
source = "../../../modules/resources/test-role-2"
}
In this scenario terraform is trying to create two policies for each user, but I want them to use the same resource.
Is there a way to keep the code clean, so not all resources are in the same file so that a resource is identified, and the same resource can be used in multiple modules? Or it's a tree like structure where sibling modules cannot share the same child? Yes, I could define the policy first, and pass down the properties needed to child modules where I create the users, but what if I want to have a many to many relationship between them so multiple roles share the same multiple policies?
I can think of a few ways to do this:
Option 1: Move the use of the policy module up to the parent level, and have your parent (root) Terraform code look like this:
# create-policy
module "my-policy" {
source = "../../../modules/resources/policy"
}
# create-roles
module "role-1" {
source = "../../../modules/resources/test-role-1"
policy = module.my-policy.policy
}
module "role-2" {
source = "../../../modules/resources/test-role-2"
policy = module.my-policy.policy
}
Option 2: Output the policy from the role modules, and also make it an optional input variable of the modules:
variable "policy" {
default = null # Make the variable optional
}
module "policy" {
# Create the policy, only if one wasn't passed in
count = var.policy == null ? 1 : 0
source = "../test-policy"
}
locals {
# Create a variable with the value of either the passed-in policy,
# or the one we are creating
my-policy = var.policy == null ? module.policy[0].policy : var.policy
}
resource "aws_iam_role_policy_attachment" "attach_secrets_manager_read_to_role" {
role = aws_iam_role.test_role_2.name
policy_arn = local.my-policy
}
output "policy" {
value = locals.my-policy
}
Then your root code could look like this:
module "role-1" {
source = "../../../modules/resources/test-role-1"
}
module "role-2" {
source = "../../../modules/resources/test-role-2"
policy = module.role-1.policy
}
The first module wouldn't get an input, so it would create a new policy. The second module would get an input, so it would use it instead of re-creating the policy.
I also highly recommend looking at the source code for some of the official AWS Terraform modules, like this one. Reading the source code for those really helped me understand how to create reusable Terraform modules.
I have sample code below which creates an IAM role, a policy document, attachment of policy document and then the attachment of that policy to role.
resource "aws_iam_role" "aws_snsANDsqsTeam" {
name = "aws_snsANDsqsTeam"
assume_role_policy = data.aws_iam_policy_document.production-okta-trust-relationship.json
}
data "aws_iam_policy_document" "sns-and-sqs-policy" {
statement {
sid = "AllowToPublishToSns"
effect = "Allow"
actions = [
"sns:Publish",
]
resources = [
data.resource.arn,
]
}
statement {
sid = "AllowToSubscribeFromSqs"
effect = "Allow"
actions = [
"sqs:changeMessageVisibility*",
"sqs:SendMessage",
"sqs:ReceiveMessage",
"sqs:GetQueue*",
"sqs:DeleteMessage",
]
resources = [
data.resource.arn,
]
}
}
resource "aws_iam_policy" "sns-and-sqs" {
name = "sns-and-sqs-policy"
policy = data.aws_iam_policy_document.sns-and-sqs-policy.json
}
resource "aws_iam_role_policy_attachment" "sns-and-sqs-role" {
role = "aws_snsANDsqsTeam"
policy_arn = aws_iam_policy.sns-and-sqs.arn
}
Now below is the directory tree that I am trying to get
Now I want the policy document and policy code to be moved to the developer.tf file under shared/iam folder so it will look like this
data "aws_iam_policy_document" "sns-and-sqs-policy" {
statement {
sid = "AllowToPublishToSns"
effect = "Allow"
actions = [
"sns:Publish",
]
resources = [
data.resource.arn,
]
}
statement {
sid = "AllowToSubscribeFromSqs"
effect = "Allow"
actions = [
"sqs:changeMessageVisibility*",
"sqs:SendMessage",
"sqs:ReceiveMessage",
"sqs:GetQueue*",
"sqs:DeleteMessage",
]
resources = [
data.resource.arn,
]
}
}
resource "aws_iam_policy" "sns-and-sqs" {
name = "sns-and-sqs-policy"
policy = data.aws_iam_policy_document.sns-and-sqs-policy.json
}
and have the role creation and policy attachment code in main.tf file under iam-platform-security folder, so the code will look like this:
resource "aws_iam_role" "aws_snsANDsqsTeam" {
name = "aws_snsANDsqsTeam"
assume_role_policy = data.aws_iam_policy_document.production-okta-trust-relationship.json
}
resource "aws_iam_role_policy_attachment" "sns-and-sqs-role" {
role = "aws_snsANDsqsTeam"
policy_arn = aws_iam_policy.sns-and-sqs.arn
}
My Question is how can I reference a policy which is under shared/iam folder to attach it to a role I created in main.tf file under the folder iam-platform-security. The goal is to create policies separately in the shared/iam folder and roles under team/sub-team folders ( like iam-platform-security, iam-platform-architecture,iam-platform-debug etc etc) and then create attachments so policies remains separately as standalone.
Can somebody help me on this.
How can I reference the policy document in main.tf file in different directory.
You have to use modules so that you can separate your parent TF code from other code, such as your IAM related code in a different folder.
I'm trying to create an az ad app and credential for each entry in a locals set.
The objects in the locals set have values that are needed for both resources, but my issue is the credentials resource needs values from both the locals object as well as the ad application.
This would be easy normally, but I am using a for_each which is complicated, and the value of each for the credential resource is the ad application. Is there any way I can get access to the each of az app resource but from the credential resource?
locals {
github_repos_with_apps = {
tftesting_testing = {
repo = "tftesting-testing"
environment = "tfplan"
}
}
}
resource "azuread_application" "aadapp" {
for_each = local.github_repos_with_apps
display_name = join("-", ["github-actions", each.value.repo, each.value.environment])
owners = [data.azuread_client_config.current.object_id]
}
resource "azuread_application_federated_identity_credential" "cred" {
for_each = azuread_application.aadapp
application_object_id = each.value.object_id
display_name = "my-repo-deploy"
description = "Deployments for my-repo"
audiences = ["api://AzureADTokenExchange"]
issuer = "https://token.actions.githubusercontent.com"
subject = "repo:my-org/${each.value.<something?>.repo}:environment:${each.value.<something?>.environment}"
}
In the snippet above I need the cred resource to access aadapp.object_id but also reference the locals value in order to get rep and environment. Since both cred and aadapp both use for_each the meaning of each.value changes. I'd like to reference the each.value of aadapp from cred.
My problem line is the subject value in the cred resource:
subject = "repo:my-org/${each.value.<something?>.repo}:environment:${each.value.<something?>.environment}"
I think I may have to use modules to accomplish this, but I feel there is a quicker way, like being able to store a temporary value on aadapp that would let me reference it.
After scouring some examples I did find out how to achieve this.
If I change all resources to use for_each = local.github_repos_with_apps, I can then use 'each.key` as a lookup to get the other associated resources like so:
application_object_id = resource.azuread_application.aadapp[each.key].object_id
This allows the cred resource to reference the locals values directly
subject = "repo:my-org/${each.value.repo}:environment:${each.value.environment}"
Full code:
locals {
github_repos_with_apps = {
first_test : {
repo = "tftesting-testing"
environment = "tfplan"
}
second_test : {
repo = "bleep-testing"
environment = "tfplan"
}
}
}
resource "azuread_application" "aadapp" {
for_each = local.github_repos_with_apps
display_name = join("-", ["github-actions", each.value.repo, each.value.environment])
owners = [data.azuread_client_config.current.object_id]
lifecycle {
ignore_changes = [
required_resource_access
]
}
}
resource "azuread_application_federated_identity_credential" "cred" {
for_each = local.github_repos_with_apps
application_object_id = resource.azuread_application.aadapp[each.key].object_id
display_name = each.value.repo
description = "Deployments for my-repo"
audiences = ["api://AzureADTokenExchange"]
issuer = "https://token.actions.githubusercontent.com"
subject = "repo:my-org/${each.value.repo}:environment:${each.value.environment}"
}
I'm trying to create a module in Terraform for creating Azure resources and facing some issues. This module creates a resource group, subnet, vnet and Role bindings. I see that the below code creates the resources twice because of the loop. Does the for_each loop work in such a way that the entire resource or module block will be executed each time it loops? I'm new to Terraform and come from a Java background.
Also, ideally would like to use the flatten inside the module without locals possibly, any way to do that? Code is below.
locals {
groupsbyrole = flatten([
for roleName, groupList in var.testproject1_role_assignments : [
for groupName in groupList : {
role_name = roleName
group_name = groupName
}
]
])
}
module "testproject1" {
source = "C:\\Users\\ebo1h8h\\Documents\\Project\\Automation\\Terraform\\Code\\Azure\\modules\\sandbox-module"
short_name = "testproj"
# Resource Group Variables
az_rg_location = "eastus"
az_tags = {
Environment = "Sandbox"
CostCenter = "Department"
ResourceOwner = "Vikram"
Project = "testproj"
Role = "Resource Group"
}
address_space = ["10.0.0.0/16"]
subnet_prefixes = ["10.0.1.0/24"]
subnet_names = ["a-npe-snet01-sbox"]
vnet_location = var.az_rg_location
for_each = {
for group in local.groupsbyrole : "${group.role_name}.${group.group_name}}" => group
}
principal_id = each.value.group_name
role_definition_name = each.value.role_name
}
And here is the role_assignments variable
variable "testproject1_role_assignments" {
type = map(list(string))
default = {
"Contributor" = ["prod-azure-contrib-sbox", "gcp-org-network-engineering"],
"Owner" = ["gcp-org-cloud-delivery"]
}
}
The above code creates 12 resources when it should be only 6. The only was I was able to get around this is have the resource "azurerm_role_assignment" "role_assignment" as a separate module. Ideally, I want to pass the role assignments variable in each of the module to be created so that it creates a set of resources.
Any pointers on how to achieve that?
Thanks,
The docs state
If a resource or module block includes a for_each argument whose value is a map or a set of strings, Terraform will create one instance for each member of that map or set.
So in your scenario you are creating 3 instances of the module, whereas it sounds like you want to pass in the local.groupsbyrole object as a variable in the module and only attach the for_each to the resources you want multiple instances of.
Sidenote: You could simplify your local by adding group like below:
locals {
groupsbyrole = flatten([
for roleName, groupList in var.testproject1_role_assignments : [
for groupName in groupList : {
role_name = roleName
group_name = groupName
group = "${roleName}.${groupName}"
}
]
])
}
Tip: I find adding an output to see the shape of the object whilst developing can also be useful
output "test_output" {
value = local.groupsbyrole
}
Then when you run plan you will see your object
test_output = [
+ {
+ group = "Contributor.prod-azure-contrib-sbox"
+ group_name = "prod-azure-contrib-sbox"
+ role_name = "Contributor"
},
+ {
+ group = "Contributor.gcp-org-network-engineering"
+ group_name = "gcp-org-network-engineering"
+ role_name = "Contributor"
},
+ {
+ group = "Owner.gcp-org-cloud-delivery"
+ group_name = "gcp-org-cloud-delivery"
+ role_name = "Owner"
},
]
I’m trying to write some code which would take an input structure like this:
projects = {
"project1" = {
namespaces = ["mynamespace1"]
},
"project2" = {
namespaces = ["mynamespace2", "mynamespace3"]
}
}
and provision multiple resources with for_each which would result in this:
resource "rancher2_project" "project1" {
provider = rancher2.admin
cluster_id = module.k8s_cluster.cluster_id
wait_for_cluster = true
}
resource "rancher2_project" "project2" {
provider = rancher2.admin
cluster_id = module.k8s_cluster.cluster_id
wait_for_cluster = true
}
resource "rancher2_namespace" "mynamespace1" {
provider = rancher2.admin
project_id = rancher2_project.project1.id
depends_on = [rancher2_project.project1]
}
resource "rancher2_namespace" "mynamespace2" {
provider = rancher2.admin
project_id = rancher2_project.project2.id
depends_on = [rancher2_project.project2]
}
resource "rancher2_namespace" "mynamespace3" {
provider = rancher2.admin
project_id = rancher2_project.project2.id
depends_on = [rancher2_project.project2]
}
namespaces are dependent on Projects and the generate id needs to be passed into namespace.
Is there any good way of doing this dynamically ? We might have a lot of Projects/namespaces.
Thanks for any help and advise.
The typical answer for systematically generating multiple instances of a resource based on a data structure is resource for_each. The main requirement for resource for_each is to have a map which contains one element per resource instance you want to create.
In your case it seems like you need one rancher2_project per project and then one rancher2_namespace for each pair of project and namespaces. Your current data structure is therefore already sufficient for the rancher2_project resource:
resource "rancher2_project" "example" {
for_each = var.projects
provider = rancher2.admin
cluster_id = module.k8s_cluster.cluster_id
wait_for_cluster = true
}
The above will declare two resource instances with the following addresses:
rancher2_project.example["project1"]
rancher2_project.example["project2"]
You don't currently have a map that has one element per namespace, so it will take some more work to derive a suitable value from your input data structure. A common pattern for this situation is flattening nested structures for for_each using the flatten function:
locals {
project_namespaces = flatten([
for pk, proj in var.projects : [
for nsk in proj.namespaces : {
project_key = pk
namespace_key = ns
project_id = rancher2_project.example[pk].id
}
]
])
}
resource "rancher2_namespace" "example" {
for_each = {
for obj in local.project_namespaces :
"${obj.project_key}.${obj.namespace_key}" => obj
}
provider = rancher2.admin
project_id = each.value.project_id
}
This produces a list of objects representing all of the project and namespace pairs, and then the for_each argument transforms it into a map using compound keys that include both the project and namespace keys to ensure that they will all be unique. The resulting instances will therefore have the following addresses:
rancher2_namespace.example["project1.mynamespace1"]
rancher2_namespace.example["project2.mynamespace2"]
rancher2_namespace.example["project2.mynamespace3"]
This seems to work too:
resource "rancher2_namespace" "example" {
count = length(local.project_namespaces)
provider = rancher2.admin
name = local.project_namespaces[count.index].namespace_name
project_id = local.project_namespaces[count.index].project_id
}