as i am a new bot to terraform, i am trying to create lambda permissions to multiple lambda functions using terraform.
main.tf
module "lambda1_s3_events" {
source = "./terraform-aws-modules/lambda/aws"
statement_id = "AllowS3Invoke"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.lambda1.function_name
principal = "s3.amazonaws.com"
source_arn = "arn:aws:s3:::${module.s3_bucket.name}"
}
module "lambda2_s3_events" {
source = "./terraform-aws-modules/lambda/aws"
statement_id = "AllowS3Invoke"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.lambda2.function_name
principal = "s3.amazonaws.com"
source_arn = "arn:aws:s3:::${module.s3_bucket.name}"
}
module "lambda3_s3_events" {
source = "./terraform-aws-modules/lambda/aws"
statement_id = "AllowS3Invoke"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.lambda3.function_name
principal = "s3.amazonaws.com"
source_arn = "arn:aws:s3:::${module.s3_bucket.name}"
}
and instead of creating multiple lambda permission modules as showed above. how can we create this three as one ?
i have tried add three blocks to configure
you can use a for_each meta argument on modules also, similar like resources.
module "lambda1_s3_events" {
source = "./terraform-aws-modules/lambda/aws"
for_each = toset(local.lambda_functions)
statement_id = "AllowS3Invoke"
action = "lambda:InvokeFunction"
function_name = each.value
principal = "s3.amazonaws.com"
source_arn = "arn:aws:s3:::${module.s3_bucket.name}"
}
locals {
lambda_functions = [
aws_lambda_function.lambda1.function_name,
aws_lambda_function.lambda2.function_name,
aws_lambda_function.lambda3.function_name,
## update this list with your lambda function names ##
]
}
you can do a couple of more possibilities with for_each here but I hope you get an idea.
This will enable you to use only one module call but loop over all all lambda functions.
CONSIDERATION: Module support for for_each was added in Terraform 0.13; previous versions can only use it with resources.
Related
I realised that terraform modules are recreating its resources per module declaration. So the way to reference a resource created in a module can only be done from the module, if it's defined as output. I'm looking for a way where I can reuse a module not in the way so it's recreating resources.
Imagine a scenario where I have three terraform modules.
One is creating an IAM policy (AWS), second is creating an IAM role, third is creating a different IAM role, and both roles share the same IAM policy.
In code:
# policy
resource "aws_iam_policy" "secrets_manager_read_policy" {
name = "SecretsManagerRead"
description = "Read only access to secrets manager"
policy = {} # just to shorten demonstration
}
output "policy" {
value = aws_iam_policy.secrets_manager_read_policy
}
# test-role-1
resource "aws_iam_role" "test_role_1" {
name = "test-role-1"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = "sts:AssumeRole"
Effect = "Allow"
Sid = ""
Principal = {
Service = "ecs-tasks.amazonaws.com"
}
},
]
})
}
module "policy" {
source = "../test-policy"
}
resource "aws_iam_role_policy_attachment" "attach_secrets_manager_read_to_role" {
role = aws_iam_role.test_role_1.name
policy_arn = module.policy.policy.arn
}
# test-role-2
resource "aws_iam_role" "test_role_2" {
name = "test-role-2"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = "sts:AssumeRole"
Effect = "Allow"
Sid = ""
Principal = {
Service = "ecs-tasks.amazonaws.com"
}
},
]
})
}
module "policy" {
source = "../test-policy"
}
resource "aws_iam_role_policy_attachment" "attach_secrets_manager_read_to_role" {
role = aws_iam_role.test_role_2.name
policy_arn = module.policy.policy.arn
}
# create-roles
module "role-1" {
source = "../../../modules/resources/test-role-1"
}
module "role-2" {
source = "../../../modules/resources/test-role-2"
}
In this scenario terraform is trying to create two policies for each user, but I want them to use the same resource.
Is there a way to keep the code clean, so not all resources are in the same file so that a resource is identified, and the same resource can be used in multiple modules? Or it's a tree like structure where sibling modules cannot share the same child? Yes, I could define the policy first, and pass down the properties needed to child modules where I create the users, but what if I want to have a many to many relationship between them so multiple roles share the same multiple policies?
I can think of a few ways to do this:
Option 1: Move the use of the policy module up to the parent level, and have your parent (root) Terraform code look like this:
# create-policy
module "my-policy" {
source = "../../../modules/resources/policy"
}
# create-roles
module "role-1" {
source = "../../../modules/resources/test-role-1"
policy = module.my-policy.policy
}
module "role-2" {
source = "../../../modules/resources/test-role-2"
policy = module.my-policy.policy
}
Option 2: Output the policy from the role modules, and also make it an optional input variable of the modules:
variable "policy" {
default = null # Make the variable optional
}
module "policy" {
# Create the policy, only if one wasn't passed in
count = var.policy == null ? 1 : 0
source = "../test-policy"
}
locals {
# Create a variable with the value of either the passed-in policy,
# or the one we are creating
my-policy = var.policy == null ? module.policy[0].policy : var.policy
}
resource "aws_iam_role_policy_attachment" "attach_secrets_manager_read_to_role" {
role = aws_iam_role.test_role_2.name
policy_arn = local.my-policy
}
output "policy" {
value = locals.my-policy
}
Then your root code could look like this:
module "role-1" {
source = "../../../modules/resources/test-role-1"
}
module "role-2" {
source = "../../../modules/resources/test-role-2"
policy = module.role-1.policy
}
The first module wouldn't get an input, so it would create a new policy. The second module would get an input, so it would use it instead of re-creating the policy.
I also highly recommend looking at the source code for some of the official AWS Terraform modules, like this one. Reading the source code for those really helped me understand how to create reusable Terraform modules.
I'm trying to create an az ad app and credential for each entry in a locals set.
The objects in the locals set have values that are needed for both resources, but my issue is the credentials resource needs values from both the locals object as well as the ad application.
This would be easy normally, but I am using a for_each which is complicated, and the value of each for the credential resource is the ad application. Is there any way I can get access to the each of az app resource but from the credential resource?
locals {
github_repos_with_apps = {
tftesting_testing = {
repo = "tftesting-testing"
environment = "tfplan"
}
}
}
resource "azuread_application" "aadapp" {
for_each = local.github_repos_with_apps
display_name = join("-", ["github-actions", each.value.repo, each.value.environment])
owners = [data.azuread_client_config.current.object_id]
}
resource "azuread_application_federated_identity_credential" "cred" {
for_each = azuread_application.aadapp
application_object_id = each.value.object_id
display_name = "my-repo-deploy"
description = "Deployments for my-repo"
audiences = ["api://AzureADTokenExchange"]
issuer = "https://token.actions.githubusercontent.com"
subject = "repo:my-org/${each.value.<something?>.repo}:environment:${each.value.<something?>.environment}"
}
In the snippet above I need the cred resource to access aadapp.object_id but also reference the locals value in order to get rep and environment. Since both cred and aadapp both use for_each the meaning of each.value changes. I'd like to reference the each.value of aadapp from cred.
My problem line is the subject value in the cred resource:
subject = "repo:my-org/${each.value.<something?>.repo}:environment:${each.value.<something?>.environment}"
I think I may have to use modules to accomplish this, but I feel there is a quicker way, like being able to store a temporary value on aadapp that would let me reference it.
After scouring some examples I did find out how to achieve this.
If I change all resources to use for_each = local.github_repos_with_apps, I can then use 'each.key` as a lookup to get the other associated resources like so:
application_object_id = resource.azuread_application.aadapp[each.key].object_id
This allows the cred resource to reference the locals values directly
subject = "repo:my-org/${each.value.repo}:environment:${each.value.environment}"
Full code:
locals {
github_repos_with_apps = {
first_test : {
repo = "tftesting-testing"
environment = "tfplan"
}
second_test : {
repo = "bleep-testing"
environment = "tfplan"
}
}
}
resource "azuread_application" "aadapp" {
for_each = local.github_repos_with_apps
display_name = join("-", ["github-actions", each.value.repo, each.value.environment])
owners = [data.azuread_client_config.current.object_id]
lifecycle {
ignore_changes = [
required_resource_access
]
}
}
resource "azuread_application_federated_identity_credential" "cred" {
for_each = local.github_repos_with_apps
application_object_id = resource.azuread_application.aadapp[each.key].object_id
display_name = each.value.repo
description = "Deployments for my-repo"
audiences = ["api://AzureADTokenExchange"]
issuer = "https://token.actions.githubusercontent.com"
subject = "repo:my-org/${each.value.repo}:environment:${each.value.environment}"
}
I'm trying to create a module in Terraform for creating Azure resources and facing some issues. This module creates a resource group, subnet, vnet and Role bindings. I see that the below code creates the resources twice because of the loop. Does the for_each loop work in such a way that the entire resource or module block will be executed each time it loops? I'm new to Terraform and come from a Java background.
Also, ideally would like to use the flatten inside the module without locals possibly, any way to do that? Code is below.
locals {
groupsbyrole = flatten([
for roleName, groupList in var.testproject1_role_assignments : [
for groupName in groupList : {
role_name = roleName
group_name = groupName
}
]
])
}
module "testproject1" {
source = "C:\\Users\\ebo1h8h\\Documents\\Project\\Automation\\Terraform\\Code\\Azure\\modules\\sandbox-module"
short_name = "testproj"
# Resource Group Variables
az_rg_location = "eastus"
az_tags = {
Environment = "Sandbox"
CostCenter = "Department"
ResourceOwner = "Vikram"
Project = "testproj"
Role = "Resource Group"
}
address_space = ["10.0.0.0/16"]
subnet_prefixes = ["10.0.1.0/24"]
subnet_names = ["a-npe-snet01-sbox"]
vnet_location = var.az_rg_location
for_each = {
for group in local.groupsbyrole : "${group.role_name}.${group.group_name}}" => group
}
principal_id = each.value.group_name
role_definition_name = each.value.role_name
}
And here is the role_assignments variable
variable "testproject1_role_assignments" {
type = map(list(string))
default = {
"Contributor" = ["prod-azure-contrib-sbox", "gcp-org-network-engineering"],
"Owner" = ["gcp-org-cloud-delivery"]
}
}
The above code creates 12 resources when it should be only 6. The only was I was able to get around this is have the resource "azurerm_role_assignment" "role_assignment" as a separate module. Ideally, I want to pass the role assignments variable in each of the module to be created so that it creates a set of resources.
Any pointers on how to achieve that?
Thanks,
The docs state
If a resource or module block includes a for_each argument whose value is a map or a set of strings, Terraform will create one instance for each member of that map or set.
So in your scenario you are creating 3 instances of the module, whereas it sounds like you want to pass in the local.groupsbyrole object as a variable in the module and only attach the for_each to the resources you want multiple instances of.
Sidenote: You could simplify your local by adding group like below:
locals {
groupsbyrole = flatten([
for roleName, groupList in var.testproject1_role_assignments : [
for groupName in groupList : {
role_name = roleName
group_name = groupName
group = "${roleName}.${groupName}"
}
]
])
}
Tip: I find adding an output to see the shape of the object whilst developing can also be useful
output "test_output" {
value = local.groupsbyrole
}
Then when you run plan you will see your object
test_output = [
+ {
+ group = "Contributor.prod-azure-contrib-sbox"
+ group_name = "prod-azure-contrib-sbox"
+ role_name = "Contributor"
},
+ {
+ group = "Contributor.gcp-org-network-engineering"
+ group_name = "gcp-org-network-engineering"
+ role_name = "Contributor"
},
+ {
+ group = "Owner.gcp-org-cloud-delivery"
+ group_name = "gcp-org-cloud-delivery"
+ role_name = "Owner"
},
]
I have a list of variables (name of lambda function). I want to generate lambda function on aws lambda. Most of them have same configuration but 1 of them have different of memory size and environment variables. After I finish the block resource (with for_each calling that block) how can I call that reference to change configuration.
This is file main.tf in module lambda
resource "aws_lambda_function" "lambda_function" {
function_name = <existed var>
s3_bucket = <existed var>
s3_key = <existed var>
runtime = <existed var>
handler = <existed var>
memory_size = 512
timeout = 30
role = <existed var>
environment {
variables = {
ENV = "${var.env}"
POOL_ID = "${var.pooid}"
}
}
This is main.tf file in root level
module "aws_lambda_nodejs" {
source = "../../Modules/Lambda"
for_each = var.lambda_nodejs_api_path_part
env = var.env
function_name = each.key
s3_bucket = module.aws_s3_bucket.lambda_bucket_id
s3_key = "${each.key}.zip"
runtime = var.nodejs_runtime
handler = var.nodejs_handler
role = module.bootstrap.lambda_role_arn
aws_api_gateway_rest_api_id = module.aws_api_gateway.aws_api_gateway_rest_api_id
aws_api_gateway_method = "*/"
aws_api_gateway_resource_path = each.value
aws_region = var.aws_region
account_id = var.account_id
pooid = var.pooid
}
My target is:
var.lambda_nodejs_api_path_part = ["functionA", "functionB", "functionC",..etc]
function A
{
memory_size = 1536,
environment = {
ENV=dev,
poolId=eyjudb123,
configpath="/env/cfg"
}
}
other function
{
memory_size = 512,
environment = {
ENV=dev,
poolId=eyjudb123,
}
}
After I finish the block resource (with for_each calling that block) how can I call that reference to change configuration.
This is not how Terraform works. In Terraform you define the desired state for each resource and that exactly once. Terraform then manages for you to get the resources in that state. There is no iterative approach like "first do this for all resources and then do that for other resources".
What you might want to do to achieve your goal is
Add memory size as an additional variable to your lambda module.
Pass different values to that variable for different instances.
root module main.tf
module "aws_lambda_nodejs" {
source = "../../Modules/Lambda"
for_each = var.lambda_nodejs_api_path_part
env = var.env
function_name = each.key
memory_size = each.key == "function A" ? 1536 : 512
...
}
Modules/Lambda/main.tf
resource "aws_lambda_function" "pabx_function" {
...
memory_size = var.memory_size
...
}
Modules/Lambda/variables.tf
variable "memory_size" {
type = number
}
I’m trying to write some code which would take an input structure like this:
projects = {
"project1" = {
namespaces = ["mynamespace1"]
},
"project2" = {
namespaces = ["mynamespace2", "mynamespace3"]
}
}
and provision multiple resources with for_each which would result in this:
resource "rancher2_project" "project1" {
provider = rancher2.admin
cluster_id = module.k8s_cluster.cluster_id
wait_for_cluster = true
}
resource "rancher2_project" "project2" {
provider = rancher2.admin
cluster_id = module.k8s_cluster.cluster_id
wait_for_cluster = true
}
resource "rancher2_namespace" "mynamespace1" {
provider = rancher2.admin
project_id = rancher2_project.project1.id
depends_on = [rancher2_project.project1]
}
resource "rancher2_namespace" "mynamespace2" {
provider = rancher2.admin
project_id = rancher2_project.project2.id
depends_on = [rancher2_project.project2]
}
resource "rancher2_namespace" "mynamespace3" {
provider = rancher2.admin
project_id = rancher2_project.project2.id
depends_on = [rancher2_project.project2]
}
namespaces are dependent on Projects and the generate id needs to be passed into namespace.
Is there any good way of doing this dynamically ? We might have a lot of Projects/namespaces.
Thanks for any help and advise.
The typical answer for systematically generating multiple instances of a resource based on a data structure is resource for_each. The main requirement for resource for_each is to have a map which contains one element per resource instance you want to create.
In your case it seems like you need one rancher2_project per project and then one rancher2_namespace for each pair of project and namespaces. Your current data structure is therefore already sufficient for the rancher2_project resource:
resource "rancher2_project" "example" {
for_each = var.projects
provider = rancher2.admin
cluster_id = module.k8s_cluster.cluster_id
wait_for_cluster = true
}
The above will declare two resource instances with the following addresses:
rancher2_project.example["project1"]
rancher2_project.example["project2"]
You don't currently have a map that has one element per namespace, so it will take some more work to derive a suitable value from your input data structure. A common pattern for this situation is flattening nested structures for for_each using the flatten function:
locals {
project_namespaces = flatten([
for pk, proj in var.projects : [
for nsk in proj.namespaces : {
project_key = pk
namespace_key = ns
project_id = rancher2_project.example[pk].id
}
]
])
}
resource "rancher2_namespace" "example" {
for_each = {
for obj in local.project_namespaces :
"${obj.project_key}.${obj.namespace_key}" => obj
}
provider = rancher2.admin
project_id = each.value.project_id
}
This produces a list of objects representing all of the project and namespace pairs, and then the for_each argument transforms it into a map using compound keys that include both the project and namespace keys to ensure that they will all be unique. The resulting instances will therefore have the following addresses:
rancher2_namespace.example["project1.mynamespace1"]
rancher2_namespace.example["project2.mynamespace2"]
rancher2_namespace.example["project2.mynamespace3"]
This seems to work too:
resource "rancher2_namespace" "example" {
count = length(local.project_namespaces)
provider = rancher2.admin
name = local.project_namespaces[count.index].namespace_name
project_id = local.project_namespaces[count.index].project_id
}