One of my modules - let it be a - has an output definition as
output "data_table_arn" {
value = aws_dynamodb_table.data_table.*.arn
}
This is accessed one level above in module b
module "b" {
source = "../c"
data_lookup_table_arn = module.a.data_table_arn
The code above makes the variable module.a.data_table_arn accessible to module c through the variable data_lookup_table_arn. And now I am trying to access it in module c in an aws policy document definition
data "aws_iam_policy_document" "dynamo-read-policy-document" {
count = local.one_if_uses_dynamo
statement {
actions = ["dynamodb:GetItem"]
resources = [
var.data_table_arn
]
}
}
The exception I am getting is
Inappropriate value for attribute "resources": element 0: string required.
I want to debug this thing somehow and I cannot find a way to inspect the outputs or the variables defined inside a module. I can list the resources using terraform state list <resource_name> but this is not what I really want to do here.
How can I inspect the variables and outputs of a TF module?
Is this even possible?
Can you see anything faulty with my approach
I though of also using a heredoc instead of a aws_iam_policy_document but I would still need to make use of the output mentioned above - but this time in an interpolation I guess. Is this better or worse? They should be the same thing right?
Related
Looked for an answer for this question in multiple posts but found solutions for a simpler scenarios only.
Let's say I have a module called a and this module depends on multiple different modules, let's call them b and c.
module "a" {
source = ./a
depends_on = [module.B, module.C]
b_id = module.b.id
c_id = module.c.id
}
module "b" {
source = ./b
}
module "c" {
source = ./c
}
In the naive scenario both modules b and c will be created before we create module a, but what if I want to let the user of module a to create it's own instances of b and c and just pass their ids to a without us creating b and c.
I'm aware of the count trick on resources detailed here, but it works only for resources and not modules and to make it work here I will need to apply it to every resource inside the modules which will create a cumbersome code.
I also saw terraform conditional creation of objects but this solution works good if the main module/resource has just one conditional attribute, what if we have 10 modules that can be provisioned to us by the user or need to be created inline?
You can use count in module blocks as long as you are using Terraform v0.13 or later:
module "a" {
source = "./a"
b_id = one(module.b[*].id)
c_id = one(module.c[*].id)
}
module "b" {
source = "./b"
count = 1
}
module "c" {
source = "./c"
count = 0
}
The use of the one function here is to transform what would naturally be a list of either zero or one elements into a single value that may be null. Therefore your module "a" must be ready to accept null as a valid value for both b_id and c_id here, to handle the cases where there are zero instances of those modules.
Note that referring to module.b and module.c in the arguments of module "a" automatically declares dependencies on all of the outputs of those modules, and so it would be redundant to also include depends_on in the module "a" block. depends_on should be used only for "hidden dependencies", which are not reflected in the data flow between resources.
I am trying to create a GCP project with this:
module "project-factory" {
source = "terraform-google-modules/project-factory/google"
version = "11.2.3"
name = var.project_name
random_project_id = "true"
org_id = var.organization_id
folder_id = var.folder_id
billing_account = var.billing_account
activate_apis = [
"iam.googleapis.com",
"run.googleapis.com"
]
}
After that, I am trying to create a service account, like so:
module "service_accounts" {
source = "terraform-google-modules/service-accounts/google"
version = "4.0.3"
project_id = module.project-factory.project_id
generate_keys = "true"
names = ["backend-runner"]
project_roles = [
"${module.project-factory.project_id}=>roles/cloudsql.client",
"${module.project-factory.project_id}=>roles/pubsub.publisher"
]
}
To be honest, I am fairly new to Terraform. I have read a few answers on the topic (this and this) but I am unable to understand how that would apply here.
I am getting the error:
│ Error: Invalid for_each argument
│
│ on .terraform/modules/pubsub-exporter-service-account/main.tf line 47, in resource "google_project_iam_member" "project-roles":
│ 47: for_each = local.project_roles_map_data
│ ├────────────────
│ │ local.project_roles_map_data will be known only after apply
│
│ The "for_each" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work around this, use the
│ -target argument to first apply only the resources that the for_each depends on.
Looking forward to learn more about Terraform through this challenge.
With only parts of the configuration visible here I'm guessing a little bit, but let's see. You mentioned that you'd like to learn more about Terraform as part of this exercise, so I'm going to go into a lot of detail about the chain here to explain why I'm recommending what I'm going to recommend, though you can skip to the end if you find this extra detail uninteresting.
We'll start with that first module's definition of its project_id output value:
output "project_id" {
value = module.project-factory.project_id
}
module.project-factory here is referring to a nested module call, so we need to look one level deeper in the nested module terraform-google-modules/project-factory/google//modules/core_project_factory:
output "project_id" {
value = module.project_services.project_id
depends_on = [
module.project_services,
google_project.main,
google_compute_shared_vpc_service_project.shared_vpc_attachment,
google_compute_shared_vpc_host_project.shared_vpc_host,
]
}
Another nested module call! 😬 That one declares its project_id like this:
output "project_id" {
description = "The GCP project you want to enable APIs on"
value = element(concat([for v in google_project_service.project_services : v.project], [var.project_id]), 0)
}
Phew! 😅 Finally an actual resource. This expression in this case seems to be taking the project attribute of a google_project_service resource instance, or potentially taking it from var.project_id if that resource was disabled in this instance of the module. Let's have a look at the google_project_service.project_services definition:
resource "google_project_service" "project_services" {
for_each = local.services
project = var.project_id
service = each.value
disable_on_destroy = var.disable_services_on_destroy
disable_dependent_services = var.disable_dependent_services
}
project here is set to var.project_id, so it seems like either way this innermost project_id output just reflects back the value of the project_id input variable, so we need to jump back up one level and look at the module call to this module to see what that was set to:
module "project_services" {
source = "../project_services"
project_id = google_project.main.project_id
activate_apis = local.activate_apis
activate_api_identities = var.activate_api_identities
disable_services_on_destroy = var.disable_services_on_destroy
disable_dependent_services = var.disable_dependent_services
}
project_id is set to the project_id attribute of google_project.main:
resource "google_project" "main" {
name = var.name
project_id = local.temp_project_id
org_id = local.project_org_id
folder_id = local.project_folder_id
billing_account = var.billing_account
auto_create_network = var.auto_create_network
labels = var.labels
}
project_id here is set to local.temp_project_id, which is declared further up in the same file:
temp_project_id = var.random_project_id ? format(
"%s-%s",
local.base_project_id,
random_id.random_project_id_suffix.hex,
) : local.base_project_id
This expression includes a reference to random_id.random_project_id_suffix.hex, and .hex is a result attribute from random_id, and so its value won't be known until apply time due to how that random_id resource type is implemented. (It generates a random value during the apply step and saves it in the state so it'll stay consistent on future runs.)
This means that (after all of this indirection) module.project-factory.project_id in your module is not a value defined statically in the configuration, and might instead be decided dynamically during the apply step. That means it's not an appropriate value to use as part of the instance key of a resource, and thus not appropriate to use as a key in a for_each map.
Unfortunately the use of for_each here is hidden inside this other module terraform-google-modules/service-accounts/google, and so we'll need to have a look at that one too and see how it's making use of the project_roles input variable. First, let's look at the specific resource block the error message was talking about:
resource "google_project_iam_member" "project-roles" {
for_each = local.project_roles_map_data
project = element(
split(
"=>",
each.value.role
),
0,
)
role = element(
split(
"=>",
each.value.role
),
1,
)
member = "serviceAccount:${google_service_account.service_accounts[each.value.name].email}"
}
There's a couple somewhat-complex things going on here, but the most relevant thing for what we're looking at here is that this resource configuration is creating multiple instances based on the content of local.project_roles_map_data. Let's look at local.project_roles_map_data now:
project_roles_map_data = zipmap(
[for pair in local.name_role_pairs : "${pair[0]}-${pair[1]}"],
[for pair in local.name_role_pairs : {
name = pair[0]
role = pair[1]
}]
)
A little more complexity here that isn't super important to what we're looking for; the main thing to consider here is that this is constructing a map whose keys are built from element zero and element one of local.name_role_pairs, which is declared directly above, along with local.names that it refers to:
names = toset(var.names)
name_role_pairs = setproduct(local.names, toset(var.project_roles))
So what we've learned here is that the values in var.names and the values in var.project_roles both contribute to the keys of the for_each on that resource, which means that neither of those variable values should contain anything decided dynamically during the apply step.
However, we've also learned (above) that the project and role arguments of google_project_iam_member.project-roles are derived from the prefixes of elements in the two lists you provided as names and project_roles in your own module call.
Let's return back to where we started then, with all of this extra information in mind:
module "service_accounts" {
source = "terraform-google-modules/service-accounts/google"
version = "4.0.3"
project_id = module.project-factory.project_id
generate_keys = "true"
names = ["backend-runner"]
project_roles = [
"${module.project-factory.project_id}=>roles/cloudsql.client",
"${module.project-factory.project_id}=>roles/pubsub.publisher"
]
}
We've learned that names and project_roles must both contain only static values decided in the configuration, and so it isn't appropriate to use module.project-factory.project_id because that won't be known until the random project ID has been generated during the apply step.
However, we also know that this module is expecting the prefix of each item in project_roles (the part before the =>) to be a valid project ID, so there isn't any other value that would be reasonable to use there.
Therefore we're at a bit of an empasse: this second module has a rather awkward design decision that it's trying to derive a both a local instance key and a reference to a real remote object from the same value, and those two situations have conflicting requirements. But this isn't a module you created, so you can't easily modify it to address that design quirk.
Given that, I see two possible approaches to move forward, neither ideal but both workable with some caveats:
You could take the approach the error message offered as a workaround, asking Terraform to plan and apply the resources in the first module alone first, and then plan and apply the rest on a subsequent run once the project ID is already decided and recorded in the state:
terraform apply -target=module.factory
terraform apply
Although it's annoying to have to do this initial create in two steps, it does at least only matter for the initial creation of this infrastructure. If you update it later then you won't need to repeat this two-step process unless you've changed the configuration in a way that requires generating a new project ID.
While working through the above we saw that this approach of generating and returning a random project ID was optional based on that first module's var.random_project_id, which you set to "true" in your configuration. Without that, the project_id output would be just a copy of your given name argument, which seems to be statically defined by reference to a root module variable.
Unless you particularly need that random suffix on your project ID, you could leave random_project_id unset and thus just get the project ID set to the same static value as your var.project_name, which should then be an acceptable value to use as a for_each key.
Ideally this second module would be designed to separate the values it's using for instance keys from the values it's using to refer to real remote objects, and thus it would be possible to use the random-suffixed name for the remote object but a statically-defined name for the local object. If this were a module under your control then I would've suggested a design change like that, but I assume the current unusual design of that third-party module (packing multiple values into a single string with a delimiter) is a compromise resulting from wanting to retain backward compatibility with an earlier iteration of the module.
I am using Terraform to build Fortigate resources within a custom child module I've defined. Each instance of the child module will build 12 unique VIPs using the fortios_firewall_vip resource.
Inside the root module, I'm attempting to define a VIP Group using the fortios_firewall_vipgrp resource, which would include ALL of the VIPs built from ALL instances of the child module. I seem to be struggling with how to build the member{} block within the vipgrp resource.
Here is a breakdown of the folder structure:
root/
| _
fortinet.tf
modules/instance
| _
main.tf
outputs.tf
The main.tf file is where I'm building the 12 unique VIPs for each instance, and the outputs.tf file is where I'm saving all of the VIP names to reference in the fortinet.tf root file.
I've tried to save the VIP names as a list and as a map, but I'm not sure which is the better approach.
Example of list output:
output "vips" {
value = [
fortios_firewall_vip.vip1.name,
fortios_firewall_vip.vip2.name,
etc...
]
}
Example of map output:
output "vips" {
value = {
name = fortios_firewall_vip.vip1.name
name = fortios_firewall_vip.vip2.name
etc...
}
}
In the root fortinet.tf file, I'm defining a single VIP Group with the member block and trying to "splat" all instances VIPs in there.
I thought perhaps with the map outputs, I could simply merge them all into one giant block like so:
resource "fortios_firewall_vipgrp" "vipgrp" {
name = "example-vipgrp"
member = merge(module.instance[*].vips)
}
However, I'm receiving an error sayingAn argument named "member" is not expected here. Did you mean to define a block of type "member"?
I'm a newb with Terraform, so I'm struggling with what terminology I should be Googling for assistance here... Is this even possible?
Any input would be extremely helpful. Thanks!
member is a block, not argument. So it should be:
resource "fortios_firewall_vipgrp" "vipgrp" {
name = "example-vipgrp"
member {
# this merge here probably also need changing,
# but my answer covers your error of missing member block,
# not whether your merge is correct or not.
name = merge(module.instance[*].vips)
}
}
Original reference - Quit condition on Terraform blueprint
Is it still possible to make conditional check like in the above question
resource "null_resource" "condition_checker" {
count = "${var.variable == 1 ? 0 : 1}"
"Insert your custom error message" = true
}
Similar format does not work in terraform 0.12 and 0.13 and I could not find any reference to removal of this feature. Is it possible to make a check like this 0.12 or 0.13?
Currently it is still not possible to validate inputs that require access to more than a variable. (The validation block only allows access to the validated variable.)
A hacky validation is still possible using the external data source:
data "external" "check_valid" {
count = var.to_test == true && some_other_condition ? 1 : 0
program = ["sh", "-c", ">&2 echo Condition must be satisfied when to_test is true; exit 1"]
}
This condition is checked before terraform asks for approval of a plan.
On the output it looks like this:
Error: failed to execute "sh": Condition must be satisfied when to_test is true
on variables.tf line 1, in data "external" "check_valid":
1: data "external" "check_valid" {
What you're referring to here was never an actual Terraform feature, but rather an example of exploiting a bug in an earlier version of Terraform to get a result that Terraform had no explicit support for.
With that said, modern versions of Terraform have support for custom variable validation rules which allow you to write out variable validation checks directly inside the corresponding variable block. For example:
variable "variable" {
type = number
validation {
condition = var.variable == 1
error_message = "Variable value must always be 1."
}
}
With that said, I just copied your contrived example from the question here, so this would require some adaptation for a real example. Note also that variable validation rules can only depend on the variable value and other constants, so you can't use this for more complicated checks such as those which involve two different variables. For that sort of situation, I'd recommend refactoring so that the values that are related arrive in a single variable of a object type, and then the validation can be for whether that object is valid.
In my lambda.tf, I have a data resource
data "template_file" "handler" {
template = "${file("${path.module}/templates/handler.js")}"
vars = {
ENDPOINT = "${var.domain}"
PASSWORD = "${var.password}"
}
}
However - I'm encountering a syntax error:
Error: failed to render : <template_file>:280,49-50: Extra characters after interpolation expression; Expected a closing brace to end the interpolation expression, but found extra characters.
on ../docs/lambda.tf line 1, in data "template_file" "handler":
1: data "template_file" "handler" {
Is interpolation inside an interpolation allowed for Terraform? If so - any suggestions on pointing towards where the error is would be greatly appreciated.
Terraform v0.12.9.
Provider "aws" version "~> 2.7"
Not exactly clear what your template file looks like or what you are trying to do, so here are a couple different answers.
You can escape interpolation with double dollar signs: $${foo} will be rendered as a literal ${foo}.
Terraform does not allow dynamic construction of variable names, because it needs to be able to analyze the configuration statically (that is, without evaluating any expressions) in order to determine which order the expressions must be resolved in.
Terraform supports a map data structure that can be used to achieve this effect.
variable "var1" {
default = "value1"
}
variable "var2" {
default = "value2"
}
locals {
var3 = "${var.var1}_${var.var2}"
values = {
"value1_value2" = "local1"
"value2_value3" = "local2"
"value3_value4" = "local3"
}
}
output "val_output" {
value = "${local.values[local.var3]}"
}
If neither is what you are looking for, you need to share your template file or a modified version that duplicates the error.
The template_file data source continues to exist for users of Terraform 0.11 and earlier, but since you are using a Terraform 0.12 release I'd recommend using the templatefile function instead. Because it's built directly into Terraform, it is able to produce better error messages.
To use it, you can replace your references to data.template_file.handler.rendered with a direct call to templatefile. If you are using that rendered result in multiple locations, you can assign the templatefile result to a local value and reference that in multiple places instead.
templatefile("${path.module}/templates/handler.js", {
ENDPOINT = var.domain
PASSWORD = var.password
})
The error message you saw suggests that there's a syntax error in your template itself, but because template_file is implemented in a separate provider it's reporting that syntax error in an unhelpful way, pointing to a particular source location but not including the relevant source code snippet.
If you use templatefile instead, Terraform can hopefully report this syntax error itself and give better information about it.
Either way, it seems like the syntax error is on line 280 column 49 of your handler.js file and is caused by Terraform's template engine expecting to find the } to close a ${ ... } interpolation sequence but finding something else instead. If you correct that syntax error, template rendering should succeed by either approach.