I had a flat structure of all my .tf files and want to migrate to a folder (i.e. module) based set up so that my code is clearer.
For example I have moved my instance and elastic IP (eip) definitions in separate folders
/terraform
../instance
../instance.tf
../eip
../eip.tf
In my instance.tf:
resource "aws_instance" "rancher-node-production" {}
In my eip.tf:
module "instance" {
source = "../instance"
}
resource "aws_eip" "rancher-node-production-eip" {
instance = "${module.instance.rancher-node-production.id}"
However when running terraform plan:
Error: resource 'aws_eip.rancher-node-production-eip' config: "rancher-node-production.id" is not a valid output for module "instance"
Think of modules as black boxes that you can't "reach" into. To get data out of a module, that module needs to export that data with an output. So in your case, you need to declare the rancher-node-production id as an output of the instance module.
If you look at the error you're getting, that's exactly what it's saying: rancher-node-production.id is not a valid output of the module (because you never defined it as an output).
Anyway here's what it would look like.
# instance.tf
resource "aws_instance" "rancher-node-production" {}
output "rancher-node-production" {
value = {
id = "${aws_instance.rancher-node-production.id}"
}
}
Hope that fixes it for you.
Related
I have the following structure
module "policies" {
source = "../../../../path/to/my/custom/modules/groups"
for_each = var.config.policies
name = each.key
policy = each.value
}
module "groups" {
source = "../../../../path/to/my/custom/modules/groups"
for_each = var.config.groups
name = each.key
type = each.value.type
policies = each.value.policies
depends_on = [
module.policies
]
}
Policies and groups are declared in a yaml file from which through yamldecode the corresponding variables to for_each are created.
Is there any way to make sure that the policies passed to policies = each.value.policies of the groups module DO exist?
I mean, OK I have the depends_on clause, but I want to also provision for typos in the yaml file and other similar situations.
The usual way to declare a dependency on an external object (managed elsewhere) in Terraform is to use a data block using a data source defined by the provider responsible for that object. If the goal is only to verify that the object exists then it's enough to declare the data source and then have your downstream object's configuration refer to anything about its result, just so Terraform can see that the data source is a dependency and so should be resolved first.
Unfortunately it seems like the hashicorp/vault provider doesn't currently have a data source for declaring a dependency on a policy, although there is a feature request for it.
Assuming that it did exist then the pattern might look something like this:
data "vault_policy" "needed" {
for_each = var.config.policies
name = each.value
}
module "policies" {
source = "../../../../path/to/my/custom/modules/groups"
for_each = var.config.policies
name = each.key
# Accessing this indirectly via the data resource tells
# Terraform that it must complete the data lookup before
# planning anything which depends on this "policy" argument.
policy = data.vault_policy.needed[each.key].name
}
Without a data source for this particular object type I don't think there will be an elegant way to solve this, but you may be able to work around it by using a more general data source like hashicorp/external's external data source for collecting data by running an external program that prints JSON.
Again because you don't actually seem to need any specific data from the policy and only want to check whether it exists, it would be sufficient to write an external program which queries vault and then exists with an unsuccessful status if the request fails, or prints an empty JSON object {} if the request succeeds.
data "external" "vault_policy" {
for_each = var.config.policies
program = ["${path.module}/query-vault"]
query = {
policy_name = each.value
}
}
module "policies" {
source = "../../../../path/to/my/custom/modules/groups"
for_each = var.config.policies
name = each.key
policy = data.external.vault_policy.query.policy_name
}
I'm not familiar enough with Vault to suggest a specific implementation of this query-vault program, but you may be able to use a shell script wrapping the vault CLI program if you follow the advice in Processing JSON in shell scripts. You only need to do the input parsing part of that, because your result would be communicated either by exit 1 to signal failure or echo '{}' followed by exiting successfully to signal success.
I have a terraform plan that defines most of my BQ environment.
I'm working on a cross-region deployment which will replicate some of my tables to multiple regions.
Rather than copy pasting the same module in every place that I need it at, I'd like to define the module in one place and just call that on every configuration that needs it.
Example I have the following file structure
./cross_region_tables
-> tables.tf
./foo
-> tables.tf
./bar
-> tables.tf
I'd like to define some_module in ./cross_region_tables/tables.tf like so
output "some_module" {
x = something
region = var.region
}
Then I'd like just call some_module from ./foo/tables.tf
The problem is that I don't know how to call this specific module, since ./cross_region_tables/tables.tf will contain several table definitions (as output objects). I know how to import a child module, but I don't know how to call a specific output within that child module
I've solved the issue by adding a module object to the child module with a variable for the region, then calling the child from each regional configuration and passing the region as a variable.
in child folder main.tf:
variable "region" = {}
module "foo" {
x = "something"
y = "something_else"
region = var.region
}
in regional folder for regionX
variable "region" = {
default = regionX
}
module "child" {
source = "../path/to/child"
region = var.region
}
in regional folder for regionY
variable "region" = {
default = regionY
}
module "child" {
source = "../path/to/child"
region = var.region
}
repeat for as many regions as necessary.
You can pass the provider to your modules and each provider with a different
region...
That is well documented here:
https://www.terraform.io/language/modules/develop/providers#passing-providers-explicitly
# The default "aws" configuration is used for AWS resources in the root
# module where no explicit provider instance is selected.
provider "aws" {
region = "us-west-1"
}
# An alternate configuration is also defined for a different
# region, using the alias "usw2".
provider "aws" {
alias = "usw2"
region = "us-west-2"
}
# An example child module is instantiated with the alternate configuration,
# so any AWS resources it defines will use the us-west-2 region.
module "example" {
source = "./example"
providers = {
aws = aws.usw2
}
}
The other part is what you mentioned:
The problem is that I don't know how to call this specific module, since ./cross_region_tables/tables.tf will contain several table definitions
Resources within that module (cross_region_tables) can be turned off/on with variables
I have the following state, with an inital vpc module. I am trying to pass the vpc id to an instance module to separate out the infrastructure.
terraform state list
module.my-vpc.aws_default_route_table.clear-default
module.my-vpc.aws_internet_gateway.gw
module.my-vpc.aws_route_table.rt
module.my-vpc.aws_route_table_association.a
module.my-vpc.aws_route_table_association.b
module.my-vpc.aws_subnet.UAT-vpc-subnet1
module.my-vpc.aws_subnet.UAT-vpc-subnet2
module.my-vpc.aws_vpc.UAT-vpc
Created output - within the root/main tf calling the modules.
output "vpc_id" {
value = module.my-vpc ----- attributes do not sem to work?
}
To pass this on to another module -
module "app" {
source = "./modules/app/"
vpc_id = module.my-vpc.vpc_id
}
Can not get this to work?
I believe this is the correct output format, can anyone enlighten me otherwise please?
You need first to make output from within your vpc module, and you can't use module keyword there.
So what you need to do is reference the source directly there. For example :
output "vpc_id" {
value = aws_vpc.my_vpc.id
}
Since you are writing this as a part of the vpc module you will have access to vpc id with similar command to this.
Then you can reference that output inside your other module like you wrote :
module "app" {
source = "./modules/app/"
vpc_id = module.my-vpc.vpc_id
}
I'm creating a terraform module which is working fine. However, when I use it multiple times it creates multiples roles and policies which are literally the same.
I'm thinking if there is a way for the module to create a role when I call it for the first time and keep using the same role for the subsequent modules
No, Terraform does not support this. Your best bet is to create the shared resources outside the module (or in a separate module), and then pass them in as input arguments into the module you're creating multiple times.
I like the approach of having a module for "shared" resources, because then you can pass that entire module in as an input argument into any module that uses those shared resources.
EDIT: Sample code for shared modules:
main.tf
module "mod1" {
source = "./mymodule1"
}
module "mod2" {
source = "./mymodule2"
input_module = module.mod1
}
output "mod2" {
value = module.mod2
}
mymodule1/main.tf
output "some_field" {
value = "foo"
}
mymodule2/main.tf
variable "input_module" {}
output "module_that_was_input" {
value = var.input_module
}
Result:
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
mod2 = {
"module_that_was_input" = {
"some_field" = "foo"
}
}
I have a terraform project I am working on. In it, I want a file to contain many variables. I want these variables to be accessible from any module of the project. I have looked in the docs and on a udemy course but still don't see how to do this. How does one do this in terraform? Thanks!
I don't think this is possible. There are several discussions about this at Github, but this is not something the Hashicorp team wants.
In general we're against the particular solution of Global Variables, since it makes the input -> resources -> output flow of Modules less explicit, and explicitness is a core design goal.
I know, we have to repeat a lot of variables between different modules
I try to do this
Define a the set of common variables e.g. common_tags at the bottom/top of the variables.tf file for all modules. Usually, your tech ops admin/cloud admin will have a template project created for this.
For each module, I add the following as the last item
global_var = var.global_var_val
An example is common tags. In any root/child module I combine them using merge() function. I hope that makes sense.
You can use -var-file to pass the overrides. Example:
terraform -chdir=staging/000base apply -var-file ../../global.tfvars
Other way can be dedicated shared module.
For example you have module with:
output "my_var" {
value = "apollo440"
}
And in other modules you can use it like:
module "gvars" {
# source = "git#github.com:user/terraform-global-vars.git?ref=v0.0.1-alpha"
source="../../../modules/global_vars"
}
provider "something" {
name="${module.gvars.my_var}"
}
Third way could be to use https://registry.terraform.io/providers/hashicorp/http/latest/docs/data-sources/http to query some http endpoint.
Ouh... and forth way could be to utilize Vault Provider https://registry.terraform.io/providers/hashicorp/vault/latest/docs
For pathing constants between lots of modules we can use the following simple way.
/modules/global_constants/outputs.tf:
Describe module with global constants as outputs:
output "parameter_1" {
value = "value_1"
sensitive = true
}
output "parameter_2" {
value = "value_2"
sensitive = true
}
/example_1.tf
After we can use in any *.tf
module "global_settings" {
source = "./modules/global_constants"
}
data "azurerm_key_vault" "keyvault" {
name = module.global_settings.parameter_1
resource_group_name = module.global_settings.parameter_2
}
/modules/module2/main.tf
Or in other any modules:
module "global_settings" {
source = "../global_constants"
}
data "azurerm_key_vault" "keyvault" {
name = module.global_settings.parameter_1
resource_group_name = module.global_settings.parameter_2
}