Declaring same module twice: duplicate ressources - terraform

I am working on a 100% terraform project and I am trying to use the output value from one module into another module. Based on different StackOverflow posts, the most popular way to import the output from module a to module b is to reference the module a inside module b such as:
modules/b/main.tf
module "a" {
source = "./modules/a"
}
After that, you can access output variables from module a inside module b.
My project structure
├── main.tf # declaring all my modules here
├── modules
│ ├── accounts
│ │ ├── main.tf
│ │ └── variables.tf
│ └── organizations
│ ├── main.tf
│ ├── outputs.tf # the var. that I wanna use in accounts
│ └── variables.tf
├── providers.tf
├── variables.tf
└── versions.tf
So my issue is I am declaring all my modules in my main.tf
main.tf
module "organizations" {
source = "./modules/organizations"
}
module "accounts" {
source = "./modules/accounts"
}
However, I need to use one output of module/organizations into module/accounts. And the only way I found to do that is to have (another) organizations module in my modules/accounts/main.tf
modules/accounts/main.tf
module "organizations" {
source = "../organizations"
}
resource "aws_organizations_account" "this" {
name = "uuuu"
email = "udduu#gmail.com"
parent_id = module.organizations.sandbox_organizational_unit_id #HERE
}
But since I already have an organizations module in my main.tf, it's creating/deleting resources in my organization module twice.
organisations/main.tf
data "aws_organizations_organization" "root" {}
locals {
root_id = data.aws_organizations_organization.root.roots[0].id
}
resource "aws_organizations_organizational_unit" "sandboxs" {
name = var.aws_sandboxs_unit_name
parent_id = local.root_id
}
organisations/outputs.tf
output "sandbox_organizational_unit_id" {
value = aws_organizations_organizational_unit.sandboxs.id
description = "ID of the Sandboxs OU"
sensitive = false
}

Neither of your modules should explicitely refer to the other one. Instead, they should declare what kind of variable they expect as input (via a variable key), and what output they provide in return (via the output key)
Then in your main.tf, you can plug everything together:
module "organizations" {
source = "./modules/organizations"
some_variable = module.accounts.some_output
}
module "accounts" {
source = "./modules/accounts"
}
in "organizations", a some_variable must be declared as input: variable some_variable {}
in accounts , a some_output must be declared as output: output some_output { value = ... }

Related

How can I use locals defined in terragrunt.hcl in Terraform files?

I've created this folder structure:
.
├── main.tf
└── terragrunt.hcl
# FILE: terragrunt.hcl
include {
path = find_in_parent_folders()
}
locals {
common_vars = read_terragrunt_config(find_in_parent_folders("common.hcl"))
cluster_name = local.common_vars.locals.cluster_name
}
terraform {
source = "./main.tf"
}
# FILE: main.tf
module "tags" {
source = "..."
eks_cluster_names = [local.cluster_name]
}
module "vpc" {
source = "..."
aws_region = local.common_vars.locals.aws_region
...
vpc_custom_tags = module.tags.vpc_eks_tags
...
}
But for every local. I am trying to use I get an error:
A local value with the name "blabla" has not been declared
So now I am trying to figure out a way to make this work. I considered following how-to-access-terragrunt-variables-in-terraform-code, but I didn't want to create a variables.tf. Also, another problem is that I would have to redefine all outputs from modules in main.tf, isn't there a nicer way to do this?
Is there a structure that is a good practice I could follow? How could I "propagate" these locals in terragrunt.hcl to main.tf?
Sorry to disappoint but you do have to create a variables.tf - that is standard terraform. You define the input variables you need in there for your terraform configuration, and in terragrunt you fill these.
So your terragrunt file should look something like:
# FILE: terragrunt.hcl
locals {
common_vars = read_terragrunt_config(find_in_parent_folders("common.hcl"))
cluster_name = local.common_vars.locals.cluster_name
}
terraform {
source = "./main.tf"
}
inputs = {
cluster_name = local.cluster_name
aws_region = local.common_vars.locals.aws_region
}
And your terraform main should look like this:
# FILE: main.tf
module "tags" {
source = "..."
eks_cluster_names = var.cluster_name
}
module "vpc" {
source = "..."
aws_region = var.aws_region
...
vpc_custom_tags = module.tags.vpc_eks_tags
...
}
And your variables.tf would then look like:
variable "aws_region" {
type = string
}
variable "cluster_name" {
type = string
}
Additionally, you probably also need to create a provider.tf and a backend configuration to get this to run.
Terragrunt calls directly TF modules. Meaning get rid of main.tf and use just Terragrunt to wire your modules. There needs to be a separated subfolder (component) with terragrunt.hcl per TF module.
Your project structure will look like this:
.
├── terragrunt.hcl
├── tags
│   └── terragrunt.hcl
└── vpc
   └── terragrunt.hcl
Feel free to have a look at how that works and how the variables are passed across the modules at my example here.

how to get terragrunt to read tfvars files into dependent modules

Anyone know how to get terragrunt to read tfvars files into dependent modules? If I declare all my tfvars as inputs in my root terragrunt.hcl, everything works fine, but of course then I can’t customize them by environment. I tried adding the extra_arguments block, but the variables aren’t declared in the root module. They’re declared in the dependent module and I don’t want to have to declare them in both places.
Here’s my setup:
// terraform/terragrunt.hcl
terraform {
extra_arguments "common_vars" {
commands = ["plan", "apply"]
arguments = [
"-var-file=${find_in_parent_folders("account.tfvars")}",
"-var-file=./terraform.tfvars"
]
}
}
locals {
environment_vars = read_terragrunt_config(find_in_parent_folders("account.hcl"))
bucket = local.environment_vars.locals.bucket
}
remote_state {
backend = "s3"
generate = {
path = "backend.tf"
if_exists = "overwrite_terragrunt"
}
config = {
key = "${path_relative_to_include()}/terraform.tfstate"
region = "us-east-1"
bucket = local.bucket
}
}
dependencies {
paths = ["../../../shared/services", "../../../shared/core"]
}
// terraform/accounts/dev/account.tfvars
aws_region = "us-east-1"
// terraform/accounts/dev/william/terraform.tfvars
aws_vpc_cidr = "10.1.0.0/16"
// terraform/accounts/dev/william/terragrunt.hcl
include {
path = find_in_parent_folders()
}
This doesn't work because the variable values don't actually get passed to the dependent modules. I got this back when I tried to run a terragrunt plan:
''' TERMINAL OUTPUT
$ terragrunt plan
No changes. Infrastructure is up-to-date.
This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.
Warning: Value for undeclared variable
The root module does not declare a variable named
"aws_region" but a value was found in file
"/Users/williamjeffries/code/parachute/infrastructure/terraform/accounts/dev/account.tfvars".
To use this value, add a "variable" block to the configuration.
Using a variables file to set an undeclared variable is deprecated and will
become an error in a future release. If you wish to provide certain "global"
settings to all configurations in your organization, use TF_VAR_...
environment variables to set these instead.
Actually there were 26 such warnings, I’ve only pasted in one here but you get the idea. It seems like there should be some way to solve this with a terragrunt generate block but I'm not sure how. Any ideas?
I have been following the documentation here where it suggested to have directory structure:
live
├── prod
│ ├── app
│ │ └── terragrunt.hcl
│ ├── mysql
│ │ └── terragrunt.hcl
│ └── vpc
│ └── terragrunt.hcl
├── qa
│ ├── app
│ │ └── terragrunt.hcl
etc...
and
# content of qa/app/terragrunt.hcl
terraform {
# Deploy version v0.0.3 in qa
source = "git::git#github.com:foo/modules.git//app?ref=v0.0.3"
}
inputs = {
# tfvars for qa
instance_count = 3
instance_type = "t2.micro"
}
and
# content of prod/app/terragrunt.hcl
terraform {
# Deploy version v0.0.3 in prod
source = "git::git#github.com:foo/modules.git//app?ref=v0.0.3"
}
inputs = {
# tfvars for prod
instance_count = 20
instance_type = "t2.2xlarge"
}
and then the source could be within the same git repo., i.e: just app directory. You could then customize module app by different environment (and even different versions in different environments)

can't declare map variables in child modules in terraform 0.12

I used to have (working) map variables in terraform, but after upgrading to terraform 0.12 I keep getting errors of the form:
Error: Invalid value for module argument
on main.tf line 84, in module "gke":
84: gke_label = "var.gke_label"
The given value is not suitable for child module variable "gke_label" defined
at gke/variables.tf:40,1-19: map of any single type required.
I don't understand how to upgrade these map variables. Documentation on this is not particularly clear (to me).
My set-up is as follows:
I have a terraform folder structure:
├── infrastructure
│   ├── backend
│   │   ├── subnet
│   │   │   ├── main.tf
│   │   │   ├── outputs.tf
│   │   │   └── variables.tf
│   │   └── vpc
│   │   ├── main.tf
│   │   └── outputs.tf
│   ├── backend.tf
│   ├── backend.tfvars
│   ├── gke
│   │   ├── main.tf
│   │   ├── outputs.tf
│   │   └── variables.tf
│   ├── main.tf
│   ├── outputs.tf
│   ├── variables.tf
│   └── versions.tf
within main.tf I had / have (among others):
module "gke" {
source = "./gke"
region = "var.region"
min_master_version = "var.min_master_version"
node_version = "var.node_version"
gke_num_nodes = "var.gke_num_nodes" # [MAP VARIABLE]
vpc_name = "module.vpc.vpc_name"
subnet_name = "module.subnet.subnet_name"
gke_master_user = "var.gke_master_user"
gke_master_pass = "var.gke_master_pass"
gke_node_machine_type = "var.gke_node_machine_type"
gke_label = "var.gke_label" # [MAP VARIABLE]
}
and in variables.tf (among others)
variable "gke_label" {
default = {
prod = "prod"
dev = "dev"
}
variable "gke_num_nodes" {
default = {
prod = 2
dev = 1
}
description = "Number of nodes in each GKE cluster zone"
}
within gke/variables.tf I had:
variable "gke_num_nodes" {
type = map
description = "Number of nodes in each GKE cluster zone"
}
variable gke_label {
type = map
description = "label"
}
This used to work fine, but with the upgrade to terraform 0.12 this results in:
Error: Invalid value for module argument
on main.tf line 78, in module "gke":
78: gke_num_nodes = "var.gke_num_nodes"
The given value is not suitable for child module variable "gke_num_nodes"
defined at gke/variables.tf:15,1-25: map of any single type required.
Error: Invalid value for module argument
on main.tf line 84, in module "gke":
84: gke_label = "var.gke_label"
The given value is not suitable for child module variable "gke_label" defined
at gke/variables.tf:40,1-19: map of any single type required.
I changed in gke/variables.tf (same for num_nodes)
variable gke_label {
type = map(any)
description = "label"
}
but the error remains
Error: Invalid value for module argument
on main.tf line 84, in module "gke":
84: gke_label = "var.gke_label"
The given value is not suitable for child module variable "gke_label" defined
at gke/variables.tf:40,1-19: map of any single type required.
How do I update these map variables to terraform 0.12?
This Terraform 0.12 code will assign the value as expected (not a literal string):
gke_num_nodes = var.gke_num_node
In either Terraform 0.11.x or Terraform 0.12, if you use quotes around your variable assignments without interpolation, they will be treated as strings.
gke_num_nodes = "var.gke_num_node"
The code above will assign the literal string "var.gke_num_node" to gke_num_nodes in the module, instead of assigning the value of var.gke_num_nodes as you intend. Since string is not assignable to map(any), Terraform outputs the type error you presented:
Error: Invalid value for module argument
on main.tf line 78, in module "gke":
78: gke_num_nodes = "var.gke_num_nodes"
In Terraform 0.11.x and earlier, you would use string interpolation with ${} to get the value of a variable:
gke_num_nodes = "${var.gke_num_node}"
That kind of expression is deprecated in Terraform 0.12, but will still work in most cases. Do not use string interpolation in Terraform 0.12 unless you are building a string from multiple variable.
You leapt halfway to Terraform 0.12 by removing the ${}. Leap the remaining gap by removing the quotes so your variable assignments will work as expected:
gke_num_nodes = var.gke_num_node
Here is the entire module block, corrected to remove the quotes:
module "gke" {
source = "./gke"
region = var.region
min_master_version = var.min_master_version
node_version = var.node_version
gke_num_nodes = var.gke_num_node # [MAP VARIABLE]
vpc_name = module.vpc.vpc_name
subnet_name = module.subnet.subnet_name
gke_master_user = var.gke_master_user
gke_master_pass = var.gke_master_pass
gke_node_machine_type = var.gke_node_machine_type
gke_label = var.gke_label # [MAP VARIABLE]
}

Using output in another module or resource

I've seen a good amount of posts that talk about passing a module's output into another module. For some reason I can't get this to work.
I can get the output of the module without any issues
$ terraform output
this_sg_id = sg-xxxxxxxxxxxxxxxxx
However, when I call the module in the resource or into another module, it asks me for the Security group ID.
$ terraform plan
var.vpc_security_group_ids
Security Group ID
Enter a value:
Here's my file structure:
── dev
│ └── service
│ └── dev_instance
│ ├── main.tf
│ ├── outputs.tf
│ ├── variables.tf
├── modules
│ ├── ec2
│ │ ├── build_ec2.tf
│ │ ├── outputs.tf
│ │ └── variables.tf
│ └── sg
│ ├── build_sg.tf
│ ├── outputs.tf
│ └── variables.tf
Not sure if this is correct but in dev/service/dev_instance/main.tf:
module "build_sg" {
source = "../../../modules/sg/"
vpc_id = var.vpc_id
sg_name = var.sg_name
sg_description = var.sg_description
sg_tag = var.sg_tag
sg_tcp_ports = var.sg_tcp_ports
sg_tcp_cidrs = var.sg_tcp_cidrs
sg_udp_ports = var.sg_udp_ports
sg_udp_cidrs = var.sg_udp_cidrs
sg_all_ports = var.sg_all_ports
sg_all_cidrs = var.sg_all_cidrs
}
module "build_ec2" {
source = "../../../modules/ec2/"
vpc_security_group_ids = ["${module.build_sg.this_sg_id}"]
}
In dev/service/dev_instance/output.tf:
output "this_sg_id" {
description = "The security group ID"
value = "${module.build_sg.this_sg_id}"
}
My ec2 module build_ec2.tf file has the following:
resource "aws_instance" "ec2" {
vpc_security_group_ids = ["${module.build_sg.this_sg_id}"]
}
You have a var "vpc_security_group_ids" defined somewhere, I assume in one of your variables.tf files. Terraform doesn't automatically know to fill that in with the output from a module. You need to remove the var definition and just use the module output reference in your template.
Variables are used to pass in values from the command line. They are not tied to module outputs in any way. If you expect values to come from a module you are using then you should not be also defining that value as a variable.
I also think, you need to remove var definition from your variable tf file and use only module output reference.

attach security group created in another app

folder structure.
I am creating the following for 2 seperate applications using same modules in terragrunt
LB
Instances
Security Groups
my question is how do I reference a security group created for app1 in app2?
eg.
in app1
I can references it as
security_groups = ["${aws_security_group.sec_group_A.id}"]
how can I refer the same security group in app2?
resource "aws_security_group" "sec_group_A" {
name = "sec_group_A"
...
...
}
resource "aws_elb" "bar" {
name = "foobar-terraform-elb"
security_groups = ["${aws_security_group.sec_group_A.id}"]
...
...
}
In app2, you can:
data "aws_security_group" "other" {
name = "sec_group_A"
}
and then use the ID:
resource "aws_elb" "bar" {
name = "foobar-terraform-elb"
security_groups = ["${data.aws_security_group.other.id}"]
...
...
}
(caveat for using data is that you are running two separate terraform applys - one configuration creates the group, and other configuration references the group).
I have no experience of using terragrunt, but normally I would be calling my modules from a "main.tf" file in the root of the project. An example folder structure is below
.
├── main.tf
└── modules
├── app1
│ ├── main.tf
│ ├── outputs.tf
│ └── variables.tf
└── app2
├── main.tf
├── outputs.tf
└── variables.tf
My app1 outputs.tf declares a security group A output
output "sec_group_a" { value = "${aws_security_group.sec_group_A}" }
I can then call this output in my main.tf file in the root of the project. This would look something like the below
module "app1" {
source = "./modules/app1"
...
// Pass in my variables
}
module "app2" {
source = "./modules/app2"
sec_group_A = "${module.app1.sec_group_A}"
...
//Pass in the rest of my variables
}
Finally inside of the app2 module you can call this as you would any other variable.
resource "aws_elb" "bar" {
name = "foobar-terraform-elb"
security_groups = ["${var.sec_group_A.id}"]
...
...
}
I'd read up on modules here https://www.terraform.io/docs/modules/index.html to get a better understanding of how they fit together.
Alternatively you can grab the data from your remote state (if you have one configured) as long as sec_group_A declared as an output in app1. See https://www.terraform.io/docs/providers/terraform/d/remote_state.html

Resources