I inherited a codebase with all providers stored inside modules and am having a lot of trouble moving the providers out so that I can remove the resources created from modules.
The current design violates the rules outlined here: https://www.terraform.io/docs/configuration/providers.html and makes removing modules impossible.
My understanding of the migration steps is:
Create a provider for use at the top-level
Update module resources to use providers stored outside of the module
Remove module (with top-level provider persisting)
Example module
An example /route53-alias-record/main.ts is:
variable "evaluate_target_health" {
default = true
}
data "terraform_remote_state" "env" {
backend = "s3"
config = {
bucket = "<bucket>"
key = "infra-${var.environment}-${var.target}.tfstate"
region = "<region>"
}
}
provider "aws" {
region = data.terraform_remote_state.env.outputs.region
allowed_account_ids = data.terraform_remote_state.env.outputs.allowed_accounts
assume_role {
role_arn = data.terraform_remote_state.env.outputs.aws_account_role
}
}
resource "aws_route53_record" "alias" {
zone_id = data.terraform_remote_state.env.outputs.public_zone_id
name = var.fqdn
type = "A"
alias {
name = var.alias_name
zone_id = var.zone_id
evaluate_target_health = var.evaluate_target_health
}
}
Starting usage
module "api-dns-alias" {
source = "../environment/infra/modules/route53-alias-record"
environment = "${var.environment}"
zone_id = "${module.service.lb_zone_id}"
alias_name = "${module.service.lb_dns_name}"
fqdn = "${var.environment}.example.com"
}
Provider overriding
## Same as inside module
provider "aws" {
region = data.terraform_remote_state.env.outputs.region
allowed_account_ids = data.terraform_remote_state.env.outputs.allowed_accounts
assume_role {
role_arn = data.terraform_remote_state.env.outputs.aws_account_role
}
}
module "api-dns-alias" {
source = "../environment/infra/modules/route53-alias-record"
environment = "${var.environment}"
zone_id = "${module.service.lb_zone_id}"
alias_name = "${module.service.lb_dns_name}"
fqdn = "${var.environment}.example.com"
providers = {
aws = aws ## <-- pass in explicitly
}
}
I was able to safely deploy with the providers set, but I do not believe that they are being used inside the module, which means the handshake still fails when I remove the module and the resources cannot be deleted.
I am looking for the steps needed to migrate to an outside provider so that I can safely remove resources.
I am currently working with terraform 0.12.24
Related
I have deployed a cloud run application for currently two domains with a load balancer, which is already running. Now this setup needs to be rolled out to other domains. Because the resource setup is always the same, I face some issues:
I want to prevent repeating code (which is managed through a for_each)
Still there are some domain-specific values to cover, which i tried through a mapping table
Referencing resources, which are created with for_each in another resource
The first issue I solved like this, which seems to work:
Old:
resource "google_cloud_run_service" "cr_domain1" {
name = "cr-domain1"
location = "europe-west6"
project = "my_project"
template {
...
}
}
resource "google_cloud_run_service" "cr_domain2" {
name = "cr-domain2"
location = "europe-southwest1"
project = "my_project"
template {
...
}
}
New:
resource "google_cloud_run_service" "cr" {
for_each = toset( ["domain1", "domain2"] )
name = "cr-${each_key}"
location = "tdb" # This is my second issue
project = "my_project"
template {
...
}
}
Regarding second issue I still need domain-specific location setup, which I tried to solve like this, but I am getting errors:
variable "cr_location" {
type = list(object({
domain1 = string
domain2 = string
}))
default = [{
domain1 = "europe-west6"
domain2 = "europe-southwest1"
}]
}
resource "google_cloud_run_service" "cr" {
for_each = toset( ["domain1", "domain2"] )
name = "cr-${each_key}"
location = "${var.cr_location[0]}.${each.key}"
project = "my_project"
template {
...
}
}
Error is "Cannot include the given value in a string template: string required". But I have already declared it as a string in my variable "cr_location". Any idea what's the issue here? The expected output should be:
location = "europe-west6" # For domain1
location = "europe-southwest1" # For domain2
Also regarding issue 3 I do not understand how to referencing resources, which are created with for_each in another resource. So before my for_each in the cloud run resource block (see issue 1) I had this 2 resources:
resource "google_cloud_run_service" "cr_domain1"
resource "google_cloud_run_service" "cr_domain2"
Now I only have resource "google_cloud_run_service" "cr". But in my loadbalancer.tf I still have to references to the old namings (last coderow within "service"):
resource "google_compute_region_network_endpoint_group" "backendneg" {
for_each = toset( ["domain1", "domain2"] )
name = "backendneg-${each.key}"
project = "my_project"
network_endpoint_type = "SERVERLESS"
region = "${var.cr_location[0]}.${each.key}" # Here same issues as issue 2
cloud_run {
service = google_cloud_run_service.cr_domain1.name # Old reference
}
}
So if there is no "cr_domain1" anymore how do I reference to this resource? My issue is that I have to create over 20 resources like that and I couldn't figure it out how to do it. I appreciate any guideline here.
What I would suggest here is to try and refactor the variable because it is making a lot of things harder than they should be. So I would go for this kind of a variable definition:
variable "cr_location" {
type = map(string)
default = {
domain1 = "europe-west6"
domain2 = "europe-southwest1"
}
}
Then, the rest should be easy to create:
resource "google_cloud_run_service" "cr" {
for_each = var.cr_location
name = "cr-${each.key}"
location = each.value
project = "my_project"
template {
...
}
}
And for the network endpoint resource:
resource "google_compute_region_network_endpoint_group" "backendneg" {
for_each = var.cr_location
name = "backendneg-${each.key}"
project = "my_project"
network_endpoint_type = "SERVERLESS"
region = each.value
cloud_run {
service = google_cloud_run_service.cr[each.key].name
}
}
You could even try resource chaining with for_each [1] to make sure you are doing this for all the Cloud Run resources created:
resource "google_compute_region_network_endpoint_group" "backendneg" {
for_each = google_cloud_run_service.cr
name = "backendneg-${each.key}"
project = "my_project"
network_endpoint_type = "SERVERLESS"
region = each.value.location
cloud_run {
service = each.value.name
}
}
[1] https://www.terraform.io/language/meta-arguments/for_each#chaining-for_each-between-resources
I have a list of objects in Terraform called users and the structure is the following:
variable "accounts" {
type = list(object({
id = string #used in assume_role
email = string
is_enabled = bool
}))
}
What I am trying to achieve now is to create a simple S3 bucket in each of those AWS accounts (if the is_enabled is true). I was able to do it for a single account but I am not sure if there is a way to loop over a provider?
Code for a single account - main.tf
provider "aws" {
alias = "new_account"
region = "eu-west-3"
assume_role {
role_arn = "arn:aws:iam::${aws_organizations_account.this.id}:role/OrganizationAccountAccessRole"
session_name = "new_account_creation"
}
}
resource "aws_s3_bucket" "bucket" {
provider = aws.new_account
bucket = "new-account-bucket-${aws_organizations_account.this.id}"
acl = "private"
}
You need to define one provider for each aws account you want to use:
Create a module (i.e. a directory), where your bucket configuration lives:
├── main.tf
└── module
└── bucket.tf
bucket.tf should contain the resource definition: resource "aws_s3_bucket" "bucket" {...}
In main.tf , define multiple aws providers and call the module with each of them:
provider "aws" {
alias = "account1"
region = "eu-west-1"
...
}
provider "aws" {
alias = "account2"
region = "us-west-1"
...
}
module "my_module" {
source = "./module"
providers = {
aws.account1 = aws.account1
aws.account2 = aws.account2
}
}
I guess you could also get fancy by creating a variable containing the providers, and pass it to the module invocation (you could probably also use a filter on the list to take into account the is_enabled flag)
More details about providers: https://www.terraform.io/docs/language/modules/develop/providers.html
Found what I was looking for here: https://github.com/hashicorp/terraform/issues/19932
Thanks Bryan Karaffa
## Just some data... a list(map())
locals {
aws_accounts = [
{ "aws_account_id": "123456789012", "foo_value": "foo", "bar_value": "bar" },
{ "aws_account_id": "987654321098", "foo_value": "foofoo", "bar_value": "barbar" },
]
}
## Here's the proposed magic... `provider.for_each`
provider "aws" {
for_each = local.aws_accounts
alias = each.value.aws_account_id
assume_role {
role_arn = "arn:aws:iam::${each.value.aws_account_id}:role/TerraformAccessRole"
}
}
## Modules reference the provider dynamically using `each.value.aws_account_id`
module "foo" {
source = "./foo"
for_each = local.aws_accounts
providers = {
aws = "aws.${each.value.aws_account_id}"
}
foo = each.value.foo_value
}
module "bar" {
source = "./bar"
for_each = local.aws_accounts
providers = {
aws = "aws.${each.value.aws_account_id}"
}
bar = each.value.bar_value
}
Issue summary:
Providers not being passed to submodule
Issue description:
Hello,
I'm trying to pass providers to a submodule module from the root as recommended my Hashicorp, especially now as I need to loop through the root module, using for_each. However I'm getting an error that indicates that the submodule isn't getting the provider passed down to it.
Does anyone have any guidance on what I'm doing wrong?
Thank you for your time
error:
Error: missing provider
module.vpc_peering.provider["registry.terraform.io/hashicorp/aws"].requester
code:
main.tf
# Requestors's credentials
provider "aws" {
alias = "requester"
region = var.aws_region
assume_role {
role_arn = local.workspace_role_arn_requester
}
}
# Accepter's credentials
provider "aws" {
alias = "accepter"
region = var.aws_region
assume_role {
role_arn = local.workspace_role_arn_accepter
}
}
#################################################
# VPC peer from Admin to Current
#################################################
module "vpc_peering" {
for_each = toset(local.accepter_ids)
source = "./modules/peer"
providers = {
aws.requester = aws.requester
aws.accepter = aws.accepter
}
modules/peer/admin-peer.tf
module "vpc_peering_cross_account" {
source = "git::https://github.com/YouLend/terraform-aws-vpc-peering-multi-account?ref=aws_profile_accepter_version_0.13"
providers = {
aws.requester = aws.requester
aws.accepter = aws.accepter
}
I got it working but for those that are experiencing the same issue, this comment on github explains what needs to be done
https://github.com/hashicorp/terraform/issues/17399#issuecomment-367342717
essentially you need an empty provider block in each module that intends to pass the providers on so in my above example this code needs to go into modules/peer/admin-peer.tf
provider "aws" {
}
provider "aws" {
alias = "requester"
}
provider "aws" {
alias = "accepter"
}
Running Terraform 0.11.7 and getting the following error:
module.frontend_cfg.var.web_acl: Resource 'data.terraform_remote_state.waf' does not have attribute 'waf_nonprod_id' for variable 'data.terraform_remote_state.waf.waf_nonprod_id'
Below is the terraform file:
module "frontend_cfg"
{
source = "../../../../modules/s3_fe/developers"
region = "us-east-1"
dev_shortname = "cfg"
web_acl = "${data.terraform_remote_state.waf.waf_nonprod_id}"
}
data "terraform_remote_state" "waf" {
backend = "local"
config = {
name = "../../../global/waf/terraform.tfstate"
}
}
The file which creates the tfstate file referenced above is below. This file has had no issues building.
resource "aws_waf_web_acl" "waf_fe_nonprod"
{
name = "fe_nonprod_waf"
metric_name = "fenonprodwaf"
default_action
{
type = "ALLOW"
}
}
output waf_nonprod_id
{
value = "${aws_waf_web_acl.waf_fe_nonprod.id}"
}
I will spare the full output of the cloudfront file, however, the following covers the text:
resource "aws_cloudfront_distribution" "fe_distribution"
{
web_acl_id = "${var.web_acl}"
}
If I put the ID of the waf ID into the web_acl variable, it works just fine, so I suspect the issue is something to do with the way I am calling data. This appears to match documentation though.
Use path instead of name in terraform_remote_state,
https://www.terraform.io/docs/backends/types/local.html
data "terraform_remote_state" "waf" {
backend = "local"
config = {
path = "../../../global/waf/terraform.tfstate"
}
}
or
data "terraform_remote_state" "waf" {
backend = "local"
config = {
path = "${path.module}/../../../global/waf/terraform.tfstate"
}
}
I tested it with terraform version 0.11.7 and 0.11.14
If you upgrade terraform to version 0.12.x, syntax using remote_state ouput has changed.
So change
web_acl = "${data.terraform_remote_state.waf.waf_nonprod_id}"
to
web_acl = data.terraform_remote_state.waf.outputs.waf_nonprod_id
or
web_acl = "${data.terraform_remote_state.waf.outputs.waf_nonprod_id}"
We are trying to create Terraform modules for below activities in AWS, so that we can use them where ever that is required.
VPC creation
Subnets creation
Instance creation etc.
But while creating these modules we have to define the provider in all above listed modules. So we decided to create one more module for provider so that we can call that provider module in other modules (VPC, Subnet, etc.).
Issue in above approach is that it is not taking provider value, and asking for the user input for region.
Terraform configuration is as follow:
$HOME/modules/providers/main.tf
provider "aws" {
region = "${var.region}"
}
$HOME/modules/providers/variables.tf
variable "region" {}
$HOME/modules/vpc/main.tf
module "provider" {
source = "../../modules/providers"
region = "${var.region}"
}
resource "aws_vpc" "vpc" {
cidr_block = "${var.vpc_cidr}"
tags = {
"name" = "${var.environment}_McD_VPC"
}
}
$HOME/modules/vpc/variables.tf
variable "vpc_cidr" {}
variable "environment" {}
variable "region" {}
$HOME/main.tf
module "dev_vpc" {
source = "modules/vpc"
vpc_cidr = "${var.vpc_cidr}"
environment = "${var.environment}"
region = "${var.region}"
}
$HOME/variables.tf
variable "vpc_cidr" {
default = "192.168.0.0/16"
}
variable "environment" {
default = "dev"
}
variable "region" {
default = "ap-south-1"
}
Then when running terraform plan command at $HOME/ location it is not taking provider value and instead asking for the user input for region.
I need help from the Terraform experts, what approach we should follow to address below concerns:
Wrap provider in a Terraform module
Handle multiple region use case using provider module or any other way.
I knew a long time back that it wasn't possible to do this because Terraform built a graph that required a provider for any resource before it included any dependencies and it didn't used to be possible to force a dependency on a module.
However since Terraform 0.8 it is now possible to set a dependency on modules with the following syntax:
module "network" {
# ...
}
resource "aws_instance" "foo" {
# ...
depends_on = ["module.network"]
}
However, if I try that with your setup by changing modules/vpc/main.tf to look something like this:
module "aws_provider" {
source = "../../modules/providers"
region = "${var.region}"
}
resource "aws_vpc" "vpc" {
cidr_block = "${var.vpc_cidr}"
tags = {
"name" = "${var.environment}_McD_VPC"
}
depends_on = ["module.aws_provider"]
}
And run terraform graph | dot -Tpng > graph.png against it it looks like the graph doesn't change at all from when the explicit dependency isn't there.
This seems like it might be a potential bug in the graph building stage in Terraform that should probably be raised as an issue but I don't know the core code base well enough to spot where the change needs to be.
For our usage we use symlinks heavily in our Terraform code base, some of which is historic from before Terraform supported other ways of doing things but could work for you here.
We simply define the provider in a single .tf file (such as environment.tf) along with any other generic config needed for every place you would ever run Terraform (ie not at a module level) and then symlink this into each location. That allows us to define the provider in a single place with overridable variables if necessary.
Step 1
Add region alias in the main.tf file where you gonna execute the terraform plan.
provider "aws" {
region = "eu-west-1"
alias = "main"
}
provider "aws" {
region = "us-east-1"
alias = "useast1"
}
Step 2
Add providers block inside your module definition block
module "lambda_edge_rule" {
providers = {
aws = aws.useast1
}
source = "../../../terraform_modules/lambda"
tags = var.tags
}
Step 3
Define "aws" as providers inside your module. ( source = ../../../terraform_modules/lambda")
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = ">= 2.7.0"
}
}
}
resource "aws_lambda_function" "lambda" {
function_name = "blablabla"
.
.
.
.
.
.
.
}
Note: Terraform version v1.0.5 as of now.