Issue summary:
Providers not being passed to submodule
Issue description:
Hello,
I'm trying to pass providers to a submodule module from the root as recommended my Hashicorp, especially now as I need to loop through the root module, using for_each. However I'm getting an error that indicates that the submodule isn't getting the provider passed down to it.
Does anyone have any guidance on what I'm doing wrong?
Thank you for your time
error:
Error: missing provider
module.vpc_peering.provider["registry.terraform.io/hashicorp/aws"].requester
code:
main.tf
# Requestors's credentials
provider "aws" {
alias = "requester"
region = var.aws_region
assume_role {
role_arn = local.workspace_role_arn_requester
}
}
# Accepter's credentials
provider "aws" {
alias = "accepter"
region = var.aws_region
assume_role {
role_arn = local.workspace_role_arn_accepter
}
}
#################################################
# VPC peer from Admin to Current
#################################################
module "vpc_peering" {
for_each = toset(local.accepter_ids)
source = "./modules/peer"
providers = {
aws.requester = aws.requester
aws.accepter = aws.accepter
}
modules/peer/admin-peer.tf
module "vpc_peering_cross_account" {
source = "git::https://github.com/YouLend/terraform-aws-vpc-peering-multi-account?ref=aws_profile_accepter_version_0.13"
providers = {
aws.requester = aws.requester
aws.accepter = aws.accepter
}
I got it working but for those that are experiencing the same issue, this comment on github explains what needs to be done
https://github.com/hashicorp/terraform/issues/17399#issuecomment-367342717
essentially you need an empty provider block in each module that intends to pass the providers on so in my above example this code needs to go into modules/peer/admin-peer.tf
provider "aws" {
}
provider "aws" {
alias = "requester"
}
provider "aws" {
alias = "accepter"
}
Related
I have created a module I want to use across multiple providers (just two AWS providers for 2 regions). How can I set a resource's provider value via variable from a calling module? I am calling a module codebuild.tf (which I want to be region agnostic) from a MGMT module named cicd.tf - Folder structure:
main.tf
/MGMT/
-> cicd.tf
/modules/codebuild/
-> codebuild.tf
main.tf:
terraform {
required_version = ">= 1.0.10"
backend "s3" {
}
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 3.0"
}
}
}
# default AWS provider for MGMT resources in us-east-1 and global
provider "aws" {
region = "us-east-1"
}
# DEV Account resources in us-east-1 and global
provider "aws" {
region = "us-east-1"
assume_role {
role_arn = "arn:aws:iam::accountid:role/dev-rolename"
}
alias = "dev_us-east-1"
}
# DEV Account resources in us-west-2 and global
provider "aws" {
region = "us-west-2"
assume_role {
role_arn = "arn:aws:iam::accountid:role/dev-rolename"
}
alias = "dev_us-west-2"
}
module "MGMT" {
source = "./MGMT"
count = var.aws_env == "MGMT" ? 1 : 0
aws_env = var.aws_env
}
When I build my TF, its under the MGMT AWS account which uses the the default aws provider that doesn't have an alias - I am then trying to set a provider with an AWS IAM Role (that's cross account) when I am calling the module (I made the resource a module because I want to run it in multiple regions):
/MGMT/cicd.tf:
# DEV in cicd.tf
# create the codebuild resource in the assumed role's us-east-1 region
module "dev_cicd_iac_us_east_1" {
source = "../modules/codebuild/"
input_aws_provider = "aws.dev_us-east-1"
input_aws_env = var.dev_aws_env
}
# create the codebuild resource in the assumed role's us-west-2 region
module "dev_cicd_iac_us_west_2" {
source = "../modules/codebuild/"
input_aws_provider = "aws.dev_us-west_2"
input_aws_env = var.dev_aws_env
}
/modules/codebuild/codebuild.tf:
# Code Build resource here
variable "input_aws_provider" {}
variable "input_aws_env" {}
resource "aws_codebuild_project" "codebuild-iac" {
provider = tostring(var.input_aws_provider) # trying to make it a string, with just the var there it looks for a var provider
name = "${var.input_aws_env}-CodeBuild-IaC"
# etc...
}
I get the following error when I plan the above:
│ Error: Invalid provider reference
│ On modules/codebuild/codebuild.tf line 25: Provider argument requires
│ a provider name followed by an optional alias, like "aws.foo".
How can I make the provider value a proper reference to the aws provider defined in main.tf while still using a MGMT folder/module file named cicd.tf?
I want to use the resource "data" in Terraform for example for an sns topic but I don't want too look for a resource in the aws-account, for which I'm deploying my other resources. It should look up to my other aws-account (in the same organization) and find resources in there. Is there a way to make this happen?
data "aws_sns_topic" "topic_alarms_data" {
name = "topic_alarms"
}
Define an aws provider with credentials to the remote account:
# Default provider that you use:
provider "aws" {
region = var.context.aws_region
assume_role {
role_arn = format("arn:aws:iam::%s:role/TerraformRole", var.account_id)
}
}
provider "aws" {
alias = "remote"
region = var.context.aws_region
assume_role {
role_arn = format("arn:aws:iam::%s:role/TerraformRole", var.remote_account_id)
}
}
data "aws_sns_topic" "topic_alarms_data" {
provider = aws.remote
name = "topic_alarms"
}
Now the topics are loaded from the second provider.
I have a list of objects in Terraform called users and the structure is the following:
variable "accounts" {
type = list(object({
id = string #used in assume_role
email = string
is_enabled = bool
}))
}
What I am trying to achieve now is to create a simple S3 bucket in each of those AWS accounts (if the is_enabled is true). I was able to do it for a single account but I am not sure if there is a way to loop over a provider?
Code for a single account - main.tf
provider "aws" {
alias = "new_account"
region = "eu-west-3"
assume_role {
role_arn = "arn:aws:iam::${aws_organizations_account.this.id}:role/OrganizationAccountAccessRole"
session_name = "new_account_creation"
}
}
resource "aws_s3_bucket" "bucket" {
provider = aws.new_account
bucket = "new-account-bucket-${aws_organizations_account.this.id}"
acl = "private"
}
You need to define one provider for each aws account you want to use:
Create a module (i.e. a directory), where your bucket configuration lives:
├── main.tf
└── module
└── bucket.tf
bucket.tf should contain the resource definition: resource "aws_s3_bucket" "bucket" {...}
In main.tf , define multiple aws providers and call the module with each of them:
provider "aws" {
alias = "account1"
region = "eu-west-1"
...
}
provider "aws" {
alias = "account2"
region = "us-west-1"
...
}
module "my_module" {
source = "./module"
providers = {
aws.account1 = aws.account1
aws.account2 = aws.account2
}
}
I guess you could also get fancy by creating a variable containing the providers, and pass it to the module invocation (you could probably also use a filter on the list to take into account the is_enabled flag)
More details about providers: https://www.terraform.io/docs/language/modules/develop/providers.html
Found what I was looking for here: https://github.com/hashicorp/terraform/issues/19932
Thanks Bryan Karaffa
## Just some data... a list(map())
locals {
aws_accounts = [
{ "aws_account_id": "123456789012", "foo_value": "foo", "bar_value": "bar" },
{ "aws_account_id": "987654321098", "foo_value": "foofoo", "bar_value": "barbar" },
]
}
## Here's the proposed magic... `provider.for_each`
provider "aws" {
for_each = local.aws_accounts
alias = each.value.aws_account_id
assume_role {
role_arn = "arn:aws:iam::${each.value.aws_account_id}:role/TerraformAccessRole"
}
}
## Modules reference the provider dynamically using `each.value.aws_account_id`
module "foo" {
source = "./foo"
for_each = local.aws_accounts
providers = {
aws = "aws.${each.value.aws_account_id}"
}
foo = each.value.foo_value
}
module "bar" {
source = "./bar"
for_each = local.aws_accounts
providers = {
aws = "aws.${each.value.aws_account_id}"
}
bar = each.value.bar_value
}
I am following this guide https://github.com/hashicorp/terraform/blob/v0.13/website/docs/configuration/modules.html.md#passing-providers-explicitly
This is a brand new Terraform project ie no resources and an empty state file! The version of terraform is Terraform v0.14.5.
In the calling module I have my providers set up as follows
terraform {
backend "azurerm" {
}
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = ">= 2.68.0"
}
}
}
provider "azurerm" {
alias = "lz"
subscription_id = "xxx-xxx-xxx-xxx"
features {}
}
provider "azurerm" {
alias = "prd"
subscription_id = "xxx-xxx-xxx-xxx"
features {}
}
And I am calling modules passing multiple providers like this
module "prod" {
source = "../../../Terraform/modules/LandingZone"
providers = {
azurerm.lz = azurerm.lz
azurerm.prd = azurerm.prd
}
}
In the called module I have this in the providers.tf
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = ">= 2.68.0"
}
}
backend "azurerm" {
}
}
provider "azurerm" {
alias = "lz"
features{}
}
provider "azurerm" {
alias = "prd"
features{}
}
Up to this point init and plan works fine. However when I try to create resources like this
resource "azurerm_resource_group" "this" {
provider= azurerm.lz
for_each = var.rg_names
name = each.value
location = "Australia Southeast"
}
I get this error message
Error: Provider configuration not present
To work with module.trn.module.rg.azurerm_resource_group.this its original
provider configuration at
module.trn.module.rg.provider["registry.terraform.io/hashicorp/azurerm"].lz is
required, but it has been removed. This occurs when a provider configuration
is removed while objects created by that provider still exist in the state.
Re-add the provider configuration to destroy
module.trn.module.rg.azurerm_resource_group.this, after which you can remove
the provider configuration again.
From the guide you have linked,
A proxy configuration block is one that contains only the alias argument.
Your provider blocks in your module have more than just the alias argument, so they are probably not being set up as proxy provider configurations. Try removing the features{} part from the provider blocks inside your module.
Referring to this guide you should declare configuration_aliases inside terraform block for required providers. You can try this also.
I inherited a codebase with all providers stored inside modules and am having a lot of trouble moving the providers out so that I can remove the resources created from modules.
The current design violates the rules outlined here: https://www.terraform.io/docs/configuration/providers.html and makes removing modules impossible.
My understanding of the migration steps is:
Create a provider for use at the top-level
Update module resources to use providers stored outside of the module
Remove module (with top-level provider persisting)
Example module
An example /route53-alias-record/main.ts is:
variable "evaluate_target_health" {
default = true
}
data "terraform_remote_state" "env" {
backend = "s3"
config = {
bucket = "<bucket>"
key = "infra-${var.environment}-${var.target}.tfstate"
region = "<region>"
}
}
provider "aws" {
region = data.terraform_remote_state.env.outputs.region
allowed_account_ids = data.terraform_remote_state.env.outputs.allowed_accounts
assume_role {
role_arn = data.terraform_remote_state.env.outputs.aws_account_role
}
}
resource "aws_route53_record" "alias" {
zone_id = data.terraform_remote_state.env.outputs.public_zone_id
name = var.fqdn
type = "A"
alias {
name = var.alias_name
zone_id = var.zone_id
evaluate_target_health = var.evaluate_target_health
}
}
Starting usage
module "api-dns-alias" {
source = "../environment/infra/modules/route53-alias-record"
environment = "${var.environment}"
zone_id = "${module.service.lb_zone_id}"
alias_name = "${module.service.lb_dns_name}"
fqdn = "${var.environment}.example.com"
}
Provider overriding
## Same as inside module
provider "aws" {
region = data.terraform_remote_state.env.outputs.region
allowed_account_ids = data.terraform_remote_state.env.outputs.allowed_accounts
assume_role {
role_arn = data.terraform_remote_state.env.outputs.aws_account_role
}
}
module "api-dns-alias" {
source = "../environment/infra/modules/route53-alias-record"
environment = "${var.environment}"
zone_id = "${module.service.lb_zone_id}"
alias_name = "${module.service.lb_dns_name}"
fqdn = "${var.environment}.example.com"
providers = {
aws = aws ## <-- pass in explicitly
}
}
I was able to safely deploy with the providers set, but I do not believe that they are being used inside the module, which means the handshake still fails when I remove the module and the resources cannot be deleted.
I am looking for the steps needed to migrate to an outside provider so that I can safely remove resources.
I am currently working with terraform 0.12.24