Terraform 0.13 - Modules, for_each and providers - terraform

UPDATED
I am trying to provision multiple SQL databases in Azure using Terraform.
My child module has the following code that provisions a SQL database:
providers.tf
// default provider
provider "azurerm" {
alias = "main"
features {}
}
// The provider that can access the storage account to store diagnostics
provider "azurerm" {
alias = "storage_account"
features {}
}
sql_db.tf
resource "azurerm_mssql_database" "default" {
name = var.name
base_name = var.base_name
...
tags = var.tags
provider = azurerm.main
}
data.tf
data "azurerm_storage_account" "storage" {
name = var.storage_account_name
resource_group_name = var.storage_account_rg
provider = azurerm.storage_account
}
I am calling this module in my main.tf file as follows where I want to provision multiple SQL databases using a for_each:
module "sql_db" {
for_each = var.sql_db
source = "...../sql_db.git"
base_name = each.value.base_name
name = each.value.name
providers = {
azurerm.main = azurerm.main
azurerm.storage_account = azurerm.storage_account
}
}
provider "azurerm" {
features {}
version = "=2.20.0"
}
// default provider
provider "azurerm" {
alias = "main"
features {}
}
provider "azurerm" {
alias = "storage_account"
features {}
}
When I run plan, I get the following error:
Error: Module does not support for_each
on main.tf line 35, in module "sql_db":
35: for_each = var.sql_db
Module "sql_db" cannot be used with for_each because it contains a nested
provider configuration for "azurerm.main", at
.terraform\modules\sql_db\providers.tf:2,10-19.
This module can be made compatible with for_each by changing it to receive all
of its provider configurations from the calling module, by using the
"providers" argument in the calling module block.
Error: Module does not support for_each
on main.tf line 35, in module "sql_db":
35: for_each = var.sql_db
Module "sql_db" cannot be used with for_each because it contains a nested
provider configuration for "azurerm.storage_account", at
.terraform\modules\sql_db\providers.tf:8,10-19.
This module can be made compatible with for_each by changing it to receive all
of its provider configurations from the calling module, by using the
"providers" argument in the calling module block.

The simple answer is, it's not supported. From the Terraform documentation:
A module containing its own provider configurations is not compatible with the for_each, count, and depends_on arguments that were introduced in Terraform v0.13.
HashiCorp has been absolutely adamant that providers can never be declared dynamically, which is why they allow neither a for_each/count within a provider block, nor a for_each/count on a module that contains a provider block.

Related

Using azure azurerm_resource_provider_registration with newly created subscription

I am working on creating Azure landing zone and part of that is to enable/disable resource providers on the newly created subscriptions.
I have tried to used alias with a variable but i am getting error that i cant use variable in an alias so is there any way through which i can use this feature on multiple subscription
This is my code main.tf
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "=3.0.0"
}
}
}
#list of providers i want to register
locals {
# List is compiled from here
# https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-services-resource-providers
provider_list = [
"Microsoft.Storage"
]
provider_map = { for p in local.provider_list : p => p }
}
# Registering a default provider here and skipping registration
# as i will do it later
provider "azurerm" {
features {}
skip_provider_registration = true
}
# I am creating a subscription here with same alias as the name
# the subscription is being created under and EA enrollment but
# any type of subscription will do
resource "azurerm_subscription" "feature_subscription" {
billing_scope_id = "/providers/Microsoft.Billing/billingAccounts/xxx/enrollmentAccounts/xx"
alias = var.temp_alias # "test-provider-registration"
subscription_name = "test-provider-registration"
}
#this is what i have created to point out my azurerm_resource_provider_registration
#module i am using variable in alias which is failing
provider "azurerm" {
alias = var.temp_alias
subscription_id = azurerm_subscription.feature_subscription.id
features {
}
skip_provider_registration = true
}
#module through which i am registering the resource providers
module "azurerm_resource_provider_registration-provider" {
source = "../modules/azurerm_resource_provider_registration"
providers = {
azurerm = azurerm.test-provider-registration
}
feature_list = local.provider_map
}
#the module code is mentioned here
#resource "azurerm_resource_provider_registration" "provider" {
# for_each = var.feature_list
# name = each.value
#}
I am getting this error when i run it
There are some problems with the configuration, described below.
The Terraform configuration must be valid before initialization so that
Terraform can determine which modules and providers need to be installed.
╷
Error: Variables not allowed
│
On main.tf line 25: Variables may not be used here.
╵
╷
Error: Unsuitable value type
│
On main.tf line 25: Unsuitable value: value must be known
There is a workaround available like using this
resource "null_resource" "provider_registration" {
for_each = local.provider_map
provisioner "local-exec" {
command = "az account set --subscription ${data.azurerm_subscription.subscription.subscription_id} && az provider register --namespace ${each.value}"
}
}
but i want to use the state file for the resource registration if possible as i have more subscriptions in a loop
Error: Unsuitable value type | Unsuitable value: value must be known
Need to check:
This problem usually occurs with module sources or versions. When invoking a module, using variables instead of passing direct values for the source and some other arguments causes this error. terraform initdoesn't take variable inputs with modules in backend state configuration.
Note: And also include required version for modules.
To make it work, use direct values for module block instead of accessing from other resources.
Pass provider explicitly to make it work as detailed in terraform providers.
providers = {
azurerm = azurerm.test-provider-registration
}
feature_list = local.provider_map
}
}
module "azurerm_resource_provider_registration-provider" {
source ="../modules/azurerm_resource_provider_registration"
version = <required version> //Use latest one
After checking the above conditions, I tried same in my environment and it was initialized successfully:
Refer aliasing for the deployment in multiple subscriptions detailed by #Jeff Brown

Terraform - Error: Provider configuration not present (passing multiple Azure providers)

I am following this guide https://github.com/hashicorp/terraform/blob/v0.13/website/docs/configuration/modules.html.md#passing-providers-explicitly
This is a brand new Terraform project ie no resources and an empty state file! The version of terraform is Terraform v0.14.5.
In the calling module I have my providers set up as follows
terraform {
backend "azurerm" {
}
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = ">= 2.68.0"
}
}
}
provider "azurerm" {
alias = "lz"
subscription_id = "xxx-xxx-xxx-xxx"
features {}
}
provider "azurerm" {
alias = "prd"
subscription_id = "xxx-xxx-xxx-xxx"
features {}
}
And I am calling modules passing multiple providers like this
module "prod" {
source = "../../../Terraform/modules/LandingZone"
providers = {
azurerm.lz = azurerm.lz
azurerm.prd = azurerm.prd
}
}
In the called module I have this in the providers.tf
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = ">= 2.68.0"
}
}
backend "azurerm" {
}
}
provider "azurerm" {
alias = "lz"
features{}
}
provider "azurerm" {
alias = "prd"
features{}
}
Up to this point init and plan works fine. However when I try to create resources like this
resource "azurerm_resource_group" "this" {
provider= azurerm.lz
for_each = var.rg_names
name = each.value
location = "Australia Southeast"
}
I get this error message
Error: Provider configuration not present
To work with module.trn.module.rg.azurerm_resource_group.this its original
provider configuration at
module.trn.module.rg.provider["registry.terraform.io/hashicorp/azurerm"].lz is
required, but it has been removed. This occurs when a provider configuration
is removed while objects created by that provider still exist in the state.
Re-add the provider configuration to destroy
module.trn.module.rg.azurerm_resource_group.this, after which you can remove
the provider configuration again.
From the guide you have linked,
A proxy configuration block is one that contains only the alias argument.
Your provider blocks in your module have more than just the alias argument, so they are probably not being set up as proxy provider configurations. Try removing the features{} part from the provider blocks inside your module.
Referring to this guide you should declare configuration_aliases inside terraform block for required providers. You can try this also.

Referencing terraform_remote_state from within module

Using Terraform v0.13.5
I have a module which has some outputs derived from some sub modules within it, e.g.:
module "egressvnet" {
source = "../modules/vnet/egress"
}
output "subnet" {
value = module.egressvnet.subnet
}
terraform output confirms that what I expect to be outputted is.
Within another terraform setup I would like to reference the outputs from the above.
So I have this in my terraform config:
data "terraform_remote_state" "network" {
backend = "azurerm"
config = {
resource_group_name = "xxx"
storage_account_name = "xxx"
container_name = "terraform"
key = "network.tfstate"
}
}
module "web" {
source = "../modules/web"
subnet_id = terraform_remote_state.network.outputs.subnet
}
Is what I'm trying to do possible?
But when I do a plan I get this error:
Error: Reference to undeclared resource
on base.tf line 111, in module "web":
111: subnet_id = terraform_remote_state.network.outputs.subnet
A managed resource "terraform_remote_state" "network" has not been declared in
the root module.
Since your terraform_remote_state is a data source, you should refer to it using data.:
subnet_id = data.terraform_remote_state.network.outputs.subnet

Terraform azure Incompatible provider version

When i'm trying to initialize terraform, i'm getting following error only with vnet module,
But terraform initilization working with azure_resource_group, azure_virtual_machine modules
Terraform 0.13 and earlier allowed provider version constraints inside the
provider configuration block, but that is now deprecated and will be removed
in a future version of Terraform. To silence this warning, move the provider
version constraint into the required_providers block.
Error: Failed to query available provider packages
Could not retrieve the list of available versions for provider
hashicorp/azure: provider registry registry.terraform.io does not have a
provider named registry.terraform.io/hashicorp/azure
If you have just upgraded directly from Terraform v0.12 to Terraform v0.14
then please upgrade to Terraform v0.13 first and follow the upgrade guide for
that release, which might help you address this problem.
Did you intend to use terraform-providers/azure? If so, you must specify that
source address in each module which requires that provider. To see which
modules are currently depending on hashicorp/azure,
enter image description here
enter image description here
main.tf
resource "azurerm_virtual_network" "vnet" {
for_each = { for n in var.networks : n.name => n }
name = each.value.name
address_space = each.value.address_space
location = each.value.location
resource_group_name = each.value.rg_name
dynamic "subnet" {
for_each = each.value.subnets
content {
name = subnet.value.name
address_prefix = subnet.value.address_prefixes
}
}
}
variables.tf
variable networks {
type = list(object({
name = string
address_space = list(string)
rg_name = string
location = string
subnets = list(object({
name = string
address_prefixes = string
}))
}))
}
module (main.tf)
module "azurevnet"{
source = "./vnet"
networks = var.networks
}
provider.tf
provider "azurerm" {
version = "=2.37.0"
}
As I know, the registry terraform-providers/azure is a deprecated provider. This model will not add new things anymore and Azure already change into ARM model. So I recommend you use the terraform-providers/azurerm model only and it supports more Azure features.
Update:
And use the azurerm model, the directory structure would look like this:
main.tf
module "azurevnet" {
source = "./vnet"
networks = var.networks
}
providers.tf
provider "azurerm" {
features {}
version = "=2.37.0"
}
vnet/main.tf
variable "networks" {}
resource "azurerm_virtual_network" "vnet" {
for_each = { for n in var.networks : n.name => n }
name = each.value.name
address_space = each.value.address_space
location = each.value.location
resource_group_name = each.value.rg_name
dynamic "subnet" {
for_each = each.value.subnets
content {
name = subnet.value.name
address_prefix = subnet.value.address_prefixes
}
}
}
I only give the code for VNet, but other resources will be in the same format. And you can also do not use the providers.tf file and put the content into the main.tf file.

How to create provide modules that support multiple AWS regions?

We are trying to create Terraform modules for below activities in AWS, so that we can use them where ever that is required.
VPC creation
Subnets creation
Instance creation etc.
But while creating these modules we have to define the provider in all above listed modules. So we decided to create one more module for provider so that we can call that provider module in other modules (VPC, Subnet, etc.).
Issue in above approach is that it is not taking provider value, and asking for the user input for region.
Terraform configuration is as follow:
$HOME/modules/providers/main.tf
provider "aws" {
region = "${var.region}"
}
$HOME/modules/providers/variables.tf
variable "region" {}
$HOME/modules/vpc/main.tf
module "provider" {
source = "../../modules/providers"
region = "${var.region}"
}
resource "aws_vpc" "vpc" {
cidr_block = "${var.vpc_cidr}"
tags = {
"name" = "${var.environment}_McD_VPC"
}
}
$HOME/modules/vpc/variables.tf
variable "vpc_cidr" {}
variable "environment" {}
variable "region" {}
$HOME/main.tf
module "dev_vpc" {
source = "modules/vpc"
vpc_cidr = "${var.vpc_cidr}"
environment = "${var.environment}"
region = "${var.region}"
}
$HOME/variables.tf
variable "vpc_cidr" {
default = "192.168.0.0/16"
}
variable "environment" {
default = "dev"
}
variable "region" {
default = "ap-south-1"
}
Then when running terraform plan command at $HOME/ location it is not taking provider value and instead asking for the user input for region.
I need help from the Terraform experts, what approach we should follow to address below concerns:
Wrap provider in a Terraform module
Handle multiple region use case using provider module or any other way.
I knew a long time back that it wasn't possible to do this because Terraform built a graph that required a provider for any resource before it included any dependencies and it didn't used to be possible to force a dependency on a module.
However since Terraform 0.8 it is now possible to set a dependency on modules with the following syntax:
module "network" {
# ...
}
resource "aws_instance" "foo" {
# ...
depends_on = ["module.network"]
}
However, if I try that with your setup by changing modules/vpc/main.tf to look something like this:
module "aws_provider" {
source = "../../modules/providers"
region = "${var.region}"
}
resource "aws_vpc" "vpc" {
cidr_block = "${var.vpc_cidr}"
tags = {
"name" = "${var.environment}_McD_VPC"
}
depends_on = ["module.aws_provider"]
}
And run terraform graph | dot -Tpng > graph.png against it it looks like the graph doesn't change at all from when the explicit dependency isn't there.
This seems like it might be a potential bug in the graph building stage in Terraform that should probably be raised as an issue but I don't know the core code base well enough to spot where the change needs to be.
For our usage we use symlinks heavily in our Terraform code base, some of which is historic from before Terraform supported other ways of doing things but could work for you here.
We simply define the provider in a single .tf file (such as environment.tf) along with any other generic config needed for every place you would ever run Terraform (ie not at a module level) and then symlink this into each location. That allows us to define the provider in a single place with overridable variables if necessary.
Step 1
Add region alias in the main.tf file where you gonna execute the terraform plan.
provider "aws" {
region = "eu-west-1"
alias = "main"
}
provider "aws" {
region = "us-east-1"
alias = "useast1"
}
Step 2
Add providers block inside your module definition block
module "lambda_edge_rule" {
providers = {
aws = aws.useast1
}
source = "../../../terraform_modules/lambda"
tags = var.tags
}
Step 3
Define "aws" as providers inside your module. ( source = ../../../terraform_modules/lambda")
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = ">= 2.7.0"
}
}
}
resource "aws_lambda_function" "lambda" {
function_name = "blablabla"
.
.
.
.
.
.
.
}
Note: Terraform version v1.0.5 as of now.

Resources