Using multiple provider version in terraform - terraform

Im trying to use two provider version with in terraform, though Im getting the below error
Error: Failed to query available provider packages
Could not retrieve the list of available versions for provider hashicorp/aws:
no available releases match the given constraints >= 3.71.0, 3.71.0, 4.6.0
Here is what Im trying to do. I have a terraform file, which uses multiple module. And in one module alone, I need to use aws provider version 4.6.0. On other modules, I need to stick to currently applied provider version, which is 3.71.0
Terraform version: 0.13.6
Im defining a constraint in terraform file, so "hashicorp/aws" can be anything above 3.71.0. Below is what is defined:
"aws": {
"version": ">= 3.71.0",
"assume_role": {
"role_arn": "....",
"session_name": "..."
},
terraform file calls more than 10 module, and module 0 to 9 provider config is
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "= 3.71.0"
}
}
}
and 10th module provider config is
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "= 4.6.0"
}
}
}
Anything Im missing?
Note : I have already referred this post - Multiple provider versions with Terraform, though not sure, if its technically not possible, and something Im doing it wrong

We use a separate provider block for each and every region, this is how we handle it:
provider "aws" {
region = "us-west-2"
}
provider "aws" {
alias = "east-2-provider"
region = "us-east-2"
version = "~> 4.0"
}
provider "aws" {
alias = "east-1-provider"
region = "us-east-1"
version = "~> 3.74"
}
When we are using a module we use it as below:
module "example-1" {
source = "./example"
providers = {
aws = east-1-provider
}
}
module "example-2" {
source = "./example"
providers = {
aws = east-2-provider
}
}

Related

Terraform: Failed to query available provider packages (Azapi)

I try to use the Azure/Azapi Provider within my Terraform project but after I add the provider and run terraform init, I get the following error:
Error: Failed to query available provider packages
Could not retrieve the list of available versions for provider hashicorp/azapi: provider registry registry.terraform.io does not have a provider named registry.terraform.io/hashicorp/azapi
This is how my providers.tf looks like:
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "=3.16.0"
}
azapi = {
source = "azure/azapi"
version = "=0.4.0"
}
}
required_version = "=1.2.6"
}
provider "azurerm" {
features {}
}
provider "azapi" {
}
When I run terraform providers, I can see that the provider has a wrong registry URL within my module:
├── module.az-aca-env
│ └── provider[registry.terraform.io/hashicorp/azapi]
I would expect something like registry.terraform.io/azure/azapi.
Any ideas?
Okay, I found a workaround. I have to add a providers.tf inside my module directory (/modules/az-aca-env) with the following content:
terraform {
required_providers {
azapi = {
source = "Azure/azapi"
version = "=0.4.0"
}
}
}
After adding it, the terraform init works ✅.
You have a typo in the provider name, it is Azure/azapi as per documentation [1]:
terraform {
required_providers {
azapi = {
source = "Azure/azapi"
version = "0.4.0"
}
}
}
provider "azapi" {
# Configuration options
}
You can always see how to use the provider if you click on the big purple button in the top right-hand corner saying USE PROVIDER.
[1] https://registry.terraform.io/providers/azure/azapi/latest/docs

While creating Azure App service via terraform throwing an error An argument named "zone_redundant" is not expected here

I'm trying to create a zone redundant azure app service for high availability, but terraform validate throwing an error An argument named "zone_redundant" is not expected here.
My configuration looks like below
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "=2.46.0"
}
}
}
resource "azurerm_app_service_plan" "example" {
name = "app-demo"
location = "Australia East"
resource_group_name = "rg-app-service"
kind = "Linux"
reserved = true
zone_redundant = true
sku {
tier = "PremiumV2"
size = "P1v2"
capacity = "3"
}
}
I'm not sure what I'm missing here. Can anyone please advise me on this ?
Reference
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/app_service_plan#zone_redundant
You are using Terraform azurerm provider version 2.46.0
zone_redundant option in azurerm_app_service_plan Terraform resources was added in Terraform azurerm provider version 2.74.0, that's why you are getting error "An argument named "zone_redundant" is not expected here."
Please update Terraform azurerm provider version in your code:
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "=2.74.0"
}
}
}

Terraform - Error: Provider configuration not present (passing multiple Azure providers)

I am following this guide https://github.com/hashicorp/terraform/blob/v0.13/website/docs/configuration/modules.html.md#passing-providers-explicitly
This is a brand new Terraform project ie no resources and an empty state file! The version of terraform is Terraform v0.14.5.
In the calling module I have my providers set up as follows
terraform {
backend "azurerm" {
}
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = ">= 2.68.0"
}
}
}
provider "azurerm" {
alias = "lz"
subscription_id = "xxx-xxx-xxx-xxx"
features {}
}
provider "azurerm" {
alias = "prd"
subscription_id = "xxx-xxx-xxx-xxx"
features {}
}
And I am calling modules passing multiple providers like this
module "prod" {
source = "../../../Terraform/modules/LandingZone"
providers = {
azurerm.lz = azurerm.lz
azurerm.prd = azurerm.prd
}
}
In the called module I have this in the providers.tf
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = ">= 2.68.0"
}
}
backend "azurerm" {
}
}
provider "azurerm" {
alias = "lz"
features{}
}
provider "azurerm" {
alias = "prd"
features{}
}
Up to this point init and plan works fine. However when I try to create resources like this
resource "azurerm_resource_group" "this" {
provider= azurerm.lz
for_each = var.rg_names
name = each.value
location = "Australia Southeast"
}
I get this error message
Error: Provider configuration not present
To work with module.trn.module.rg.azurerm_resource_group.this its original
provider configuration at
module.trn.module.rg.provider["registry.terraform.io/hashicorp/azurerm"].lz is
required, but it has been removed. This occurs when a provider configuration
is removed while objects created by that provider still exist in the state.
Re-add the provider configuration to destroy
module.trn.module.rg.azurerm_resource_group.this, after which you can remove
the provider configuration again.
From the guide you have linked,
A proxy configuration block is one that contains only the alias argument.
Your provider blocks in your module have more than just the alias argument, so they are probably not being set up as proxy provider configurations. Try removing the features{} part from the provider blocks inside your module.
Referring to this guide you should declare configuration_aliases inside terraform block for required providers. You can try this also.

Terraform workspace states in different s3 buckets?

I use terraform to provision resources in dev and prod environments. These environments live on two different aws accounts. I had my state locally but I want to push it to s3 now. The problem is that terraform stores the state for the def and prod envs in the same s3 bucket is it possible to separate them? If not what are some alternative solutions without splitting my terraform codebase?
I have a bash wrapper around terraform and create a state file per account for separation of concerns. I also break the automation into many components to keep the state small so that performance does not suffer and downloading and uploading the state to the bucket :
function set_backend () {
local STATE_PATH=$1
if [[ $BACKEND == "s3" ]]; then
cat << EOF > ./backend.tf
terraform {
backend "s3" {
bucket = "${TF_VAR_state_bucket}"
dynamodb_table = "${DYNAMODB_STATE_TABLE}"
key = "terraform/$STATE_PATH/terraform.tfstate"
region = "$REGION"
encrypt = "true"
}
}
provider "aws" {
region = "$REGION"
version = "1.51.0"
}
provider "aws" {
region = "$DR_REGION"
version = "1.51.0"
alias = "dr"
}
provider "archive" { version = "1.1.0" }
provider "external" { version = "1.0.0" }
provider "local" { version = "1.1.0" }
provider "null" { version = "1.0.0" }
provider "random" { version = "2.0.0" }
provider "template" { version = "1.0.0" }
provider "tls" { version = "1.2.0" }
EOF
fi
}
Terragrunt is a great tool to use for managing terraform state files for different environments and to store state files in different buckets, instead of to use terraform workspace.
Useful links,
https://transcend.io/blog/why-we-use-terragrunt
https://blog.gruntwork.io/how-to-manage-terraform-state-28f5697e68fa

How to create provide modules that support multiple AWS regions?

We are trying to create Terraform modules for below activities in AWS, so that we can use them where ever that is required.
VPC creation
Subnets creation
Instance creation etc.
But while creating these modules we have to define the provider in all above listed modules. So we decided to create one more module for provider so that we can call that provider module in other modules (VPC, Subnet, etc.).
Issue in above approach is that it is not taking provider value, and asking for the user input for region.
Terraform configuration is as follow:
$HOME/modules/providers/main.tf
provider "aws" {
region = "${var.region}"
}
$HOME/modules/providers/variables.tf
variable "region" {}
$HOME/modules/vpc/main.tf
module "provider" {
source = "../../modules/providers"
region = "${var.region}"
}
resource "aws_vpc" "vpc" {
cidr_block = "${var.vpc_cidr}"
tags = {
"name" = "${var.environment}_McD_VPC"
}
}
$HOME/modules/vpc/variables.tf
variable "vpc_cidr" {}
variable "environment" {}
variable "region" {}
$HOME/main.tf
module "dev_vpc" {
source = "modules/vpc"
vpc_cidr = "${var.vpc_cidr}"
environment = "${var.environment}"
region = "${var.region}"
}
$HOME/variables.tf
variable "vpc_cidr" {
default = "192.168.0.0/16"
}
variable "environment" {
default = "dev"
}
variable "region" {
default = "ap-south-1"
}
Then when running terraform plan command at $HOME/ location it is not taking provider value and instead asking for the user input for region.
I need help from the Terraform experts, what approach we should follow to address below concerns:
Wrap provider in a Terraform module
Handle multiple region use case using provider module or any other way.
I knew a long time back that it wasn't possible to do this because Terraform built a graph that required a provider for any resource before it included any dependencies and it didn't used to be possible to force a dependency on a module.
However since Terraform 0.8 it is now possible to set a dependency on modules with the following syntax:
module "network" {
# ...
}
resource "aws_instance" "foo" {
# ...
depends_on = ["module.network"]
}
However, if I try that with your setup by changing modules/vpc/main.tf to look something like this:
module "aws_provider" {
source = "../../modules/providers"
region = "${var.region}"
}
resource "aws_vpc" "vpc" {
cidr_block = "${var.vpc_cidr}"
tags = {
"name" = "${var.environment}_McD_VPC"
}
depends_on = ["module.aws_provider"]
}
And run terraform graph | dot -Tpng > graph.png against it it looks like the graph doesn't change at all from when the explicit dependency isn't there.
This seems like it might be a potential bug in the graph building stage in Terraform that should probably be raised as an issue but I don't know the core code base well enough to spot where the change needs to be.
For our usage we use symlinks heavily in our Terraform code base, some of which is historic from before Terraform supported other ways of doing things but could work for you here.
We simply define the provider in a single .tf file (such as environment.tf) along with any other generic config needed for every place you would ever run Terraform (ie not at a module level) and then symlink this into each location. That allows us to define the provider in a single place with overridable variables if necessary.
Step 1
Add region alias in the main.tf file where you gonna execute the terraform plan.
provider "aws" {
region = "eu-west-1"
alias = "main"
}
provider "aws" {
region = "us-east-1"
alias = "useast1"
}
Step 2
Add providers block inside your module definition block
module "lambda_edge_rule" {
providers = {
aws = aws.useast1
}
source = "../../../terraform_modules/lambda"
tags = var.tags
}
Step 3
Define "aws" as providers inside your module. ( source = ../../../terraform_modules/lambda")
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = ">= 2.7.0"
}
}
}
resource "aws_lambda_function" "lambda" {
function_name = "blablabla"
.
.
.
.
.
.
.
}
Note: Terraform version v1.0.5 as of now.

Resources