Using terragrunt generate provider block causes conflicts with require providers block in module - terraform

Question

I'm using Terragrunt with Terraform version 0.14.8.
My project uses mono repo structure as it is a project requirement to package Terragrunt files and Terraform modules together in a single package.
Folder structure:
project root:
├── environments
│   └── prd
│   ├── rds-cluster
│   │   └── terragrunt.hcl
│   └── terragrunt.hcl
└── modules
├── rds-cluster
│   ├── README.md
│   ├── main.tf
│   ├── output.tf
│   └── variables.tf
└── secretsmanager-secret
├── README.md
├── main.tf
├── output.tf
└── variables.tf
In prd/terragrunt.hcl I define the remote state block and the generate provider block.
remote_state {
backend = "s3"
...
}
generate "provider" {
path = "provider.tf"
if_exists = "overwrite_terragrunt"
contents = <<EOF
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 3.0"
}
}
}
provider "aws" {
region = "ca-central-1"
}
EOF
}
In environments/prd/rds-cluster/terragrunt.hcl, I defined the following:
include {
path = find_in_parent_folders()
}
terraform {
source = "../../../modules//rds-cluster"
}
inputs = {
...
}
In modules/rds-cluster/main.tf, I defined the following:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = ">= 3.0"
}
}
}
// RDS related resources...
My problem is that when I try to run terragrunt plan under environments/prd/rds-cluster, I get the following error message:
Error: Duplicate required providers configuration
on provider.tf line 3, in terraform:
3: required_providers {
A module may have only one required providers configuration. The required
providers were previously configured at main.tf:2,3-21.
I can resolve this by declaring the version within the provider block as shown here. However, the version attribute in provider blocks has been deprecated in Terraform 0.13; Terraform recommends the use of the required_providers sub-block under terraform block instead.
Does anyone know what I need to do to use the new required_providers block for my aws provider?

Answer 1

As you've seen, Terraform expects each module to have only one definition of its required providers, which is intended to avoid a situation where it's unclear why Terraform is detecting a particular when the declarations are spread among multiple files.
However, to support this sort of piecemeal code generation use-case Terraform has an advanced feature called Override Files which allows you to explicitly mark certain files for a different mode of processing where they selectively override particular definitions from other files, rather than creating entirely new definitions.
The details of this mechanism depend on which block type you're overriding, but the section on Merging terraform blocks` discusses the behavior relevant to your particular situation:
If the required_providers argument is set, its value is merged on an element-by-element basis, which allows an override block to adjust the constraint for a single provider without affecting the constraints for other providers.
In both the required_version and required_providers settings, each override constraint entirely replaces the constraints for the same component in the original block. If both the base block and the override block both set required_version then the constraints in the base block are entirely ignored.
The practical implication of the above is that if you have an override file with a required_providers block that includes an entry for the AWS provider then Terraform will treat it as a full replacement for any similar entry already present in a non-override file, but it won't affect other provider requirements entries which do not appear in the override file at all.
Putting all of this together, you should be able to get the result you were looking for by asking Terragrunt to name this generated file provider_override.tf instead of just provider.tf, which will then activate the override file processing behavior and thus allow this generated file to override any existing definition of AWS provider requirements, while allowing the configurations to retain any other provider requirements they might also be defining.