Managing slight differences in outcomes with terraform modules - terraform

Question

We inherited a terraform + modules layout like this where the databases for an environment (AWS RDS) are provisioned slightly differently depending on whether terraform is invoked on the main branch vs any feature/* branches in our CI/CD pipeliens.
☡ tree -P main.tf
.
├── feature-database
│   ├── dev
│   │   └── main.tf
│   └── modules
│   └── database
│   └── main.tf
└── main-database
├── dev
│   └── main.tf
└── modules
└── database
└── main.tf
8 directories, 4 files
The feature-database module provisions an RDS instance from a snapshot of the RDS instance created by main-database - apart from this difference, everything else in the feature-database module is an exact copy-paste of main-database.
It seems like a code-smell to have 2 very similar modules (i.e. */modules/database/main.tf) that are 95% identical to each other. We have concerns about maintaining, testing, deployments in this approach and want to restructure to make this DRY.
So the questions naturally are.
What would be a good (ideally terraform) way to manage these differences in provisioning depending on the environment? Is conditional execution a possibility or Do we just accept this as an overhead we manage as different mostly-identical modules?
Are there some out-of-the-box solutions with tools/approaches to help with something like this?

Answer 1

Non-idempotent operations such as creating and using snapshots/images are unfortunately not an ideal situation for Terraform's execution model, since they lend themselves more to an imperative execution model ("create a new instance using this particular snapshot" (where this particular is likely to change for each deployment) vs. "there should be an instance").
However, it is possible in principle to write such a thing. Without seeing the details of those modules it's hard to give specific advice, but at a high-level I'd expect to see the unified module have an optional input variable representing a snapshot ID, and then have the module vary its behavior based on whether that variable is set:
variable "source_snapshot_id" {
type = string
# This means that the variable is optional but
# it doesn't have a default value.
default = null
}
resource "aws_db_instance" "example" {
# ...
# If this variable isn't set then the value here
# will be null, which is the same as not setting
# snapshot_identifier at all.
snapshot_identifier = var.source_snapshot_id
}
The root module would then need to call this module twice and wire the result of the first instance into the second instance. Perhaps that would look something like this:
module "main_database" {
source = "../modules/database"
# ...
}
resource "aws_db_snapshot" "example" {
db_instance_identifier = module.main_database.instance_id
db_snapshot_identifier = "${module.main_database.instance_id}-feature-snapshot"
}
module "feature_database" {
source = "../modules/database"
source_snapshot_id = aws_db_snapshot.example.db_snapshot_identifier
# ...
}
On the first apply of this configuration, Terraform would first create the "main database", then immediately create a snapshot of it, and then create the "feature database" using that snapshot. In order for that to be useful the module would presumably need to encapsulate some actions to put some schema and possibly some data into the database, or else the snapshot would just be of an empty database. If those actions involve some other resources alongside the main aws_db_instance then you can encapsulate the correct ordering by declaring additional dependencies on the instance_id output value I presumed in my example above:
output "instance_id" {
# This reference serves as an implicit dependency
# on the DB instance itself.
value = aws_db_instance.example.id
# ...but if you have other resources that arrange
# for the database to have interesting data inside
# it then you'll likely want to declare those
# dependencies too, so that the root module won't
# start trying to create a snapshot until the
# database contents are ready
depends_on = [
aws_db_instance_role_association.example,
null_resource.example,
# ...
]
}
I've focused on the general Terraform patterns here rather than on the specific details of RDS, because I'm not super familiar with these particular resource types, but hopefully even if I got any details wrong above you can still see the general idea here and adapt it to your specific situation.