I'm using Terragrunt within a mono-repo as below for multiple projects and environments:
projects/
├─ terragrunt.hcl
├─ teamA/
│ ├─ dev/
│ │ ├─ some-service/
│ │ │ ├─ terragrunt.hcl
│ ├─ prod/
│ │ ├─ some-service/
│ │ │ ├─ terragrunt.hcl
├─ teamB/
│ ├─ dev/
│ │ ├─ another-service/
│ │ │ ├─ terragrunt.hcl
│ ├─ prod/
│ │ ├─ another-service/
│ │ │ ├─ terragrunt.hcl
I've read that when using Terragrunt it's best practice to just run the plan/apply command on the module(s) that need to be deployed rather than run-all apply which will recursively run apply on all modules which can sometimes break things (?).
I've also been informed if we run apply over everything, there are some resources that get rebuilt and could potentially wipe any manual changes teams have made. In a bit of a pickle.
However I'm not familiar with how to go about this in terms of a CI/CD pipeline and only executing commands on updated modules.
I would also assume then that dependencies need to be very explicitly stated in the .hcl files for each projects module so as to not try and deploy something that requires something else to already exist.
Flow
Engineer clones code
Engineer create and checkouts out new branch, eg. fix/thing
Engineer updates projects/teamB/dev/another-service/terragrunt.hcl
Engineer commits code and pushes back to remote along with branch
Engineer opens pull request
This triggers terragrunt plan on projects/teamB/dev/another-service/terragrunt.hcl some how
Engineer commits another change to projects/teamA/prod/some-service/terragrunt.hcl
this re-triggers a plan on only the affected modules
projects/teamB/dev/another-service/terragrunt.hcl
projects/teamA/prod/some-service/terragrunt.hcl
PR is approved and merged
this triggers an apply on only the affected modules:
projects/teamB/dev/another-service/terragrunt.hcl
projects/teamA/prod/some-service/terragrunt.hcl
Some answers I have seen involve adding instructions to the commit or PR for the CI/CD tool to parse. Another one mentioned running git diff and parsing the folders/files that have changed.
Has anyone deployed something similar with terragrunt or even terraform?
Related
I am a little stuck on the best way to deliver the following deployments.
For clarity, child terragrunt.hcl files are pulling from a private repo.
.
├── env01
│ ├── resource_group
│ │ └── terragrunt.hcl
│ ├── virtual_network
│ │ └── terragrunt.hcl
│ └── subnet
│ └── terragrunt.hcl
├── env02
│ ├── resource_group
│ │ └── terragrunt.hcl
│ ├── virtual_network
│ │ └── terragrunt.hcl
│ └── subnet
│ └── terragrunt.hcl
└── terragrunt.hcl
How can I, in the above terragrunt layout, merge the child terragrunt.hcl files into a single state file, at present, we have a state file per terragrunt.hcl per resource, which, while works, feels overkill for me.
Do I need to do this at the terraform end? Creating modules for the env01/env02 deployments or is there a way to achieve some kind of terraform merge when deploying?
Hope that makes sense...
I'm new to Terraform and have been spending some time reading up and had a couple of questions related to the best way to structure the code. My plan is to store my Terraform code all in a single repository for all my projects. The infrastructure (Azure) consists of Virtual Machines and an AKS cluster. 98% of our VMs are all exactly the same except for subscription (uat/prod), size, resource group, name and maybe few differences in data disks. My first idea was to create a single environment for virtual machines with multiple .tfvars files for each of the VMs I want to manage? Like this:
└── terraform/
├── virtual_machines/
│ ├── main.tf
│ ├── variables.tf
│ ├── outputs.tf
│ ├── vm1.tfvars
│ ├── vm2.tfvars
│ └── vm3.tfvars
└── aks/
└── aks_project/
├── main.tf
├── variables.tf
└── outputs.tf
Then I can just specify what .tfvars files to point to when applying?
The second idea would be to put the shared vm code in a module then create a directory for each VM like this:
└── terraform/
├── virtual_machines/
│ ├── modules/
│ │ └── virtual_machines/
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ ├── vm1/
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ ├── vm2/
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ └── vm3/
│ ├── main.tf
│ ├── variables.tf
│ └── outputs.tf
└── aks/
└── aks_project/
├── main.tf
├── variables.tf
└── outputs.tf
With each VM directory sourcing from the modules and including the variable values needed for each?
Are either of these approaches make sense for what I'm looking to do? What would you suggest for or use for organizing your terraform code?
Your second approach looks like a better start, as it applies Hashicorp's recommendation of using modules whenever it's possible.
Then you can simply define each VM in a separate .tf file (or group all of them in one), each one of these files should use your local VM module that sets your defaults, and then provide multiple tfvars for each environment.
So you would have something like:
.
├── main.tf
└── virtual_machines
├── modules
│ └── virtual_machines
│ ├── README.md
│ ├── main.tf
│ ├── outputs.tf
│ └── variables.tf
├── dev.tfvars
├── uat.tfvars
├── prod.tfvars
├── variables.tf
├── locals.tf
├── vm1.tf
└── vm2.tf
Where vm1.tf is using your local module and sets its custom parameters, for example:
module "vm1" {
source = "./modules/virtual_machines"
name = "my-vm1-name"
environment = var.environment
}
My var.environment here is defined by my tfvars file which will be picked up for each environment, if I'm on dev I would run my plan or apply with dev.tfvars.
Read more at:
https://cloud.google.com/docs/terraform/best-practices-for-terraform#module-structure
https://www.hashicorp.com/blog/structuring-hashicorp-terraform-configuration-for-production#considerations-for-defining-modules
https://developer.hashicorp.com/terraform/tutorials/modules/pattern-module-creation?in=terraform%2Fmodules&product_intent=terraform
https://www.terraform-best-practices.com/examples/terraform
Also, you may want to consider options like terragrunt for this scenario of configuration DRY.
I hope this answer helps, it's based on my opinions and my experience. You may have a different use case, and a different setup, and you may have other answers from smarter people suggesting different approaches since there's nothing such as a "Best terraform project structure".
I'm working on a terragrunt code base for the first time, having used terraform a lot in the past without terragrunt. I'm a bit confused as to the structure terragrunt seems to enforce. I would usually organise my terraform thus:
main.tf
--> module
main.tf
--> module2
main.tf
This is listed as best practice on the terraform docs:
The Root Module
Terraform always runs in the context of a single root module. A
complete Terraform configuration consists of a root module and the
tree of child modules (which includes the modules called by the root
module, any modules called by those modules, etc.).
Source
But none of the terragrunt structures seem to represent this. It seems to be designed so that each module is independent and run using the run-all command.
This seems problematic to me, from the existing code base I can see that this initialises terraform for every module and I'd say causes issues with sharing secrets between modules. So I'd prefer to work with one root module and multiple child modules.
I can't find a terragrunt pattern that will allow me to do this?
I'm also confused as to how this responsibility is decomposed, do I actually structure my terraform (as above) or do I need an extra root .hcl file?
I'm after something a little like this I guess
└── live
├── prod
│ ├── terragrunt.hcl
│ ├── app
│ │ └── terragrunt.hcl
│ ├── mysql
│ │ └── terragrunt.hcl
│ └── vpc
│ └── terragrunt.hcl
├── qa
│ ├── terragrunt.hcl
│ ├── app
│ │ └── terragrunt.hcl
│ ├── mysql
│ │ └── terragrunt.hcl
│ └── vpc
│ └── terragrunt.hcl
└── stage
├── terragrunt.hcl
├── app
│ └── terragrunt.hcl
├── mysql
│ └── terragrunt.hcl
└── vpc
└── terragrunt.hcl
But this example just talks about specifying the provider block and nothing about a root main.tf. So I'm lost?
Each TF module you used to use consumed inputs, created resources and provided outputs. The wiring of the modules was done via main.tf you are referring to in plain Terraform.
In your case with Terragrunt now, the wiring will be done by terragrunt.hcl files. On the module level (e.g.live/prod/app/terragrunt.hcl) you could define the module dependencies, i.e. where are the input values for this modules input variables, e.g.:
inputs {
username = dependency.iam.output.user.name
}
With this in mind, you might or might not use root-level terragrunt.hcl files. If you want to invoke the parent-folder terragrunt.hcl code, you need to add the following block into your module:
include "root" {
path = find_in_parent_folders()
expose = true
}
See the docs for this function here.
What is best folder structure for a project that includes:
2 database (mongodb, influxdb)
socket io
...
It all depends on the personal preferences. Make your folder structure the way you are most comfortable working with.
For example, I love my code to be split in submodules or simply said to be in different directories, accordingly to the job the code is used for.
I'd personally go with a couple of folders:
.{src}
├── controllers # All controller operations according the routes are stored here
│ ├── authController.ts # Handles authentication requests
│ ├── usersController.ts # Handles users route requests
│ └── ...
│
├── database # All database connections are stored here. For example you have two databases
│ ├── db.ts # Initialize DB connection
│ └── ...
│
├── middleware
│ ├── authenticated.ts # Decode and verify JWT token
│ ├── error.ts # Common Error Handler
│ ├── logger.ts # Control logging levels
│ └── ...
│
├── models # Simple descriptor of the database tables
│ ├── usersModel.ts # DB model for users
│ └── ...
│
├── schema # Schemas that are used for CRUD operations with the models
│ ├── users.ts # DB Schema for users
│ └── ...
│
├── listeners
│ ├── socketsManager.ts # Socket listeners/emitters handle
│ └── ...
│
├── app.ts # Entry file for the project
├── .env # Store environment variables
├── routes.ts # All routes initializer
└── ...
Although, you might not like the naming or the order I've put on, you can create your own one you feel comfortable with. Another thing you might do is take a look at some conventions that are up on the internet or some of the most popular projects like frameworks.
It's all depending on your needs and what you feel comfortable working with, after all.
I am using Terraform to deploy an 3-tiers application on Azure: Frontend, Backend, and DB. It works well.
The application is a product, and I need several instances for several customers.
My problem is, I have a hard time understanding applying several sets of values (variables.tf) to the same script, in order to get several environments.
I would like to have the following structure:
main.tf
customer1
variables.tf
customer2
variables.tf
And select whether I deploy customer1 or customer2. I read about terraform workspaces, and started creating one workspace per customer. But I don't understand how to apply a different set of values for the same scripts depending on the working workspace. It's difficult to find a comprehensive example online, what I am trying to do should be the bread and butter of terraform.
Thanks!
As both of you and #ydaetskcoR knew, workspace is one choice. If it is not suitable for your business. There is another way for you, and I would recommend it for your use case.
Terragrunt is a thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules.
So in your case, it can be easily used to manage different customers as below structure.
└── customer1
├── prod
│ ├── app
│ │ └── terraform.tfvars
│ ├── mysql
│ │ └── terraform.tfvars
│ └── vpc
│ └── terraform.tfvars
└── stage
├── app
│ └── terraform.tfvars
├── mysql
│ └── terraform.tfvars
└── vpc
└── terraform.tfvars
└── customer2
├── prod
│ ├── app
│ │ └── terraform.tfvars
│ ├── mysql
│ │ └── terraform.tfvars
│ └── vpc
│ └── terraform.tfvars
└── stage
├── app
│ └── terraform.tfvars
├── mysql
│ └── terraform.tfvars
└── vpc
└── terraform.tfvars