I am a little stuck on the best way to deliver the following deployments.
For clarity, child terragrunt.hcl files are pulling from a private repo.
.
├── env01
│ ├── resource_group
│ │ └── terragrunt.hcl
│ ├── virtual_network
│ │ └── terragrunt.hcl
│ └── subnet
│ └── terragrunt.hcl
├── env02
│ ├── resource_group
│ │ └── terragrunt.hcl
│ ├── virtual_network
│ │ └── terragrunt.hcl
│ └── subnet
│ └── terragrunt.hcl
└── terragrunt.hcl
How can I, in the above terragrunt layout, merge the child terragrunt.hcl files into a single state file, at present, we have a state file per terragrunt.hcl per resource, which, while works, feels overkill for me.
Do I need to do this at the terraform end? Creating modules for the env01/env02 deployments or is there a way to achieve some kind of terraform merge when deploying?
Hope that makes sense...
Related
I'm new to Terraform and have been spending some time reading up and had a couple of questions related to the best way to structure the code. My plan is to store my Terraform code all in a single repository for all my projects. The infrastructure (Azure) consists of Virtual Machines and an AKS cluster. 98% of our VMs are all exactly the same except for subscription (uat/prod), size, resource group, name and maybe few differences in data disks. My first idea was to create a single environment for virtual machines with multiple .tfvars files for each of the VMs I want to manage? Like this:
└── terraform/
├── virtual_machines/
│ ├── main.tf
│ ├── variables.tf
│ ├── outputs.tf
│ ├── vm1.tfvars
│ ├── vm2.tfvars
│ └── vm3.tfvars
└── aks/
└── aks_project/
├── main.tf
├── variables.tf
└── outputs.tf
Then I can just specify what .tfvars files to point to when applying?
The second idea would be to put the shared vm code in a module then create a directory for each VM like this:
└── terraform/
├── virtual_machines/
│ ├── modules/
│ │ └── virtual_machines/
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ ├── vm1/
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ ├── vm2/
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ └── vm3/
│ ├── main.tf
│ ├── variables.tf
│ └── outputs.tf
└── aks/
└── aks_project/
├── main.tf
├── variables.tf
└── outputs.tf
With each VM directory sourcing from the modules and including the variable values needed for each?
Are either of these approaches make sense for what I'm looking to do? What would you suggest for or use for organizing your terraform code?
Your second approach looks like a better start, as it applies Hashicorp's recommendation of using modules whenever it's possible.
Then you can simply define each VM in a separate .tf file (or group all of them in one), each one of these files should use your local VM module that sets your defaults, and then provide multiple tfvars for each environment.
So you would have something like:
.
├── main.tf
└── virtual_machines
├── modules
│ └── virtual_machines
│ ├── README.md
│ ├── main.tf
│ ├── outputs.tf
│ └── variables.tf
├── dev.tfvars
├── uat.tfvars
├── prod.tfvars
├── variables.tf
├── locals.tf
├── vm1.tf
└── vm2.tf
Where vm1.tf is using your local module and sets its custom parameters, for example:
module "vm1" {
source = "./modules/virtual_machines"
name = "my-vm1-name"
environment = var.environment
}
My var.environment here is defined by my tfvars file which will be picked up for each environment, if I'm on dev I would run my plan or apply with dev.tfvars.
Read more at:
https://cloud.google.com/docs/terraform/best-practices-for-terraform#module-structure
https://www.hashicorp.com/blog/structuring-hashicorp-terraform-configuration-for-production#considerations-for-defining-modules
https://developer.hashicorp.com/terraform/tutorials/modules/pattern-module-creation?in=terraform%2Fmodules&product_intent=terraform
https://www.terraform-best-practices.com/examples/terraform
Also, you may want to consider options like terragrunt for this scenario of configuration DRY.
I hope this answer helps, it's based on my opinions and my experience. You may have a different use case, and a different setup, and you may have other answers from smarter people suggesting different approaches since there's nothing such as a "Best terraform project structure".
My current project tree for cypress looks something like this:
├── cypress
│ ├── OtherProjectFolder
│ │ ├── frontend
│ │ │ └── TestUI.feature
│ ├── pages_objects
│ │ ├── mainPage.js
│ └── step_definitions
│ │ └── Testui.js
│ ├── e2e
│ │ ├── backend
│ │ │ └── TestBackend.feature
│ ├── pages_objects
│ │ ├── backendPage.js
│ └── step_definitions
│ │ └── TestBackend.js
Essentially I want to define all my step definitions in a different director, and all my page objects in a different directory, because I have many project to automate.
Here is my current cucumber preprocessor look like in package.json:
"cypress-cucumber-preprocessor": {
"nonGlobalStepDefinitions": false,
"step_definitions": "cypress/e2e"
}
If I change the path of the stepsDefinition to "cypress/OtherProjectFolder", this time it does not picked the steps in e2e. If I just type "cypress" I get this error. Please check attached screenshot. I'm wondering if there is a way to make stepDefinitions global?
try:
"cypress-cucumber-preprocessor": {
"nonGlobalStepDefinitions": false,
"stepDefinitions": "cypress/your_folder_name/*.js"
}
you can of course add more path after that.
Good to look at:
https://github.com/badeball/cypress-cucumber-preprocessor/blob/master/docs/step-definitions.md
What would be the best place for cross-environment Terraform configuration files in Terragrunt? Let's say that the directory structure looks like this:
└── live
├── terragrunt.hcl
├── prod
│ ├── app
│ │ └── terragrunt.hcl
│ ├── mysql
│ │ └── terragrunt.hcl
│ └── vpc
│ └── terragrunt.hcl
├── qa
│ ├── app
│ │ └── terragrunt.hcl
│ ├── mysql
│ │ └── terragrunt.hcl
│ └── vpc
│ └── terragrunt.hcl
└── stage
├── app
│ └── terragrunt.hcl
├── mysql
│ └── terragrunt.hcl
└── vpc
└── terragrunt.hcl
All nested terragrunt.hcl files are sourcing corresponding modules, eg. prod/app/terragrunt.hcl has a source in modules/app where terraform configs live:
modules
├── app
│ └── ...terraform configs
├── mysql
│ └── ...terraform configs
└── vpc
└── ...terraform configs
This works fine. But where in this directory structure should common terraform configs live (terragrunt documentation says about common terragrunt config to dry the configuration)? Modules seems to be a good place for reusable (among environments) configurations...
I am using Terraform to deploy an 3-tiers application on Azure: Frontend, Backend, and DB. It works well.
The application is a product, and I need several instances for several customers.
My problem is, I have a hard time understanding applying several sets of values (variables.tf) to the same script, in order to get several environments.
I would like to have the following structure:
main.tf
customer1
variables.tf
customer2
variables.tf
And select whether I deploy customer1 or customer2. I read about terraform workspaces, and started creating one workspace per customer. But I don't understand how to apply a different set of values for the same scripts depending on the working workspace. It's difficult to find a comprehensive example online, what I am trying to do should be the bread and butter of terraform.
Thanks!
As both of you and #ydaetskcoR knew, workspace is one choice. If it is not suitable for your business. There is another way for you, and I would recommend it for your use case.
Terragrunt is a thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules.
So in your case, it can be easily used to manage different customers as below structure.
└── customer1
├── prod
│ ├── app
│ │ └── terraform.tfvars
│ ├── mysql
│ │ └── terraform.tfvars
│ └── vpc
│ └── terraform.tfvars
└── stage
├── app
│ └── terraform.tfvars
├── mysql
│ └── terraform.tfvars
└── vpc
└── terraform.tfvars
└── customer2
├── prod
│ ├── app
│ │ └── terraform.tfvars
│ ├── mysql
│ │ └── terraform.tfvars
│ └── vpc
│ └── terraform.tfvars
└── stage
├── app
│ └── terraform.tfvars
├── mysql
│ └── terraform.tfvars
└── vpc
└── terraform.tfvars
My team uses a Puppet architecture which currently accommodates a single application in multiple environments (vagrant, staging, production.)
We now want to expand the scope of this setup to support additional applications. Many of them will use a subset of the existing modules we've already defined, and others will call for new modules to be defined (which may or may not be shared.)
What is the most appropriate Puppet architecture, for supporting multiple environments of multiple applications?
In such an architecture, each application would amount to a module, presumably. What's the best way of (file-) structurally differentiating between a module which is an application, and a module which is a dependency of one or more modules?
Could it be as simple as adding a third modules folder under a top-level applications folder, for example? Or is there a better tiering strategy?
Research so far hasn't turned up any best-practice examples / boilerplates, e.g. via example42 or puppetlabs on GitHub.
Our file structure:
puppet
├── environments
│ ├── production → manifests → init.pp
│ ├── staging → manifests → init.pp
│ └── vagrant → manifests → init.pp
├── hiera.yaml
├── hieradata
│ ├── accounts.yaml
│ ├── common.yaml
│ └── environments
│ ├── production.yaml
│ ├── staging.yaml
│ └── vagrant.yaml
├── modules
│ ├── acl [..]
│ ├── newrelic [..]
│ ├── nginx [..]
│ └── puma [..]
└── vendor
├── Puppetfile
├── Puppetfile.lock
└── modules [..]
I'm sure there are a lot of opinions on what the 'most appropriate' solution for this is, but I'll give you mine.
Puppet is actually designed to support multiple applications in multiple environments right out of the box, with some notable caveats:
All common dependencies (within a single environment) must be pinned to the same version
So if you have three applications that need Apache, you can only have one Apache module
All applications can be referenced using a distinctive name
I.E. If you have three different node.js applications that require their own module, you would need three uniquely named modules (or manifests) for them
You are willing to tackle the upkeep/maintenance of updating dependencies for multiple applications simultaneously
If app1 needs to update an Apache module dependency, you're willing to make sure that apps 2-* remain compatible
The other thing to keep in mind is that Puppet's terminology for 'environment' is an acknowledged misnomer. Most well operated environments I have seen actually have distinct Puppet masters in each of their true 'environments' (vagrant/dev/stage/prod) in order to avoid the perils of environment leakage as well as test out upgrades to the Puppet infrastructure (You should have somewhere to test an upgrade to your Puppet version that doesn't instantly impact your production)
Therefore, this frees up the Puppet 'environment directories' to operate free of the true 'environment' concept, and should be considered 'a collection of modules at a particular revision' instead of an 'environment'. You do still need to be cognizant of environment leakage, but this does open up a potential avenue for splitting up your modules.
Another concept you will want to keep in mind is Roles and Profiles (Well discussed by Gary Larizza, Adrien Thebo, and Craig Dunn). These help enable separating business logic from technology management modules. You can then handle dependency ordering and business oriented logic separate from the code/modules for managing individual components.
With all of these concepts in place, here are two architectural layouts that may be a good fit in your use case:
Environments by application
puppet
├── environments (Managed by r10k/code manager)
│ ├── app1
│ │ └── modules
│ │ ├── profiles [..]
│ │ └── app1_specific_component [..]
│ ├── app2
│ │ └── modules
│ │ ├── profiles [..]
│ │ └── app2_specific_component [..]
│ └── app3
│ └── modules
│ ├── profiles [..]
│ └── app3_specific_component [..]
├── hiera.yaml
├── hieradata
│ ├── accounts.yaml
│ ├── common.yaml
│ └── applications
│ ├── app1
│ │ ├── default.yaml
│ │ └── environments (server environments)
│ │ ├── vagrant
│ │ │ └── roles
│ │ │ ├── role1.yaml
│ │ │ ├── role2.yaml
│ │ │ └── role3.yaml
│ │ ├── stg
│ │ │ └── roles
│ │ │ ├── role1.yaml
│ │ │ ├── role2.yaml
│ │ │ └── role3.yaml
│ │ └── prd
│ │ └── roles
│ │ ├── role1.yaml
│ │ ├── role2.yaml
│ │ └── role3.yaml
│ ├── app2
│ │ ├── default.yaml
│ │ └── environments
│ │ ├── vagrant
│ │ │ └── roles
│ │ │ ├── role1.yaml
│ │ │ ├── role2.yaml
│ │ │ └── role3.yaml
│ │ ├── stg
│ │ │ └── roles
│ │ │ ├── role1.yaml
│ │ │ ├── role2.yaml
│ │ │ └── role3.yaml
│ │ └── prd
│ │ └── roles
│ │ ├── role1.yaml
│ │ ├── role2.yaml
│ │ └── role3.yaml
│ └── app3
│ ├── default.yaml
│ └── environments
│ ├── vagrant
│ │ └── roles
│ │ ├── role1.yaml
│ │ ├── role2.yaml
│ │ └── role3.yaml
│ ├── stg
│ │ └── roles
│ │ ├── role1.yaml
│ │ ├── role2.yaml
│ │ └── role3.yaml
│ └── prd
│ └── roles
│ ├── role1.yaml
│ ├── role2.yaml
│ └── role3.yaml
├── modules (These are common to all environments, to prevent leakage)
│ ├── acl [..]
│ ├── newrelic [..]
│ ├── nginx [..]
│ └── puma [..]
└── vendor
├── Puppetfile
├── Puppetfile.lock
└── modules [..]
Environments as a 'release' (for iteration on Puppet code over time)
puppet
├── environments (Managed by r10k/code manager)
│ ├── release_1
│ │ └── modules
│ │ ├── profiles [..]
│ │ ├── app1_specific_component [..]
│ │ ├── app2_specific_component [..]
│ │ ├── app2_specific_component [..]
│ │ ├── acl [..] (v1)
│ │ ├── newrelic [..]
│ │ ├── nginx [..]
│ │ └── puma [..]
│ ├── release_2
│ │ └── modules
│ │ ├── profiles [..]
│ │ ├── app1_specific_component [..]
│ │ ├── app2_specific_component [..]
│ │ ├── app2_specific_component [..]
│ │ ├── acl [..] (v1.1)
│ │ ├── newrelic [..]
│ │ ├── nginx [..]
│ │ ├── puma [..]
│ │ └── some_new_thing_for_release_2 [..]
│ └── release_3
│ └── modules
│ ├── profiles [..]
│ ├── app1_specific_component [..]
│ ├── app2_specific_component [..]
│ ├── app2_specific_component [..]
│ ├── acl [..] (v2.0)
│ ├── newrelic [..]
│ ├── nginx [..]
│ ├── puma [..]
│ ├── some_new_thing_for_release_2 [..]
│ └── some_new_thing_for_release_3 [..]
├── hiera.yaml
├── hieradata
│ ├── accounts.yaml
│ ├── common.yaml
│ ├── environments
│ │ ├── release_1.yaml
│ │ ├── release_2.yaml
│ │ └── release_3.yaml
│ └── roles
│ ├── role1
│ │ ├── default.yaml
│ │ ├── environments (server environments)
│ │ │ ├── vagrant
│ │ │ │ ├── defaults.yaml
│ │ │ │ └── release (optional, only if absolutely necessary)
│ │ │ │ ├── release_1.yaml
│ │ │ │ ├── release_2.yaml
│ │ │ │ └── release_3.yaml
│ │ │ ├── stg
│ │ │ │ ├── defaults.yaml
│ │ │ │ └── release (optional)
│ │ │ │ ├── release_1.yaml
│ │ │ │ ├── release_2.yaml
│ │ │ │ └── release_3.yaml
│ │ │ └── prd
│ │ │ ├── defaults.yaml
│ │ │ └── release (optional)
│ │ │ ├── release_1.yaml
│ │ │ ├── release_2.yaml
│ │ │ └── release_3.yaml
│ ├── role2
│ │ ├── default.yaml
│ │ └── environments
│ │ ├── vagrant
│ │ │ └── defaults.yaml
│ │ ├── stg
│ │ │ └── defaults.yaml
│ │ └── prd
│ │ └── defaults.yaml
│ └── role3
│ └── default.yaml
├── modules (Anything with ruby libraries should go here to prevent leakage)
│ ├── stdlib [..]
└── vendor
├── Puppetfile
├── Puppetfile.lock
└── modules [..]
Keep in mind that the nesting order (release/environment/role etc...) is flexible based on what makes the most sense for your implementation (and some can be eliminated if you're not going to use them).
I encourage you to take this information as merely a starting point, and not a concrete 'do this for instant success'. Having a highly skilled Puppet Architect work with you to understand your precise needs and environments will end up in a far better tuned and appropriate solution than the assumptions and 'cookie cutter' type solutions you are likely to find online (including mine).