I want to manage multiple environments with difference in instance size and instance cost. Can we use Terragrunt used for that purpose. Also is Terragrunt used in Azure?
Terragrunt is just a wrapper for Terraform script, not related to the actual provider. Terragrunt worked better for Terraform < 0.12 but now I would not recommend it.
For handling multiple environments you can:
Use Terraform Workspaces (available for Azure backend)
Use a smart layout of directories and modules. Check here - with Azure example.
Yes it is available. You have this nice article explaining how to use it on Azure. But, i would not recommend you to go in that direction. Most of the Terragrunt features have been adopted by Terraform Enterprise. Terragrunt allows you to work faster than Terraform for some cases but it has the main issues than Terraform. In my opinion, the main benefits are the plan-all/apply-all command if you have a lot of different sub-resources grouped by folders, but you can work on and achieve the same with Terraform.
Related
I have 3 environments to manage via Terraform: dev, staging, prod.
An example of use case is below:
create a "common" service account for each environment (sa-xxx#dev + sa-xxx#staging + sa-xxx#prod)
create a "dev-specific" role for this sa-xxx#dev SA
create a "staging-specific" role for this sa-xxx#staging SA
create a "prod-specific" role for this sa-xxx#prod SA
How can I easily manage common & specific resources for each environment?
Terraform is very simple if all environments are equals, but for specificities it looks more complicated. The goal is have a structural way to manage it, and then to avoid:
code duplication in 3 distinct folders
"count" conditions in each tf resource definition
It should be possible for Terraform to look into current root folder UNION dev/staging/prod folder (depending on the environment).
The need is very simple but implementation seems so difficult.
Thanks for help ! :)
This is a pretty broad question and so it's hard to answer specifically, but one general answer to this question is to make use of shared modules as a means for sharing code between your separate configurations.
The Module Composition guide describes some different patterns that might help you in your goal. The idea would be to make each of your configurations share modules wherever it makes sense for them to do so but to also potentially use different modules -- or the same modules but with different relationships/cardinalities -- so that your configurations can represent both what is the same and what is different between each of them.
One way would be to put shared resources in a common configuration managed in a remote state. Then in other configurations, you can refer to the shared, remote state using terraform_remote_state data source.
I am pretty new to Terraform and Terraform Cloud and I'm looking at the best way to structure my Terraform Cloud Workspaces.
Use Case: Relatively simple webapp, RDS, ECS/Fargate etc
I am currently evaluating with the following workspaces:
ECR
Database
AppCore (ECS Cluster, ALB, etc etc)
ECS Service/Tasks
Benefits: Small blast radius, logical chunks, can use Terraform when updating to new ECS task definitions.
I thought I found a doc on Terraforms site that suggested breaking workspaces similar to this was their recommended approach but I can't seem to find it again at the moment.
Is this good? bad? I've heard putting all your infrastructure in a single workspace can make things painful later.
Any ideas, thoughts or suggestions greatly appreciated!
Are there any benefits to spawn a new AWKS EKS cluster by using terraform or eksctl?
Are there some long-term maintenance benefits of one vs another?
Well, although I haven't actually tried this out with Terraform, I can definitely say that the eksctl way is not recommended. At least not if you're interested in manageing your infrastructure as code.
With eksctl, most changes to an existing cluster need to be made with specific eksctl commands. Just changing the (declarative) cluster.yaml (or whatever you name) does not apply anything relevant. You want to scale a nodeGroup? Well, please use eksctl scale nodegroup, as changing the size in the YAML file is not applying anything. I think you get the pattern.
It's really sad that, of all companies, Weaveworks, the "inventors" of GitOps, provide a tool that does not even support basic IaC :(
I would highly recommend using terraform. It is declarative and provides an interface that can be used to support all of your infrastructure and not just your EKS cluster(s).
The time and effort you put into learning terraform and implementing it in your pipeline can be easily re-used for other infrastructure needs unlike eksctl.
Not sure there is going to be a right or wrong answer for this one, but I am just interested how people manage Terraform in the real world? In terms of do you use modules, different environments and collaborations.
At the moment we are planning on having a production, dev and test environments. All similar.
Now at the moment I have made my terraform files in a way that define individual components of AWS, so say one for, VPC, IAM, EC2, Monitoring (CloudWatch + CloudTrail + CloudConfig) etc. And there is one variable file and .tfvars for the above, so the files are portable (all environments will be the same). So if you need to change something its all in one place. Also means if we have a specific project running I can create a tf file defining all the resource for the project and drop it in, then once its completed remove it.
Each environment has its own folder structure on our Terraform server.
Is this too simplistic? I keep looking at module.
Also does anyone have experience of collaboration with Terraform, as in different teams? I have been looking at things like Atlantis to tie into GitHub, so any changes need to be approved. But also at the sametime with the correct IAM role I can limit what Terraform can change.
Like I said may not be a wrong of right answer just interested in how people are managing terraform and their experiences.
Thanks
My answer is just an use case...
We are using terraform for an application deployed for several customers each having small specific configuration features.
We have only one CVS repository. We don't use CVS branches mechanism.
For each folder, we have remote states at least to share states between developers.
We are using one global folder having remote states also to share states between customers configurations
We are using one folder per customer and using workspaces (former environment) for each context for each customer (prod:blue/green, stage)
For common infrastructure chunks shared by all customers, we use module
We mainly use variables to reduce the number of specific files in each customer folders.
Hope this will help you...
I would like to use Terraform programmatically like an API/function calls to create and teardown infrastructure in multiple specific steps. e.g reserve a couple of eips, add an instance to a region and assign one of the IPs all in separate steps. Terraform will currently run locally and not on a server.
I would like to know if there is a recommended way/best practices for creating the configuration to support this? So far it seems that my options are:
Properly define input/output, heavily rely on resource separation, modules, the count parameter and interpolation.
Generate the configuration files as JSON which appears to be less common
Thanks!
Instead of using Terraform directly, I would recommend a 3rd party build/deploy tool such as Jenkins, Bamboo, Travis CI, etc. to manage the release of your infrastructure managed by Terraform. Reason being is that you should treat your Terraform code in the exact same manner as you would application code (i.e. have a proper build/release pipeline). As an added bonus, these tools come integrated with a standard api that can be used to execute your build and deploy processes.
If you choose not to create a build/deploy pipeline, your other options are to use a tool such as RunDeck which allows you to execute arbitrary commands on a server. It also has the added bonus of having a excellent privilege control system to only allow specified users to execute commands. Your other option could be to upgrade from the Open Source version of Terraform to the Pro/Premium version. This version includes an integrated GUI and extensive API.
As for best practices for using an API to automate creation/teardown of your infrastructure with Terraform, the best practices are the same regardless of what tools you are using. You mentioned some good practices such as clearly defining input/output and creating a separation of concerns which are excellent practices! Some others I can recommend are:
Create all of your infrastructure code with idempotency in mind.
Use modules to separate the common shared portions of your code. This reduces the number of places that you will have to update code and therefore the number of points of error when pushing an update.
Write your code with scalability in mind from the beginning. It is much simpler to start with this than to adjust later on when it is too late.