I am pretty new to Terraform and Terraform Cloud and I'm looking at the best way to structure my Terraform Cloud Workspaces.
Use Case: Relatively simple webapp, RDS, ECS/Fargate etc
I am currently evaluating with the following workspaces:
ECR
Database
AppCore (ECS Cluster, ALB, etc etc)
ECS Service/Tasks
Benefits: Small blast radius, logical chunks, can use Terraform when updating to new ECS task definitions.
I thought I found a doc on Terraforms site that suggested breaking workspaces similar to this was their recommended approach but I can't seem to find it again at the moment.
Is this good? bad? I've heard putting all your infrastructure in a single workspace can make things painful later.
Any ideas, thoughts or suggestions greatly appreciated!
Related
I am wondering what would be the best way to manage big set of zones in route53.
The things i wanna benefit here are:
have zones under version control
manage zones via code like for example tool Terraform or maybe developing some cli tool for this.
no manual changes to zones
my problem starts that I have big number of zones in route53 and is hard to track changes and who did what when things start breaking, and i am not sure how i could migrate all into Terraform.
Is there a way to dump current setup into TF ?
Or maybe there is some other better idea how to mange this ?
Are there any benefits to spawn a new AWKS EKS cluster by using terraform or eksctl?
Are there some long-term maintenance benefits of one vs another?
Well, although I haven't actually tried this out with Terraform, I can definitely say that the eksctl way is not recommended. At least not if you're interested in manageing your infrastructure as code.
With eksctl, most changes to an existing cluster need to be made with specific eksctl commands. Just changing the (declarative) cluster.yaml (or whatever you name) does not apply anything relevant. You want to scale a nodeGroup? Well, please use eksctl scale nodegroup, as changing the size in the YAML file is not applying anything. I think you get the pattern.
It's really sad that, of all companies, Weaveworks, the "inventors" of GitOps, provide a tool that does not even support basic IaC :(
I would highly recommend using terraform. It is declarative and provides an interface that can be used to support all of your infrastructure and not just your EKS cluster(s).
The time and effort you put into learning terraform and implementing it in your pipeline can be easily re-used for other infrastructure needs unlike eksctl.
I am having difficulty in understanding whether to use aws cdk or terraform cdktf. From my work so far, it appears to me that aws cdk has more robust patterns and constructs which will comply with the well architected framework, all available out of the box. Terraform cdktf will require hand crafting many of such constructs and patterns.
An example in point is the construct ecs_patterns.ApplicationLoadBalancedEc2Service which does a lot of heavy lifting in spinning up a industrial strength infrastructure for EC2 based ECS service. I can't find the equivalent of that in Terraform cdktf and it appears I will have to manually assemble and connect all the infra elements.
However, Terraform apparently has the advantage of working with several different cloud vendors, and therefore I want to do due diligence before choosing one of the other.
Therefore, I would like to know if my understanding is correct and if I am not missing something really important. Any other advice / considerations in this matter are highly appreciated
Thanks
The AWS CDK is limited to only AWS as a cloud, whereas you can use CDKTF with any / most of the clouds since terraform providers exist for most of them. The AWS CDK has a diverse ecosystem of constructs that can be used, which is a plus. CDKTF will eventually support these via the CDKTF AWS Adapter.
I'm with a dilema here about which SE site to ask this question so please help me out if it should be somewhere else.
I've been looking into Infrastructure as Code solutions.
Didn't like Terraform too much. The lack of intellisense makes discoberability harder than programmers have been used to.
I've been considering ARM templates. I like it that the templates are made available as we create resources in the portal but it seems way less readable and harder to maintain afterwards.
Then I found out Pulumi and love their idea compared to Terraform. The way I see it, they're approach is also declarative like the above options but we can use decent programming languages to get the job done.
The for loops is a must.
Cool, I like that! But since we like using C# (or other alternatives), then why don't we SDKs to manage our infrastructure as code?
Pulumi has compared themselves with cloud SKDs by positioning their solution as much safer advocating that, if we just use a cloud SDK ourselves, then our solution wouldn't be that reliable.
To what extent is this really true, I wonder?
Last year, I wrote some libraries that used Azure service bus queues/topics. There were several integration tests that would run in parallel and I needed to isolate them by creating new queues/topics and used Microsoft.Azure.ServiceBus.Management.ManagementClient to do this.
It really didn't seem like I had to learn anything at all.
Going to the point now. Not discarding Pulumi's innovation which I think is great:
Will Pulumi's really add that much benefit compared to using Azure SDKs?
What's been your experience with it?
A Pulumi developer here, so I'm definitely biased. I suspect the SO community may find your question violating some of the guidance, but I hope my answer survives :)
One upside of using Pulumi is that you get access to multiple providers with consistent developer experience. You may be using exclusively Azure, but you might at some point start combining it with things like building and publishing Docker images, deploying Kubernetes applications, or Datadog dashboards. All can be done from the same program or solution.
Now, the biggest difference with imperative SDKs is the notion of desired-state configuration. A Pulumi program describes the graph of resources and dependencies between them (what), not the steps to provision them (how). When you have an environment that lives for months and years, there's a big difference between evolving a single definition with baby steps and applying incremental changes (Pulumi) and writing a bunch of update scripts/programs to bring each environment to the new state (SDK).
How do you maintain multiple environments that may be similar but still different? (production vs staging vs test vs dev) How do you make sure that your short-lived infra that you created for nightly tests reflects the reality of production? What happens when an SDK program fails in the middle - can you retry running it again or will it create duplicate resources/fail with another error? How do you get a simple overview of changes over time in git? Concurrency control? Change history?
All the things above are baked into Pulumi and require manual consideration with a cloud SDK.
I want to manage multiple environments with difference in instance size and instance cost. Can we use Terragrunt used for that purpose. Also is Terragrunt used in Azure?
Terragrunt is just a wrapper for Terraform script, not related to the actual provider. Terragrunt worked better for Terraform < 0.12 but now I would not recommend it.
For handling multiple environments you can:
Use Terraform Workspaces (available for Azure backend)
Use a smart layout of directories and modules. Check here - with Azure example.
Yes it is available. You have this nice article explaining how to use it on Azure. But, i would not recommend you to go in that direction. Most of the Terragrunt features have been adopted by Terraform Enterprise. Terragrunt allows you to work faster than Terraform for some cases but it has the main issues than Terraform. In my opinion, the main benefits are the plan-all/apply-all command if you have a lot of different sub-resources grouped by folders, but you can work on and achieve the same with Terraform.