I am having difficulty in understanding whether to use aws cdk or terraform cdktf. From my work so far, it appears to me that aws cdk has more robust patterns and constructs which will comply with the well architected framework, all available out of the box. Terraform cdktf will require hand crafting many of such constructs and patterns.
An example in point is the construct ecs_patterns.ApplicationLoadBalancedEc2Service which does a lot of heavy lifting in spinning up a industrial strength infrastructure for EC2 based ECS service. I can't find the equivalent of that in Terraform cdktf and it appears I will have to manually assemble and connect all the infra elements.
However, Terraform apparently has the advantage of working with several different cloud vendors, and therefore I want to do due diligence before choosing one of the other.
Therefore, I would like to know if my understanding is correct and if I am not missing something really important. Any other advice / considerations in this matter are highly appreciated
Thanks
The AWS CDK is limited to only AWS as a cloud, whereas you can use CDKTF with any / most of the clouds since terraform providers exist for most of them. The AWS CDK has a diverse ecosystem of constructs that can be used, which is a plus. CDKTF will eventually support these via the CDKTF AWS Adapter.
Related
I am pretty new to Terraform and Terraform Cloud and I'm looking at the best way to structure my Terraform Cloud Workspaces.
Use Case: Relatively simple webapp, RDS, ECS/Fargate etc
I am currently evaluating with the following workspaces:
ECR
Database
AppCore (ECS Cluster, ALB, etc etc)
ECS Service/Tasks
Benefits: Small blast radius, logical chunks, can use Terraform when updating to new ECS task definitions.
I thought I found a doc on Terraforms site that suggested breaking workspaces similar to this was their recommended approach but I can't seem to find it again at the moment.
Is this good? bad? I've heard putting all your infrastructure in a single workspace can make things painful later.
Any ideas, thoughts or suggestions greatly appreciated!
While I'm trying to train new people on Terraform, I always find it quite cumbersome to have to deal with real infrastructure.
First, because it involves finding a non-sensitive cloud account or creating a new one, creating an identity for the new user (including setting-up some security stuff like two FA, ...), which could take some times (especially if you are in a traditional corporate environment where finding a CB to make payments is almost impossible).
Second, because as you are creating real infrastructure, you rapidly come into quirks that are impeding the learning curve, like the time it takes to create various types of infrastructure, the cost associated with some stuff, the need to deprovision them afterward since they are just tests, ...
Are you aware of any sandbox environment where it would be very easy to create infrastructure with Terraform (even not a real one), in order to concentrate on Terraform and stop wasting time on "side-stuff"? Do you share the same struggle?
Thanks in advance
Terraform does support LocalStack which is:
LocalStack provides an easy-to-use test/mocking framework for developing Cloud applications. It spins up a testing environment on your local machine that provides the same functionality and APIs as the real AWS cloud environment.
So you could set it up and test it how it would suit your teaching requirements.
If you are in academia and are working with AWS, AWS offers AWS Educate for students for free. Thus, you could also use that for sandbox if possible.
Are there any benefits to spawn a new AWKS EKS cluster by using terraform or eksctl?
Are there some long-term maintenance benefits of one vs another?
Well, although I haven't actually tried this out with Terraform, I can definitely say that the eksctl way is not recommended. At least not if you're interested in manageing your infrastructure as code.
With eksctl, most changes to an existing cluster need to be made with specific eksctl commands. Just changing the (declarative) cluster.yaml (or whatever you name) does not apply anything relevant. You want to scale a nodeGroup? Well, please use eksctl scale nodegroup, as changing the size in the YAML file is not applying anything. I think you get the pattern.
It's really sad that, of all companies, Weaveworks, the "inventors" of GitOps, provide a tool that does not even support basic IaC :(
I would highly recommend using terraform. It is declarative and provides an interface that can be used to support all of your infrastructure and not just your EKS cluster(s).
The time and effort you put into learning terraform and implementing it in your pipeline can be easily re-used for other infrastructure needs unlike eksctl.
Need an expert's input who has used cloudformation, sam and serverless framework to deploy nodejs app.
Please advise which is the best path to take I have used serverless framework but not sam or cloudformation, while I agree that it simplifies the process, I wish to learn more on the underlying configuration.
I am leaning towards cloudformation just because both the frameworks transform the code to cloudformation templates. Please correct me if I am wrong and appreciate the best resources to learn the same.
SAM is basically an extension of Cloudformation. If you know SAM, you basically know Cloudformation. SAM can and should be used in conjunction w/ Cloudformation for testing locally.
Serverless is an abstraction layer on top of Cloudformation. It helps to expedite app creation and deployment. It falls short if you are doing more advanced configurations
I always lean towards Cloudformation (or SAM) because it is provided by the CSP (ie AWS). This means everything new will be automatically available, rather than waiting for an abstraction layer (like serverless) to bake in support.
I recommend the Serverless Framework. The advantages are several:
Much easier syntax than CloudFormation
More portable than CloudFormation (you can change between cloud providers)
Ability to write raw CloudFormation inline in serverless.yml if needed
Easy to deploy (just run sls deploy)
There is a sample project at my gitlab profile if you are interested (for GoLang but the principles are the same as for other runtimes) https://gitlab.com/montao/aws-lambda-go-gitlab-sls
LostJon is absolutely correct in that the support for both CloudFormation and SAM is immediately available as it is natively provided by AWS. I am leaning to CloudFormation for the reason that is has a deterministic outcome, which is desired in a DevOps landscape. There are no transformations unless you explicitly add transformations to your CloudFormation stack.
When you deploy a CloudFormation template with SAM, you will be able to view the CloudFormation template in the AWS console, but you can also choose to show the rendered template (e.g. after transformations) so that you can learn what SAM does for you underwater. You could copy the rendered resources and use them as pure CloudFormation. This allows you to speed up development, while not losing the benefit of knowledge of the underlying configuration.
I am looking at using Amazon cloud services (EC2, S3 etc) for hosting. I've been looking at the JSON metadata that can be specified to configure various instances and the complexity has me concerned. Is there a dsl that will generate valid JSON metadata and more importantly validate the entries?
Unfortunately, I drew a blank after searching for this recently. I'm using Amazon Web Services CloudFormation (is that the JSON metadata you're talking about?).
There are a couple of issues with CloudFormation JSON files:
I'm at well over 1,500 lines and it's impossible to read,
You can't express everything the API gives you, notably in the area of Virtual Private Clouds,
There are lots of bugs that are taking a long time to fix, Elastic Load Balancers losing HTTPS information, for example.
So I've been using straight-up API calls in Scala using the Java API. It's actually really nice.
The Java API has a flavor of "setters" starting with with that return this so they can be chained. In Scala, you can use these to act like a poor-man's DSL. So you can do things like
val updateRequest = new UpdateAutoScalingGroupRequest()
.withAutoScalingGroupName(group.getAutoScalingGroupName)
.withAvailabilityZones(subnetAZsOfOurVPC)
.withVPCZoneIdentifier(subnetNamesOfOurVPC)
as.updateAutoScalingGroup(updateRequest)
Other things are easy to do in Scala with the Java API. For example, group all your Subnets by VPC in a Map, simply do
val subnetsByVPC = ec2.describeSubnets(new DescribeSubnetsRequest).getSubnets.groupBy(_.getVpcId)
In case anyone is still looking for the AWS CloudFormation DSL - we've been using the Ruby DSL for CloudFormation:
https://github.com/bazaarvoice/cloudformation-ruby-dsl
This neat project provides a tool to convert your existing CloudFormation template(s) to the Ruby DSL
It will generate valid JSON output
Validating Ruby template entries is similar to validating a regular CloudFormation template (see cfn-validate-template)
Your template becomes Ruby code, so it's easy to have reusable modules (DRY)
You can define local variables
You can have comments in your DSL template
Greatly improved readability
Greatly reduced DSL template size
The CloudFormation request template body size limits are annoying - we have to upload our large CloudFormation templates to S3, then create/update stacks using their S3 URLs.
There is now, though I haven't used it yet: Coffin a CoffeeScript DSL for CloudFormation.
If you're not talking about CloudFormation, but instead more general API talking, then the nicest interface I've found is AWS' own aws-sdk ruby gem. Unlike the other SDKs that they publish which are quite nicely-done-but-crude make-client/make-request/get-response/look-at-result affairs, the ruby SDK wraps a nicer domain-model over the top, so you interact via collections at a higher abstract level.
It also has quite-nice performance features in that you can cache responses to save on round-trips, if you know you don't need fresh responses.
I have CloudFormation templates upwards of 3000 lines. I have found that putting comments in the JSON helps tremendously!! You just have to strip it out before using it. There is a validator that would validate the template and strip out the comments: http://cloudformation-validation.com/
No, as of Feb 2022, AWS still does not provide any domain-specific language for infrastructure as code. They have nothing similar to Azure's Bicep or Terraform's HCL.
This really surprises me, as I generally think of AWS as being more expensive, but ahead of the curve, compared to other major competitors (Azure and GCP).
However, Cloud Formation now allows both JSON and YAML formats. Slight improvement?? IMHO, not really when you have an entire repo that represents your entire cloud stack. If you're using AWS, use Terraform to manage IaC.