Need an expert's input who has used cloudformation, sam and serverless framework to deploy nodejs app.
Please advise which is the best path to take I have used serverless framework but not sam or cloudformation, while I agree that it simplifies the process, I wish to learn more on the underlying configuration.
I am leaning towards cloudformation just because both the frameworks transform the code to cloudformation templates. Please correct me if I am wrong and appreciate the best resources to learn the same.
SAM is basically an extension of Cloudformation. If you know SAM, you basically know Cloudformation. SAM can and should be used in conjunction w/ Cloudformation for testing locally.
Serverless is an abstraction layer on top of Cloudformation. It helps to expedite app creation and deployment. It falls short if you are doing more advanced configurations
I always lean towards Cloudformation (or SAM) because it is provided by the CSP (ie AWS). This means everything new will be automatically available, rather than waiting for an abstraction layer (like serverless) to bake in support.
I recommend the Serverless Framework. The advantages are several:
Much easier syntax than CloudFormation
More portable than CloudFormation (you can change between cloud providers)
Ability to write raw CloudFormation inline in serverless.yml if needed
Easy to deploy (just run sls deploy)
There is a sample project at my gitlab profile if you are interested (for GoLang but the principles are the same as for other runtimes) https://gitlab.com/montao/aws-lambda-go-gitlab-sls
LostJon is absolutely correct in that the support for both CloudFormation and SAM is immediately available as it is natively provided by AWS. I am leaning to CloudFormation for the reason that is has a deterministic outcome, which is desired in a DevOps landscape. There are no transformations unless you explicitly add transformations to your CloudFormation stack.
When you deploy a CloudFormation template with SAM, you will be able to view the CloudFormation template in the AWS console, but you can also choose to show the rendered template (e.g. after transformations) so that you can learn what SAM does for you underwater. You could copy the rendered resources and use them as pure CloudFormation. This allows you to speed up development, while not losing the benefit of knowledge of the underlying configuration.
Related
I have a short question. The serverless.yml of the serverless framework is a wrapper of aws cloudformation yml or how does it work?
SAM and serverless frameworks have in common that they generate CloudFormation.
they both abstract CloudFormation so that you need to write less code to build serverless applications (in the case of SAM) and to deploy Lambda functions (for both SAM and Serverless).
Yes they are abstraction, however serverless allows you to granularize more.
I am having difficulty in understanding whether to use aws cdk or terraform cdktf. From my work so far, it appears to me that aws cdk has more robust patterns and constructs which will comply with the well architected framework, all available out of the box. Terraform cdktf will require hand crafting many of such constructs and patterns.
An example in point is the construct ecs_patterns.ApplicationLoadBalancedEc2Service which does a lot of heavy lifting in spinning up a industrial strength infrastructure for EC2 based ECS service. I can't find the equivalent of that in Terraform cdktf and it appears I will have to manually assemble and connect all the infra elements.
However, Terraform apparently has the advantage of working with several different cloud vendors, and therefore I want to do due diligence before choosing one of the other.
Therefore, I would like to know if my understanding is correct and if I am not missing something really important. Any other advice / considerations in this matter are highly appreciated
Thanks
The AWS CDK is limited to only AWS as a cloud, whereas you can use CDKTF with any / most of the clouds since terraform providers exist for most of them. The AWS CDK has a diverse ecosystem of constructs that can be used, which is a plus. CDKTF will eventually support these via the CDKTF AWS Adapter.
Ideally I'd like to have something like AWS Lambda on our internal servers. Write functions instead of monolith apps, use any programming language, communicate via sockets with other functions.
How to provide sufficiently sandboxed accounts for developers, where they can only write "functions" and not having to worry about underlying infrastructure at all? So ideally, they'd never use a terminal at all, no bash, no apt-get.
We have been using Serverless Framework with Offline Plugin to emulate and facilitate AWS Lambda development locally. It improves productivity and also supports DevOps.
You can now use OpenWhisk or Kubeless for this.
You can try using fission.io which is an open source serverless framework for Kubernetes and create your own Kubernetes cluster. https://github.com/fission/fission
Or if you don't want to fuss with configuration or set up try writing functions using Clay.run - Disclaimer I'm a co-founder of Clay. You won't need to do any dev-ops just write code and it's automatically live at an HTTPS endpoint.
How are the different FaaS or Serverless Architecture offerings of the major cloud providers effectively implemented in terms of backend and infrastructure?
AWS Lambda: Java, Node.js, C#, Python, binaries
Azure Functions: JavaScript, C#, Python, PHP, Bash, Batch, PowerShell
Google Cloud Functions: JavaScript
Google Cloud Functions seems to be based on a managed Node.js environment according to here: https://cloud.google.com/functions/docs/concepts/overview which seems to be different and a limiting factor.
AWS and Azure support a multitude of different runtimes which may look like an ideal docker use case? At least the underlying logical core concept is very close to process isolation/containerization respectively scaling of these which would lead us to Kubernetes/Mesos? It wouldn't be very efficient to invent custom solutions nowadays, wouldn't it?
Does anyone know what is at work behind the scenes?
I had a discussion with an AWS Engineer a while back and he confirmed my suspicion that AWS Lambda is run's on top of docker. Can't comment on azure or google's infrastructure though...
I am looking at using Amazon cloud services (EC2, S3 etc) for hosting. I've been looking at the JSON metadata that can be specified to configure various instances and the complexity has me concerned. Is there a dsl that will generate valid JSON metadata and more importantly validate the entries?
Unfortunately, I drew a blank after searching for this recently. I'm using Amazon Web Services CloudFormation (is that the JSON metadata you're talking about?).
There are a couple of issues with CloudFormation JSON files:
I'm at well over 1,500 lines and it's impossible to read,
You can't express everything the API gives you, notably in the area of Virtual Private Clouds,
There are lots of bugs that are taking a long time to fix, Elastic Load Balancers losing HTTPS information, for example.
So I've been using straight-up API calls in Scala using the Java API. It's actually really nice.
The Java API has a flavor of "setters" starting with with that return this so they can be chained. In Scala, you can use these to act like a poor-man's DSL. So you can do things like
val updateRequest = new UpdateAutoScalingGroupRequest()
.withAutoScalingGroupName(group.getAutoScalingGroupName)
.withAvailabilityZones(subnetAZsOfOurVPC)
.withVPCZoneIdentifier(subnetNamesOfOurVPC)
as.updateAutoScalingGroup(updateRequest)
Other things are easy to do in Scala with the Java API. For example, group all your Subnets by VPC in a Map, simply do
val subnetsByVPC = ec2.describeSubnets(new DescribeSubnetsRequest).getSubnets.groupBy(_.getVpcId)
In case anyone is still looking for the AWS CloudFormation DSL - we've been using the Ruby DSL for CloudFormation:
https://github.com/bazaarvoice/cloudformation-ruby-dsl
This neat project provides a tool to convert your existing CloudFormation template(s) to the Ruby DSL
It will generate valid JSON output
Validating Ruby template entries is similar to validating a regular CloudFormation template (see cfn-validate-template)
Your template becomes Ruby code, so it's easy to have reusable modules (DRY)
You can define local variables
You can have comments in your DSL template
Greatly improved readability
Greatly reduced DSL template size
The CloudFormation request template body size limits are annoying - we have to upload our large CloudFormation templates to S3, then create/update stacks using their S3 URLs.
There is now, though I haven't used it yet: Coffin a CoffeeScript DSL for CloudFormation.
If you're not talking about CloudFormation, but instead more general API talking, then the nicest interface I've found is AWS' own aws-sdk ruby gem. Unlike the other SDKs that they publish which are quite nicely-done-but-crude make-client/make-request/get-response/look-at-result affairs, the ruby SDK wraps a nicer domain-model over the top, so you interact via collections at a higher abstract level.
It also has quite-nice performance features in that you can cache responses to save on round-trips, if you know you don't need fresh responses.
I have CloudFormation templates upwards of 3000 lines. I have found that putting comments in the JSON helps tremendously!! You just have to strip it out before using it. There is a validator that would validate the template and strip out the comments: http://cloudformation-validation.com/
No, as of Feb 2022, AWS still does not provide any domain-specific language for infrastructure as code. They have nothing similar to Azure's Bicep or Terraform's HCL.
This really surprises me, as I generally think of AWS as being more expensive, but ahead of the curve, compared to other major competitors (Azure and GCP).
However, Cloud Formation now allows both JSON and YAML formats. Slight improvement?? IMHO, not really when you have an entire repo that represents your entire cloud stack. If you're using AWS, use Terraform to manage IaC.