How close is dynamodb-local to being the same thing as the AWS cloud version?
I'm primarily looking for a comparison on the interface(s) and functionality used by applications, though any other caveats would be helpful (say, IAM, IaC, or indexing).
I have used this for local testing with Jest (https://github.com/shelfio/jest-dynamodb). For mocking in unit tests for code that interfaces with DynamoDB it works wonderfully. Very easy to provision tables and indices for testing purposes.
There are some important caveats to keep in mind (AWS lists them here: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.UsageNotes.html), and IAM policies aren't really going to work as you aren't running on AWS infrastructure.
Related
I am looking to use GCP for a micro-services application. After comparing AWS and GCP I have decided to go with Google because one major requirement for the project is to schedule tasks to run in the future (Cloud Tasks) which AWS does not seem to offer an equivalent of.
I am planning on containerizing my services and deploying to GCP using Cloud Run with a Redis cluster running as well for caching.
I understand that you cannot have multiple Firestore instances running in one project. Does this mean that all if my services will be using the same database?
I was looking to follow a model (possible on AWS) where each service had its own database instance that it reached out to.
Is this pattern possible on GCP?
Firestore indeed is for the moment limited to a single database instance per project. For performance that is usually not a problem, but for isolation such as your use-case, that can indeed be a reason to look elsewhere.
Firebase's original Realtime Database does allow multiple instances per project, and recently added a REST API for provisioning database instances. Since each Google Cloud Project can be toggled to also be a Firebase project, you could consider that.
Does this mean that all if my services will be using the same database?
I don't know all details of your case. Do you think that you can deploy a "microservice" per project? Not ideal, especially if they are to communicate using PubSub, but may be an option. In that case every "microservice" may get its own Firestore if that is a requirement.
I don't think one should consider GCP project as some kind of "hard boundaries". For me they are just another level of granularity - in addition to folders, etc.
There might be some benefits for "one microservice - one project" appraoch as well. For example, less dependent lifecycles, better (more accurate) security, may be simpler development workflows...
While I'm trying to train new people on Terraform, I always find it quite cumbersome to have to deal with real infrastructure.
First, because it involves finding a non-sensitive cloud account or creating a new one, creating an identity for the new user (including setting-up some security stuff like two FA, ...), which could take some times (especially if you are in a traditional corporate environment where finding a CB to make payments is almost impossible).
Second, because as you are creating real infrastructure, you rapidly come into quirks that are impeding the learning curve, like the time it takes to create various types of infrastructure, the cost associated with some stuff, the need to deprovision them afterward since they are just tests, ...
Are you aware of any sandbox environment where it would be very easy to create infrastructure with Terraform (even not a real one), in order to concentrate on Terraform and stop wasting time on "side-stuff"? Do you share the same struggle?
Thanks in advance
Terraform does support LocalStack which is:
LocalStack provides an easy-to-use test/mocking framework for developing Cloud applications. It spins up a testing environment on your local machine that provides the same functionality and APIs as the real AWS cloud environment.
So you could set it up and test it how it would suit your teaching requirements.
If you are in academia and are working with AWS, AWS offers AWS Educate for students for free. Thus, you could also use that for sandbox if possible.
I am having difficulty in understanding whether to use aws cdk or terraform cdktf. From my work so far, it appears to me that aws cdk has more robust patterns and constructs which will comply with the well architected framework, all available out of the box. Terraform cdktf will require hand crafting many of such constructs and patterns.
An example in point is the construct ecs_patterns.ApplicationLoadBalancedEc2Service which does a lot of heavy lifting in spinning up a industrial strength infrastructure for EC2 based ECS service. I can't find the equivalent of that in Terraform cdktf and it appears I will have to manually assemble and connect all the infra elements.
However, Terraform apparently has the advantage of working with several different cloud vendors, and therefore I want to do due diligence before choosing one of the other.
Therefore, I would like to know if my understanding is correct and if I am not missing something really important. Any other advice / considerations in this matter are highly appreciated
Thanks
The AWS CDK is limited to only AWS as a cloud, whereas you can use CDKTF with any / most of the clouds since terraform providers exist for most of them. The AWS CDK has a diverse ecosystem of constructs that can be used, which is a plus. CDKTF will eventually support these via the CDKTF AWS Adapter.
Ideally I'd like to have something like AWS Lambda on our internal servers. Write functions instead of monolith apps, use any programming language, communicate via sockets with other functions.
How to provide sufficiently sandboxed accounts for developers, where they can only write "functions" and not having to worry about underlying infrastructure at all? So ideally, they'd never use a terminal at all, no bash, no apt-get.
We have been using Serverless Framework with Offline Plugin to emulate and facilitate AWS Lambda development locally. It improves productivity and also supports DevOps.
You can now use OpenWhisk or Kubeless for this.
You can try using fission.io which is an open source serverless framework for Kubernetes and create your own Kubernetes cluster. https://github.com/fission/fission
Or if you don't want to fuss with configuration or set up try writing functions using Clay.run - Disclaimer I'm a co-founder of Clay. You won't need to do any dev-ops just write code and it's automatically live at an HTTPS endpoint.
How are the different FaaS or Serverless Architecture offerings of the major cloud providers effectively implemented in terms of backend and infrastructure?
AWS Lambda: Java, Node.js, C#, Python, binaries
Azure Functions: JavaScript, C#, Python, PHP, Bash, Batch, PowerShell
Google Cloud Functions: JavaScript
Google Cloud Functions seems to be based on a managed Node.js environment according to here: https://cloud.google.com/functions/docs/concepts/overview which seems to be different and a limiting factor.
AWS and Azure support a multitude of different runtimes which may look like an ideal docker use case? At least the underlying logical core concept is very close to process isolation/containerization respectively scaling of these which would lead us to Kubernetes/Mesos? It wouldn't be very efficient to invent custom solutions nowadays, wouldn't it?
Does anyone know what is at work behind the scenes?
I had a discussion with an AWS Engineer a while back and he confirmed my suspicion that AWS Lambda is run's on top of docker. Can't comment on azure or google's infrastructure though...