Ideally I'd like to have something like AWS Lambda on our internal servers. Write functions instead of monolith apps, use any programming language, communicate via sockets with other functions.
How to provide sufficiently sandboxed accounts for developers, where they can only write "functions" and not having to worry about underlying infrastructure at all? So ideally, they'd never use a terminal at all, no bash, no apt-get.
We have been using Serverless Framework with Offline Plugin to emulate and facilitate AWS Lambda development locally. It improves productivity and also supports DevOps.
You can now use OpenWhisk or Kubeless for this.
You can try using fission.io which is an open source serverless framework for Kubernetes and create your own Kubernetes cluster. https://github.com/fission/fission
Or if you don't want to fuss with configuration or set up try writing functions using Clay.run - Disclaimer I'm a co-founder of Clay. You won't need to do any dev-ops just write code and it's automatically live at an HTTPS endpoint.
Related
I am looking to use GCP for a micro-services application. After comparing AWS and GCP I have decided to go with Google because one major requirement for the project is to schedule tasks to run in the future (Cloud Tasks) which AWS does not seem to offer an equivalent of.
I am planning on containerizing my services and deploying to GCP using Cloud Run with a Redis cluster running as well for caching.
I understand that you cannot have multiple Firestore instances running in one project. Does this mean that all if my services will be using the same database?
I was looking to follow a model (possible on AWS) where each service had its own database instance that it reached out to.
Is this pattern possible on GCP?
Firestore indeed is for the moment limited to a single database instance per project. For performance that is usually not a problem, but for isolation such as your use-case, that can indeed be a reason to look elsewhere.
Firebase's original Realtime Database does allow multiple instances per project, and recently added a REST API for provisioning database instances. Since each Google Cloud Project can be toggled to also be a Firebase project, you could consider that.
Does this mean that all if my services will be using the same database?
I don't know all details of your case. Do you think that you can deploy a "microservice" per project? Not ideal, especially if they are to communicate using PubSub, but may be an option. In that case every "microservice" may get its own Firestore if that is a requirement.
I don't think one should consider GCP project as some kind of "hard boundaries". For me they are just another level of granularity - in addition to folders, etc.
There might be some benefits for "one microservice - one project" appraoch as well. For example, less dependent lifecycles, better (more accurate) security, may be simpler development workflows...
How are the different FaaS or Serverless Architecture offerings of the major cloud providers effectively implemented in terms of backend and infrastructure?
AWS Lambda: Java, Node.js, C#, Python, binaries
Azure Functions: JavaScript, C#, Python, PHP, Bash, Batch, PowerShell
Google Cloud Functions: JavaScript
Google Cloud Functions seems to be based on a managed Node.js environment according to here: https://cloud.google.com/functions/docs/concepts/overview which seems to be different and a limiting factor.
AWS and Azure support a multitude of different runtimes which may look like an ideal docker use case? At least the underlying logical core concept is very close to process isolation/containerization respectively scaling of these which would lead us to Kubernetes/Mesos? It wouldn't be very efficient to invent custom solutions nowadays, wouldn't it?
Does anyone know what is at work behind the scenes?
I had a discussion with an AWS Engineer a while back and he confirmed my suspicion that AWS Lambda is run's on top of docker. Can't comment on azure or google's infrastructure though...
I've been bitten by the test-driven infrastructure bug. My current project is using Azure, including SQL Azure, Azure tables, cloud services, and mobile services. Configuring an entire environment is somewhat complex. Now I'm looking for a testing framework that I can use to verify that the environment is configured correctly. Something like "Confirm that there's a mobile service endpoint named foo, that is has APNS and GCM endpoints, and that there is a Google API key and Apple push certificate associated." There is more, but that is complex enough that existing tools don't seem to cover it but simple enough to describe in a single sentence.
Because of the number of products, I have to use both the PowerShell module and the cross-platform CLI to script the setup. The cross-platform CLI looks like the easiest way to get data out (it uses Node and can easily dump JSON data), but I'm at a loss as to how to even start with testing JSON dumps from a Node module that was never really intended to be used as a module.
The PowerShell module is buggy and doesn't have any ability to read mobile services information.
There is a ruby gem for managing Azure, but it's very limited. So my hope of being able to work all in Ruby was dashed. There too, I'm not sure how one would use ServerSpec to test a remote node without actually running anything on the remote node.
I'd like to stay within the realm of something that would be understandable by another Azure developer (e.g. JavaScript, PowerShell, and potentially Ruby) and not have to start from scratch with something like Erlang or Brainf**k.
Corey - big area of ongoing build out on Azure right now which is why you are finding limited support. Resource Manager is aimed at driving programmable infrastructure (http://azure.microsoft.com/en-us/documentation/articles/xplat-cli-azure-resource-manager/) but doesn't yet encapsulate all Azure service offerings.
There is also the Management Libraries (for .Net) - http://www.bradygaster.com/post/getting-started-with-the-windows-azure-management-libraries or at the most basic of levels there is the pure REST API that you can code directly against if there are bits missing from the above (which is likely) - http://msdn.microsoft.com/en-us/library/azure/ee460799.aspx
Compared to plain vanilla PhP/MySQL, what's the upside of Cloud?
A typical block of contents would be approximately 30,000 snippets of text, each 300 characters or less in length.
I'm looking at some good documents on buckets and objects and wondering if there's any reason for me to dive into all that.
Just a rough idea would be appreciated. Am I barking up the wrong tree even thinking of Cloud for this?
p.s. just guessing: is the way to go to run MySQL in the Cloud?
It will depend on the cloud service you choose. On the cloud you can choose between an IaaS, a PaaS or a SaaS.
On an IaaS you will get an infrastructure as a service where you need to install MySQL, the web server, ...
On a PaaS, all these services could be enabled just with click of your mouse and you will just use the service without taking care of the config or the installation process.
This blog article will give you an idea about how to use a MySQL database on a PaaS.
Regarding the web server, for PHP could be something really easy like zip your project and use a command to deploy your application without any config. See here an example.
I'm currently evaluating the different cloud service providers to determine which is best for the use of NodeJS backend with Socket.IO implementation that is speedy. While i know that is really dependent on the physical location of their servers, i hope there is one in south-east asia.
I've already deployed on the following:
Amazon EC2
Heroku
Appfog (doesnt seem like it works on it)
Any other good ones to recommend? Would be good if they have a free tier much like how AWS works.
Look forward to your opinions!
Note that at this time Heroku does not support web sockets, so any use of socket.io will have to degrade to a long-polling approach.
With your use of Socket.io in mind, I would recommend using Amazon EC2. The main advantage of many other hosting options in the past has been their easy of deployment and management of all your resources (think Heroku). Amazon has recently launched OpsWorks, which you can use to take care of most of that for you. If you don't have any complex, custom requirements then the basic Node set up should work for you.
Keep in mind that if you are using OpsWorks there are a few small things to keep in mind. Some of them are discussed in this question. Has anyone been successful deploying a node (express) app with Amazon OpsWorks?
#Nick Mitchinson gave an amazing answer. Also you can have a look into the Elastic beanstalk. AWS recently released Paas version of Nodejs.
Websocket is now supported on Heroku by the way.