I have a stateless app with 3 component say x,y,z in the same code base. Each component will be run by checking on the environment variable. I want to deploy it on Kubernetes on GCP using kind: Deployment yaml config with 3 replica pods. How can I make sure each component has a single dedicated pod to it?
Can it be done on single deployment file?
As #Ivan Aracki mentioned in the comments, the best way would be distinguish each application component with appropriate deployment object in order to guarantee a Pod assignment here.
As Ivan, suggested above, deploy three deployments one each for x, y and z.
You can use the same image for the three deployments, just pass different environment variable/value for each deployment to deploy specific component. You might have to build some logic in the container start up script to retrieve the value from the environment variable and start the desired component
As I understand your requirements stated that you have three processes application code base inside one solution. Nit sure weather three components you mentioned are independent process components or just layer front end , service , DAL etc or even tiers e.g. typical 3 tier architecture application with front end web , API and backend tier but let's call three microservices or services for simplicity...
Whichever the case is , best practices of docker , kubernetes hosted microservices pattern recommends :
container per process small app (not monolethe)
though there can be multiple containers per pod, suggested is keep one per pod - you can possibly have three containers inside pod
You can have three pods one each for your component app provided these apps can be refactored into three separate independent processes.
Having one yaml file per service and include all related objects inside seperated by --- on seperate line
Three containers inside single pod or three pods per service would be easily accessible to each other
Hope this helps.
Related
We are using ECS Fargate containers to deploy all of our services (~10) and want to follow Blue/Green Deployment.
We have deployed all the services under BLUE flag where target groups are pointing to the services.
In CICD, New Target groups are created and having slightly different forward rules to allow testing without any issue.
Now, my System is running with 2 kind of target groups, services and task definition -
tg_blue, service_blue, task_blue → pointing to old containers and serving live traffic
tg_green, service_green, task_green → pointing to new containers and do not have any traffic.
All above steps are done in Terraform.
Now, I want to switch the traffic and here I am stuck, How to Switch the Traffic and How the next Deployment will look like?
I would go for AWS native solution if no important reasons against. I have on my mind CodeDeploy. It switches in automatic way between TGroups.
Without CDeploy, you need to implement weighted balancing among two TGroups and adjust them later on. That is extra work.
Whole flow is quite good explained on this YT video.
I've been following this particular blog post as a guide.
https://blog.gruntwork.io/how-to-manage-terraform-state-28f5697e68fa
So I understand the need to have state files per environment but is anyone going the extra length and using a state file per component? For example, we have modules for our application stack. To keep this example simple, let's say I have two modules.
App1 - contains a LAMP stack
App2 - LAMP stack plus rabbitmq and redis
Each environment (dev, uat and prod) would have both app stack. Would you have a state file for each component, giving you 6 state files?
I've used this method in the past - I had 3 separate components and 3 different environments. The different components had different rates of change and some dependencies between then so I used a remote state datasource to share output variables between them.
The biggest problem with this approach is the management overhead of maintaining the different environments/components.
An alternative could be to use different components in conjuction with workspaces
I am building a micro-services project using docker.
one of my micro-services is a listener that should get data from various number of sources.
What i'm trying to achieve is the ability to start and stop getting data from sources dynamically.
For example in this drawing, i have 3 sources connected to 3 dockers.
My problem starts because i need to create another docker instance when a new source is available. In this example lets say source #4 is now available and i need to get his data (I know when a new source became available) but i want it to be scaled automatically (with source #4 information for listening)
I came up with two solutions, each has advantages and disadvantages:
1) Create a docker pool of a large number of docker running the listener service and every time a new source is available send a message (using rabbitmq but i think less relevant) to an available docker to start getting data.
in this solution i'm a little bit afraid of the memory consumption of the docker images running for no reason - but it is not a very complex solution.
2) Whenever a new source is becoming available create a new docker (with different environment variables)
With this solution i have a problem creating the docker.
At this moment i have achieved this one, but the service that is starting the dockers (lets call it manager) is just a regular nodejs application that is executing commands on the same server - and i need it to be inside a docker container also.
So the problem here is that i couldn't manage create an ssh connection from the main docker to create my new Docker.
I am not quite sure that both of my solutions are on track and would really appreciate any suggestions for my problem.
Your question is a bit unclear, but if you just want to scale a service horizontally you should look into a container orchestration technology that will allow you that - For example Kubernetes. I recommend reading the introduction.
All you would need to do for adding additional service containers is to update the number of desired replicas in the Deployment configuration. For more information read this.
Using kubernetes (or short k8s) you will benefit from deployment automation, self healing and service discovery as well as load balancing capabilities in addition to the horizontal scalability.
There are other orchestration alternatives, too ( e.g. Docker Swarm), but I would recommend to look into kubernetes first.
Let me know if that solves your issue or if you have additional requirements that weren't so clear in your original question.
Links for your follow up questions:
1 - Run kubectl commands inside container
2 - Kubernetes autoscaling based on custom metrics
3 - Env variables in Pods
I have created 5 x ARM templates that combined deploys my application. Currently I have separate Templates/parameter files for the various assets (1 x servicebus, 1 x sql server, 1 x eventhub, etc)
Is this OK or should I merge them into 1 x template, 1 x parameter file that deploys everything?
Pro & cons? What is best practice here?
Its always advised to have seperate JSON File for azuredeploy.json and azuredeploy.parameters.json.
Reason:
Azuredeploy is the json file which actually holds your resouces and paramaters.json holds your paramaters. You can have one azuredeploy.json file and have multiple paramaters.json files. Like for example let say you different environements, Dev/Test/Prod, then you have seperate azuredeploy-Dev.paramaters.json, azuredeploy-Test.paramaters.json and so and so forth; you get the idea.
You can either merger seperate json files, one for service bus, one for VMs, etc. this will help when you want multiple people to work on seperate sections of your Resource group. Else you can merge them together.
BottomLine: You are the architect, do it as you want, whichever makes your life easy.
You should approach this from the deployment view.
First answer yourself few question:
How separate resources such as ASB, SqlServer, Event hub are impacting your app? can your app run independently while all above are unavailable?
How often do you plan to deploy? I assume you are going to implement some sort of Continuous deployment.
How often will you provision a new environment.
so long story short.
Anything that will have minimum (0) downtime on your app during deployment/disaster recovery, should be considered along with the fact anyone from the street can take you scripts and have your app running in reasonable time, say 30 min max.
I have a java web application, which has two layers( business layer and presentation layer both layers has their own war files). I want to deploy this into AWS using AWS CodeDeploy.and I am using RDS MySQL as database.
can anyone tell me how can I deploy this application using CodeDeploy? Do I need to have two different appspec.yml file for the deployment?
Thanks in Advance.
This really depends on how you want to deploy your code. You need exactly one appspec per bundle you want to deploy.
If you want to deploy your businesses layer and your presentation layer separately, you will need two different deployments, two separate (logical) applications in CodeDeploy, two bundles, each with their own appspec. Each appspec would be responsible for stopping, configuring, and starting only one of the layers.
If you want to deploy both layers as part of the same deployment then you will have one deployment, one (logical) application in CodeDeploy, one bundle, which would have only one appspec. That appspec would be responsible for managing the lifecycle of both of you layers.