I'm learning AWS ECS and able to run the task/containers using the ECS instances. But the problem is there are certain scripts that I was executing to the underlying EC2 instances (provisioned through ECS instances) manually after doing SSH (port 22) to them. Is there any better way which would automate this process of running the following scripts whenever the ECS cluster setup is done or whenever the autoscaling kicks-in:-
sudo sysctl -w vm.max_map_count=262144
In your launch template (LT) or launch configuration (LC) for the autoscaling group of container instances, you can specify user_data. This is place where you can add your bootstrap code for the instances.
You are already probably using the user_data, as your instances need to be provided with ecs cluster name to which they belong. So you can create new LT/LC with the sysctl command to execute when the instances are launched by your autoscaling group.
Related
I have few ec2 machines running created from aws console manually. They are not under terrafrom.
If i want to use terrafrom now to create new VPC and ec2 machine does it will delete my old machines ?
No, Terraform will not delete old machines in aws (created by aws console). That is because whenever you run terraform script to create something, it will create state file which acts as reference copy for terraform. In your case if you create any resources (like ec2 VMs) in AWS, you will end up having both machines (the one created by manually and second set created by terraform)
Read here more about terraform
https://learn.hashicorp.com/tutorials/terraform/infrastructure-as-code?in=terraform/aws-get-started
Need to copy a local file (ad-hoc or during a release process) into a VMSS/Node or at least the VMSS's attached disk.
How can you copy a local file into a remote directory location on the VMSS/Node? Specifically from the command-line so that it can happen in a release pipeline (PowerShell etc).
I've read examples of using SCP but with no information on how to specifically do this with a VMSS in Azure. username#hostname doesn't really apply here or am I missing something?
I imagine every time it scales, the file previously copied will be available in every VM so this does not need to happen on every scale event?
You can set up SSH to an AKS node as detailed here using a privileged container pod. Once you have SSH-d into the node, in case of Linux nodes, from a different terminal shell you can copy files from your local machine to the AKS node using:
kubectl get pods
NAME READY STATUS RESTARTS AGE
node-debugger-aks-nodepool1-xxxxxxxx-vmssxxxxxx-xxxxx 1/1 Running 0 21s
kubectl cp /source/path node-debugger-aks-nodepoolname-xxxxxxxx-vmssxxxxxx-xxxxx:/host/path/to/destination
[Note: In the destination please remember to prepend the desired destination path on the host with /host]
In case of Windows nodes, once the SSH connection is set up as detailed here, you can copy files from your local machine to the Windows node using:
scp -o 'ProxyCommand ssh -p 2022 -W %h:%p azureuser#127.0.0.1' /source/path azureuser#<node-private-IP>:/path/to/destination
Reference for kubectl cp command
I imagine every time it scales, the file previously copied will be available in every VM so this does not need to happen on every scale event?
To the contrary, when the AKS node pool scales out, the VMSS instances are created from the VMSS Model. This Model is defined by the Microsoft.ContainerService Resource Provider. More on VMSS model and instance view here.
When you make any changes to the node's file system, the changes shall be available only to that corresponding VMSS instance. Such manual changes are not persisted if the node undergoes a node image upgrade, Kubernetes version upgrade or reconcile operation. Also if this node gets scaled down by the AKS cluster, the changes will be lost.
Instead we recommend using DaemonSets with Read-Write hostPath volume mounts where it can add files to on the host node. Since Daemonset is a Kubernetes construct and the Daemonset controller creates one replica of the Daemonset on each node (except virtual nodes; Reference) and there shall consistently be aviable even if the node undergoes an update or reconcile operation. When the node pool is scaled up new nodes shall also get a replica of the DaemonSet each.
For Azure Virtual Machine Scale Sets in general, the easiest ways to copy files between your local machine and an Azure VMSS instance would be:
SCP: if the VMSS was created with --public-ip-per-vm parameter to the az vmss create command or with API version of the Microsoft.Compute/virtualMachineScaleSets resource is at least 2017-03-30, and a publicIpAddressConfiguration JSON property to the scale set ipConfigurations section is added. For example:
"publicIpAddressConfiguration": {
"name": "pub1",
"properties": {
"idleTimeoutInMinutes": 15
}
}
If the VMSS instances do not have a public IP of their own or are assigned public IP addresses from an Azure Load Balancer (which has the VMSS as its backend pool) then create a jumpbox VM in the same virtual network as the VMSS. You can now SCP between your local machine and the jumpbox VM using the jumpbox VM's public IP and between the jumpbox VM and the VMSS instances using private IP addresses.
Assume that I have a pod active and contains only one active container initially.
This container is a nodejs application in typescript and shows user interface when opened in browser.
Can this container create another container on-demand/dynamically within the SAME POD ?
How can we achieve this? Please advise.
Also, will reusing npm modules like https://www.npmjs.com/package/kubernetes-client help in creating such containers within the same pod?
Can this container create another container on-demand/dynamically within the SAME POD ? How can we achieve this?
No, the containers within a Pod is declared in the PodTemplate that need to be declared upfront before the pod is created. More specific, what use case do you have? What are you trying to achieve?
Also, will reusing npm modules like https://www.npmjs.com/package/kubernetes-client help in creating such containers within the same pod?
A kubernetes client library is useful for interacting with the ApiServer, e.g. for deploying new applications or Pods. But the Kubernetes deployment unit is a Pod - that is the smallest unit you work with. To change a Pod, you create a new one and terminated the previous one.
I am trying to get all the Amazon EC2 instances with some specific tags like environment and service in auto scaling group using the AWS CLI .
As of now I included only one tag. How can I include both the tags and I need the full information of ID like Availability zone, launch configuration, instances, name, etc.
How can I do that?
I am using query like:
aws autoscaling describe-auto-scaling-groups --query "AutoScalingGroups[? Tags[? (Key=='Environment') && Value=='staging']]".AutoScalingGroupName
Some of the information you seek (eg Launch Configuration) can be obtained from the Auto Scaling Group (using a command similar to what you provided above), while some of the information relates to the instances that are launched within the auto scaling group (eg Availability Zone).
Here is a command that will return information about instances in a specific Amazon EC2 auto scaling group (eg my-autoscaling-group):
aws ec2 describe-instances --filter Name=tag:aws:autoscaling:groupName,Values=my-autoscaling-group --query "Reservations[*].Instances[*].[InstanceId,Placement.AvailabilityZone,Tags[?Key=='Name']|[0].Value]"
i have a spark multimaster cluster managed by zookeeper and i want to move this multimaster arch to aws ecs, aws EMR is not an option,
the question is how does ecs load balance will work, taking in to the account that every component(spark-master,spark-worker,zookeeper,livy) is deployed in a different service ecs .. thanks
it can not be done spark components are connected thru a config file in compile time, so the solution is to use task definition loopback interface which provides an static ip number(127.0.0.1) and hostname(localhost), so e.g worker connecting to master will be spark://127.0.0.1:7077