I was wondering how GCP process the creation of an autoscaling group. Do they start with the minimum instances specified and evaluate the load and then adjust or the evaluate the load before launching the autoscaling group ?
max_replicas = 5
min_replicas = 2
I just want to know if the size at the creation is fixed
Thanks
You can do autoscaling on compute engine with 4 ways
When you choose an option that your needs and you can choose min and max instance number. The cluster will starting with your minimum VM settings. When your load increase than your cluster's VMs number will increase up to your max instance.
Related
Google cloud load balancer monitoring dashboard provides number of healthy nodes:
Is it any metric / MQL which can be used to create an alert if one node considered as down?
There is no direct metric for the Load Balancer's backends, but what you can do is to create a Monitoring group containing them, and then set up an Uptime check for monitoring and alerting; you can follow these steps to accomplish that:
Tag all the backend instances:
gcloud compute instances add-tags <instance_name> --tags=lb-backend --zone <instance_zone>
Create the monitoring group using Resource Type = gce_instance AND the tag as discriminators:
Create the Uptime check and alerting policy for that group:
I have created a azure virtual machine scale set with two instances and also opted the Availability zone option ( selected all three available choices (1, 2,3). After the deployment of Scale set i am able to see two instances of virtual machines are created and visible in azure portal. Now I am confused to build my understanding and need help in below two questions -
1- If only two instances get created in this scenario then how they spread in three selected availability zones (1 , 2 & 3).
2- If same scale set with instances are getting created , how can i see other remaining 4 instances( as two are already visible) in the portal .
Q1. If only two instances get created in this scenario then how they spread in three selected availability zones (1, 2 & 3).
As I know, the availability zone is available for each instance. It means each instance is effectively distributed across three fault domains and three update domains when you select three zones. You use the CLI command to list all the instances to see which zone the instances in:
az vmss list-instances -g group_name -n vmss_name
Q2. If the same scale set with instances are getting created, how can I see other remaining 4 instances( as two are already visible) in the portal?
You cannot see the replicate of the instance in the portal, they are managed by the Azure platform. You also need not care about that.
Foreword
When you create a Kubernetes cluster on AKS you specify the type of VMs you want to use for your nodes (--node-vm-size). I read that you can't change this after you create the Kubernetes cluster, which would mean that you'd be scaling vertically instead of horizontally whenever you add resources.
However, you can create different node pools in an AKS cluster that use different types of VMs for your nodes. So, I thought, if you want to "change" the type of VM that you chose initially, maybe add a new node pool and remove the old one ("nodepool1")?
I tried that through the following steps:
Create a node pool named "stda1v2" with a VM type of "Standard_A1_v2"
Delete "nodepool1" (az aks nodepool delete --cluster-name ... -g ... -n nodepool1
Unfortunately I was met with Primary agentpool cannot be deleted.
Question
What is the purpose of the "primary agentpool" which cannot be deleted, and does it matter (a lot) what type of VM I choose when I create the AKS cluster (in a real world scenario)?
Can I create other node pools and let the primary one live its life? Will it cause trouble in the future if I have node pools that use larger VMs for its nodes but the primary one still using "Standard_A1_v2" for example?
Primary node pool is the first nodepool in the cluster and you cannot delete it, because its currently not supported. You can create and delete additional node pools and just let primary be as it is. It will not create any trouble.
For the primary node pool I suggest picking a VM size that makes more sense in a long run (since you cannot change it). B-series would be a good fit, since they are cheap and CPU\mem ratio is good for average workloads.
ps. You can always scale primary node pool to 0 nodes, cordon the node and shut it down. You will have to repeat this post upgrade, but otherwise it will work
It looks like this functionality was introduced around the time of your question, allowing you to add new system nodepools and delete old ones, including the initial nodepool. After encountering the same error message myself while trying to tidy up a cluster, I discovered I had to set another nodepool to a system type in order to delete the first.
There's more info about it here, but in short, Azure nodepools are split into two types ('modes' as they call it): System and User. When creating a single pool to begin with, it will be of a system type (favouring system pod scheduling -- so it might be good to have a dedicated pool of a node or two for system use, then a second user nodepool for the actual app pods).
So if you wish to delete your only system pool, you need to first create another nodepool with the --mode switch set to 'system' (with your preferred VM size etc.), then you'll be able to delete the first (and nodepool modes can't be changed after the fact, only on creation).
Spark needs lots of resources to does its job. Kubernetes is great environment for resource management. How many Spark PODs do you run per node to have the best resource utilization?
Trying to run Spark Cluster on Kubernetes Cluster.
It depends on many factors. We need to know how much resources do you have and how much is being consumed by the pods. To do so you need to setup a Metrics-server.
Metrics Server is a cluster-wide aggregator of resource usage data.
Next step is to setup HPA.
The Horizontal Pod Autoscaler automatically scales the number of pods in a replication controller, deployment or replica set based on observed CPU utilization or other custom metrics. HPA normally fetches metrics from a series of aggregated APIs:
metrics.k8s.io
custom.metrics.k8s.io
external.metrics.k8s.io
How to make it work?
HPA is being supported by kubectl by default:
kubectl create - creates a new autoscaler
kubectl get hpa - lists your autoscalers
kubectl describe hpa - gets a detailed description of autoscalers
kubectl delete - deletes an autoscaler
Example:
kubectl autoscale rs foo --min=2 --max=5 --cpu-percent=80 creates an autoscaler for replication set foo, with target CPU utilization set to 80% and the number of replicas between 2 and 5. You can and should adjust all values to your needs.
Here is a detailed documentation of how to use kubectl autoscale command.
Please let me know if you find that useful.
I am learning and setting up auoscale configuration for our production application. I want to know while setting scale-down which instance id to use. e.g. my configuration uses maximum number of instances as 3. I can put scale-up policy on instance id 1, but how can I put scale down policy for instance 2 and instance 3 which are still to be started.
PS: I understand for 2 instances, I can put policy on instance 1 and it will go down if load subsides.
you need to specify ami id for template ami only. all will be scaled up and down based on this single ami id. PLEASE NOTE: it is an ami id or image id that will be utilized and not an instance id.
The scale will be a range from 1 to 3 and will be configured in your autoscaling group with these tags: --min-size 1 --max-size 3 based on the metrics that you supply in the autoscaling policy.
With a minimum size of 1 instance, 1 instance will always be running. With a maximum size of 3 instances, 3 can run. There is not a need to scale non running instances.
Here is a complete tutorial:
http://www.cardinalpath.com/autoscaling-your-website-with-amazon-web-services-part-2/