I'm experiencing poor on Azure web apps when scaling out. Azure automatically picks up an issue from a set of rules I have set up (like CPU > 80% for 5 minutes). Then it increases the number of VMs by 1. Here is a screenshot of the settings on Azure:
This part works fine. But while it is scaling to one additional VM, I'm experiencing poor performance on the entire website. When the new VM is up and running, everything seems to work as expected.
I have previously observed similar issues when scaling up and down the VM size but this is "just" a new VM.
I have already tried to create application initialization in my Web.config:
<applicationInitialization doAppInitAfterRestart="true">
<add initializationPage="/" />
</applicationInitialization>
Still poor performance while scaling out. Does anyone know what this could be caused by? I wouldn't expect a website to perform worse while scaling. I know that the performance probably isn't that great when the VM reaches 80% CPU. But the performance is definitely worse as soon as Azure starts scaling.
Kindly ensure that the margin between the scale out\in are not small. I have seen cases where the auto scale flapping was the issue.
To avoid a "flapping" situation, where scale-in and scale-out actions continually go back and forth. It is recommended to choose an adequate margin between the scale-out and in thresholds.
Just a adding a few best practices guide for you to review the metric threshold.
Ensure the maximum and minimum values are different and have an adequate margin between them - E.g- If you have a setting that has minimum=2, maximum=2 and the current instance count is 2, no scale action can occur. Keep an adequate margin between the maximum and minimum instance counts, which are inclusive. Autoscale always scales between these limits.
Always use a scale-out and scale-in rule combination that performs an increase and decrease - If you use only one part of the combination, autoscale will only take action in a single direction (scale out, or in) until it reaches the maximum, or minimum instance counts of defined in the profile.
This is not optimal, ideally you want your resource to scale up at times of high usage to ensure availability. Similarly, at times of low usage you want your resource to scale down, so you can realize cost savings. I suggest you review the threshold for the metrics and example outlined in this documentation Autoscale best practices, and let us know if it helps.
Related
I am designing a learning management system and inflow for the website is more in some cases and less in another time. I would like to know about the getting the vCPU's which are scaled up to make it down after the stipulated time. I found a document regarding scaling up but didn't find a way to scale it down.
Any help is appreciated.
There is a chance of auto scaling for the normal services in azure cloud services, that means for stipulated time you can increase or decrease as mentioned in the link.
When it comes for vCPU which is cannot be performed automatically. vCPU can be scaled up based on the request criteria and in the same manner we need to request the support team to scale those down to the normal.
There is no specific procedure to make the auto scaling for vCPU operations. We can increase the capacity of core, but to reduce to the normal, we need to approach the support system for manual changing. You can change it from 10 cores to next level 16 cores, but cannot be performed automatic scaling down from 16 cores to 10 cores.
It is clear that the auto-scale service allows to automatically scale-in and scale-out an application horizontally. And that I can manually scale vertically my application too by MANUALLY increasing/decreasing the memory.
Is there a way to AUTOMATICALLY increase and decrease the memory associated to the node.js instances based on some rules?
Note Bluemix charges the application based on GB * Hour. So by large you will be charged similarly among vertical scaling and horizontal scaling. However, vertical scaling does improves memory usage efficiency because there is less memory overhead (e.g., you load the node.js runtime only once rather than twice ore more). But horizontal scaling also has its merits:
Better availability due to increased app instances;
Better concurrency due to distributed processes;
Potential better exploration of CPU resource (because of the way CGroup works for CPU allocation).
So if your application is memory-hungry, allocating large memory for each instance would make sense. Otherwise if the app is CPU hungry, then horizontal scaling may work better. You can do some benchmark to evaluate the response time and throughput impact of both options.
The Auto-Scaling add-on in Bluemix monitors the chosen resources against their policies and increases or decreases the number of instances, not vertical scaling (memory).
Why does your node app's memory requirements grow? Can you offload some of it by using a database or cashing service? Relying on increasing memory when needed is currently a bad practice because it will require a small downtime as your application restarts.
As Ram mentioned. The Auto-Scaling service doesn't currently support vertical scaling.
You can scale horizontally by discrete numbers of instances or by a % of the total number of instances.
See the docs for which metrics are supported by each application type
edit: typo!
I have been digging without being able to wrap my head around this. It seems like once a role is deployed, you are charged for it in full, whether or not you scale it up or down?
Why would anyone scale down with this? I don't see the incentive to not just leave the role with all possible instances used to max?
I can see why an availability set, with several roles, might want to distribute the cores between them depending on load. But is that all it's for?
You pay the price of one instance of choosen size (A0 to D14) multiplied per the number of instance that are running.
Try with the Azure Princing Calculator, the number of instance increases charges.
When you try to use Autoscaling it clearly states :
Autoscale is saving you XX% off your current bill
I am using MVC3, ASP.NET4.5, C#, Razor, EF6.1, SQL Azure
I have been doing some load testing using JMeter, and I have found some surprising results.
I have a test of 30 concurrent users, ramping up over 10 secs. The test plan is fairly simple:
1) Login
2) Navigate to page
3) Do query
4) Navigate back
5) Logout.
I am using "small" "standard" instances.
I have found that when my Azure setup is configured to "autoscale", it behaves like my test with one "small" instance with no autoscale. When I setup two "small" instances with no autoscale, it goes twice as fast, or rather the average process time per request is 2x, over the test. So it appears that it is NOT autoscaling. I have tried setting the CPU trigger to a lower target ie 40-70. Still no joy.
On further investigation, when "Autoscale" was first introduced, it seems it evaluated the metrics over the previous hour, and now I see references to "10 minutes". I thought that once the CPU started hitting the target value, then it immediately triggered the new instance, which must be the whole point of "autoscale". If I have a burst of concurrent usage, I need the extra instances now, hence a reason for using a PAAS . Since my test took less than 10 minutes, "Autoscale" never kicked in. So what should be the time that Autoscale takes to kick in?
Thanks.
Azure will check the CPU metric every 5 minutes, and if it exceeds the threshold that is set, will increase the instance count at that point.
Interestingly, Azure will decrease instance counts after 2 hours of remaining below the threshold.
Source: How to Scale Websites
Quoted relevant section:
Note: When Scale by Metric is enabled, Microsoft Azure checks the CPU
of your website once every five minutes and adds instances as needed
at that point in time. If CPU usage is low, Microsoft Azure will
remove instances once every two hours to ensure that your website
remains performant. Generally, putting the minimum instance count at 1
is appropriate. However, if you have sudden usage spikes on your
website, be sure that you have a sufficient minimum number of
instances to handle the load. For example, if you have a sudden spike
of traffic during the 5 minute interval before Microsoft Azure checks
your CPU usage, your site might not be responsive during that time. If
you expect sudden, large amounts of traffic, set the minimum instance
count higher to anticipate these bursts.
It is now possible in the new Azure portal (https://portal.azure.com) to configure scaling based upon different metrics:
CPU
Memory usage
Data in/out
Http queue length
Disk Queue length
And also to configure scale up time and scale down time. In the graph it will show you the current amount of instances (solid line) vs your max configured (dashed line) and your configured metrics. When the metric exceeds the line (=configured scale up for that given metric) it will scale up & vica versa.
I am using a azure websites solution with 20 websites. Hosted on 4 cores, 8 GB RAM standard instance. I would like to know how I could do scaling in Azure websites and when to do it ?
Also I am reading some values from the new azure portal.
Can someone guide me on the values that I see here ?
Thank you
Averages
The Avg % is telling you, on average, how much of that resource is being used. So, if you have 8GB of ram, and you are typically using 66% of it, then you are averaging 5.28 Gb of ram used. Same goes for the CPU average listed below.
For the totals, I have no idea.
You're not using much of the CPU available to you here, but you are definitely taking advantage of the RAM. I'm not sure of what kind of web application you are running though, so it's dificult to determine what could be causing this.
Scaling
In terms of scaling, I always suggest starting with a small machine, then gradually scaling up.
Based on your usage, I'd drop to a machine that has fewer CPU cores, but more available RAM. From within your dashboard, you can see how to scale by clicking no your web app, then scrolling down. Click on the scale tab and it should appear as it does below:
You can now adjust what you want to scale by. The default setting is CPU Percentage, but that isn't particularly useful in this case. Instead, select Schedule and performance rules and a new panel wioll appear. On the right hand side, select Metric name and look for Memory Percentage.
In your particular case, this is helpful as we saw that your RAM is consistently being used.
Look at Action and you will want to Increase count by and change the number of VMs to 1. What this does is when your RAM reaches a certain usage %, Azure will auto-scale and generate a new VM for you. After a cool down period of 5 minutes (the default, listed at the bottom), your machine will revert to 1 machine.
Conclusion
With these settings, each time your website uses <= (Select your percentage) of RAM, Azure will increase the size of your machines.
In your case, I suggest using fewer cores, but more RAM.
Make sure you save your settings, with the Save button above.
Scott Hanselman as a great blog post on how to make sense of all of this.