I'm new to Azure and auto-scaling and have some questions I have not find answers to.
Today I use a VM and a Service Bus queue. The queue revives jobs only at beginning of each hour during office hours. Because it is important to execute this jobs quickly I want to scale out VM instances before the jobs arrives. And then scale in after the work is done(around 15 minutes later).
In my Service Bus Queue I have a MLQ(multilevel queue). I want to scale in to 1 VM as fast the queue with highest priority is empty.
So, my questions is:
How can I scale out VM instances 10 minutes before each our during office hours? How can I Scale in when the highest prioritized queue is empty? Is using Azure Scale Set a good approach? What techniques can be used to achieve this scaling rules?
Thanks in advance
Related
We have a solution where we use an Azure Storage Queue to process messages that take approx 6 minutes.
I've read that the maximum batchSize of Queue messages concurrently processed are 32 per VM.
If the function app scales out to multiple VMs, each VM could run one instance of each queue-triggered function.
https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-queue?tabs=in-process%2Cextensionv5%2Cextensionv3&pivots=programming-language-csharp#host-json
How does that translate to Azure Functions Premium plan?
Lets say we want to be able to process 64 messages at once using Azure Functions Premium plan with Always ready instances. If we have 2 ready instances, can they process 2 * 32 concurrent messages? Or do they underwater really need to be on seperate VM's and 2 instances will not do anything different?
In the Premium plan, you can have your app always ready on a specified number of instances. The maximum number of always ready instances is 20. When events begin to trigger the app, they are first routed to the always ready instances.
https://learn.microsoft.com/en-us/azure/azure-functions/functions-premium-plan?tabs=portal#always-ready-instances
Yes. In Azure Functions premium plan, if you have pre-warmed instance, then that is given a dedicated VM instance. So, if you had 2 VM instances running your function app, then they can process 2*(batchSize + newBatchThreshold) concurrent Queue messages!
The Azure platform scales the function app onto new VM as the existing instances gets more busy.
I have an Azure Subscription that is used by our automation to spin up VMSS. Based on logging, it appears that every VMSS request takes about 25 mins to complete. It seems like an exorbitant amount of time to complete.
Where can I find historical trends on how long each of my VMSS request took on my Azure subscription?
When I spin a single VM using the same image & SKU it only takes a few minutes. Why does it take 10 times the time when I request it through VMSS?
If you are talking about Azure Control plane operations - you can check activity log, its available under the resource group and under the vmss resource.
thats how the platform is built. vmss is a bit more complex than a single vm, so takes more time to create it.
I have a queue with 6 million messages to be processed in azure. I have a function app running in an app plan with a queue trigger. The app service plan will scale out to additional instances when the CPU gets too high. My app plan is running around 10-20% cpu usage. I was previously having an issue where the function was blocking due to the network calls inside the function which caused CPU to skyrocket and my app service plan would scale out. I resolved this by using async code, so now I have this low cpu usage and a very fast function execution time.
I am currently seeing about 1k messages processed a minute, sometimes it will go up to 2k for a bit and then back down. Are there any reasons why my function wouldn't scale up?
Your question also holds the answer:
The app service plan will scale out to additional instances when the CPU gets too high.
and
My app plan is running around 10-20% cpu usage.
Long story short: I think you're scaling on the wrong metric. As #evilSnobu suggested, you could scale on the number of messages in your queue. Some extra info on Consumption Plans that might be interesting:
Runtime scaling
Azure Functions uses a component called the scale controller to monitor the rate of events and determine whether to scale out or scale down. The scale controller uses heuristics for each trigger type. For example, when you're using an Azure Queue storage trigger, it scales based on the queue length and the age of the oldest queue message.
Taken from Azure Functions Consumption and App Service plans
I have one big task to do every day, with no need to scale, that takes about 30 minutes and is DB, processor and memory intensive.
This means actual 16h/month of computation time.
WebJobs require constantly running WebSite 744h/month
WebRole is also constantly running 744h/month
Azure Batch - suited for scaled storage input - storage output
processing (or that is how I understand it)
Stopped cloud service still cost you. Setting instance count to 0 is not available. And paying for 728h/month unused computation time looks like madness. Only thing I can imagine is automatic deployment of cloud service every day and automatic deletion of deployment once task is finished, but this also looks like madness.
Are there any options for this scenario in Azure?
Cloud service will be charged continuously until the deployment is deleted. Yes you can delete it every day and redeploy...
Azure VMs in Stopped (Deallocated) status, does not incur any charge. You can shut them down in portal or by script when you don't need them.
I think there is a large difference in billing if you only use it 62h/month. Would you consider switch this deployment to VM? WorkerRole and VMs can be placed on the same subnet, they can still connect to each other.
Ok Azure Experts,
I have a task that only needs to run once every week - this is a long
running task that can take 2-3 days to run.
I have set up a worker role to scale based on a queue. On the day
that we want the task to start - we populate the queue (using a Web
Job).
During the rest of the time, when the queue is empty, I want the
worker roles to shut down - but I cannot scale down to 0 instances.
Originally, we wanted to do this with a Web Job, but the website shuts down from time to time - abruptly turning off my webjob - is this supposed to happen? Even with Keep-Alive turned on? Also, you cannot stop a triggered Web Job from running - so if we want the process to stop - we need to turn off the Web site - not ideal.
How do I scale my instances down to zero?
* Alternatives solutions are also welcome.
Trying to minimize cost here - why pay for a worker role that isn't doing anything?
It is not possible to scale-down a Worker Role to 0 instances at this time. Even if you STOP the worker role, you're still incurring charges for STOPPED instances.
However, behavior that you're looking for, is possible with Virtual Machines. If you shutdown (STOP & DEALLOCATE) a virtual machine, you're not paying fees for that machine.
Now, the only challenge is to stop/start the VM based on a queue count. I don't recall if Azure portal's native scaling supports scaling down to 0 instances for VMs. However, if you use AzureWatch, you should be able to get this done without any issues. Disclaimer: I am affiliated with AzureWatch.
HTH