I want to run a fully redundant failover environment in Azure. I have my webapps covered with an Azure Traffic Manager. I can run my storage queues in two different accounts. All is covered. The only thing I can't figure out is how timetriggers should be handled? I can't have them fire in both regions. Does anyone here have an idea or solution I could try?
Time triggers will run only instance at a time even though you have configured for multiple instances scale out. The timer trigger uses a storage lock to ensure that there is only one timer instance when a function app scales out to multiple instances. If two function apps share the same identifying configuration and each uses a timer trigger, only one timer runs.
Related
we have created azure durable function with timer trigger. We are not using Azure front door. Currently, function is deployed at east US2 & Central US region with Active -Active configuration.
Here problem is, both functions are executing and processing same data twice which is incorrect. I want to setup this configuration as Active-Passive but how it should be architect?
There is no out-of-the-box solution for this. You have to synchronize the work yourself by setting a lock for example, preventing the other timer from doing work when the work is already done. Or you disable the timer trigger function in the failover region, which means that in case of a failover you need to enable them yourself. Of course you could script that.
See also this blogpost describing the solution contained in this answer.
When would I prefer Azure Functions to Azure Container Instances, considering they both offer the possibility to perform run-once tasks and they bill on consumption?
Also, reading this Microsoft Learn Module:
Serverless compute can be thought of as a function as a service (FaaS), or a microservice that is hosted on a cloud platform.
Azure Functions is a platform that allows you to run plain code (instead of containers). The strength of Azure Functions is the rich set of bindings (input- and output bindings) it supports. If you want to execute a piece of code when something happen (e. g. a blob was added to a storage Account, a timer gets triggered, ....) then I definitely would go with Azure Functions.
If you want to run some container-based workload for a short period of time and you don't have an orchestrator (like Azure Kubernetes Services) in place - Azure Container Instances makes sense.
Take a look at this from Microsoft doc
Source: https://learn.microsoft.com/en-us/dotnet/architecture/modernize-with-azure-containers/modernize-existing-apps-to-cloud-optimized/choosing-azure-compute-options-for-container-based-applications
If you would like to simplify application development model where your application architecture has microservices that are more granular, such that various functionalities are reduced typically to a single function then, Azure functions can be considered for usage.
In case, the solution needs some extension to existing azure application with event trigger based use cases , the azure functions can be better choice . Here, the specific code (function) shall be invoked only for specific event or trigger as per requirement and the function instances are created and destroyed on demand (compute on demand - function as a service (FaaS) ).
More often, the event driven architecture is seen in IoT where typically you can define a specific trigger that causes execution of Azure function. Accordingly, Azure functions have its place in IoT ecosystem as well.
If the solution has fast bursting and scaling requirement, then container Instances can be used whereas if the requirement is predictable scaling then, VMs can be used.
Azure function avoids allocation of extra resources (VMs) and also the cost is considered only when the function is processing work. Here, we need not take care of infrastructure such as where the code is going to execute, server configuration, memory etc. For ACI, the cost is per-second where it is accounted based on the time the container runs - CaaS(Container As A Service).
ACI enables for quickly spawning a container for performing the operation and deletion of it when done where the cost is only for few hours of usage rather than a dedicated VM which would be of high cost. ACI enables one to run a container by avoiding dependency on orchestrators like Kubernetes in scenarios where we would not need orchestration functions like service discovery, mesh and co-ordination features.
The key difference is that, in case of Azure function, the function is the unit of work whereas in container instance, the entire container contains the unit of work. So, Azure functions start and end based on event triggers whereas the microservices in containers shall get executed the entire time.
The processing / execution time also plays a critical role where if the event handler function consumes processing time of 10 minutes or more to execute, it is better to host in VM because the maximum timeout that is configurable for functions is 10 minutes.
There are typical solutions that utilize both the functionalities such that Azure function shall be triggered for minimal processing / decision making and inturn can invoke container instance for specific burst processing / complete processing.
Also, ACI along with AKS form a powerful deployment model for microservices where AKS can be for typical deployment of microservices and ACIs for handling the burst workloads thereby reducing the challenges in management of scaling and ensuring effective utilization of the per second usage cost model.
Can we use Azure Functions along with Azure Batch? Please Advise.
I am working on a POC to decide which one to use for our background processes.
I too was in similar dilemma till I tried both of them for my use case.
The major difference between the two is that Azure Function has a hard timeout limit of I guess 10 minutes which you can not exceed. What I mean is that if your script/execution runs beyond 10 minutes then Azure function will kill it automatically.
Whereas Azure batch is essentially a configuration of pools or VMs in which you can run long running jobs where you are not bothered about the time of its execution. Essentially they are old VMs (low costs too). Difference between batch and Azure VMs is that Azure VMs have high speed VMs but in batch you can configure the periodic jobs where in Azure VMs you need to code in such a way that it executed like a periodic job
And yes it is possible to use Functions with Azure batch. You can configure your script as HTTP trigger in Function which you can call (get/post) through Azure Batch VMs.
Hope it helps.
May be we should expand this topic to Azure services for Batch processing in general. I did come across an article from Microsoft that goes through these options in general (which includes Web Jobs, and Kubernetes options).
But, frankly, even after reading the article; the confusion remains. For example, Azure Batches can be scheduled; but not sure if they can be triggered based on other Azure services like how Azure web jobs handles it. I get a feeling that Azure Batch is pitched where you need high + parallel computing at low costs. Because, none of the other options directly allow you to low-priority and low-cost compute instances. Correct me please!
#AzureBatch #AzureWebJobs #AzureAKS #AzureFunctions
We have written a high scalable Cloudservice for MS Azure with two roles: "WebsiteRole" and "WebsiteWorkerRole". For better performance we deploy this Cloudservice in multiple regions (2x US, 2x EU, 1x JP). We have different configuration files for each region (EuWestProductive.azurePubxml, ServiceConfiguration.CloudEuWest.cscfg, Web.ReleaseEuWest.config).
Now the Problem: In each Region we have running the "WebsiteRole" and "WebsiteWorkerRole". But the "WebsiteWorkerRole" has only very small tasks, so that one extra small instance in one region is more than enough.
We tried to set the Role instance count to zero (ServiceConfiguration.CloudEuWest.cscfg). But this is not allowed:
Azure Feedback: Allow a Role instance count of 0
Is there an other way to remove a role when deploy the Cloudservice?
No, as you've discovered, a cloud service does not allow for scale to zero. You have to effectively remove the deployment. To have the minimum change to what you already have in place you could separate the two roles into two different deployments. Then have an Azure Automation Script, or set of scripts run elsewhere, that handles deploying the worker role when needed and decommissioning when it's not needed.
Depending on the type of workload that worker is doing you could also look at taking another route of using something like Azure Automation to perform the work. This is especially true if it's a small amount of processing that occurs only a few times a day. You're charged by the minute for the automation script, so just make sure it's going to run less than the actual current instance does.
It really boils down to what that worker is doing, how much processing it really needs to do, how much resources it needs and how often it needs to be running. There are a lot of options, such as Azure Automation, another thread on the web role, a separate cloud service deployment, etc. Each with their own pros and cons. One option might even to look at the new Azure Functions they just announced (in preview and charged by the execution).
The short answer is separate the worker from the WebSiteRole deployment, then decide the best hosting mechanism for that worker role making sure that the option includes the ability to only run when you need it to.
Thanks #MikeWo, your idea to separate the deployments was great!
I have verified this with an small example project and it works just fine. Now it is also possible to change the VM size and other configurations per region.
(Comments do not allow images)
Can anybody explain the difference between Azure Web Jobs and Azure Scheduler
Azure Web Jobs
Only available on Azure Websites
It is used to run code at particular intervals. E.g. a console application every day
Used to trigger and run workloads.
Mainly recommended for workloads that either scale with the website or are relatively small.
Can be persistently running if "Always On" selected, otherwise you will get the 20 min timeout.
The code that needs to be run and schedule are defined together.
Azure Scheduler
Is not tied to Websites or Cloud Services
It allows you to call a website or add a message to a storage queue
Used for triggering events or triggering small workloads (e.g. add to queue), usually to trigger larger workloads
Mainly recommended for triggering more complex workloads.
This is only a trigger, and a separate function listening to trigger events (e.g. queue's) needs to be coded separately.
For many instances I prefer to use the scheduler to push to a storage queue and a worker role on each instance takes off the queue. This keeps tasks controlled granularly and can also move up or down in scale outside of your website.
With WebJobs they scale up and down with your site and hence your background tasks can become over taxed if your website is experiencing low traffic and scaled down.
Azure Scheduler - Provides a way to easily schedule http calls in a well-defined schedule, like every hour, every Friday at 9:00 am, Once a day, ...
Azure WebJobs - Provides a way to run small to medium work load (in the form of a script: .exe, .cmd, .sh, .js, ...) at the same context of an Azure Website (but can be hosted even with an empty website).
While a WebJob can run continuously (with a process that has a while loop) and Azure will make sure this WebJob is always running (with "Always On" set).
There is also an integration between Azure scheduler and Azure WebJobs where you have a WebJob that is running some finite work and the schduler is responsible for scheduling this work (invoking the WebJob).
So in summary, the scheduler is about scheduling work and WebJobs is about running work load.