I currently have a webjob that runs when user does specific action.
When the webjob is running it cannot accept a new request until its done with the current process.
Is it possible to run webjob simultaneously, meaning be able to process a job for multiple clients at a time?
Absolutely. But how are you triggering your WebJob when a user performs a specific action? If you trigger a method within your WebJob via a QueueTrigger that listens for a message on an Azure Storage Queue, you can have multiple threads processing messages at once. You can do something similar using Service Bus queues as well.
Check out the docs in the parallel execution section here.
Related
We are currently using Azure more like IAAS, rather than cloud services.
As a start, I would like to utilise Azure Messaging Queues to process some database actions and Web API calls.
I am assuming I would need to write another piece of code that subscribes to the queues, so when messages arrive, it knows to process the transaction?
Is that piece of code, a console app? runs on a scheduled task? a windows service? or a function app within azure?
What is the Best Practice for this architecture?
You can write console app and schedule web job to monitor the queue. However better way is to use Azure Functions. You don't have to monitor the queue then. Whenever the message arrives in the queue it will trigger Azure Function and you can process the message. The benefit is it's server-less.
I do have a AzureWorker that receives SMTP messages from TCP ports and pushes them to queues. Other threads pick up these messages from the queues and process them. Currently, process threads have their queue polling logic. Simply they check the queues and increase wait interval if the queues are empty.
I want to simplify the queue logic and make use of other Webjobs functionalities in this AzureWorker.
Is it possible to start a WebJobs thread in this AzureWorker and let that thread handle the details? Are there any limitations that I need to know?
Azure Worker Roles are a feature of Azure Cloud Services. Azure Web Jobs are a feature of Azure App Service. They are both built to provide similar ability to run background process tasks within the context of your application. Although, since they are features of different Azure services they can't be run together like you are asking in a nested fashion.
Is it possible to start a WebJobs thread in this AzureWorker and let that thread handle the details?
I agree with Chris Pietschmann, it does not enable us to start WebJobs thread directly in Azure Worker Role.
Other threads pick up these messages from the queues and process them. Currently, process threads have their queue polling logic. Simply they check the queues and increase wait interval if the queues are empty.
I want to simplify the queue logic and make use of other Webjobs functionalities in this AzureWorker.
If you’d like to complete this task by using WebJobs, you could write a program and run as a WebJobs in your Azure App Service. And WebJobs API provides a way to dynamically start/stop WebJobs via REST API, you could use it to manage your WebJobs in your Worker Role.
I'm using Azure webjobs with queue-triggered functions (which rely on the Azure webjobs sdk) to perform some background processing work. Within the webjobs I make various connects to a SQL Azure database (using PetaPoco which uses System.Data.SqlClient).
I want to be purposeful in my database connection strategy - specifically because there are some concurrency issues inherent to the environment.
One concurrency scenario is with the SDK's BatchSize property that you can set for queue-triggered webjobs. It's my understanding that setting BatchSize > 1 results in multiple instances of the queue-triggered function running within the same webjob process.
The second concurrency scenario is the website scale-out scenario where you're running multiple instances of the webjob itself. These of course are in different processes.
In my website I have a database connection per request (the machine handles connection pooling by default). No problems there.
How should I treat connections in the webjob scenario, accounting for the concurrency scenarios described above? Webjobs are of course just long-lived console processes (these are continuous webjobs). Should I create a database connection when my webjob starts and simply re-use that connection through the webjob's lifetime? Should I instantiate and close connections per function when I need them?
These are the types of things I'm trying to understand.
Webjobs are of course just long-lived console processes (these are continuous webjobs).
The main process is the long-lived processes , but for trigged sub- process will be released after the triggered function is executed. It means that connection will also be released automatically in the sub-process. For best program practices that we 'd better close it manually before exit function.
The second concurrency scenario is the website scale-out scenario where you're running multiple instances of the webjob itself. These of course are in different processes.
WebJob SDK queue trigger will automatically prevents a queue triggered by multiple instances.
If your web app runs on multiple instances, a continuous WebJob runs on each machine, and each machine will wait for triggers and attempt to run functions. The WebJobs SDK queue trigger automatically prevents a function from processing a queue message multiple times; functions do not have to be written to be idempotent. However, if you want to ensure that only one instance of a function runs even when there are multiple instances of the host web app, you can use the Singleton attribute.
It's my understanding that setting BatchSize > 1 results in multiple instances of the queue-triggered function running within the same webjob process
BatchSize it means that how many queue messages that can be picked up simutaneouly to be executed in Parallel in a WebJob.
How to use Azure queue storage with the WebJobs SDK induling parallel execution and multiple instances, we could get more info from the doucment.
When I read the documentation for azure WebJobs, it found below statement For Continuous WebJobs to run reliably and on all instances refer to image below
My WebJob workflow:
Need to prepare the report for the newly created user in my application at 12 AM EST and send me the report in Email in daily occurrence.This time is changeable by UI so I need to run job continuously to find and run schedule at selected time
My Question
If WebJob runs all instance say two instances in running now for my web app.
Will I receive two email such that WebJob in each instance prepare the report and send to me?
Will get only one email irrespective of how many WebJobs are running?
A Continuous WebJob by default runs on all instances of your App Service Plan.
Whether your WebJob will run twice depends on how you implemented it. A Service Bus Queue listening WebJob will only run a queue message once, no matter how many instances (though if it fails, it'll run more than once).
If you wish, you can also make the WebJob run on a single instance by including a settings.job file in your WebJob with the following content:
{
"is_singleton": true
}
I've got a continuously running WebJob on my auto-scale Azure website.
My WebJob is a simple console application with a while(true) loop that subscribes to certain messages on the Azure service bus and processes them. I don't want to process the same message twice, so when the web site is scaled an another WebJob is started I need for it to detect that another instance is already running and just sit there doing nothing until it's either killed (by scaling down again) or the other instance is killed. In the last scenario the second WebJob should detect that's the other is no longer take over.
Any takers?
You should create a queue (either using the Service Bus or storage queues) and pull the jobs off (create and manage a lease to the message) and process them from there. If that lease is managed properly, the job should only get processed once although you should make sure it's idempotent just in case as there are fringe cases where it will be processed more than once.