I am trying to understand the best practice when hosting a message queue consumer in a azure worker role. I have many different type of message consumers that subscribe to different azure service bus subscriptions (or queue if you like to call it). I am wondering if I should instantiate multiple threads for each consumer in one Worker Role or should I deploy to multiple Worker Role for each consumer.
This is really dependent on your app and workload. If you have tasks that are I/O-blocked, then you should be running several threads; otherwise, you'll have a virtual machine instance which isn't being used efficiently. If it's primarily CPU-based, you may find that you can run efficiently with a lower number of threads.
You should only scale out your worker instances if you can't handle the capacity in a single instance (or if you need high-availability, in which you'd need at least two instances). Just remember that a worker role instance is a full VM, so adding one VM per queue consumer scales in cost, and you still might not see great throughput in an I/O-bound app (or one that blocks on other things such as a Web Service call).
You'll need to do a bit of experimenting to see how many threads to work with on the worker side.
Related
I do have a AzureWorker that receives SMTP messages from TCP ports and pushes them to queues. Other threads pick up these messages from the queues and process them. Currently, process threads have their queue polling logic. Simply they check the queues and increase wait interval if the queues are empty.
I want to simplify the queue logic and make use of other Webjobs functionalities in this AzureWorker.
Is it possible to start a WebJobs thread in this AzureWorker and let that thread handle the details? Are there any limitations that I need to know?
Azure Worker Roles are a feature of Azure Cloud Services. Azure Web Jobs are a feature of Azure App Service. They are both built to provide similar ability to run background process tasks within the context of your application. Although, since they are features of different Azure services they can't be run together like you are asking in a nested fashion.
Is it possible to start a WebJobs thread in this AzureWorker and let that thread handle the details?
I agree with Chris Pietschmann, it does not enable us to start WebJobs thread directly in Azure Worker Role.
Other threads pick up these messages from the queues and process them. Currently, process threads have their queue polling logic. Simply they check the queues and increase wait interval if the queues are empty.
I want to simplify the queue logic and make use of other Webjobs functionalities in this AzureWorker.
If you’d like to complete this task by using WebJobs, you could write a program and run as a WebJobs in your Azure App Service. And WebJobs API provides a way to dynamically start/stop WebJobs via REST API, you could use it to manage your WebJobs in your Worker Role.
We need API at Azure that would store messages sent to it (broker) via HTTP in case my system (Cloud Service) unavailable or DB is busy. It's not easy to change what exact message will be sent. What ways to make such a broker at Azure?
Service Bus Queue looks interesting but it needs Shared Access Signatures as far as I understand.
Another WebRole should be a solution but it needs time to implement.
Virtual Machine with some tool (MSMQ?) seems a way but it requires maintenance.
What do you think?
Your scenario is a prime candidate for applying a Queue-Centric Work Pattern.
From http://www.asp.net/aspnet/overview/developing-apps-with-windows-azure/building-real-world-cloud-apps-with-windows-azure/queue-centric-work-pattern:
If either your Worker(s) or Database become unavailable, messages are still placed in durable storage and consumed later.
The Task Queue can take the form of an Azure Storage Queue or a Service Bus Queue. In every great design, the least complex component that does the job wins. In this case that would be Azure Storage Queues, durable, reliable, very few moving parts. Unless you absolutely need precision FIFO ordering, in which case you go with Service Bus.
From https://msdn.microsoft.com/en-us/library/dn568101.aspx:
This solution offers the following benefits:
It enables an inherently load-leveled system that can handle wide variations in the volume of requests sent by application instances. The queue acts as a buffer between the application instances and the consumer service instances, which can help to minimize the impact on availability and responsiveness for both the application and the service instances (as described by the Queue-based Load Leveling pattern). Handling a message that requires some long-running processing to be performed does not prevent other messages from being handled concurrently by other instances of the consumer service.
It improves reliability. If a producer communicates directly with a consumer instead of using this pattern, but does not monitor the consumer, there is a high probability that messages could be lost or fail to be processed if the consumer fails. In this pattern messages are not sent to a specific service instance, a failed service instance will not block a producer, and messages can be processed by any working service instance.
It does not require complex coordination between the consumers, or between the producer and the consumer instances. The message queue ensures that each message is delivered at least once.
It is scalable. The system can dynamically increase or decrease the number of instances of the consumer service as the volume of messages fluctuates.
It can improve resiliency if the message queue provides transactional read operations. If a consumer service instance reads and processes the message as part of a transactional operation, and if this consumer service instance subsequently fails, this pattern can ensure that the message will be returned to the queue to be picked up and handled by another instance of the consumer service.
Given you can't change the client, I would proxy the call. Recreate the API using the API Management Service in Azure, and change the web url to point to the API Management Service proxy.
The proxy can then easily delegate to a function application like Aravind mentioned in the comments to your question by using the API Management Service policies.
I'm wondering how I can get certain behavior with Azure Service Bus topics and subscriptions. For our app we have 1 topic and many subscriptions. Subscriptions use filters. When a message is published it can be filtered by one or more subscriptions.
We already use peeklock and maxconcurrentcalls of 1, but that does not prevent multiple consumers from running in parallel.
What we want to do is let multiple consumer instances subscribe to the same subscription. But
* Only one instance may process a message at a given time. (in that way the order of processing is garanteed).
* We hope that those instances are load balanced.
So, at the end we want to load balance our consumers while keeping message order and no parallel processing.
How can we achieve this?
Edit: I want to make sure that I have failover of the consumers. And hopefully the consumers are load balanced: I don't want that all active instances are on 1 machine.
Multiple consumers will work in parallel, so it is very difficult to let just one work and have the others standby.
The solution is a single started consumer.
Failover of a consumer should be build on the consumer side, Azure has no idea of the state of your consumer.
So a second machine should check if the first one is still working, if not, take over, and make sure to lock the first one out in case this machine starts consuming again.
I need to create a worker role to process messages from multiple queues. I realized that there's a cloud project Worker Role with Service Bus Queue. Can I create multiple Queue Clients using this one or should I separate in N Worker Roles?
There is no restriction on how many Queues you can process from a single Worker role, you can create several QueueClients and then kick off Receive calls in parallel or register with OnMessage.
From an application perspective you need to think about isolation and scaling in that if you have queues with different types of workloads or different priorities then having different processing backends can provide more flexibility.
I want to create an Azure application which does the following:
User is presented with a MVC 4 website (web role) which shows a list of commands.
When the user selects a command, it is broadcast to all worker roles.
Worker roles process the task, store the results and notify web role
Web role displays the combined results of the worker roles
From what I've been reading there seem to be two ways of doing this: the Windows Azure Service Bus or using Queues. Each worker role also stores the results in the database.
The Service Bus seems more appropriate with its publish/subscribe model, so all worker roles would get the same command and roughly the same time. Queues seem easier to use though.
Can the service bus be used locally with the emulator when developing? I am using a free trial and cannot keep the application constantly whilst still developing. Also, when using queues how can you notify the web role that processing is complete?
I agree. ServiceBus is a better choice for this messaging requirement. You could, with some effort, do the same with queues. But, you'll be writing a lot of code to implement things that the ServiceBus already gives you.
There is not a local emulator for ServiceBus like there is for the Azure Strorage service (queues/tables/blobs). However, you could still use the ServiceBus for messaging between roles while they are running locally in your development environment.
As for your last question about notifying the web role that processing is complete, there are a several ways to go here. Just a few thoughts (not exhaustive list)...
Table storage where the web role can periodically check the status of the unit of work.
Another ServiceBus Queue/topic for completed work.
Internal endpoints. You'll have to have logic to know if it's just an update from worker role N or if it is indicating a completed unit of work for all worker roles.
I agree with Rick's answer, but would also add the following things to think about:
If you choose the Service Bus Topic approach then as each worker role comes online it would need to generate a subscription to the topic. You'll need to think about subscription maintenance of when one of the workers has a failure and is recycled, or any number of reasons why a subscription may be out there.
Telling the web role that all the workers are complete is interesting. The options Rick provides are good ones, but you'll need to think about some things here. It means that the web role needs to know just how many workers are out there or some other mechanism to decide when all have reported done. You could have the situation of five worker roles receieving a message and start working, then one of them starts to repeatedly fail processing. The other four report their completion but now the web role is waiting on the fifth. How long do you wait for a reply? Can you continue? What if you just told the system to scale down and while the web role thinks there are 5 there is now only 4. These are things you'll need to to think about and they all depend on your requirements.
Based on your question, you could use either queue service and get good results. But each of them are going to have different challenges to overcome as well as advantages.
Some advantages of service bus queues is that it provides blocking receipt with a persistent connection (up to 100 connections), it can monitor messages for completion, and it can send larger messages (256KB).
Some advantages of storage queues over the service bus solution is that it's slightly faster (if 15 ms matters to you), you can use a single storage system (since you'll probably be using Storage for blob and table services anyways), and simple auto-scaling. If you need to auto-scale your worker roles based on the load, passing the the requests through a storage queue makes auto-scaling trivial -- you just setup auto-scaling in the Azure Cloud Service UI under the scale tab.
A more in-depth comparison of the two azure queue services can be found here: http://msdn.microsoft.com/en-us/library/hh767287.aspx
Also, when using queues how can you notify the web role that processing is complete?
For the Azure Storage Queues solution, I've written a library that can help: https://github.com/brentrossen/AzureDistributedService.
It provides a proxy layer that facilitates RPC style communication from web roles to worker roles and back through Storage Queues.