Sync parallel operation in Azure - azure

Question

I need some achitectural suggestions about an Azure application. So, there is a queue with items, let's say it is [A, B, A, B, D].
Each distinct item in the queue will get a random category assigned and it is possible to have the same item multiple time in the queue. The category assignment is done by some worker roles which do the following: if the item has already a category assigned, it will add the item to the category, otherwise it will create a new category and the add the item. So it goes like:
D: has category? no. Create category 123. Assign [D, 123]
B: has category? no. Create category 435. Assign [B, 435]
A: has category? no. Create category 154. Assign [A, 154]
B: has category? yes. Assign [B, 435] (category already created)
... etc ...
My dillema is: how do I syncronize workers so that the same item doesn't get two categories? If two workers pick item B them it would be possible to have two categories for "B".

Answer 1

The only way to ensure that you don't get duplicates is to have a lock on the assigning of categories that can be accessed from both instances. The most popular way of doing this in Azure is with a lease on a blob in storage. If your items are of type Foo and you're passing the Id of the Foo through the queue, the pseudo code would look something like this:
int fooId = GetIdFromQueue();
Foo myFoo = LoadFooFromStorage(fooId);
if (myFoo.Category == null)
{
CreateLockBlobIfNoExistForFoo(fooId);
while (not GetLockOnBlobForFoo(fooId))
{
WaitForSomeTime();
}
// Need to reload the underlying item as another thread may have
// been assigning the category while we were waiting on the lock
Foo myFoo = LoadFooFromStorage(fooId);
if (myFoo.Category == null)
{
myFoo.Category = GetRandomCategory();
SaveFoo(myFoo);
}
ReleaseLease(fooId);
}
You'll need to look up some specifics on blob leases, but hopefully that's enough to get you started.

Answer 2

Maintain your Item/Category list in an azure table that is accessible to your worker roles, but this is still likely to end up in duplicates without some kind of throttling. For throttling, for example, set your GetMessage() in a timer loop with a reasonable wait (1-3 seconds) - and before each call to GetMessage(), call PeekMessages(5) to view but not dequeue the next 5 messages. Loop through them and assign categories any unassigned items and store them in the Azure table before calling GetMessage().