I have a web role with multiple instances. I'd like to send an email to my customers with their statistics every morning at 7AM.
My problem is the following: if I use a Cron job to do the work mails will be sent multiple times.
How can I configure my instances in order to send only one email to each of my customers?
Per my experience, I think you can try to use a unique instance id to be ensure there is a single instance works for emailing as cron job.
Here is a simple code.
import com.microsoft.windowsazure.serviceruntime.RoleEnvironment;
String instanceId = RoleEnvironment.getCurrentRoleInstance().getId();
if("<instace-id-for-running-cronjob>".equals(instanceId)) {
// Coding for cron job
.....
}
As reference, please see the description of the function RoleInstance.getId below.
Returns the ID of this instance.
The returned ID is unique to the application domain of the role's instance. If an instance is terminated and has been configured to restart automatically, the restarted instance will have the same ID as the terminated instance.
More details for using the class above from Azure SDK for Java, please see the list of classes below.
RoleEnvironment
RoleInstance
Hope it helps.
Typically the way to solve this problem with Multi-instance Cloud Services would be to use Master Election pattern. What you would do is at 7:00 AM (or whenever your CRON job will fire), all your instances will try to acquire lease on a blob. However only one instance will be successful in acquiring the lease while other instances will fail with PreConditionFailed (419) error (make sure you catch this exception!). The instance that acquires the lease is essentially a Master and that instance will send out email.
Obviously the problem with this approach is that what if this Master instance fails to deliver the messages. But I guess that's another question :).
Related
Given: One Worker role + several Quartz.net jobs. Quartz jobs are host-agnostic and are executed in Worker role.
A worker role can be scaled to multiple instances.
The Goal is: have the ability to define what job to run in what instance at runtime (or define it with configuration only, no code changes). For example:
MyRole Instance 1: Job1, Job2, Job3
MyRole Instance 2: Job4,
MyRole Instance 3: Job4,
MyRole Instance 4: Job4,
MyRole Instance 5: Job5, Job6
In my example Job4 receives a lot of load. So I'd like it to run on more instances. And I also want it to be scalable at runtime (or at least via configuration, w/o code changes).
AFAIK it is not possible to have azure configuration per instance (only per role itself). Search online on similar issues haven't given any results.
Question: Did anybody have similar situation? What would be the best approach? Any other design advice is very appreciated. Thanks.
This is more of an architectural problem then one specific to Azure. The most common solution is to set up a "broker" that each instance/process reaches out to and asks for its individual workload. The challenge, regardless of the platform you are deploying the solution is how to a) identify the broker and b) ensure the "state" information being managed by the broker is persisted in case the broker process dies.
The most common approach to address these concerns in Azure is the use of a blob with a lease on it that allows the broker to be 'self elected' (the first process to get the lease is the broker), and stores both the address of the broker and the broker's state (metadata stored within the blob). You then put the logic for assigning jobs into this broker in a way that best suits your task distribution needs.
I'm running this Azure Cloudservice.
To make sure I can leverage the Servicebus in a safe way,
I have implemented the advice of this blog post: http://blogs.msdn.com/b/clemensv/archive/2012/07/30/transactions-in-windows-azure-with-service-bus-an-email-discussion.aspx
Instead of polling a database table constantly, I send a message to a queue to trigger a background process to look in the database and send the message.
But of course this alone, would not be safe, so I also have to periodically check the database on a schedule to make sure, I didn't miss anything.
But since I'm running multiple instances, I would like to spread this schedule, to make it more efficient, I thought I could do this, by getting the last integer from the Instance name. I know how to get this number using Microsoft.WindowsAzure.ServiceRuntime.RoleEnvironment.CurrentRoleInstance.Id
But I don't know how to get the total number of instances, I need that, in order to spread the schedule equally.
Does anybody know how to get this, without having to use Azure's management API's ?
Try Role.Instances property. Here's code from this page only to find all instances:
foreach (RoleInstance roleInst in RoleEnvironment.CurrentRoleInstance.Role.Instances)
{
Trace.WriteLine("Instance ID: " + roleInst.Id);
}
in my service deployment i have two roles.. a web role and a worker role.. in my on_start() method of webrole im enumerating the instances of the worker roles and creating a tcp connection on some internal end point. but very often it fails because the instances of the worker role havent started yet.
so the question is that can i know wether the instances have started or can i wait for instances of the worker role to start in some way?
herez the code
public override bool OnStart()
{
// For information on handling configuration changes
// see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.
ConnectionStatics.ConnectRouterToWorkers();
Messaging.KeepReadingMessages.Start();
return base.OnStart();
}
I'd recommend building retry logic into your loop so when its unable to establish the connection, it just sleep and retries it again later. Not only will this address your startup issue, but it will help you address changes to the service topology as instances are added/removed by the fabric controller (which can happen for numerous reasons).
Taken a step further, you might be able to leverage the RoleEnvironmentChanging and RoleEnvironmentChanged events to provide notification of when an instances is added/dropped. But I haven't leveraged this personally and can't say with any certain how these methods may or may not reflect the "ready state" of particular instances.
Another option would be to have the worker roles put a message in a queue when they are started up. Then you could just check the queue and wait for it to post a message there.
I see how Azure checks the status of my worker role periodically, but how?
There is no method in RoleEntryPoint to do that, and I'm taking a look on Microsoft.WindowsAzure.ServiceRuntime's classes with ILSpy but I don't see anything relevant.
Here's a blog post that describes how the Windows Azure Fabric Controller monitors instance health.
Aside from that, the controller calls a StatusCheck event, every 15 seconds, that you can handle. If you want to pull yourself out of the load balancer (maybe based on some internal data your instance has), you just call SetBusy() on the RoleEnvironmentStatusCheckEventArgs object. This takes you out of the load balancer until the next check.
I think the mechanism is the same as the one used for WebRoles
the Azure RoleEnvironment performs a StatusCheck - see http://msdn.microsoft.com/en-us/library/microsoft.windowsazure.serviceruntime.roleenvironment.statuscheck.aspx
If you want to tell the service you are busy then call SetBusy() when this event fires
I have two instances of a worker role.
I want to run a sub-task (on a Thread Pool thread) only on one of the Worker Role instances.
My initial idea was to do something like this:
ThreadPool.QueueUserWorkItem((o) =>
{
if (RoleEnvironment.CurrentRoleInstance.Id == RoleEnvironment.Roles[RoleEnvironment.CurrentRoleInstance.Role.Name].Instances.First().Id)
{
emailWorker.Start();
}
});
However, the above code relies on Role.Instances collection always returning the instances in the same order. Is this the case? or can the items be returned in any order?
Is there another approved way of running a task on one role instance only?
Joe, the solution you are looking for typically rely on:
either acquiring on lease (similar to a lock, but with an expiration) on a specific blob using the Blob Storage as a synchronization point between your role instances.
or queuing / dequeuing a message from the Queue Storage, which is usually the suggested pattern to delay long running operations such as sending an email.
Either ways, you need to go through the Azure Storage to make it work. I suggest to have a look at Lokad.Cloud, as we have designed this open-source framework precisely to handle this sort of situations.
If they need to be doing different things, then it sounds to me like you don't have 2 instances of a single worker role. In reality you have 2 different worker roles.
Especially when looking at the scalability of your application, processes need to be able to run on more than one instance. What happens when that task that you only want to run on one role gets large enough that it needs to scale to 2 or more role instances?
One of the benefits of developing for Azure is that you get scalability automatically if you design your app properly. If makes you work extra to get something that's not scalable, which is what you're trying to do.
What triggers this task to be started? If you use a message on Queue Storage (as suggested by Joannes) then only one worker role will pick up the message and process it and it doesn't matter which instance of your worker role does that.
So for now if you've got one worker role that's doing the sub task and another worker role that's doing everything else, just add 2 worker roles to your Azure solution. However, even if you do that, the worker role that processes the sub task should be written in such a way that if you ever scale it to run more than a single instance that it will run properly. In that case, you might as well stick with the single worker role and code for processing messages off the Queue to start your sub task.