My cloud service is deployed with two instances by default and has auto-scaling feature on which makes Azure to deploy additional instances as needed. I need to signal all instances currently running to invalidate their internal caches. Ideally by POSTing to a public facing url. But Azure load balancer will route the request randomly to one of the instances only.
I probably can utilize Service Bus topics/subscriptions to solve the problem but that looks like overkill. Is there an easier way?
From https://msdn.microsoft.com/library/azure/microsoft.windowsazure.serviceruntime.roleenvironment.changing.aspx:
RoleEnvironment.Changing Event
Occurs before a change to the service configuration is applied to the running instances of a role.
The Changing event and the Changed event are used together to identify and manage configuration changes to the service model. By using the Changing event, an instance can respond to a configuration change [...]
To be honest i can't remember if this is how you do it, but i believe i'm pretty close:
private void RoleEnvironmentChanging(object sender, RoleEnvironmentChangingEventArgs e)
{
if ((e.Changes.Any(change => change is RoleEnvironmentConfigurationSettingChange)))
{
// Deal with e.Cancel
// Invalidate cache
}
}
Here's more on what events are triggered on configuration change:
https://azure.microsoft.com/en-us/blog/responding-to-role-topology-changes/
The following diagram shows which events fire in an example scenario containing a single role. 2 instances are deployed initially, the deployment is then scaled to 4 instances, then back down to 3, and finally the deployment is stopped.
Related
I have an Azure logic app that correctly creates an Azure Container Instance. The container starts, does its job and terminates. I need to collect its logs with the appropriate connector and write them to an azure blob.
I have all the pieces in place but I do not know how to wait for the container to terminate before using the "get logs of container" connector to collect logs.
If the container job would last a predictable amount of time, I could use the Delay connector before getting the logs and it would suffice (I've tried with short jobs and it works well).
But my jobs may last several hours, depending on some external factors, so the Delay technique does not work.
I've tried with the "Until" connector, together with delay and the "get properties of a container group" container to wait until the state of the container is not "terminated", but without success (maybe I did it wrong). Anyway this can be quite expensive, since every "check" is billed.
How can I wait for the container to terminate before asking for its logs?
thanks.
starting from Charles Xu's answer, the correct sequence when setting the variable is
this uses the "state" container instance variable instead of "provisioning state". The latter is about the creation of the container group, the first is about the state of the container instance, which is what I need.
I added a delay to decrease the number of (paid) runs of the connector.
If you want to get the logs of a container group and nobody knows when it can terminate. In the logic app, you can use a variable to store the state of the container group and then use an until control the loop which will get the terminate of the container group until it comes true.
Here are the steps:
Create a container group;
Get the provision state of the container group;
Initialize a variable to store the provision state of the container group;
In the untile, get the provision state of the container group until the state is equal to terminate;
Get the logs of the container group.
The whole structure:
The Initialize variable and Until steps:
Since you mentioned using "Until" in logic app maybe expensive, here I provide another solution for your reference.
We can create a time trigger azure function and set the cron expression every 1 minutes, create a service plan(free tier) for the function, so we don't need to pay for the running cost of the function(but may pay for the storage of the function).
The function will run every 1 minute. In function we need to get the properties of the container by using this rest api and then if the state is "terminated", call the logic app request to trigger the logic app(the logic app should be created as "When a HTTP request is received").
The function code(in local, before deployed to azure) should be as below:
using System;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Extensions.Logging;
namespace hurytimeFun
{
public static class Function1
{
[FunctionName("Function1")]
public static void Run([TimerTrigger("0 */1 * * * *")]TimerInfo myTimer, ILogger log)
{
log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
//1. using rest api to get the properties of your container
//2. if the state is "terminated"
// call the logic app request to trigger the logic app
}
}
}
Hope it would be helpful to your problem~
Once a container terminates, everything is lost. You could install the log-analytics-containers from the Azure marketplace for logging and log before the container terminates
I'm trying to find a solution that I can use to perform virus scanning on files that have been uploaded to Azure blob storage. I wanted to know if it is possible to copy the file to local storage on a Worker Role instance, call Antimalware for Azure Cloud Services to perform the scan on that specific file, and then depending on whether the file is clean, process the file accordingly.
If the Worker Role cannot call the scan programmatically, is there a definitive way to check if a file has been scanned and whether it is clean or not once it has been copied to local storage (I don't know if the service does a real-time scan when new files are added, or only runs on a schedule)?
There isn't a direct API that we've found, but the anti-malware services conform to the standards used by Windows desktop virus checkers in that they implement the IAttachmentExecute COM API.
So we ended up implementing a file upload service that writes the uploaded file to a Quarantine local resource, then calling the IAttachmentExecute API. If the file is infected then, depending on the anti-malware service in use, it will either throw an exception, silently delete the file or mark it as inaccessible. So by attempting to read the first byte of the file, we can test if the file remains accessible.
var type = Type.GetTypeFromCLSID(new Guid("4125DD96-E03A-4103-8F70-E0597D803B9C"));
var svc = (IAttachmentExecute)Activator.CreateInstance(type);
try {
svc.SetClientGuid(ref clientGuid);
svc.SetLocalPath(path);
svc.Save();
}
finally
{
svc.ClearClientState();
}
using (var fileStream = File.OpenRead(path))
{
fileStream.ReadByte();
}
[Guid("73DB1241-1E85-4581-8E4F-A81E1D0F8C57")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
public interface IAttachmentExecute
{
void SetClientGuid(ref Guid guid);
void SetLocalPath(string pszLocalPath);
void Save();
void ClearClientState();
}
I think the best way for you to know is simply take an Azure VM (IaaS) and activate Microsoft Antimalware extension. Then you may log into it and do all the necessary check and tests against the service.
Later, you will apply all this into the Worker Role (there is a similar PaaS extension available for that, calles PaaSAntimalware).
See the next excerpt from https://msdn.microsoft.com/en-us/library/azure/dn832621.aspx:
"In PaaS, the VM agent is called GuestAgent, and is always available on Web and Worker Role VMs. (For more information, see Azure Role Architecture.) The VM agent for Role VMs can now add extensions to the cloud service VMs in the same way that it does for persistent Virtual Machines.
The biggest difference between VM Extensions on role VMs and persistent VMs is that with role VMs, extensions are added to the cloud service first and then to the deployments within that cloud service.
Use the Get-AzureServiceAvailableExtension cmdlet to list all available role VM extensions."
The scenario I have in mind is this: Service Bus is used for instance-to-instance communication, so a Subscription is unique per service instance. The end result is that if an instance does not shut down gracefully, its subscription does not get deleted.
When a service instance "dies" and restarts, previous contents of the subscription are irrelevant and can be discarded.
So, is there a way to set a "time to live" for Service Bus Subscription or simulate something similar, without having to resort to some custom orphan detection mechanism?
Starting with Azure SDK 2.0 this works as expected.
Also, contrary to other reports, in my testing, subscription does not get deleted as long as there is a pending receiver listening to that subscription.
var description = new SubscriptionDescription(topicPath, subscriptionId);
description.AutoDeleteOnIdle = TimeSpan.FromSeconds(600);
namespaceManager.CreateSubscription(description);
that exact feature is on the backlog for one of the next releases. that said, in azure you could use the instance-id fro the role environment to create the name of your subscription and thus have a restarting instance reuse a subscription. the instance-id names are stable.
Edit: The feature is AutoDeleteOnIdle https://learn.microsoft.com/en-us/dotnet/api/microsoft.servicebus.messaging.subscriptiondescription
I had the exact same problem, preview solving it was released beginning of 2013: http://msdn.microsoft.com/en-us/library/microsoft.servicebus.messaging.subscriptiondescription.autodeleteonidle.aspx
It's very easy to use (see example below). Unfortunately it seems that the subscription times out if there is no message published for the AutoDeleteOnIdle period, even if you have some process awaiting for messages (according to Azure Servicebus AutoDeleteOnIdle).
NamespaceManager manager=NamespaceManager.CreateFromConnectionString(serviceBusConnectionString);
if(!manager.SubscriptionExists(topic,subscriptionName))
{
manager.CreateSubscription(new SubscriptionDescription(topic,subscriptionName) {
AutoDeleteOnIdle=TimeSpan.FromDays(2)
});
}
Background: I've deployed an MVC3 application to 2 Azure Web Role instances, but I'm confused as to how I can test out the possibility of one of these instances failing.
Is there a way that I can test to ensure that my Web Role code works seamlessly when one of my instances is taken offline?
Can I manually stop one of them? Or somehow configure the load balancer to force all traffic to one of the servers?
Thanks!
If you have RDP access enabled to your instances you can very easily remove one or more instance out from LoadBalancer even when the instance is running healthy without writing any line of code. You just need to RDP to your instance and then use PowerShell scripts to take the instance off from loadbalancer. In my following blog I have described the exact procedure:
http://blogs.msdn.com/b/avkashchauhan/archive/2012/01/27/windows-azure-troubleshooting-taking-specific-windows-azure-instance-offline.aspx
The above details also help to run load testing by removing N instances from total M instances .
If you use the StatusCheck event to set the status to Busy, your instance won't receive any more requests from the loadbalancer. You might want to write some code to send messages to your instances that will set them as busy for some time (using queues for example).
public override bool OnStart()
{
RoleEnvironment.StatusCheck += RoleEnvironmentStatusCheck;
return base.OnStart();
}
// Use the busy object to indicate that the status of the role instance must be Busy
private volatile bool busy = true;
private void RoleEnvironmentStatusCheck(object sender, RoleInstanceStatusCheckEventArgs e)
{
If (this.busy)
{
// Sets the status of the role instance to Busy for a short interval.
// If you want the role instance to remain busy, add code to
// continue to call the SetBusy method
e.SetBusy();
}
}
Reference: http://msdn.microsoft.com/en-us/library/microsoft.windowsazure.serviceruntime.roleenvironment.statuscheck.aspx
Besides that you can also simply reboot your instance (using the Windows Azure portal or Remote Desktop).
Steve Marx (aka smarx) developed a tool called WazMonkey. It is the azure counterpart of the tool Chaos Monkey developed by the netflix team to simulate broken instances in amazon AWS. WazMonkey terminates instances of a windows azure cloud service randomly to test the resilience of a cloud applications.
I'm doing my first project but large one on developing Azure Application with Intergration Component.
Currently most of the integration are done using SSIS Packages and would like to transform them on to Worker Role in Azure.
Could someone please help me to understand the following queries regarding Worker Role please?
Is there way to start or stop the Worker role (just like SSIS or Windows Schedulers) via GUI? If not how to achieve this?
How do I know my worker role has been running or not running (including why it's not running ie. logs)
How do I spin multiple worker role based on time (i.e. (9:00AM to 11:00AM spin 4 roles and scale down on quiet period)
Does the following code creates any poison message or dead lock (if multiple there are 10,000 messages to process and every 5 seconds the new thread (Processsing.run) is started?
while(true)
{
var thread = new Thread(Run);
thread.start();
Thread.Sleep(5000);
Trace.WriteLine("Working", "Information");
}
public class PhotoProcessing
{
public static void Run()
{
// Read from queue
CloudQueueMessage msg =
Storage.Queue.GetNextMessage();
while(msg != null)
{
string[] message = msg.AsString.Split('$');
if(message.Length == 2)
{
AddWatermark(message[0], message[1]);
}
// Message has been read so remove it
Storage.Queue.DeleteMessage(msg);
// Get next message if any
msg = Storage.Queue.GetNextMessage();
}
}
Is there way to start or stop the Worker role (just like SSIS or Windows Schedulers) via GUI? If not how to achieve this?
There are actually many ways to achieve this. You can use Windows Azure Portal to do or you could use 3rd party tools (like our Cloud Storage Studio) or you could write your own application using Windows Azure Service Management API (http://msdn.microsoft.com/en-us/library/ee460799.aspx)
How do I know my worker role has been running or not running (including why it's not running ie. logs)
Again you could use one of the GUI based tools to see the status of your roles. As far as why the roles are not running, you would need to enable Windows Azure Diagnostics in your worker role (http://msdn.microsoft.com/en-us/library/gg433048.aspx)
How do I spin multiple worker role based on time (i.e. (9:00AM to 11:00AM spin 4 roles and scale down on quiet period)
You can write your own application using Windows Azure Service Management API to do so or you could make use of 3rd party tools like AzureWatch from Paraleap or Azure Management Cmdlets (both from Microsoft and our company). While the cmdlets will get the job done, I believe Azure Watch is much more sophisticated solution. We wrote a blog post for autoscaling some days back which you can find here: http://www.cerebrata.com/Blog/post/Scale-your-Windows-Azure-instances-with-Azure-Management-Cmdlets.aspx.