Azure Web App Windows Container Terminated before gracefull shutdown - azure-web-app-service

I've created a .NET Framework Console program which starts and runs some code, then upon exit it should logout of any external services before exiting (gracefully shutdown).
Here is a sample program:
using System;
using System.Runtime.InteropServices;
using System.Threading;
namespace delayed_shutdown
{
class Program
{
public enum CtrlTypes
{
CTRL_C_EVENT = 0,
CTRL_BREAK_EVENT,
CTRL_CLOSE_EVENT,
CTRL_LOGOFF_EVENT = 5,
CTRL_SHUTDOWON_EVENT
}
[DllImport("Kernel32")]
public static extern bool SetConsoleCtrlHandler(HandlerRoutine handler, bool Add);
public delegate bool HandlerRoutine(CtrlTypes CtrlType);
public static volatile HandlerRoutine handlerRoutine = new HandlerRoutine(ConsoleCtrlCheck), true)
public static volatile ManualResetEvent exitEvent = new ManualResetEvent(false);
public static bool ConsoleCtrlCheck(CtrlTypes ctrlType)
{
switch (ctrlType)
{
case CtrlTypes.CTRL_C_EVENT:
Console.WriteLine("CTRL_C received");
exitEvent.Set();
return true;
case CtrlTypes.CTRL_CLOSE_EVENT:
Console.WriteLine("CTRL_CLOSE received");
exitEvent.Set();
return true;
case CtrlTypes.CTRL_BREAK_EVENT:
Console.WriteLine("CTRL_BREAK received");
exitEvent.Set();
return true;
case CtrlTypes.CTRL_LOGOFF_EVENT:
Console.WriteLine("CTRL_LOGOFF received");
exitEvent.Set();
return true;
case CtrlTypes.CTRL_SHUTDOWON_EVENT:
Console.WriteLine("CTRL_SHUTDOWN received");
exitEvent.Set();
return true;
default:
return false;
}
}
static int Main(string[] args)
{
if (!SetConsoleCtrlHandler(handlerRoutine))
{
Console.WriteLine("Error setting up control handler... :(");
return -1;
}
Console.WriteLine("Waiting for control event...");
exitEvent.WaitOne();
var i = 60;
Console.WriteLine($"Exiting in {i} seconds...");
while (i > 0)
{
Console.WriteLine($"{i}");
Thread.Sleep(TimeSpan.FromSeconds(1));
i--;
}
Console.WriteLine("Goodbye");
return 0;
}
}
}
I would have expected Windows Containers running as Azure App Service to trigger "docker stop" like function, which would send SIGTERM to my application.
But what happens is that Azure Web App Windows Container is terminated, after 1 sec of trying to stop the container. How do ask Azure Web App to wait X number of seconds before terminating the windows container?

We are currently working on signaling the process upon stop for Windows Containers on Azure App Service.
In Azure App Service we will default to 5 seconds for waiting for a container to exit upon shutdown but we will allow this to be configurable using the following app setting: WEBSITES_CONTAINER_STOP_TIME_LIMIT and we will allow to wait up to 2 min (WEBSITES_CONTAINER_STOP_TIME_LIMIT=00:02:00)
This capability will be deployed in the next update rollout and I hope to be available worldwide early next year and once it is out we will update our docs too, so please stay tuned.

Intercepting the SIGTERM event is something that isn't currently supported. Since App Service is tailored to HTTP workloads, I am curious though as to the reasoning of having a console app pick up such an event. If you could elaborate further, there may be an alternative such as running your console app as a Web Job instead.

Related

Azure Cloud Service: RoleEnvironment.StatusCheck event not firing

I am maintaining a legacy Cloud Services application hosted on Azure targeting .net 4.6.1. Inside the Application_Start method of the Global.asax on the Web Role we are registering an event handler for RoleEnvironment.StatusCheck however our logs are demonstrating that this event call back is never being called or triggered.
According to this blog: https://convective.wordpress.com/2010/03/18/service-runtime-in-windows-azure/ we were expecting this event to be triggered every 15 seconds and we believe this was happening however has since stopped. We expect that the stopped working around the time we installed some new DLLs into the solution (some of these dlls include: Microsoft.Rest.ClientRuntime.dll, Microsoft.Azure.Storage.Common.dll, Microsoft.Azure.Storage.Blob.dll, Microsoft.Azure.KeyVault.dll)
We've tried RDP-ing onto the VM to check the event logs but nothing obvious is there. Any suggestions on where we may be able to search for clues?
It seems your event handler is not registered. Try below code with a different approach:
public class WorkerRole : RoleEntryPoint
{
public override bool OnStart()
{
RoleEnvironment.StatusCheck += RoleEnvironmentStatusCheck;
return base.OnStart();
}
// Use the busy object to indicate that the status of the role instance must be Busy
private volatile bool busy = true;
private void RoleEnvironmentStatusCheck(object sender, RoleInstanceStatusCheckEventArgs e)
{
if (this.busy)
{
// Sets the status of the role instance to Busy for a short interval.
// If you want the role instance to remain busy, add code to
// continue to call the SetBusy method
e.SetBusy();
}
}
public override void Run()
{
Trace.TraceInformation("Worker entry point called", "Information");
while (true)
{
Thread.Sleep(10000);
}
}
public override void OnStop()
{
base.OnStop();
}
}

Webjob is causing performance issue with ASP.NET core

I created a .NET CORE console application and uploaded as a continuous mode webjob to an Azure application (ASP.NET Core). With webjob running, webapp is very slow responding to API request (Request time rise to few seconds).
WebJob code
static void Main(string[] args)
{
queueClient.RegisterMessageHandler(
async (message, token) =>
{
// Process the message
// Complete the message so that it is not received again.
// This can be done only if the queueClient is opened in ReceiveMode.PeekLock mode.
await queueClient.CompleteAsync(message.SystemProperties.LockToken);
},
new MessageHandlerOptions(exce => {
return Task.CompletedTask;
})
{ MaxConcurrentCalls = 1, AutoComplete = false });
//Console.ReadKey();
while (true) ;
}
And the processing the message operation takes few seconds.
from SCM console
Request time
The while(true) ; will pin the CPU so I suggest you don't do that.
Check the Queue message handling example for the proper way how to implement message handling: https://github.com/Azure/azure-webjobs-sdk/wiki/Queues
You will have to change your Main to:
static void Main(string[] args)
{
JobHost host = new JobHost();
host.RunAndBlock();
}
And then you can make a message handler in another file:
public static void ProcessQueueMessage([QueueTrigger("logqueue")] string logMessage, TextWriter logger)
{
logger.WriteLine(logMessage);
}
The 3.0.0 preview versions of the Microsoft.Azure.WebJobs library support .NET Standard 2.0, so it can be used with .NET Core 2.0 projects.
If you want to implement it yourself, you can check the Webjobs SDK source code for an example how they do it: https://github.com/Azure/azure-webjobs-sdk/blob/dev/src/Microsoft.Azure.WebJobs.Host/JobHost.cs

Azure web jobs - parallel message processing from queues not working properly

I need to provision SharePoint Online team rooms using azure queues and web jobs.
I have created a console application and published as continuous web job with the following settings:
config.Queues.BatchSize = 1;
config.Queues.MaxDequeueCount = 4;
config.Queues.MaxPollingInterval = TimeSpan.FromSeconds(15);
JobHost host = new JobHost();
host.RunAndBlock();
The trigger function looks like this:
public static void TriggerFunction([QueueTrigger("messagequeue")]CloudQueueMessage message)
{
ProcessQueueMsg(message.AsString);
}
Inside ProcessQueueMsg function i'm deserialising the received json message in a class and run the following operations:
I'm creating a sub site in an existing site collection;
Using Pnp provisioning engine i'm provisioning content in the sub
site (lists,upload files,permissions,quick lunch etc.).
If in the queue I have only one message to process, everything works correct.
However, when I send two messages in the queue with a few seconds delay,while the first message is processed, the next one is overwriting the class properties and the first message is finished.
Tried to run each message in a separate thread but the trigger functions are marked as succeeded before the processing of the message inside my function.This way I have no control for potential exceptions / message dequeue.
Tried also to limit the number of threads to 1 and use semaphore, but had the same behavior:
private const int NrOfThreads = 1;
private static readonly SemaphoreSlim semaphore_ = new SemaphoreSlim(NrOfThreads, NrOfThreads);
//Inside TriggerFunction
try
{
semaphore_.Wait();
new Thread(ThreadProc).Start();
}
catch (Exception e)
{
Console.Error.WriteLine(e);
}
public static void ThreadProc()
{
try
{
DoWork();
}
catch (Exception e)
{
Console.Error.WriteLine(">>> Error: {0}", e);
}
finally
{
// release a slot for another thread
semaphore_.Release();
}
}
public static void DoWork()
{
Console.WriteLine("This is a web job invocation: Process Id: {0}, Thread Id: {1}.", System.Diagnostics.Process.GetCurrentProcess().Id, Thread.CurrentThread.ManagedThreadId);
ProcessQueueMsg();
Console.WriteLine(">> Thread Done. Processing next message.");
}
Is there a way I can run my processing function for parallel messages in order to provision my sites without interfering?
Please let me know if you need more details.
Thank you in advance!
You're not passing in the config object to your JobHost on construction - that's why your config settings aren't having an effect. Change your code to:
JobHost host = new JobHost(config);
host.RunAndBlock();

How to integration test Azure Web Jobs?

I have a ASP.NET Web API application with supporting Azure Web Job with functions that are triggered by messages added to a storage queue by the API's controllers. Testing the Web API is simple enough using OWIN but how do I test the web jobs?
Do I run a console app in memory in the test runner? Execute the function directly (that wouldn't be a proper integration test though)? It is a continious job so the app doesn't exit. To make matters worse Azure Web Job-functions are void so there's no output to assert.
There is no need to run console app in memory. You can run JobHost in the memory of your integration test.
var host = new JobHost();
You could use host.Call() or host.RunAndBlock(). You would need to point to Azure storage account as webjobs are not supported in localhost.
It depends on what your function is doing, but you could manually add a message to a queue, add a blob or whatever. You could assert by querying the storage where your webjob executed result, etc.
While #boris-lipschitz is correct, when your job is continious (as op says it is), you can't do anything after calling host.RunAndBlock().
However, if you run the host in a separate thread, you can continue with the test as desired. Although, you have to do some kind of polling in the end of the test to know when the job has run.
Example
Function to be tested (A simple copy from one blob to another, triggered by created blob):
public void CopyBlob(
[BlobTrigger("input/{name}")] TextReader input,
[Blob("output/{name}")] out string output)
{
output = input.ReadToEnd();
}
Test function:
[Test]
public void CopyBlobTest()
{
var blobClient = GetBlobClient("UseDevelopmentStorage=true;");
//Start host in separate thread
var thread = new Thread(() =>
{
Thread.CurrentThread.IsBackground = true;
var host = new JobHost();
host.RunAndBlock();
});
thread.Start();
//Trigger job by writing some content to a blob
using (var stream = new MemoryStream())
using (var stringWriter = new StreamWriter(stream))
{
stringWriter.Write("TestContent");
stringWriter.Flush();
stream.Seek(0, SeekOrigin.Begin);
blobClient.UploadStream("input", "blobName", stream);
}
//Check every second for up to 20 seconds, to see if blob have been created in output and assert content if it has
var maxTries = 20;
while (maxTries-- > 0)
{
if (!blobClient.Exists("output", "blobName"))
{
Thread.Sleep(1000);
continue;
}
using (var stream = blobClient.OpenRead("output", "blobName"))
using (var streamReader = new StreamReader(stream))
{
Assert.AreEqual("TestContent", streamReader.ReadToEnd());
}
break;
}
}
I've been able to simulate this really easily by simply doing the following, and it seems to work fine for me:
private JobHost _webJob;
[OneTimeSetUp]
public void StartupFixture()
{
_webJob = Program.GetHost();
_webJob.Start();
}
[OneTimeTearDown]
public void TearDownFixture()
{
_webJob?.Stop();
}
Where the WebJob Code looks like:
public class Program
{
public static void Main()
{
var host = GetHost();
host.RunAndBlock();
}
public static JobHost GetHost()
{
...
}
}

how to restart windows service c#

I want every to hour restart windows service by c# code. I have this method
but when I but it in project installer or where? Can I put it in the same service I want to restart it?
public static void RestartService(string serviceName, int timeoutMilliseconds)
{
ServiceController service = new ServiceController(serviceName);
try
{
int millisec1 = Environment.TickCount;
TimeSpan timeout = TimeSpan.FromMilliseconds(timeoutMilliseconds);
service.Stop();
service.WaitForStatus(ServiceControllerStatus.Stopped, timeout);
// count the rest of the timeout
int millisec2 = Environment.TickCount;
timeout = TimeSpan.FromMilliseconds(timeoutMilliseconds - (millisec2 - millisec1));
service.Start();
service.WaitForStatus(ServiceControllerStatus.Running, timeout);
}
catch
{
// ...
}
}
i solved it by making a method inside the service like this
private void StopService()
{
using (ServiceController service = new ServiceController("ServiceName"))
{
try
{
TimeSpan timeout = TimeSpan.FromMilliseconds(80000);
if (!(service.Status.Equals(ServiceControllerStatus.StartPending) || service.Status.Equals(ServiceControllerStatus.StopPending)))
{
service.Stop();
service.WaitForStatus(ServiceControllerStatus.Stopped, timeout);
}
}
catch (Exception ex)
{
ExceptionLog(ex);
}
}
}
and make another service to restart it
You could add this to the Windows scheduler. Just put this function in a windows command program and schedule hourly launch of it. Also, you could have a second service to accomplish the recycling of the first service.
-rwg

Resources