I've got this simple Azure Function:
public static class MyCounter
{
public static int _timerRound = 0;
public static bool _isFirst = true;
[FunctionName("Counter")]
//[TimeoutAttribute("00:00:05")]
public async static Task Run([TimerTrigger("*/10 * * * * *")]TimerInfo myTimer, TraceWriter log, CancellationToken token)
{
try
{
log.Info($"C# Timer trigger function executed at: {DateTime.UtcNow}");
if (_isFirst)
{
log.Info("Cancellation token registered");
token.Register(async () =>
{
log.Info("Cancellation token requested");
return;
});
_isFirst = false;
}
Interlocked.Increment(ref _timerRound);
for (int i = 0; i < 10; i++)
{
log.Info($"Round: {_timerRound}, Step: {i}, cancel request:{token.IsCancellationRequested}");
await Task.Delay(500, token).ConfigureAwait(false);
}
}
catch (Exception ex)
{
log.Error("hold on, exception!", ex);
}
}
}
What I'm trying to do is capturing the CancellationToken request event when the app stops or a code redeploy happened (host shutdown event).
BTW, I've also tried to check the IsCancellationRequested property in the for loop as well. Never turns true.
The main requirement is not to loose any operation/data during the function deployments, I want to know that the app is being stopped so that I persist some data to be processed once host started again after update.
Based on your code, I tested it on my side, here are my test result:
From the above screenshots, we could find that the subsequent rounds could not handle the cancellation callback except the first round. As Fabio Cavalcante commented, I removed the _isFirst logical checking and found it could work for all rounds as follows:
Note: I simulated the shutdown of my host by disabling my function when the TimerTrigger is triggered.
Related
I am new to Azure. I would like to create a function that returns the sequence number. I have created a function using Thread mutex to lock the sequence number. I tested the below code with around 10k parallel requests. The problem is I am getting duplicates sequence number when doing the testing, mutex is not working. I am not sure what to do to avoid duplication instead generate running number for each request
Public class MySharedMutexCounter {
public static long count = 0;
public static Mutex ObjMutex = new Mutex(false,"SeqGenerator");
}
public long GetSequnceNo(){
long seqId = -1;
try{
MySharedMutexCounter.ObjMutex.waitOne();
seqId = ++MySharedMutexCounter.count;
if(seqId > 100){
MySharedMutexCounter.count = 0;
seqId = ++MySharedMutexCounter.count;
}
return seqId;
}finally{
MySharedMutexCounter.ObjMutex.RelaseMutex();
}
return -1;
}
Thing is, an azure function can scale to multiple instances running on different machines so you need a distributed lock of some kind or another way to guarantee there won't be concurrent access to the state.
How about using a Durable Entity? It is basically a piece of state that can be accessed by a Durable Function and operations against the state are performed in a safe way:
To prevent conflicts, all operations on a single entity are guaranteed to execute serially, that is, one after another.
(source)
A durable entity is like a distributed object, so other instances of the function will use the same entity.
The Developer Guide demonstrates a nice example using a counter. Kind of fits your scenario.
Hi #Peter Bons I tried the below code but taking lot of time. May be something wrong in my code. Is it possible to get the value in a fraction of second bcos I shd return the value less than a second.
[FunctionName("FunctionOrchestrator")]
public static async Task<int> RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context)
{
int currentValue = -1;
var input = context.GetInput<CounterParameter>();
if (input != null && !string.IsNullOrWhiteSpace(input.OperationName))
{
var entityId = new EntityId("Counter", "myCounter");
// Perform the requested operation on the entity
currentValue = await context.CallEntityAsync<int>(entityId, input.OperationName);
}
return currentValue;
}
[FunctionName("Counter")]
public static int Counter([EntityTrigger] IDurableEntityContext ctx, ILogger log)
{
log.LogInformation($"Request for operation {ctx.OperationName} on entity.");
switch (ctx.OperationName.Trim().ToLowerInvariant())
{
case "increment":
ctx.SetState(ctx.GetState<int>() + 1);
break;
}
// Return the latest value
return ctx.GetState<int>();
}
[FunctionName("AutoIncrement")]
public static async Task<HttpResponseMessage> HttpAutoIncrement(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")] HttpRequestMessage req,
[DurableClient] IDurableOrchestrationClient starter,
[DurableClient] IDurableEntityClient client,
ILogger log)
{
// Function input comes from the request content.
var input = new CounterParameter { OperationName = "Increment" };
string instanceId = await starter.StartNewAsync("FunctionOrchestrator", input);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
await starter.WaitForCompletionOrCreateCheckStatusResponseAsync(req, instanceId);
var entityId = new EntityId("Counter", "myCounter");
try
{
// An error will be thrown if the counter is not initialised.
var stateResponse = await client.ReadEntityStateAsync<int>(entityId);
return new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(stateResponse.EntityState.ToString())
};
}
catch (System.NullReferenceException)
{
return new HttpResponseMessage(HttpStatusCode.NotFound)
{
Content = new StringContent("Counter is not yet initialised. " +
"Initialise it by calling increment or decrement HTTP Function.")
};
}
}
I am looking at this example to run a durable function Activity after a set timeout.
https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-eternal-orchestrations
This will allow my function activity to perform processing of data, then wait exactly 1 hour before it attempts to load again. This will continue to run forever. Perfect.
However, when publishing the Function to Azure, I don't want to have to manually invoke/start the function via the associated HTTP Trigger. I just want the durable function to kickoff automatically and start processing.
Is this possible? If not, what is a suggested work around?
Thanks!
As discussed in the comments, one way of doing this would be to add a new Task in your Release pipeline.
Here is what I understood of your setup from your question:
[FunctionName("ClientFunction")]
public static async Task<HttpResponseMessage> OnHttpTriggerAsync([HttpTrigger(AuthorizationLevel.Anonymous, "post")]
HttpRequestMessage request, [OrchestrationClient] DurableOrchestrationClient starter, ILogger logger)
{
// Triggers the orchestrator.
string instanceId = await starter.StartNewAsync("OrchestratorFunction", null);
return new HttpResponseMessage(HttpStatusCode.OK);
}
[FunctionName("OrchestratorFunction")]
public static async Task DoOrchestrationThingsAsync([OrchestrationTrigger] DurableOrchestrationContext context, ILogger logger)
{
DateTime deadline = context.CurrentUtcDateTime.Add(TimeSpan.FromHours(1));
await context.CreateTimer(deadline, CancellationToken.None);
// Triggers some yout activity.
await context.CallActivityAsync("ActivityFunction", null);
}
[FunctionName("ActivityFunction")]
public static Task DoAnAwesomeActivity([ActivityTrigger] DurableActivityContext context)
{
}
Now, every time you deploy a new version of the Function App, you need the orchestrator to be run. However, I do not think it can be started by itself.
What I propose is to have a simple bash script (using curl or something else) that would call the ClientFunction at the appropriate URL.
On top of that, one of the nice things of this solution is that you could make the deployment fail if the Azure Function does not respond.
This seems to be working too.
[FunctionName("AutoStart")]
public static async Task Run([TimerTrigger("*/5 * * * * *", RunOnStartup = true, UseMonitor = false)]TimerInfo myStartTimer,
[DurableClient] IDurableClient orchestrationClient, ILogger log)
{
string instanceId = await orchestrationClient.StartNewAsync("Start_Orchestrator", null);
}
I don't know if there are hidden problems with this, but I'm experimenting now with having a TimerTrigger that runs on startup and also once a day at midnight (or whatever schedule you want). That TimerTrigger will search the list of instances for any running instances of this orchestration, terminate them, then start a new one.
private const string MyOrchestrationName = "MyOrchestration";
[FunctionName("MyOrchestration_Trigger")]
public async Task MyOrchestrationr_Trigger(
[TimerTrigger("0 0 0 * * *", RunOnStartup = true)] TimerInfo timer,
[DurableClient] IDurableOrchestrationClient starter,
ILogger log,
CancellationToken cancellationToken)
{
// Get all the instances currently running that have a status of Pending, Running, ContinuedAsNew
var instances = await starter.ListInstancesAsync(new OrchestrationStatusQueryCondition()
{
ShowInput = false,
RuntimeStatus = new List<OrchestrationRuntimeStatus>() { OrchestrationRuntimeStatus.Suspended, OrchestrationRuntimeStatus.Pending, OrchestrationRuntimeStatus.Running, OrchestrationRuntimeStatus.ContinuedAsNew }
}, cancellationToken);
// Find any instances of the current orchestration that are running.
var myInstances = instances.DurableOrchestrationState.Where(inst => inst.Name == MyOrchestrationName);
List<Task> terminateTasks = new List<Task>();
foreach (var instance in myInstances )
{
// Delete any instances that are currently running.
terminateTasks.Add(starter.TerminateAsync(instance.InstanceId, $"Restarting eternal orchestration"));
}
await Task.WhenAll(terminateTasks);
// Start the new task now that other instances have been terminated.
string instanceId = await starter.StartNewAsync(MyOrchestrationName, null);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
}
I think at least for my purposes this will be safe. Any activities that are running when you terminate will still run to completion (which is what I want in my case), so you would just kill it and restart it on a schedule.
I'm using Cucumber to test my service when it is deployed to a container. The request contains a URL the service calls when the operation is successful. What is a good way to create a Cucumber test that waits for the Http callback? The Gherkin script would look something like.
Scenario: Process Order
Given An Order has been submitted
When the Order is processed
Then the order process service calls back with a successful status message
What would the Java glue code look like?
Here's the solution I came up with using an embedded http server. In the OrderSteps.java glue code I added a class used to start the server on another thread.
private static class Callback implements Runnable
{
public void run()
{
HttpServer server;
try
{
server = HttpServer.create(new InetSocketAddress(8000), 0);
server.createContext("/callback", new CallbackHandler());
server.setExecutor(null); // creates a default executor
server.start();
}
catch (IOException e)
{
logger.debug("HTTP server loop failure.", e);
}
}
static class CallbackHandler implements HttpHandler
{
#Override
public void handle(HttpExchange t) throws IOException
{
// Read the message and set the global variable
// which informs the main test thread a callback
// has been received.
InputStream is=t.getRequestBody();
byte[] buf=new byte[1000];
int len=is.read(buf);
OrderSteps.receivedCallback=new String(buf,0,len);
String response = "Callback received.";
t.sendResponseHeaders(200, response.length());
// Send response
OutputStream os = t.getResponseBody();
os.write(response.getBytes());
os.close();
}
}
}
Then in the OrderSteps class, in the step/method that publishes the Order, the server thread is started and then the order is submitted. This guarantees the server will receive the callback, since it is listening before the order is submitted.
// Start a listener for the callback.
Thread callbackThread = new Thread(new Callback());
callbackThread.start();
In the step/method that checks if the callback was received, there's a loop checking the static variable to see if it has been set.
// Allow 5 seconds for the callback to occur.
for (int i = 0; i < 5; i++)
{
if (receivedCallback != null) {
break;
}
Thread.sleep(1000);
}
if (receivedCallback == null) fail("Callback was not received.");
assertEquals("Expected callback message", receivedCallback);
In a ServiceStack Self-Hosted service, is it possible to gracefully shutdown the service when if pending requests exist?
Use AppHost.Stop()? (derived from AppHostHttpListenerBase)
I don't think there is a built in mechanism for this, though it would be nice to see it. I use my own simplistic Graceful shutdown method.
Essentially I have a static bool, IsShuttingDown flag that is checked prior to starting each request, at the first possible opportunity in the service pipeline. (RawHttpHandlers)
If this flag is set true it means I am not wanting the service to handle any more requests, and will instead send http status 503 Unavailable to the client.
My graceful shutdown method simply sets IsShuttingDown flag and starts a timeout timer of 60 seconds to give any currently processing requests time to complete. After which the service stops calling AppHost.Stop(). (See end of question for how to do it without a timer)
My code is for ServiceStack v3, you may have to modify it slightly to get it to work with v4 if you are using that version.
In your AppHost:
public static bool IsShuttingDown = false;
public override void Configure(Funq.Container container)
{
// Other configuration options ...
// Handle the graceful shutdown response
var gracefulShutdownHandler = new CustomActionHandler((httpReq, httpRes) => {
httpRes.StatusCode = 503;
httpRes.StatusDescription = "Unavailable";
httpRes.Write("Service Unavailable");
httpRes.EndRequest();
});
SetConfig(new EndpointHostConfig {
// Other EndPoint configuration options ...
RawHttpHandlers = { httpReq => IsShuttingDown ? gracefulShutdownHandler : null }
});
}
The CustomActionHandler is just copied from here, it is responsible for handling the request. (A custom action handler is included already in v4 so it wouldn't be needed)
public class CustomActionHandler : IServiceStackHttpHandler, IHttpHandler
{
public Action<IHttpRequest, IHttpResponse> Action { get; set; }
public CustomActionHandler(Action<IHttpRequest, IHttpResponse> action)
{
if (action == null)
throw new Exception("Action was not supplied to ActionHandler");
Action = action;
}
public void ProcessRequest(IHttpRequest httpReq, IHttpResponse httpRes, string operationName)
{
Action(httpReq, httpRes);
}
public void ProcessRequest(HttpContext context)
{
ProcessRequest(context.Request.ToRequest(GetType().Name),
context.Response.ToResponse(),
GetType().Name);
}
public bool IsReusable
{
get { return false; }
}
}
I appreciate that using a timer doesn't guarantee that all requests will have ended in the 60 seconds, but it works for my needs, where most requests are handled in far far less time.
To avoid using a timer (immediate shutdown when all connections closed):
Because there is no access to the underlying connection pool, you would have to keep track of what connections are active.
For this method I would use the PreExecuteServiceFilter and PostExecuteServiceFilter to increment & decrement an active connections counter. I am thinking you would want to use Interlocked.Increment and Interlocked.Decrement to ensure thread safety of your count. I haven't tested this, and there is probably a better way.
In your AppHost:
public static int ConnectionCount;
// Configure Method
// As above but with additional count tracking.
ConnectionCount = 0;
SetConfig(new EndpointHostConfig {
// Other EndPoint configuration options ...
RawHttpHandlers = { httpReq => IsShuttingDown ? gracefulShutdownHandler : null },
// Track active connection count
PreExecuteServiceFilter = () => Interlocked.Increment(ref ConnectionCount),
PostExecuteServiceFilter = (obj, req, res) => {
Interlocked.Decrement(ref ConnectionCount);
// Check if shutting down, and if there are no more connections, stop
if(IsShuttingDown && ConnectionCount==0){
res.EndRequest(); // Ensure last request gets their data before service stops.
this.Stop();
}
},
});
Hope some of this helps anyway.
I have a windows service developed in C#. On it's Start method I have a initialization such as:
Task _backgroundTask = null;
CancellationTokenSource _backgroundCancellationSource = null;
protected override void OnStart(string[] args)
{
......
_backgroundCancellationSource = new CancellationTokenSource();
CancellationToken token = backgroundCancellationSource.Token;
_backgroundTask = new Task(() => BackgroundFoldersProcessing(token), token, TaskCreationOptions.LongRunning);
.......
}
Now the method BackgroundFoldersProcessing looks like this:
void BackgroundFoldersProcessing(CancellationToken token)
{
while (true)
{
try
{
if (token.IsCancellationRequested)
{
return;
}
DoSomeWork()
}
catch (Exception ex)
{
.........
}
}
}
Now, the Stop method is as follows:
protected override void OnStop()
{
.................
_backgroundCancellationSource.Cancel();
_backgroundTask.Wait();
_backgroundCancellationSource.Dispose();
_backgroundTask.Dispose();
_backgroundTask = null;
_backgroundCancellationSource = null;
.................
}
Now the problem is when I try to stop the service in a middle of processing, the Wait method of _backgroundTask would not stop the service until and unless the DoSomeWork() method inside the BackgroundFoldersProcessing gets completed, the Windows Service would not stop.
Is there any way, though which I can stop the service and the execution of _backgroundTask would be terminated, even though the DoSomeWork() method gets completed/executed or not? I have also tried token.ThrowIfCancellationRequested() in BackgroundFoldersProcessing method, but that also did not worked. I want that whenever I try to Stop the service from Service Control Manager (SCM), the service should be stopped immediately and the __backgroundTask should stop executing the BackgroundFoldersProcessing method and be terminated as well. How can I achieve this?
You can try use ThreadAbortException:
defining the thread:
ThreadStart threadDelegate = new ThreadStart(BackgroundFoldersProcessing);
Thread thread_ = new Thread(threadDelegate);
thread_.Start();
Add catch to BackgroundFoldersProcessing
catch (ThreadAbortException e)
{
return;
}
and when you want to shut it down use:
thread_.Abort();
thread_.Join();
Then when Abort() will be called ThreadAbortException will be thrown.