I'm quite new to AzureFunctions so excuse me for maybe a simple question...
I have got the following code which is fired when I have an item on the queue
[FunctionName("ProcessCompanies")]
public async Task ProcessCompaniesAsync([ActivityTrigger] IDurableOrchestrationContext context,
[Queue("outqueue"), StorageAccount("AzureWebJobsStorage")] ICollector<myCompanyData> msg,
ILogger log)
{
log.LogInformation("Getting companies");
var companies = await _myService.GetCompaniesAsync(); //here how to pass cancellation token?
log.LogInformation($"Found {companies.companies.Count} companies.");
companies.companies.ForEach(msg.Add);
log.LogInformation($"{companies.companies.Count} companies added to queue.");
}
How can I pass to the Async call the CancellationToken? and mainly, should I pass it? in this case I perform an HTTP API request
You can inject a CancellationToken into the function method. It is a way to get notified about a graceful shutdown of the function. See https://learn.microsoft.com/en-us/azure/azure-functions/functions-dotnet-class-library?tabs=v4%2Ccmd#cancellation-tokens
Once injected you can pass it to downstream calls. Whether you should pass it to that API is up to you. There is some short period of time before the process killed so if the call does not take long you could just try to finish it.
Related
I've implemented an advice handler attached to outbound-gateway components in order to be able to long outgoing HTTP requests before/after. Originally, this was for a Spring-Integration project using MVC/servlet underpinnings. (Original question: Logging http request in Spring integration http outbound gateway )
We are attempting a migration to webflux/reactive which means that we are using a webflux:outbound-gateway in lieu of a http:outbound-gateway.
The advice handler is an around call in the doInvoke() method:
public class MyLogger extends AbstractRequestHandlerAdvice {
// ...
#Override
protected Object doInvoke(ExecutionCallback callback, Object target, Message<?> message(
{
// ... logging before the call ...
Object result = callback.execute();
// ... logging after the call ...
return result;
}
}
Whereas I understand that the webflux:outbound-gateway call itself is supposed to be asynchronous (i.e. no threads waiting on a response), it would seem to me that the advice handler itself must be implemented synchronously (i.e. thread awaiting the response). Am I understanding correctly? Is the advice handler adding synchronicity that wouldn't otherwise be there?
Thanks for any insights on this point.
There is a ReactiveRequestHandlerAdvice for this particular use-case to wrap a reply Mono with anything you'd like: https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-endpoints.html#reactive-advice
I have created a Service Fabric Application. There is a long-running job in the RunAsync() method like "LoadData() which migrates million of records from Database to ServiceFabric dictionary".
As per the MSDN documentation, Services that want to implement a background task, which runs when the service comes up, should override this method with their logic "RunAsync(CancellationToken)"
Cancellation token is used to monitor for cancellation requests and it exists from RunAsync() and shut down the service. So I had used CancellationToken in my project.
Here is my code
RunAsync (System.Threading.CancellationToken cancellationToken);
{
LoadData(CancellationToken);
}
Async Task LoadData(CancellationToken)
{
Method1(CancellationToken) -- Aync call
Method2() -- Normal call
Method3() -- Normal call
}
As you can see, I have Method1, which is an asynchronous call and it runs as a separate thread so the same token is being passed to this method as the main thread will not aware of the child thread.
However, Method2 and Method3 are just functional calls so the CancellationToken is not passed because there are running in the context of the main thread.
I have a couple of questions on CancellationToken usage in Service Fabric.
1. Do we need to pass the CancellationToken for all methods that are being called from the Long-running method( Async/ Sync)?
2. The way how I have handled CancellationToken is correct or Do I need to modify anything?
Do we need to pass the CancellationToken for all methods that are being called from the Long-running method( Async/ Sync)?
Yes, if you are able to respect it. It makes no sense to pass it to a method if it does not do anything with it.
The way how I have handled CancellationToken is correct or Do I need to modify anything?
Hard to tell, we need to know what your code does.
If the service is going to shut down it will use the CancellationToken to cancel any pending work. If you have any async calls that accept a CancellationToken you should pass it to that method so it can exit when cancellation is requested.
If you have some other methods like Method1() and it does some long running work you can still exit when cancellation is requested by either using CancellationToken.IsCancellationRequested or CancellationToken.ThrowIfCancellationRequested. The latter will throw an OperationCanceledException if cancellation is requested.
For example, you can do this:
public void Method1(CancellationToken cancellationToken)
{
// some work
if(cancellationToken.IsCancellationRequested)
return;
// some more work
if (cancellationToken.IsCancellationRequested)
return;
// some more work
}
It is up to you to decide whether the method has some logical points to check for cancellation or that the method is so fast it won't make much difference.
By the way, this is not specific to service fabric, you can find more docs here.
We have an Azure Storage Queue which triggers an azure function once a payload/message hits the queue. The queue-triggered function invokes another durable function to process the message/payload.
Here it is the code snippet:
[FunctionName("QueueTriggerFunction")]
public Task QueueTriggerFunction(
[QueueTrigger("MyQueue", Connection = "MyStorage")]string item,
[OrchestrationClient] DurableOrchestrationClient client,
ILogger log)
=> client.StartNewAsync("Processor", JsonConvert.DeserializeObject<MyObject>(item));
And the durable function looks like the following code sample:
[FunctionName("Processor")]
public async Task ConcurrencyProcessorAsync(
[OrchestrationTrigger] DurableOrchestrationContext context,
ILogger log)
{
var myObject= context.GetInput<MyObject>();
if(ObjectProcessor(myObject) == false)
{
throw new Exception("Processor failed");
}
}
I'd like the payload to end up in the poison messages queue if the exception above is raised upon failing the ObjectProcessor method but it's not happening in reality because the exception does not bublle up through the orchestrator client. Any suggestions on how to make this exception thrown back to the caller function which is a queue-triggered one to make the payload appear in the poison messages queue?
You can't.
The QueueTriggerFunction just starts the Orchestration. After that it's life cycle ends.
I believe you can directly add your payload to poison queue using either Azure Storage Services REST API or this .Net library
Please note that name of poison queue == $"{queueName}-poison"
I have an issue with Azure Function Service Bus trigger.
The issue is Azure function cannot wait a message done before process a new message. It process Parallel, it not wait 5s before get next message. But i need it process sequencecy (as image bellow).
How can i do that?
[FunctionName("HttpStartSingle")]
public static void Run(
[ServiceBusTrigger("MyServiceBusQueue", Connection = "Connection")]string myQueueItem,
[OrchestrationClient] DurableOrchestrationClient starter,
ILogger log)
{
Console.WriteLine($"MessageId={myQueueItem}");
Thread.Sleep(5000);
}
I resolved my problem by using this config in my host.json
{
"version": "2.0",
"extensions": {
"serviceBus": {
"messageHandlerOptions": {
"maxConcurrentCalls": 1
}
}
}}
There are two approaches you can accomplish this,
(1) You are looking for Durable Function with function chaining
For background jobs you often need to ensure that only one instance of
a particular orchestrator runs at a time. This can be done in Durable
Functions by assigning a specific instance ID to an orchestrator when
creating it.
(2) Based on the messages that you are writing to Queue, you need to partition the data, that will automatically handle the order of messages which you do not need to handle manually by azure function
In general, ordered messaging is not something I'd be striving to implement since the order can and at some point will be distorted. Saying that, in some scenarios, it's required. For that, you should either use Durable Function to orchestrate your messages or use Service Bus message Sessions.
Azure Functions has recently added support for ordered message delivery (accent on the delivery part as processing can still fail). It's almost the same as the normal Function, with a slight change that you need to instruct the SDK to utilize sessions.
public async Task Run(
[ServiceBusTrigger("queue",
Connection = "ServiceBusConnectionString",
IsSessionsEnabled = true)] Message message, // Enable Sessions
ILogger log)
{
log.LogInformation($"C# ServiceBus queue trigger function processed message: {Encoding.UTF8.GetString(message.MessageId)}");
await _cosmosDbClient.Save(...);
}
Here's a post for more detials.
Warning: using sessions will require messages to be sent with a session ID, potentially requiring a change on the sending side.
I have a queue trigger azure function with DurableOrchestrationClient. I am able to start a new execution of my orchestration function, which triggers multiple activitytrigger functions and waits for them all to process. Everything works great.
My issue is that I am unable to check on the status of my orchestration function("TestFunction"). GetStatusAsync always returns as null. I need to know when the orchestration function is actually complete and process the return object (bool).
public static async void Run([QueueTrigger("photostodownload", Connection = "QueueStorage")]PhotoInfo photoInfo, [OrchestrationClient]DurableOrchestrationClient starter, TraceWriter log)
{
var x = await starter.StartNewAsync("TestFunction", photoInfo);
Thread.Sleep(2 * 1000);
var y = await starter.GetStatusAsync(x);
}
StartNewAsync enqueues a message into the control queuee, it doesn't mean that the orchestration starts immediately.
GetStatusAsync returns null if the instance either doesn't exist or has not yet started running. So, probably the orchestration just doesn't start yet during those 2 seconds of sleep that you have.
Rather than having a fixed wait timeout, you should either periodically poll the status of your orchestration, or send something like a Done event from the orchestration as the last step of the workflow.
Are you using function 1.0 or 2.0? A similar issue has been reported for Function 2.0 runtime on Github.
https://github.com/Azure/azure-functions-durable-extension/issues/126
Also when you say everything works great do you mean activityTrigger functions complete execution?
Are you running functions locally or is it deployed on Azure?