I want to use Azure Durable Functions to orchestrate my functions. This is my code (auto-generated by VS Code when you create a Durable Function) :
[FunctionName("testorchestration")]
public static async Task<List<string>> RunOrchestrator(
[OrchestrationTrigger] DurableOrchestrationContext context)
{
var outputs = new List<string>();
// Replace "hello" with the name of your Durable Activity Function.
outputs.Add(await context.CallActivityAsync<string>("testorchestration_Hello", "Tokyo"));
outputs.Add(await context.CallActivityAsync<string>("testorchestration_Hello", "Seattle"));
outputs.Add(await context.CallActivityAsync<string>("testorchestration_Hello", "London"));
// returns ["Hello Tokyo!", "Hello Seattle!", "Hello London!"]
return outputs;
}
[FunctionName("testorchestration_Hello")]
public static string SayHello([ActivityTrigger] string name, ILogger log)
{
log.LogInformation($"Saying hello to {name}.");
return $"Hello {name}!";
}
[FunctionName("testorchestration_HttpStart")]
public static async Task<HttpResponseMessage> HttpStart(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")]HttpRequestMessage req,
[OrchestrationClient]DurableOrchestrationClient starter,
ILogger log)
{
// Function input comes from the request content.
string instanceId = await starter.StartNewAsync("testorchestration", null);
return await starter.WaitForCompletionOrCreateCheckStatusResponseAsync(req,instanceId);
}
I add await starter.WaitForCompletionOrCreateCheckStatusResponseAsync(req,instanceId); because I need to wait for completion , and this is my problem.
If I wait for completion, it takes about ~2 seconds and sometimes 20 secondes (test on localhost) and I want to understand why because each function takes ~50ms to execute.
Maybe because this method calls await starter.GetStatusAsync(instanceId) everytime until the task is completed ?
Related
I am trying to build a simple orchestrator func + additional call with sub-orchestrator.
The problem is in main orchestrator and call to await ctx.CallSubOrchestratorAsync("sub-orchestration", someInput) - the task is never resolved/completed... If I change the code in sub orchestrator from await Send_SomeData(); to Send_SomeData().Wait(); then it works, but that is not what I want. As far as I know async/await is supported in orchestration funcs
Thank you!
The code is below:
Main Orchestrator
[FunctionName("main-orchestration")]
public static async Task MainOrchestrationAsync(
[OrchestrationTrigger] IDurableOrchestrationContext ctx,
ILogger log)
{
var someInput = "my msg";
await ctx.CallSubOrchestratorAsync("sub-orchestration", someInput);
log.LogWarning("finish");
}
Sub Orchestrator
[FunctionName("sub-orchestration")]
public static async Task SubOrchestrationAsync(
[OrchestrationTrigger] IDurableOrchestrationContext ctx,
ILogger log)
{
log.LogWarning("Sub Start");
await Send_SomeData(some_url, some_message);
log.LogWarning("Sub End");
}
Send_SomeData
public static async Task<string> ConnectAsync(string url, string message)
{
using var request = new HttpRequestMessage
{
RequestUri = new Uri(url),
Method = HttpMethod.Post,
Content = new FormUrlEncodedContent(new[]
{
...message...
})
};
var response = await _httpClient.SendAsync(request);
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
UPDATE
I've decided to create an example from scratch, to not miss something and get something that can be copy/paste for debugging.
Below is another example. Main End log message never appeared, so Task from .CallSubOrchestratorAsync never resolved.
What am I missing?
Thanks again!
using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Threading.Tasks;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.DurableTask;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Extensions.Logging;
namespace _Await_Issue
{
public static class Function1
{
[FunctionName("Main")]
public static async Task RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context,
ILogger log)
{
log.LogWarning("Main Start");
try
{
await context.CallSubOrchestratorAsync("Sub", null);
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
log.LogWarning("Main End");
}
[FunctionName("Sub")]
public static async Task RunSubOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context,
ILogger log)
{
log.LogWarning("Sub Start");
var data = await GetDataAsync("https://www.google.com");
log.LogWarning("Sub End");
}
[FunctionName("Start")]
public static async Task<HttpResponseMessage> HttpStart(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")]
HttpRequestMessage req,
[DurableClient] IDurableOrchestrationClient starter,
ILogger log)
{
// Function input comes from the request content.
string instanceId = await starter.StartNewAsync("Main", null);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
return starter.CreateCheckStatusResponse(req, instanceId);
}
public static async Task<string> GetDataAsync(string url)
{
var httpClient = new HttpClient();
using var request = new HttpRequestMessage
{
RequestUri = new Uri(url),
Method = HttpMethod.Get,
};
var response = await httpClient.SendAsync(request);
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
}
I found the light here ))
Answering on my own question as it might help others.
As Peter Bons pointed Send_SomeData method was under suspicion ))
The problem in general is in "Deterministic Apis" as Azure team call in the docs
Deterministic Apis Explanation
In my case Send_SomeData inside uses HttpClient to communication with external endpoint.
It's not recommended as it might result in an unexpected behavior as in my case with await...
If there are such portions of code (e.g. Send_SomeData in my case) put it in Activity and call this with context.CallActivityAsync() - (be aware that Activity funcs has limited time to execute)
There are couple of durable functions that call each other.
Main orchestration -> Sub orchestration -> Activity -> Helper async method
Each func has ILogger dependency and log on function start and on function end.
Both orchestrators duplicates "on start" message for some reason. (See pic)
Activity does not have this effect. (See pic)
Ran example below many times - same story.
I am also sure that the whole process has been triggered once.
Is this a bug in orchestrators or expected behavior?
using System;
using System.Net.Http;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.DurableTask;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Extensions.Logging;
namespace Issues
{
public static class Log_Issue
{
[FunctionName("Main")]
public static async Task RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context,
ILogger log)
{
try
{
log.LogWarning("Main Start");
await context.CallSubOrchestratorAsync("Sub", null);
log.LogWarning("Main End");
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
}
[FunctionName("Sub")]
public static async Task RunSubOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context,
ILogger log)
{
log.LogWarning("Sub Start");
var data = await context.CallActivityAsync<string>("Activity", null);
log.LogWarning("Sub End");
}
[FunctionName("Activity")]
public static async Task<string> GetDataActivity([ActivityTrigger] string name, ILogger log)
{
log.LogWarning("Activity Start");
var data = await GetDataAsync("https://www.google.com");
log.LogWarning("Activity End");
return data;
}
[FunctionName("Start")]
public static async Task<IActionResult> HttpStart(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")]
HttpRequestMessage req,
[DurableClient] IDurableOrchestrationClient starter,
ILogger log)
{
var instanceId = await starter.StartNewAsync("Main", null);
log.LogWarning($"Started orchestration with ID = '{instanceId}'.");
return new OkResult();
}
private static async Task<string> GetDataAsync(string url)
{
var httpClient = new HttpClient();
using var request = new HttpRequestMessage
{
RequestUri = new Uri(url),
Method = HttpMethod.Get,
};
var response = await httpClient.SendAsync(request);
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
}
This is expected. For example on await context.CallActivityAsync("Activity", null); The code pauses itself and may even be loaded out of memory (in order to save on cost).
Then the orchestrator waits for an event to be placed in another Azure Storage Table which the activity creates, this may occur many days later. For activities they are usually very instant but it still waits for this event to occur.
When that happens the code needs to start from where it last stopped but there is no way to do that. Therefor the code reruns from the beginning but instead of waiting for the activity to finish again it first looks in the table and sees that we already have done this activity and can continue running. If the activity function returned some value it would be returned from the await call. During both runs of the orchestrator it would log but since we only go inte the activity ones that would only be logged ones.
This is why orchestrators have to be deterministic since e.g. a random value on the first run would not be the same as during the second run. Instead we would put the random.Next() into an activity funtion so that the value is saved to Azure Table Storage to be used on subsequent reruns. The orchestrator could also be waiting for some external events which normal functions create. E.g someone has to verify their email account which could take some number of days and this is why durable functions can unload themself and restart when they are triggered by the event.
All that #FilipB said is true. It is just missing the actual code to solve it ;)
[FunctionName("Main")]
public static async Task RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context,
ILogger log)
{
log = context.CreateReplaySafeLogger(log); // this is what you should use at the start of every Orchestrator
I am trying to have some durable functions defined in separate projects, as I want to start them as suborchestrations from a root orchestration in my 'root project'.
However, it seems the actual Functions are not found, as they are part of the external project. Is this is known limitation? (I could not find this documented)
The exception that is thrown is the following: The function 'TestFunction' doesn't exist, is disabled, or is not an orchestrator function. Additional info: No orchestrator functions are currently registered!
For your interest, some code (but nothing special):
Functions project
[FunctionName("TestFunction_HttpStart")]
public static async Task<HttpResponseMessage> HttpStart(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")]
HttpRequestMessage req,
[OrchestrationClient] IDurableOrchestrationClient starter,
ILogger log)
{
// Function input comes from the request content.
string instanceId = await starter.StartNewAsync("TestFunction", null);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
return starter.CreateCheckStatusResponse(req, instanceId);
}
Referenced project
public class FunctionDefinitions
{
[FunctionName("TestFunction")]
public static async Task<List<string>> RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context)
{
var outputs = new List<string>();
// Replace "hello" with the name of your Durable Activity Function.
outputs.Add(await context.CallActivityAsync<string>("TestFunction_Hello", "Tokyo"));
outputs.Add(await context.CallActivityAsync<string>("TestFunction_Hello", "Seattle"));
outputs.Add(await context.CallActivityAsync<string>("TestFunction_Hello", "London"));
// returns ["Hello Tokyo!", "Hello Seattle!", "Hello London!"]
return outputs;
}
[FunctionName("TestFunction_Hello")]
public static async Task<string> SayHello([ActivityTrigger] string name, ILogger log)
{
log.LogInformation($"Saying hello to {name}.");
return $"Hello {name}!";
}
}
I did a test, and found it can't refer external orchestrator functions. I got the same error message as you, and it prompted me Additional info: The following are the known orchestrator functions: 'Function1'., Function1 is defined in the same project.
I have referenced the external project in the csproj file, but it still can't detect external orchestrator functions.
So, I suggest you define all durable function in the same Function app project.
I am trying to store the time stamp information in durable entities and retrieve it every time a trigger fired. Here is how I am doing it. I want the timestamp value set by the current execution to be available for the next trigger. But when the control reaches "string prevTS = await context.CallEntityAsync(entityId, "Get");" to goes back to start of the function again. What am I missing here.
I want execution to be sequential between the timer triggers.
'''
***public static class GetOpenDataRealtimeFeed
{
[FunctionName("GetOpenDataOrchestrator")]
public static async Task<List<string>> RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context, Binder binder, ILogger log)
{
var outputs = new List<string>();
var entityId = new EntityId(nameof(GetPrevLastModifiedTimestamp), "entityKey2");
string prevTS = await context.CallEntityAsync<string>(entityId, "Get");
string currentTS = DateTime.Now.ToString();
outputs.Add(currentTS);
outputs.Add(prevTS);
context.SignalEntity(entityId, "Set", currentTS);
return null;
}
//Durable entity function to get & set the last modified timestamp
[FunctionName("GetPrevLastModifiedTimestamp")]
public static void GetPrevLastModifiedTimestamp([EntityTrigger] IDurableEntityContext ctx)
{
switch (ctx.OperationName.ToLowerInvariant())
{
case "set":
ctx.SetState(ctx.GetInput<string>());
break;
case "get":
ctx.Return(ctx.GetState<string>());
break;
}
}
[FunctionName("getOpenDataRealtimeFeed_Trigger")]
public static async Task Run(
[TimerTrigger("%triggerTimer%")] TimerInfo myTimer,
[DurableClient] IDurableOrchestrationClient starter,
ILogger log)
{
// Function input comes from the request content.
string instanceId = await starter.StartNewAsync("GetOpenDataOrchestrator", null);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
}
}
}***
'''
I assume you are referring to the current line while debugging. If so, this is expected.
Since Durable Functions replays functions after awaiting a durable client call, execution won't ever go through the first round. Only the final replay will be "sequential" step overs.
I want to call another (not timer triggered) azure function from my timer triggered azure function.
It compiles but during runtime I get the error:
System.ArgumentException: 'The function 'HelloWorld' doesn't exist, is disabled, or is not an orchestrator function. Additional info: No orchestrator functions are currently registered!'
I reduced it to this tiny code snippet.
[FunctionName("HelloWorld")]
public static string HelloWorld([ActivityTrigger] string name, ILogger log)
{
return $"Hello {name}!";
}
[FunctionName("DownloadLiveList")]
public async void DownloadLiveList([DurableClient] IDurableOrchestrationClient client, [TimerTrigger("0 0 0 * * *", RunOnStartup = true)]TimerInfo myTimer, ILogger log)
{
await client.StartNewAsync<string>("HelloWorld", "Magdeburg");
}
As I took the idea from the official Microsoft example for that kind of azure function cascading, I've no clue, why the function "HelloWorld" is not registered. After uploading into azure, the function is visible in the azure portal as all other functions from the class.
Your time trigger function needs to invoke the start function written with Durable Function Framework. Here's a sample:
[FunctionName("Function1")]
public async Task Run([TimerTrigger("0 */1 * * * *")]TimerInfo myTimer, ILogger log)
{
log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
var url = "http://localhost:7071/api/Durable_Starter";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.AutomaticDecompression = DecompressionMethods.GZip;
using (HttpWebResponse response = (HttpWebResponse) await request.GetResponseAsync())
using (Stream stream = response.GetResponseStream())
using (StreamReader reader = new StreamReader(stream))
{
var html = reader.ReadToEnd();
log.LogInformation(html);
}
}
[FunctionName("Durable_Starter")]
public async Task<IActionResult> Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")]HttpRequest req, [DurableClient] IDurableClient starter, ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
string instanceId = await starter.StartNewAsync("Durable_Orchestrator");
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
var checkStatusResponse = starter.CreateCheckStatusResponse(req, instanceId);
return checkStatusResponse;
}
[FunctionName("Durable_Orchestrator")]
public async Task RunOrchestrator([OrchestrationTrigger] IDurableOrchestrationContext context, ILogger log)
{
var message = await context.CallActivityAsync<string>("HelloWorld", "Thiago");
log.LogInformation(message);
}
[FunctionName("HelloWorld")]
public string HelloWorldActivity([ActivityTrigger] string name)
{
return $"Hello {name}!";
}