I am new to Durable function(Orchestration function) and seen sample application as per Microsoft documentation.So I have few doubts.
example:
public static async Task<HttpResponseMessage> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, methods: "post",
Route = "orchestrators/{functionName}")] HttpRequestMessage req,
[OrchestrationClient] DurableOrchestrationClient starter,
string functionName,
TraceWriter log)
{
// Function input comes from the request content.
dynamic eventData = await req.Content.ReadAsAsync<object>();
string instanceId = await starter.StartNewAsync(functionName, eventData);
log.Info($"Started orchestration with ID = '{instanceId}'.");
return starter.CreateCheckStatusResponse(req, instanceId);
}
to invoke it I made HTTP POST request using postman so request processed successfully but when I configured different verb like HTTP GET it was responded with NotFound" error in console as well as request made to it with http request from browser responded with "NotFound" error in console .Why this happened?
Can I invoke any Orchestration function with in timer trigger azure function?
If not why?
UPDATE:
Some additional details about question
[FunctionName("TimerTrigger")]
public static async Task Run([TimerTrigger("0 */5 * * * *")]TimerInfo myTimer, TraceWriter log)
{//this runs for every 5minutes
using (HttpClient client = new HttpClient())
{
var content = new FormUrlEncodedContent(new[]
{
new KeyValuePair<string, string>("", "")
});
//making request to above function by http trigger
var result = await client.PostAsync("http://localhost:7071/orchestrators/E1_HelloSequence", content);
}
log.Info($"C# Timer trigger function executed at: {DateTime.Now}");
return;
}
can I make request to http trigger by timer triggred why because my durable function has long running process so if invoke orchestration function in timer trigger itself so there might be possibility of timer triggered timeout so that why I am trying to follow this approach.Is it possible to invoke by above code?
This is general Azure Functions behavior. The reason GET doesn't work is because the function in the sample is only configured to work with POST. See the [HttpTrigger] attribute in the function signature:
[HttpTrigger(AuthorizationLevel.Anonymous, methods: "post",
Route = "orchestrators/{functionName}")]
If you want to support GET, then change the methods parameter accordingly:
[HttpTrigger(AuthorizationLevel.Anonymous, methods: "get",
Route = "orchestrators/{functionName}")]
Note that Visual Studio seems to have a caching bug where making changes to route information is not properly saved when debugging locally. I opened a GitHub issue to track that here: https://github.com/Azure/Azure-Functions/issues/552
For more information on HTTP triggers, see this documentation: https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-http-webhook
If you want to trigger a Durable Function using a timer trigger, then just change the trigger type. For example:
[FunctionName("ScheduledStart")]
public static async Task RunScheduled(
[TimerTrigger("0 0 * * * *")] TimerInfo timerInfo,
[OrchestrationClient] DurableOrchestrationClient starter,
TraceWriter log)
{
string functionName = "E1_HelloSequence";
string instanceId = await starter.StartNewAsync(functionName, null);
log.Info($"Started orchestration with ID = '{instanceId}'.");
}
EDIT: If you're using Durable v2.x, then the syntax looks like this:
[FunctionName("ScheduledStart")]
public static async Task RunScheduled(
[TimerTrigger("0 0 * * * *")] TimerInfo timerInfo,
[DurableClient] IDurableClient starter,
ILogger log)
{
string functionName = "E1_HelloSequence";
string instanceId = await starter.StartNewAsync(functionName, null);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
}
For more information on Timer triggers, see this documentation: https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-timer
when I configured different verb like HTTP GET it was responded with NotFound" error in console as well as request made to it with http request from browser responded with "NotFound" error in console .Why this happened?
Because you specified your function to be triggered on POST only:
[HttpTrigger(AuthorizationLevel.Anonymous, methods: "post",
Route = "orchestrators/{functionName}")] HttpRequestMessage req,
To enable GET, add get to methods parameter.
Can I invoke any Orchestration function with in timer trigger azure function?
You can define timer-triggered function with OrchestrationClient input binding similar to your HTTP function. Sample declaration:
public static async Task Run(
[TimerTrigger("0 */1 * * * *")] TimerInfo info,
[OrchestrationClient] DurableOrchestrationClient starter)
The Microsoft Docs provide an example for an "eternal work" orchestrator to orchestrate work that needs to be done periodically but forever: https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-eternal-orchestrations?tabs=csharp#periodic-work-example
[FunctionName("EternalWorkOrchestrator")]
public static async Task Run([OrchestrationTrigger] DurableOrchestrationContext context)
{
await context.CallActivityAsync("DoWork", null);
// sleep for one hour between doing work
DateTime nextJobStart = context.CurrentUtcDateTime.AddHours(1);
await context.CreateTimer(nextJobStart, CancellationToken.None);
context.ContinueAsNew(null);
}
More info:
"IDurableClient" is a union of "IDurableEntityClient" and "IDurableOrchastrationClient". It has all the methods of the other two.
Entity Methods:
CleanEntityStorageAsync(Boolean, Boolean, CancellationToken)
ListEntitiesAsync(EntityQuery, CancellationToken)
ReadEntityStateAsync(EntityId, String, String)
SignalEntityAsync(EntityId, DateTime, String, Object, String, String)
SignalEntityAsync(EntityId, String, Object, String, String)
SignalEntityAsync(EntityId, Action)
SignalEntityAsync(EntityId, DateTime, Action)
SignalEntityAsync(String, Action)
SignalEntityAsync(String, DateTime, Action)
Orchestration Methods:
CreateCheckStatusResponse(HttpRequest, String, Boolean)
CreateCheckStatusResponse(HttpRequestMessage, String, Boolean) CreateHttpManagementPayload(String)
GetStatusAsync(String, Boolean, Boolean, Boolean) ListInstancesAsync(OrchestrationStatusQueryCondition, CancellationToken)
PurgeInstanceHistoryAsync(DateTime, Nullable, IEnumerable)
PurgeInstanceHistoryAsync(String)
RaiseEventAsync(String, String, Object)
RaiseEventAsync(String, String, String, Object, String)
RestartAsync(String, Boolean)
StartNewAsync(String, String)
StartNewAsync(String, String, T)
StartNewAsync(String, T)
TerminateAsync(String, String)
WaitForCompletionOrCreateCheckStatusResponseAsync(HttpRequest, String, Nullable, Nullable, Boolean)
WaitForCompletionOrCreateCheckStatusResponseAsync(HttpRequestMessage, String, Nullable, Nullable, Boolean)
https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.webjobs.extensions.durabletask?view=azure-dotnet
Related
I am trying to store the time stamp information in durable entities and retrieve it every time a trigger fired. Here is how I am doing it. I want the timestamp value set by the current execution to be available for the next trigger. But when the control reaches "string prevTS = await context.CallEntityAsync(entityId, "Get");" to goes back to start of the function again. What am I missing here.
I want execution to be sequential between the timer triggers.
'''
***public static class GetOpenDataRealtimeFeed
{
[FunctionName("GetOpenDataOrchestrator")]
public static async Task<List<string>> RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context, Binder binder, ILogger log)
{
var outputs = new List<string>();
var entityId = new EntityId(nameof(GetPrevLastModifiedTimestamp), "entityKey2");
string prevTS = await context.CallEntityAsync<string>(entityId, "Get");
string currentTS = DateTime.Now.ToString();
outputs.Add(currentTS);
outputs.Add(prevTS);
context.SignalEntity(entityId, "Set", currentTS);
return null;
}
//Durable entity function to get & set the last modified timestamp
[FunctionName("GetPrevLastModifiedTimestamp")]
public static void GetPrevLastModifiedTimestamp([EntityTrigger] IDurableEntityContext ctx)
{
switch (ctx.OperationName.ToLowerInvariant())
{
case "set":
ctx.SetState(ctx.GetInput<string>());
break;
case "get":
ctx.Return(ctx.GetState<string>());
break;
}
}
[FunctionName("getOpenDataRealtimeFeed_Trigger")]
public static async Task Run(
[TimerTrigger("%triggerTimer%")] TimerInfo myTimer,
[DurableClient] IDurableOrchestrationClient starter,
ILogger log)
{
// Function input comes from the request content.
string instanceId = await starter.StartNewAsync("GetOpenDataOrchestrator", null);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
}
}
}***
'''
I assume you are referring to the current line while debugging. If so, this is expected.
Since Durable Functions replays functions after awaiting a durable client call, execution won't ever go through the first round. Only the final replay will be "sequential" step overs.
I want to call another (not timer triggered) azure function from my timer triggered azure function.
It compiles but during runtime I get the error:
System.ArgumentException: 'The function 'HelloWorld' doesn't exist, is disabled, or is not an orchestrator function. Additional info: No orchestrator functions are currently registered!'
I reduced it to this tiny code snippet.
[FunctionName("HelloWorld")]
public static string HelloWorld([ActivityTrigger] string name, ILogger log)
{
return $"Hello {name}!";
}
[FunctionName("DownloadLiveList")]
public async void DownloadLiveList([DurableClient] IDurableOrchestrationClient client, [TimerTrigger("0 0 0 * * *", RunOnStartup = true)]TimerInfo myTimer, ILogger log)
{
await client.StartNewAsync<string>("HelloWorld", "Magdeburg");
}
As I took the idea from the official Microsoft example for that kind of azure function cascading, I've no clue, why the function "HelloWorld" is not registered. After uploading into azure, the function is visible in the azure portal as all other functions from the class.
Your time trigger function needs to invoke the start function written with Durable Function Framework. Here's a sample:
[FunctionName("Function1")]
public async Task Run([TimerTrigger("0 */1 * * * *")]TimerInfo myTimer, ILogger log)
{
log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
var url = "http://localhost:7071/api/Durable_Starter";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.AutomaticDecompression = DecompressionMethods.GZip;
using (HttpWebResponse response = (HttpWebResponse) await request.GetResponseAsync())
using (Stream stream = response.GetResponseStream())
using (StreamReader reader = new StreamReader(stream))
{
var html = reader.ReadToEnd();
log.LogInformation(html);
}
}
[FunctionName("Durable_Starter")]
public async Task<IActionResult> Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")]HttpRequest req, [DurableClient] IDurableClient starter, ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
string instanceId = await starter.StartNewAsync("Durable_Orchestrator");
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
var checkStatusResponse = starter.CreateCheckStatusResponse(req, instanceId);
return checkStatusResponse;
}
[FunctionName("Durable_Orchestrator")]
public async Task RunOrchestrator([OrchestrationTrigger] IDurableOrchestrationContext context, ILogger log)
{
var message = await context.CallActivityAsync<string>("HelloWorld", "Thiago");
log.LogInformation(message);
}
[FunctionName("HelloWorld")]
public string HelloWorldActivity([ActivityTrigger] string name)
{
return $"Hello {name}!";
}
there are 2 ways to read all the messages from service bus topic.
which one is recommended for azure functions
option 1 -
var messageReceiver = new MessageReceiver(SBConnString, QueueName, ReceiveMode.PeekLock);
Message message = await messageReceiver.ReceiveAsync();
option 2 -
static async Task ReceiveOrProcessMessagesAsync(Message message, CancellationToken token)
{
// Process the message
Console.WriteLine($"Received message: SequenceNumber:{message.SystemProperties.SequenceNumber} Body:{Encoding.UTF8.GetString(message.Body)}");
// Complete the message so that it is not received again.
// This can be done only if the queueClient is created in ReceiveMode.PeekLock mode (which is default).
await queueClient.CompleteAsync(message.SystemProperties.LockToken);
// Note: Use the cancellationToken passed as necessary to determine if the queueClient has already been closed.
// If queueClient has already been Closed, you may chose to not call CompleteAsync() or AbandonAsync() etc. calls
// to avoid unnecessary exceptions.
}
what should be used for azure functions http trigger ?
Check the azure binding table, you could find azure function doesn't support service bus input binding for now.
So if you want to receive messages in the function, you could only use the service bus trigger or the service bus sdk. The first sample you provide is the sdk and the second is the trigger.
And you said you want to receive messages in the http trigger function, so you have to use the Service Bus SDK to implement. You could refer to the below code.
public static class Function1
{
[FunctionName("Function1")]
public static async void Run(
[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req,
ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
string serviceBusConnectionString = Environment.GetEnvironmentVariable("servicebuscon");
var messageReceiver = new MessageReceiver(serviceBusConnectionString, "myqueue", ReceiveMode.PeekLock, null, 500);
var tempMessages = await messageReceiver.ReceiveAsync(500, TimeSpan.FromSeconds(1));
foreach (Message m1 in tempMessages)
{
log.LogInformation($"C# HTTP trigger function processed message: {Encoding.UTF8.GetString(m1.Body)}");
}
}
}
With reference to George Chen's code, you need to make a few changes in order to read all the messages from the Topic.
public static class Function1
{
[FunctionName("Function1")]
public static async void Run(
[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req,
ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
string serviceBusConnectionString = Environment.GetEnvironmentVariable("servicebuscon");
var messageReceiver = new MessageReceiver(serviceBusConnectionString, "myqueue", ReceiveMode.PeekLock, null, 500);
do
{
var tempMessages = await messageReceiver.ReceiveAsync(500, TimeSpan.FromSeconds(1));
foreach (Message m1 in tempMessages)
{
log.LogInformation($"C# HTTP trigger function processed message: {Encoding.UTF8.GetString(m1.Body)}");
}
}while(tempMessages!=null);
}
}
The above code has been modified with a do-while loop that receives all the messages from the Topic rather than the first messages in the queue.
I am looking at this example to run a durable function Activity after a set timeout.
https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-eternal-orchestrations
This will allow my function activity to perform processing of data, then wait exactly 1 hour before it attempts to load again. This will continue to run forever. Perfect.
However, when publishing the Function to Azure, I don't want to have to manually invoke/start the function via the associated HTTP Trigger. I just want the durable function to kickoff automatically and start processing.
Is this possible? If not, what is a suggested work around?
Thanks!
As discussed in the comments, one way of doing this would be to add a new Task in your Release pipeline.
Here is what I understood of your setup from your question:
[FunctionName("ClientFunction")]
public static async Task<HttpResponseMessage> OnHttpTriggerAsync([HttpTrigger(AuthorizationLevel.Anonymous, "post")]
HttpRequestMessage request, [OrchestrationClient] DurableOrchestrationClient starter, ILogger logger)
{
// Triggers the orchestrator.
string instanceId = await starter.StartNewAsync("OrchestratorFunction", null);
return new HttpResponseMessage(HttpStatusCode.OK);
}
[FunctionName("OrchestratorFunction")]
public static async Task DoOrchestrationThingsAsync([OrchestrationTrigger] DurableOrchestrationContext context, ILogger logger)
{
DateTime deadline = context.CurrentUtcDateTime.Add(TimeSpan.FromHours(1));
await context.CreateTimer(deadline, CancellationToken.None);
// Triggers some yout activity.
await context.CallActivityAsync("ActivityFunction", null);
}
[FunctionName("ActivityFunction")]
public static Task DoAnAwesomeActivity([ActivityTrigger] DurableActivityContext context)
{
}
Now, every time you deploy a new version of the Function App, you need the orchestrator to be run. However, I do not think it can be started by itself.
What I propose is to have a simple bash script (using curl or something else) that would call the ClientFunction at the appropriate URL.
On top of that, one of the nice things of this solution is that you could make the deployment fail if the Azure Function does not respond.
This seems to be working too.
[FunctionName("AutoStart")]
public static async Task Run([TimerTrigger("*/5 * * * * *", RunOnStartup = true, UseMonitor = false)]TimerInfo myStartTimer,
[DurableClient] IDurableClient orchestrationClient, ILogger log)
{
string instanceId = await orchestrationClient.StartNewAsync("Start_Orchestrator", null);
}
I don't know if there are hidden problems with this, but I'm experimenting now with having a TimerTrigger that runs on startup and also once a day at midnight (or whatever schedule you want). That TimerTrigger will search the list of instances for any running instances of this orchestration, terminate them, then start a new one.
private const string MyOrchestrationName = "MyOrchestration";
[FunctionName("MyOrchestration_Trigger")]
public async Task MyOrchestrationr_Trigger(
[TimerTrigger("0 0 0 * * *", RunOnStartup = true)] TimerInfo timer,
[DurableClient] IDurableOrchestrationClient starter,
ILogger log,
CancellationToken cancellationToken)
{
// Get all the instances currently running that have a status of Pending, Running, ContinuedAsNew
var instances = await starter.ListInstancesAsync(new OrchestrationStatusQueryCondition()
{
ShowInput = false,
RuntimeStatus = new List<OrchestrationRuntimeStatus>() { OrchestrationRuntimeStatus.Suspended, OrchestrationRuntimeStatus.Pending, OrchestrationRuntimeStatus.Running, OrchestrationRuntimeStatus.ContinuedAsNew }
}, cancellationToken);
// Find any instances of the current orchestration that are running.
var myInstances = instances.DurableOrchestrationState.Where(inst => inst.Name == MyOrchestrationName);
List<Task> terminateTasks = new List<Task>();
foreach (var instance in myInstances )
{
// Delete any instances that are currently running.
terminateTasks.Add(starter.TerminateAsync(instance.InstanceId, $"Restarting eternal orchestration"));
}
await Task.WhenAll(terminateTasks);
// Start the new task now that other instances have been terminated.
string instanceId = await starter.StartNewAsync(MyOrchestrationName, null);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
}
I think at least for my purposes this will be safe. Any activities that are running when you terminate will still run to completion (which is what I want in my case), so you would just kill it and restart it on a schedule.
I have an azure function with EventHubTrigger:
[FunctionName("TradesDataProcessStarterEh")]
public static async Task TradesDataProcessStarterEh([EventHubTrigger("aeehrobotronapiintegrationdev", Connection = "EventHubConnectionString", ConsumerGroup = "$Default")]
EventData eventData, PartitionContext partitionContext, [OrchestrationClient] DurableOrchestrationClient starter, ILogger log)
{
if (partitionContext.PartitionId != "1")
return;
var orchestrationId = await starter.StartNewAsync("O_ProcessTradesFromEventHub", eventData);
await partitionContext.CheckpointAsync();
}
The orchestrator function is receiving then the eventData:
[FunctionName("O_ProcessTradesFromEventHub")]
public static async Task ProcessTradesFromEventHub([OrchestrationTrigger] DurableOrchestrationContext context,
ILogger log)
{
if (!context.IsReplaying)
Console.WriteLine("O_ProcessTradesFromEventHub is triggered");
var eventData = context.GetInput<EventData>();
//do stuff...
}
But by execution of context.GetInput() I get an exception:
Function 'O_ProcessTradesFromEventHub (Orchestrator)' failed with an error. Reason: Newtonsoft.Json.JsonSerializationException: Unable to find a constructor to use for type Microsoft.Azure.EventHubs.EventData. A class should either have a default constructor, one constructor with arguments or a constructor marked with the JsonConstructor attribute. Path 'Body', line 1, position 81.
I can think of 3 possible solutions that you can try:
1 - Wrap EventData in your own class with a constructor (possibly via inheritance?).
2 - Try casting to object, doubt this will work but, but worth a try as it's a simple fix.
3 - Build your own DTO (Data Transfer Object) to transform EventData to <your class> and then pass <your class> to the orchestration.
I think (3) is the cleanest solution and you have full control over what you pass, unfortunately it is likely the least performant and most tedious.
Good luck!
Use LINQ to JSON - a year later but hopefully it'll save somebody else some time.
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
public static async Task Run(
[OrchestrationTrigger] DurableOrchestrationContext context, ILogger log) {
var eventData = context.GetInput<JObject>();
log.LogInformation ($"Executing tasks with eventData = {eventData}");
string step = (string)eventData.SelectToken("Step");
log.LogInformation ($"Step = {step}");
}