I am trying to locally step through (using Visual Studio 2019) a Durable fuction, but when I invoke a call to context.CallActivityWithRetryAsync("MyActivity_Activity", retryOptions, myActivityParameters) from the OrchestrationTrigger, that invocation always returns the below:
Id = [xxxx], Status = WaitingForActivation, Method = "{null}", Result = "{Not yet computed}"
My first guess is there might be a deadlock somewhere, but I have rebooted my machine and still seeing the same result. Also, I tried setting a breakpoint at the first line of the activity (MyActivity_Activity), but that wasn't hit. Any help would be greatly appreciated!
Using the following:
VS 2019 Pro
.Net Core 2.2
Microsoft.Azure.DurableTask.AzureStorage v1.6.2
Microsoft.Azure.WebJobs.Extensions.DurableTask v1.8.2
Microsoft.NET.Sdk.Functions v1.0.29
...et al.
Below are some relevant snippets of code:
[FunctionName("MyOrchestrator_HttpStart")]
public static async Task<HttpResponseMessage> HttpStart(
[HttpTrigger(AuthorizationLevel.Function, "post")]HttpRequestMessage req,
[OrchestrationClient]DurableOrchestrationClient starter, ExecutionContext executionContext)
{
string instanceId = string.Empty;
MyOrchestratorParameters myOrchestratorParameters = await req.Content.ReadAsAsync<MyOrchestratorParameters>();
instanceId = await starter.StartNewAsync("MyOrchestrator", myOrchestratorParameters);
return starter.CreateCheckStatusResponse(req, instanceId);
}
[FunctionName("MyOrchestrator")]
public static async Task RunOrchestrator(
[OrchestrationTrigger] DurableOrchestrationContext context,
ExecutionContext executionContext)
{
MyOrchestratorParameters myOrchestratorParameters =
context.GetInput<MyOrchestratorParameters>();
ValidateParameters(myOrchestratorParameters);
var delay = 3000;
var retry = 3;
var retryOptions = new
RetryOptions(TimeSpan.FromSeconds(delay), retry);
MyActivityParameters myActivityParameters = new
MyActivityParameters()
{
JobNumber = myOrchestratorParameters.JobNumber,
Stage = myOrchestratorParameters.Stage,
TemplateId = myOrchestratorParameters.TemplateId
};
myActivityResults =
context.CallActivityWithRetryAsync<MyActivityResult>
("MyActivity_Activity", retryOptions, myActivityParameters);
...
}
[FunctionName("MyActivity_Activity")]
public static async Task<MyActivityResult>
RunActivity([ActivityTrigger] DurableActivityContext
activityContext, ExecutionContext executionContext)
{
var _myActivityParameters =
activityContext.GetInput<MyActivityParameters>();
//do some stuff & return MyActivityResult object
}
I think I found the answer. In my calls to CallActivityWithRetryAsync, I needed to predicate that with an await. Once I did this, it started working!
Related
I am trying to build a simple orchestrator func + additional call with sub-orchestrator.
The problem is in main orchestrator and call to await ctx.CallSubOrchestratorAsync("sub-orchestration", someInput) - the task is never resolved/completed... If I change the code in sub orchestrator from await Send_SomeData(); to Send_SomeData().Wait(); then it works, but that is not what I want. As far as I know async/await is supported in orchestration funcs
Thank you!
The code is below:
Main Orchestrator
[FunctionName("main-orchestration")]
public static async Task MainOrchestrationAsync(
[OrchestrationTrigger] IDurableOrchestrationContext ctx,
ILogger log)
{
var someInput = "my msg";
await ctx.CallSubOrchestratorAsync("sub-orchestration", someInput);
log.LogWarning("finish");
}
Sub Orchestrator
[FunctionName("sub-orchestration")]
public static async Task SubOrchestrationAsync(
[OrchestrationTrigger] IDurableOrchestrationContext ctx,
ILogger log)
{
log.LogWarning("Sub Start");
await Send_SomeData(some_url, some_message);
log.LogWarning("Sub End");
}
Send_SomeData
public static async Task<string> ConnectAsync(string url, string message)
{
using var request = new HttpRequestMessage
{
RequestUri = new Uri(url),
Method = HttpMethod.Post,
Content = new FormUrlEncodedContent(new[]
{
...message...
})
};
var response = await _httpClient.SendAsync(request);
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
UPDATE
I've decided to create an example from scratch, to not miss something and get something that can be copy/paste for debugging.
Below is another example. Main End log message never appeared, so Task from .CallSubOrchestratorAsync never resolved.
What am I missing?
Thanks again!
using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Threading.Tasks;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.DurableTask;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Extensions.Logging;
namespace _Await_Issue
{
public static class Function1
{
[FunctionName("Main")]
public static async Task RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context,
ILogger log)
{
log.LogWarning("Main Start");
try
{
await context.CallSubOrchestratorAsync("Sub", null);
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
log.LogWarning("Main End");
}
[FunctionName("Sub")]
public static async Task RunSubOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context,
ILogger log)
{
log.LogWarning("Sub Start");
var data = await GetDataAsync("https://www.google.com");
log.LogWarning("Sub End");
}
[FunctionName("Start")]
public static async Task<HttpResponseMessage> HttpStart(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")]
HttpRequestMessage req,
[DurableClient] IDurableOrchestrationClient starter,
ILogger log)
{
// Function input comes from the request content.
string instanceId = await starter.StartNewAsync("Main", null);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
return starter.CreateCheckStatusResponse(req, instanceId);
}
public static async Task<string> GetDataAsync(string url)
{
var httpClient = new HttpClient();
using var request = new HttpRequestMessage
{
RequestUri = new Uri(url),
Method = HttpMethod.Get,
};
var response = await httpClient.SendAsync(request);
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
}
I found the light here ))
Answering on my own question as it might help others.
As Peter Bons pointed Send_SomeData method was under suspicion ))
The problem in general is in "Deterministic Apis" as Azure team call in the docs
Deterministic Apis Explanation
In my case Send_SomeData inside uses HttpClient to communication with external endpoint.
It's not recommended as it might result in an unexpected behavior as in my case with await...
If there are such portions of code (e.g. Send_SomeData in my case) put it in Activity and call this with context.CallActivityAsync() - (be aware that Activity funcs has limited time to execute)
There are couple of durable functions that call each other.
Main orchestration -> Sub orchestration -> Activity -> Helper async method
Each func has ILogger dependency and log on function start and on function end.
Both orchestrators duplicates "on start" message for some reason. (See pic)
Activity does not have this effect. (See pic)
Ran example below many times - same story.
I am also sure that the whole process has been triggered once.
Is this a bug in orchestrators or expected behavior?
using System;
using System.Net.Http;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.DurableTask;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Extensions.Logging;
namespace Issues
{
public static class Log_Issue
{
[FunctionName("Main")]
public static async Task RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context,
ILogger log)
{
try
{
log.LogWarning("Main Start");
await context.CallSubOrchestratorAsync("Sub", null);
log.LogWarning("Main End");
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
}
[FunctionName("Sub")]
public static async Task RunSubOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context,
ILogger log)
{
log.LogWarning("Sub Start");
var data = await context.CallActivityAsync<string>("Activity", null);
log.LogWarning("Sub End");
}
[FunctionName("Activity")]
public static async Task<string> GetDataActivity([ActivityTrigger] string name, ILogger log)
{
log.LogWarning("Activity Start");
var data = await GetDataAsync("https://www.google.com");
log.LogWarning("Activity End");
return data;
}
[FunctionName("Start")]
public static async Task<IActionResult> HttpStart(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")]
HttpRequestMessage req,
[DurableClient] IDurableOrchestrationClient starter,
ILogger log)
{
var instanceId = await starter.StartNewAsync("Main", null);
log.LogWarning($"Started orchestration with ID = '{instanceId}'.");
return new OkResult();
}
private static async Task<string> GetDataAsync(string url)
{
var httpClient = new HttpClient();
using var request = new HttpRequestMessage
{
RequestUri = new Uri(url),
Method = HttpMethod.Get,
};
var response = await httpClient.SendAsync(request);
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
}
This is expected. For example on await context.CallActivityAsync("Activity", null); The code pauses itself and may even be loaded out of memory (in order to save on cost).
Then the orchestrator waits for an event to be placed in another Azure Storage Table which the activity creates, this may occur many days later. For activities they are usually very instant but it still waits for this event to occur.
When that happens the code needs to start from where it last stopped but there is no way to do that. Therefor the code reruns from the beginning but instead of waiting for the activity to finish again it first looks in the table and sees that we already have done this activity and can continue running. If the activity function returned some value it would be returned from the await call. During both runs of the orchestrator it would log but since we only go inte the activity ones that would only be logged ones.
This is why orchestrators have to be deterministic since e.g. a random value on the first run would not be the same as during the second run. Instead we would put the random.Next() into an activity funtion so that the value is saved to Azure Table Storage to be used on subsequent reruns. The orchestrator could also be waiting for some external events which normal functions create. E.g someone has to verify their email account which could take some number of days and this is why durable functions can unload themself and restart when they are triggered by the event.
All that #FilipB said is true. It is just missing the actual code to solve it ;)
[FunctionName("Main")]
public static async Task RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context,
ILogger log)
{
log = context.CreateReplaySafeLogger(log); // this is what you should use at the start of every Orchestrator
I have an Azure durable function triggered by a message that then uses the client to start an Orchestration trigger which starts several activity functions. I have set breakpoints in Orchestration client , trigger and each activity function.
But, it only hits the breakpoints in Orchestration client function and the others are getting ignored. But underneath it seems to execute the activity functions although the breakpoints are not hit.
Investigative information
Programming language used = C#
Visual Studio Enterprise 2019 version = 16.8.3
Azure Functions Core Tools
Core Tools Version: 3.0.3216
Function Runtime Version: 3.0.15193.0
Below here, I have included a code snippet. (have not added every activity function)
[FunctionName(nameof(InitiateExport))]
public static async Task InitiateExport(
[ServiceBusTrigger("%ExportQueueName%", Connection = "AzureSBConnection")]Message message,
[DurableClient(TaskHub = "%FunctionHubName%")] IDurableOrchestrationClient orchestrationClient,
[Inject]IServiceProvider rootServiceProvider, ILogger log)
{
var DataQueuedDetails = JsonConvert.DeserializeObject<DataQueuedDetails>(Encoding.UTF8.GetString(message.Body));
using (var scope = rootServiceProvider.CreateScope())
{
log.LogInformation($"{nameof(ExportData)} function execution started at: {DateTime.Now}");
var services = scope.ServiceProvider;
services.ResolveRequestContext(message);
var requestContext = services.GetRequiredService<RequestContext>();
await orchestrationClient.StartNewAsync(nameof(TriggerDataExport), null, (DataQueuedDetails, requestContext));
log.LogInformation($"{nameof(ExportData)} timer triggered function execution finished at: {DateTime.Now}");
}
}
[FunctionName(nameof(TriggerDataExport))]
public static async Task TriggerDataExport(
[OrchestrationTrigger] IDurableOrchestrationContext orchestrationContext,
[Inject] IServiceProvider rootServiceProvider, ILogger log)
{
using (var scope = rootServiceProvider.CreateScope())
{
var services = scope.ServiceProvider;
var (DataOperationInfo, requestContext) = orchestrationContext.GetInput<(DataQueuedDetails, RequestContext)>();
if (!orchestrationContext.IsReplaying)
log.LogInformation($"Starting Export data Id {DataOperationInfo.Id}");
var blobServiceFactory = services.GetRequiredService<IBlobServiceFactory>();
requestContext.CustomerId = DataOperationInfo.RelatedCustomerId;
try
{
await orchestrationContext.CallActivityAsync(
nameof(UpdateJobStatus),
(DataOperationInfo.Id, DataOperationStatus.Running, string.Empty, string.Empty, requestContext));
// some other activity functions
---
---
} catch (Exception e)
{
await orchestrationContext.CallActivityAsync(
nameof(UpdateJobStatus),
(DataOperationInfo.Id, DataOperationStatus.Failed, string.Empty, string.Empty, requestContext));
}
}
}
[FunctionName(nameof(UpdateJobStatus))]
public static async Task RunAsync(
[ActivityTrigger] IDurableActivityContext activityContext,
[Inject]IServiceProvider rootServiceProvider)
{
using (var scope = rootServiceProvider.CreateScope())
{
try
{
var (DataOperationId, status, blobReference, logFileBlobId, requestContext) = activityContext.GetInput<(string, string, string, string, RequestContext)>();
var services = scope.ServiceProvider;
services.ResolveRequestContext(requestContext.CustomerId, requestContext.UserId, requestContext.UserDisplayName, requestContext.Culture);
var dataService = services.GetRequiredService<IDataService>();
var DataOperationDto = new DataOperationDto
{
Id = DataOperationId,
OperationStatusCode = status,
BlobReference = blobReference,
LogBlobReference = logFileBlobId
};
await dataService.UpdateAsync(DataOperationDto);
}
catch (Exception e)
{
throw e;
}
}
}
While you are debugging your Function, you should make sure that you are either using the local storage emulator, or an Azure Storage Account that is different from the one that is also being used by the Function already deployed in Azure.
If you are using the same storage account that another running Function is using, then it could be that parts of the execution is actually happening in Azure instead of your dev machine.
I have an Azure Function with a CosmosDB output binding, like this:
public static class ComponentDesignHttpTrigger
{
[FunctionName("ComponentDesignInserter-Http-From-ComponentDesign")]
public static IActionResult Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "fromComponentDesign")] HttpRequest request,
[CosmosDB(
databaseName: StorageFramework.CosmosDb.DatabaseId,
collectionName: Storage.ComponentDesignCollectionId,
ConnectionStringSetting = "CosmosDBConnection")] out ComponentDesign componentDesignToInsert,
ILogger log)
{
var requestBody = new StreamReader(request.Body).ReadToEnd();
componentDesignToInsert = JsonConvert.DeserializeObject<ComponentDesign>(requestBody);
return new OkObjectResult(componentDesignToInsert);
}
}
In this function componentDesignToInsert is automatically serialized and put into CosmosDB after the function finishes executing. But the default serialization does not put things in camelCase. For this Json.NET lets you provide custom serializer settings, like this:
var settings = new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver()
};
var json = JsonConvert.SerializeObject(yourObject, settings);
but I'm unsure how I can integrate this with my output binding. How can I accomplish this?
Output binding does not expose the serializer settings at this moment.
One thing you can do though, is leverage your own custom DocumentClient for the operation.
One important thing though is that the DocumentClient instance needs to be static (more details on https://github.com/Azure/azure-functions-host/wiki/Managing-Connections).
private static Lazy<DocumentClient> lazyClient = new Lazy<DocumentClient>(InitializeDocumentClient);
private static DocumentClient documentClient => lazyClient.Value;
private static DocumentClient InitializeDocumentClient()
{
// Perform any initialization here
var uri = new Uri("example");
var authKey = "authKey";
var settings = new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver()
};
return new DocumentClient(uri, authKey, settings);
}
[FunctionName("ComponentDesignInserter-Http-From-ComponentDesign")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "fromComponentDesign")] HttpRequest request,
ILogger log)
{
var requestBody = new StreamReader(request.Body).ReadToEnd();
var componentDesignToInsert = JsonConvert.DeserializeObject<ComponentDesign>(requestBody);
var collectionUri = UriFactory.GetDocumentCollectionUri(StorageFramework.CosmosDb.DatabaseId, Storage.ComponentDesignCollectionId);
await documentClient.UpsertDocumentAsync(collectionUri, componentDesignToInsert);
return new OkObjectResult(componentDesignToInsert);
}
Another option is to decorate the class with JsonProperty if that suits your scenario.
I've tried to come up with something from the example in the WebJobsSDK gitHub
var eventHubConfig = new EventHubConfiguration();
string eventHubName = "MyHubName";
eventHubConfig.AddSender(eventHubName,"Endpoint=sb://test.servicebus.windows.net/;SharedAccessKeyName=SendRule;SharedAccessKey=xxxxxxxx");
eventHubConfig.AddReceiver(eventHubName, "Endpoint=sb://test.servicebus.windows.net/;SharedAccessKeyName=ReceiveRule;SharedAccessKey=yyyyyyy");
config.UseEventHub(eventHubConfig);
JobHost host = new JobHost(config);
But I'm afraid that's not far enough for someone of my limited "skillset"!
I can find no instance of JobHostConfiguration that has a UseEventHub property (using the v1.2.0-alpha-10291 version of the Microsoft.AzureWebJobs package), so I can't pass the EventHubConfiguration to the JobHost.
I've used EventHub before, not within the WebJob context. I don't see if the EventHostProcessor is still required if using the WebJob triggering...or does the WebJob trigger essentially act as the EventHostProcessor?
Anyway, if anyone has a more complete example for a simpleton like me that would be really sweet! Thanks
From the documentation here, you should have all the information you need.
What you are missing is a reference of the Microsoft.Azure.WebJobs.ServiceBus.1.2.0-alpha-10291 nuget package.
The UseEventHub is an extension method that is declared in this package.
Otherwise your configuration seems ok.
Here is an example on how to receive or send messages from/to an EventHub:
public class BasicTest
{
public class Payload
{
public int Counter { get; set; }
}
public static void SendEvents([EventHub("MyHubName")] out Payload x)
{
x = new Payload { Counter = 100 };
}
public static void Trigger(
[EventHubTrigger("MyHubName")] Payload x,
[EventHub("MyHubName")] out Payload y)
{
x.Counter++;
y = x;
}
}
EventProcessorHost is still required, as the WebJob just provides the hosting environment for running it. As far as I know, EventProcessorHost is not integrated so deeply into WebJob, so its triggering mechanism cannot be used for processing EventHub messages. I use WebJob for running EventProcessorHost continuously:
public static void Main()
{
RunAsync().Wait();
}
private static async Task RunAsync()
{
try
{
using (var shutdownWatcher = new WebJobsShutdownWatcher())
{
await Console.Out.WriteLineAsync("Initializing...");
var eventProcessorHostName = "eventProcessorHostName";
var eventHubName = ConfigurationManager.AppSettings["eventHubName"];
var consumerGroupName = ConfigurationManager.AppSettings["eventHubConsumerGroupName"];
var eventHubConnectionString = ConfigurationManager.ConnectionStrings["EventHub"].ConnectionString;
var storageConnectionString = ConfigurationManager.ConnectionStrings["EventHubStorage"].ConnectionString;
var eventProcessorHost = new EventProcessorHost(eventProcessorHostName, eventHubName, consumerGroupName, eventHubConnectionString, storageConnectionString);
await Console.Out.WriteLineAsync("Registering event processors...");
var processorOptions = new EventProcessorOptions();
processorOptions.ExceptionReceived += ProcessorOptions_ExceptionReceived;
await eventProcessorHost.RegisterEventProcessorAsync<CustomEventProcessor>(processorOptions);
await Console.Out.WriteLineAsync("Processing...");
await Task.Delay(Timeout.Infinite, shutdownWatcher.Token);
await Console.Out.WriteLineAsync("Unregistering event processors...");
await eventProcessorHost.UnregisterEventProcessorAsync();
await Console.Out.WriteLineAsync("Finished.");
}
catch (Exception ex)
{
await HandleErrorAsync(ex);
}
}
}
private static async void ProcessorOptions_ExceptionReceived(object sender, ExceptionReceivedEventArgs e)
{
await HandleErrorAsync(e.Exception);
}
private static async Task HandleErrorAsync(Exception ex)
{
await Console.Error.WriteLineAsync($"Critical error occured: {ex.Message}{ex.StackTrace}");
}