I have an Azure durable function triggered by a message that then uses the client to start an Orchestration trigger which starts several activity functions. I have set breakpoints in Orchestration client , trigger and each activity function.
But, it only hits the breakpoints in Orchestration client function and the others are getting ignored. But underneath it seems to execute the activity functions although the breakpoints are not hit.
Investigative information
Programming language used = C#
Visual Studio Enterprise 2019 version = 16.8.3
Azure Functions Core Tools
Core Tools Version: 3.0.3216
Function Runtime Version: 3.0.15193.0
Below here, I have included a code snippet. (have not added every activity function)
[FunctionName(nameof(InitiateExport))]
public static async Task InitiateExport(
[ServiceBusTrigger("%ExportQueueName%", Connection = "AzureSBConnection")]Message message,
[DurableClient(TaskHub = "%FunctionHubName%")] IDurableOrchestrationClient orchestrationClient,
[Inject]IServiceProvider rootServiceProvider, ILogger log)
{
var DataQueuedDetails = JsonConvert.DeserializeObject<DataQueuedDetails>(Encoding.UTF8.GetString(message.Body));
using (var scope = rootServiceProvider.CreateScope())
{
log.LogInformation($"{nameof(ExportData)} function execution started at: {DateTime.Now}");
var services = scope.ServiceProvider;
services.ResolveRequestContext(message);
var requestContext = services.GetRequiredService<RequestContext>();
await orchestrationClient.StartNewAsync(nameof(TriggerDataExport), null, (DataQueuedDetails, requestContext));
log.LogInformation($"{nameof(ExportData)} timer triggered function execution finished at: {DateTime.Now}");
}
}
[FunctionName(nameof(TriggerDataExport))]
public static async Task TriggerDataExport(
[OrchestrationTrigger] IDurableOrchestrationContext orchestrationContext,
[Inject] IServiceProvider rootServiceProvider, ILogger log)
{
using (var scope = rootServiceProvider.CreateScope())
{
var services = scope.ServiceProvider;
var (DataOperationInfo, requestContext) = orchestrationContext.GetInput<(DataQueuedDetails, RequestContext)>();
if (!orchestrationContext.IsReplaying)
log.LogInformation($"Starting Export data Id {DataOperationInfo.Id}");
var blobServiceFactory = services.GetRequiredService<IBlobServiceFactory>();
requestContext.CustomerId = DataOperationInfo.RelatedCustomerId;
try
{
await orchestrationContext.CallActivityAsync(
nameof(UpdateJobStatus),
(DataOperationInfo.Id, DataOperationStatus.Running, string.Empty, string.Empty, requestContext));
// some other activity functions
---
---
} catch (Exception e)
{
await orchestrationContext.CallActivityAsync(
nameof(UpdateJobStatus),
(DataOperationInfo.Id, DataOperationStatus.Failed, string.Empty, string.Empty, requestContext));
}
}
}
[FunctionName(nameof(UpdateJobStatus))]
public static async Task RunAsync(
[ActivityTrigger] IDurableActivityContext activityContext,
[Inject]IServiceProvider rootServiceProvider)
{
using (var scope = rootServiceProvider.CreateScope())
{
try
{
var (DataOperationId, status, blobReference, logFileBlobId, requestContext) = activityContext.GetInput<(string, string, string, string, RequestContext)>();
var services = scope.ServiceProvider;
services.ResolveRequestContext(requestContext.CustomerId, requestContext.UserId, requestContext.UserDisplayName, requestContext.Culture);
var dataService = services.GetRequiredService<IDataService>();
var DataOperationDto = new DataOperationDto
{
Id = DataOperationId,
OperationStatusCode = status,
BlobReference = blobReference,
LogBlobReference = logFileBlobId
};
await dataService.UpdateAsync(DataOperationDto);
}
catch (Exception e)
{
throw e;
}
}
}
While you are debugging your Function, you should make sure that you are either using the local storage emulator, or an Azure Storage Account that is different from the one that is also being used by the Function already deployed in Azure.
If you are using the same storage account that another running Function is using, then it could be that parts of the execution is actually happening in Azure instead of your dev machine.
Related
I am looking into how the app insights work with different types of dependencies. I have a question on using the AppInsights ASP Net Core sdk with Messaging Service Bus sdk.
How can I capture messages to service bus while sending or receiving using this sdk in dependency? I understand that this is not something we would like to log all the time and I will make it configurable.
Thanks
You can create your own sender and implement the desired logic inside the SendMessageAsync method:
public class TelemetryEnabledServiceBusSender : ServiceBusSender
{
private readonly TelemetryClient telemetryClient;
internal TelemetryEnabledServiceBusSender(ServiceBusClient client, string topic, TelemetryClient telemetryClient) : base(client, topic)
{
this.telemetryClient = telemetryClient;
}
public override async Task SendMessageAsync(ServiceBusMessage message, CancellationToken cancellationToken = default)
{
telemetryClient.TrackTrace(message.Body.ToString());
await base.SendMessageAsync(message, cancellationToken);
}
}
use it like this:
var serviceBusSender = new TelemetryEnabledServiceBusSender(serviceBusClient, serviceBusData.Topic, telemetryClient);
await serviceBusSender.SendMessageAsync(message);
Logging processed messages is even simpler and can be done using the ServiceBusProcessor
serviceBusProcessor.ProcessMessageAsync += ProcessMessageAsync;
...
private async Task ProcessMessageAsync(ProcessMessageEventArgs arg)
{
telemetryClient.TrackTrace(arg.Message.Body.ToString());
...
}
Adding my approach as an answer here as it is too long to comment.
var telemetry = new DependencyTelemetry(
"Azure Service Bus",
_serviceBusClient.FullyQualifiedNamespace,
"ServiceBusReceiver.Receive",
string.Empty
);
using var operation =
_telemetryClient.StartOperation(telemetry);
try
{
var receivedMessage = await _serviceBusReceiver.ReceiveMessageAsync();
if (receivedMessage?.Body is not null)
{
message = receivedMessage.Body.ToString();
telemetry.Data = message;
// do something
}
telemetry.Success = true;
//Consider set telemetry duration
}
catch (Exception e)
{
// log exception
_telemetryClient.TrackException(e);
telemetry.Success = false;
throw;
}
finally
{
_telemetryClient.TrackTrace("Done");
_telemetryClient.StopOperation(operation);
}
Thanks to #Peter Bons
I'm using a Azure Function .Net 6 Isolated.
When I run the function on localhost from VS2022, it is 5 times faster then when I deploy it to Azure Function. Localhost is a VM hosted in Azure in the same region as the function.
I tried different Service Plans, but issue remains. (Consumption Plan, Elastic Premium EP3, Premium V2 P3v2)
Results in different regions vs. localhost:
The code is as follows:
DI - using the IHttpClientFactory (here):
public static class DataSourceServiceRegistration
{
public static IServiceCollection RegisterDataSourceServices(this IServiceCollection serviceCollection)
{
serviceCollection.AddHttpClient();
return serviceCollection;
}
}
HttpClient usage:
private readonly HttpClient _httpClient;
public EsriHttpClientAdapter(HttpClient httpClient)
{
_httpClient = httpClient ?? throw new ArgumentNullException(nameof(httpClient));
}
public async Task<JsonDocument> SendPrintServiceMessage(string url, HttpMethod httpMethod, string referer, IEnumerable<KeyValuePair<string, string>> content = null)
{
var watch = System.Diagnostics.Stopwatch.StartNew();
HttpContent httpContent = null;
if (content != null)
{
httpContent = new FormUrlEncodedContent(content);
}
var msg = new HttpRequestMessage(httpMethod, url) { Content = httpContent };
_httpClient.DefaultRequestHeaders.Referrer = new Uri(referer);
_httpClient.DefaultRequestHeaders.Add("some", "config");
_logger.LogInformation($"Before SendAsync - time {watch.ElapsedMilliseconds}");
var result = await _httpClient.SendAsync(msg);
_logger.LogInformation($"After SendAsync - time {watch.ElapsedMilliseconds}");
var response = await result.Content.ReadAsStringAsync();
_logger.LogInformation($"After ReadAsStringAsync - time {watch.ElapsedMilliseconds}");
if (result.StatusCode == HttpStatusCode.OK)
{
//do some stuff here
}
}
Application Insights is as follows:
AZURE:
Localhost:
Not sure if this is applicable to you, but hopefully it helps. If you're running under a Basic (Consumption) plan, your function will always be cold and need to spin up when being invoked by Http trigger. To circumvent this, you can set the function to Always On (if this is within your budget and scope) if on App Service Environment, Dedicated, or Premium plans. (In other words, Free Functions will always run cold.)
You can change this under Configuration > General Settings > Always On.
There's good info on how a Function runs through a cold startup at:
https://azure.microsoft.com/en-us/blog/understanding-serverless-cold-start/
I am trying to locally step through (using Visual Studio 2019) a Durable fuction, but when I invoke a call to context.CallActivityWithRetryAsync("MyActivity_Activity", retryOptions, myActivityParameters) from the OrchestrationTrigger, that invocation always returns the below:
Id = [xxxx], Status = WaitingForActivation, Method = "{null}", Result = "{Not yet computed}"
My first guess is there might be a deadlock somewhere, but I have rebooted my machine and still seeing the same result. Also, I tried setting a breakpoint at the first line of the activity (MyActivity_Activity), but that wasn't hit. Any help would be greatly appreciated!
Using the following:
VS 2019 Pro
.Net Core 2.2
Microsoft.Azure.DurableTask.AzureStorage v1.6.2
Microsoft.Azure.WebJobs.Extensions.DurableTask v1.8.2
Microsoft.NET.Sdk.Functions v1.0.29
...et al.
Below are some relevant snippets of code:
[FunctionName("MyOrchestrator_HttpStart")]
public static async Task<HttpResponseMessage> HttpStart(
[HttpTrigger(AuthorizationLevel.Function, "post")]HttpRequestMessage req,
[OrchestrationClient]DurableOrchestrationClient starter, ExecutionContext executionContext)
{
string instanceId = string.Empty;
MyOrchestratorParameters myOrchestratorParameters = await req.Content.ReadAsAsync<MyOrchestratorParameters>();
instanceId = await starter.StartNewAsync("MyOrchestrator", myOrchestratorParameters);
return starter.CreateCheckStatusResponse(req, instanceId);
}
[FunctionName("MyOrchestrator")]
public static async Task RunOrchestrator(
[OrchestrationTrigger] DurableOrchestrationContext context,
ExecutionContext executionContext)
{
MyOrchestratorParameters myOrchestratorParameters =
context.GetInput<MyOrchestratorParameters>();
ValidateParameters(myOrchestratorParameters);
var delay = 3000;
var retry = 3;
var retryOptions = new
RetryOptions(TimeSpan.FromSeconds(delay), retry);
MyActivityParameters myActivityParameters = new
MyActivityParameters()
{
JobNumber = myOrchestratorParameters.JobNumber,
Stage = myOrchestratorParameters.Stage,
TemplateId = myOrchestratorParameters.TemplateId
};
myActivityResults =
context.CallActivityWithRetryAsync<MyActivityResult>
("MyActivity_Activity", retryOptions, myActivityParameters);
...
}
[FunctionName("MyActivity_Activity")]
public static async Task<MyActivityResult>
RunActivity([ActivityTrigger] DurableActivityContext
activityContext, ExecutionContext executionContext)
{
var _myActivityParameters =
activityContext.GetInput<MyActivityParameters>();
//do some stuff & return MyActivityResult object
}
I think I found the answer. In my calls to CallActivityWithRetryAsync, I needed to predicate that with an await. Once I did this, it started working!
I've tried to come up with something from the example in the WebJobsSDK gitHub
var eventHubConfig = new EventHubConfiguration();
string eventHubName = "MyHubName";
eventHubConfig.AddSender(eventHubName,"Endpoint=sb://test.servicebus.windows.net/;SharedAccessKeyName=SendRule;SharedAccessKey=xxxxxxxx");
eventHubConfig.AddReceiver(eventHubName, "Endpoint=sb://test.servicebus.windows.net/;SharedAccessKeyName=ReceiveRule;SharedAccessKey=yyyyyyy");
config.UseEventHub(eventHubConfig);
JobHost host = new JobHost(config);
But I'm afraid that's not far enough for someone of my limited "skillset"!
I can find no instance of JobHostConfiguration that has a UseEventHub property (using the v1.2.0-alpha-10291 version of the Microsoft.AzureWebJobs package), so I can't pass the EventHubConfiguration to the JobHost.
I've used EventHub before, not within the WebJob context. I don't see if the EventHostProcessor is still required if using the WebJob triggering...or does the WebJob trigger essentially act as the EventHostProcessor?
Anyway, if anyone has a more complete example for a simpleton like me that would be really sweet! Thanks
From the documentation here, you should have all the information you need.
What you are missing is a reference of the Microsoft.Azure.WebJobs.ServiceBus.1.2.0-alpha-10291 nuget package.
The UseEventHub is an extension method that is declared in this package.
Otherwise your configuration seems ok.
Here is an example on how to receive or send messages from/to an EventHub:
public class BasicTest
{
public class Payload
{
public int Counter { get; set; }
}
public static void SendEvents([EventHub("MyHubName")] out Payload x)
{
x = new Payload { Counter = 100 };
}
public static void Trigger(
[EventHubTrigger("MyHubName")] Payload x,
[EventHub("MyHubName")] out Payload y)
{
x.Counter++;
y = x;
}
}
EventProcessorHost is still required, as the WebJob just provides the hosting environment for running it. As far as I know, EventProcessorHost is not integrated so deeply into WebJob, so its triggering mechanism cannot be used for processing EventHub messages. I use WebJob for running EventProcessorHost continuously:
public static void Main()
{
RunAsync().Wait();
}
private static async Task RunAsync()
{
try
{
using (var shutdownWatcher = new WebJobsShutdownWatcher())
{
await Console.Out.WriteLineAsync("Initializing...");
var eventProcessorHostName = "eventProcessorHostName";
var eventHubName = ConfigurationManager.AppSettings["eventHubName"];
var consumerGroupName = ConfigurationManager.AppSettings["eventHubConsumerGroupName"];
var eventHubConnectionString = ConfigurationManager.ConnectionStrings["EventHub"].ConnectionString;
var storageConnectionString = ConfigurationManager.ConnectionStrings["EventHubStorage"].ConnectionString;
var eventProcessorHost = new EventProcessorHost(eventProcessorHostName, eventHubName, consumerGroupName, eventHubConnectionString, storageConnectionString);
await Console.Out.WriteLineAsync("Registering event processors...");
var processorOptions = new EventProcessorOptions();
processorOptions.ExceptionReceived += ProcessorOptions_ExceptionReceived;
await eventProcessorHost.RegisterEventProcessorAsync<CustomEventProcessor>(processorOptions);
await Console.Out.WriteLineAsync("Processing...");
await Task.Delay(Timeout.Infinite, shutdownWatcher.Token);
await Console.Out.WriteLineAsync("Unregistering event processors...");
await eventProcessorHost.UnregisterEventProcessorAsync();
await Console.Out.WriteLineAsync("Finished.");
}
catch (Exception ex)
{
await HandleErrorAsync(ex);
}
}
}
private static async void ProcessorOptions_ExceptionReceived(object sender, ExceptionReceivedEventArgs e)
{
await HandleErrorAsync(e.Exception);
}
private static async Task HandleErrorAsync(Exception ex)
{
await Console.Error.WriteLineAsync($"Critical error occured: {ex.Message}{ex.StackTrace}");
}
I have an Azure Web Job built using the Azure SDK whose only job is to call a web service (Web API) and then log a response based on the return value (a class). The problem is that as soon as it calls the HttpClient PostAsJsonAsync method to call the service, it exits out of the web job without executing any of the response handling. My code is:
public class Result
{
// Properties ---------------------------------------------------------
public bool Success { get; set; }
public string Error { get; set; }
}
public class Functions
{
// This function will be triggered based on the schedule you have set for this WebJob
// This function will enqueue a message on an Azure Queue called queue
[NoAutomaticTrigger]
public async static void ManualTrigger(TextWriter log, int value)
{
using (var client = new HttpClient())
{
var rootUrl = ConfigurationManager.AppSettings.Get("WebJobTargetUrl");
client.BaseAddress = new System.Uri(rootUrl);
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
Console.WriteLine("Call service");
var response = await client.PostAsJsonAsync("api/Reminder/ProcessDueReminders", new { ItemID = 1 });
Console.WriteLine("After service");
var result = await response.Content.ReadAsAsync<Result>();
Console.WriteLine("After result");
if (result.Success)
Console.WriteLine("Reminders Processed");
else
Console.WriteLine("Reminder process error: " + result.Error);
}
}
}
and the execution logs from the portal are:
I believe it has something to do with the asynchronous operation but I can't figure out a pattern that will work. Any help would be appreciated.
You must define the return value of your own async method as Task instead of void.
On a related note, you should suffix the name of your method with Async. That's not going to solve the problem, but it indicates that you're using the async/await pattern.
There is probably an exception in your PostAsJsonAsync call. Try to put a try catch around it to and log the error:
try {
var response = await client.PostAsJsonAsync("api/Reminder/ProcessDueReminders", new { ItemID = 1 });
} catch (Exception ex){
Console.WriteLine("Exception: "+ ex);
}
Console.WriteLine("After service");