Azure Function / Azure Service Bus: Manually completing a message - azure

I'd like to mark a message as complete after successful completion of all activities. In this case autoCompleteMessages is set to false in host.json.
I can complete or dead letter a message from the ServiceBusTrigger function, but how do I ensure all activities succeeded?
Can it be done in the OrchestrationTrigger function?
FunctionName("QueueStart")]
public static async Task Run(
[ServiceBusTrigger("%QueueTopicName%", "Subscription", Connection = "ServiceBusConnectionString")]
ServiceBusReceivedMessage msg,
ServiceBusMessageActions messageActions,
[DurableClient] IDurableOrchestrationClient starter,
ILogger log)
{
string inputMessage = Encoding.UTF8.GetString(msg.Body);
await starter.StartNewAsync("Hello", null, inputMessage);
// can run here, but how to check if all activities succeeded?
// await messageActions.CompleteMessageAsync(msg);
// await messageActions.DeadLetterMessageAsync(msg);
}
[FunctionName("Hello")]
public static async Task<List<string>> RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context, ILogger log)
{
var outputs = new List<string>();
outputs.Add(await context.CallActivityAsync<string>("Hello_Hello", "London"));
outputs.Add(await context.CallActivityAsync<string>("Hello_Hello1", "Seattle"));
// how to mark message as complete here?
return outputs;
}
[FunctionName("Hello_Hello")]
public static string SayHello([ActivityTrigger] string name, ILogger log)
{
log.LogInformation($"Saying hello to {name}.");
return $"Hello {name}!";
}
[FunctionName("Hello_Hello1")]
public static string SayHello1([ActivityTrigger] string city, ILogger log)
{
throw new Exception("Exception from hello1");
log.LogInformation($"Saying hello1 to {city}.");
return $"Hello {city}!";
}

the following included in the ServiceBusTrigger does the trick
string instanceID = Guid.NewGuid();
await starter.StartNewAsync("Hello", instanceID, inputMessage);
var orchestratorStatus = await starter.GetStatusAsync(instanceID);
while (orchestratorStatus.RuntimeStatus == OrchestrationRuntimeStatus.Running || orchestratorStatus.RuntimeStatus == OrchestrationRuntimeStatus.Pending)
{
await Task.Delay(1000);
orchestratorStatus = await starter.GetStatusAsync(instanceID);
}
if (orchestratorStatus.RuntimeStatus == OrchestrationRuntimeStatus.Completed)
{
logger.LogInformation($"Completed orchestration with ID = '{instanceID}'.");
await messageActions.CompleteMessageAsync(msg);
}

Related

ILogger sink for Azure Serverless Functions

I am injecting ILogger into my Azure functions, and in my host.json I have the following:
{
"version": "2.0",
"logging": {
"applicationInsights": {
"samplingSettings": {
"isEnabled": true,
"maxTelemetryItemsPerSecond": 20
}
}
}
}
I then log as below:
public class EnrollmentFunctions
{
private readonly ILogger<AttestationFunctions> logger;
public EnrollmentFunctions(ILogger<AttestationFunctions> log)
{
logger = log;
}
public async Task<IActionResult> Enroll(
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "enroll")] HttpRequest req)
{
logger.LogInformation("Requesting attestation of device name {deviceName}.", req.Query["deviceName"]);
}
}
However no logs are logged as below - appreciate pointers as to what I am doing wrong and where I would see my logs please.
I've followed the below work around to achieve the desired results as described:
I've used generalized logging in my function project using Visual studio and included both ILogger and TelemetryClient in a class Genlog.cs
Genlog.cs
public class Genlog
{
public Genlog(ILogger<Genlog> logger, TelemetryClient telemetry)
{
Logger = logger ?? throw new ArgumentNullException(nameof(logger));
Telemetry = telemetry ?? throw new ArgumentNullException(nameof(telemetry));
}
public void testlog(string mylog)
{
Logger.LogInformation("TestCustomInformationLog:" + mylog);
Telemetry.TrackTrace("TestCustomTraceLog: " + mylog, Microsoft.ApplicationInsights.DataContracts.SeverityLevel.Information);
Telemetry.TrackException(new Exception(""));
}
public ILogger<Genlog> Logger { get; }
public TelemetryClient Telemetry { get; }
}
Startup.cs
#Adding configuring service of Genlog in startup.cs
public override void Configure(IFunctionsHostBuilder builder)
{
builder.Services.AddScoped<Genlog>();
}
Function.cs
public class Function1
{
private readonly Genlog genlog;
public Function1(Genlog genlog)
{
this.genlog = genlog ?? throw new ArgumentNullException(nameof(genlog));
}
[FunctionName("Function1")]
public async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req,
ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
# Using Both Ilogger & Telemetry logger in same class.
genlog.testlog("C# HTTP trigger function processed a request.");
string name = req.Query["name"];
string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
dynamic data = JsonConvert.DeserializeObject(requestBody);
name = name ?? data?.name;
string responseMessage = string.IsNullOrEmpty(name)
? "This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response."
: $"Hello, {name}. This HTTP triggered function executed successfully.";
genlog.testlog(responseMessage);
return new OkObjectResult(responseMessage);
}
}
Result
Published to Azure to check & retrieve logs:

How to call two rest APIs in a Azure Function?

I'm trying to call two rest APIs (Azure Web Service APIs) in a Azure Function. I have written the below code but When I try to run it, the first API is only getting executed but the second one isn't.
The Logic should be like if the first API gets a response status as 200 Ok then only, it should proceed to call the next API.
namespace FunctionChainingApp
{
public static class FunctionChaining
{
[FunctionName("FunctionChaining")]
public static async Task<HttpResponseMessage> Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)]HttpRequestMessage req, ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
try
{
// Call Your API
HttpClient abcdeClient = new HttpClient();
HttpRequestMessage abcdeRequest = new HttpRequestMessage(HttpMethod.Get, string.Format("https://abcde.azurewebsites.net/api/abcde/GetabcdeDetails"));
//Read Server Response
HttpResponseMessage abcdeResponse = await abcdeClient.SendAsync(abcdeRequest);
bool abcdeStatusCode200 = await abcdeResponse.Content.ReadAsAsync<bool>();
if (abcdeStatusCode200 == true)
{
// Call Your API
HttpClient vwxyzClient = new HttpClient();
HttpRequestMessage vwxyzRequest = new HttpRequestMessage(HttpMethod.Get, string.Format("https://vwxyz.azurewebsites.net/api/vwxyz/GetvwxyzDefaultDetails"));
//Read Server Response
HttpResponseMessage vwxyzResponse = await vwxyzClient.SendAsync(vwxyzRequest);
bool vwxyzStatusCode200 = await vwxyzResponse.Content.ReadAsAsync<bool>();
}
else
{
// Call Your API
abcdeRequest = new HttpRequestMessage(HttpMethod.Get, string.Format("https://abcde.azurewebsites.net/api/abcde/GetabcdeDetails"));
}
return req.CreateResponse(HttpStatusCode.OK);
}
catch (Exception ex)
{
return req.CreateResponse(HttpStatusCode.OK, "The Called Scheduler Failed : {0}", string.Format(ex.Message));
}
}
}
public class ResponseModel
{
public bool abcdeStatusCode200 { get; set; }
public bool vwxyzStatusCode200 { get; set; }
}
}
You need to separate these two functions and call them within DurableOrchestrationClient
var output = new List<string>();
output.Add(await context.CallActivityAsync<string>("CallAPI1", "test2019"));
output.Add(await context.CallActivityAsync<string>("CallAPI2", "test2"));
EXAMPLE

Add metadata to traces in Azure functions

I have an Azure function(.NET core 2.0) that runs on each PR in an ADO repo.
I would like to add the PR-ID as metadata to each trace logged by the Azure function.
(I am viewing the logs using the traces table in Azure application insights instance)
The Azure function logs traces via:
public static async Task<HttpResponseMessage> Run([HttpTrigger(AuthorizationLevel.Function, "post", Route = null)]HttpRequestMessage req, ILogger log, ExecutionContext context)
{
logger.LogInformation("Trace Message");
}
How can I add additional metadata to each trace?
Yes this is possible.
Option 1: Using a Telemetry Initializer with the help of AsyncLocal:
public class CustomTelemetryInitializer : ITelemetryInitializer
{
public static AsyncLocal<string> MyValue = new AsyncLocal<string>();
public void Initialize(ITelemetry telemetry)
{
if (telemetry is ISupportProperties propertyItem)
{
propertyItem.Properties["myProp"] = MyValue.Value;
}
}
}
You can set the value of the AsyncLocal in the function like this:
[FunctionName("Function")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)]
HttpRequest req,
ILogger log)
{
var name = req.Query["name"];
CustomTelemetryInitializer.MyValue.Value = name;
log.LogInformation("C# HTTP trigger function processed a request.");
return new OkObjectResult($"Hello, {name}");
}
When you run the function you can see the information in the portal:
You can find the whole code at this repo
Now, addressing your comments. Yes you need something extra. The ITelemetryInitializer instance needs to be registered using dependency injection. That is done in the Startup class as outlined in the documentation:
using Microsoft.ApplicationInsights.Extensibility;
using Microsoft.Azure.Functions.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection;
[assembly: FunctionsStartup(typeof(FunctionApp.Startup))]
namespace FunctionApp
{
public class Startup : FunctionsStartup
{
public override void Configure(IFunctionsHostBuilder builder)
{
builder.Services.AddSingleton<ITelemetryInitializer, CustomTelemetryInitializer>();
builder.Services.AddLogging();
}
}
}
Once registered the Application Insights SDK will use the CustomTelemetryInitializer.
Option 2
The other option does not involve any TelemetryInitializer, but you can only add properties to the generated RequestTelemetry that is added by the Azure Function App Insights integration. This is done by making using of the fact that the current TelemetryRequest is stored in the HttpContext:
[FunctionName("Function")]
public static IActionResult Run(
[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)]
HttpRequest req,
ILogger log)
{
var name = req.Query["name"];
CustomTelemetryInitializer.MyValue.Value = name;
log.LogInformation("C# HTTP trigger function processed a request.");
var telemetryItem = req.HttpContext.Features.Get<RequestTelemetry>();
telemetryItem.Properties["SetInFunc"] = name;
return new OkObjectResult($"Hello, {name}");
}
This will show up in the portal as well:
Is it a problem that the context is added only to the Request? It might, but be aware you can query all related telemetry and know what context is involved, for example:
union (traces), (requests), (dependencies), (customEvents), (exceptions)
| extend itemType = iif(itemType == 'availabilityResult',itemType,iif(itemType == 'customEvent',itemType,iif(itemType == 'dependency',itemType,iif(itemType == 'pageView',itemType,iif(itemType == 'request',itemType,iif(itemType == 'trace',itemType,iif(itemType == 'exception',itemType,"")))))))
| extend prop = customDimensions.SetInFunc
| where ((itemType == 'trace' or (itemType == 'request' or (itemType == 'pageView' or (itemType == 'customEvent' or (itemType == 'exception' or (itemType == 'dependency' or itemType == 'availabilityResult')))))))
| top 101 by timestamp desc
will show:
All telemetry that comes from the same invocation will have the same operation_Id.
This could be implemented with ITelemetry Initializer, you could add custom dimension to a specified telemetry like only for request.
1.Install the following nuget packages:
Microsoft.ApplicationInsights, version 2.11.0
Microsoft.NET.Sdk.Functions, version 1.0.29
2.The below is my test code.
[assembly: WebJobsStartup(typeof(FunctionApp54.MyStartup))]
namespace FunctionApp54
{
internal class MyTelemetryInitializer : ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
//use telemetry is RequestTelemetry to make sure only add to request
if (telemetry != null && telemetry is RequestTelemetry && !telemetry.Context.GlobalProperties.ContainsKey("testpro"))
{
telemetry.Context.GlobalProperties.Add("testpro", "testvalue");
}
}
}
public class MyStartup : IWebJobsStartup
{
public void Configure(IWebJobsBuilder builder)
{
builder.Services.AddSingleton<ITelemetryInitializer, MyTelemetryInitializer>();
}
}
public static class Function2
{
[FunctionName("Function2")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req,
ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
string name = req.Query["name"];
string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
dynamic data = JsonConvert.DeserializeObject(requestBody);
name = name ?? data?.name;
return name != null
? (ActionResult)new OkObjectResult($"Hello, {name}")
: new BadRequestObjectResult("Please pass a name on the query string or in the request body");
}
}
}

How to pass object from trigger to orchestra and activity functions

I have HTTP trigger which invokes orchestra function. In HTTP trigger function I receive json string which I deserialize into object . However I have no idea how to pass this object to orchestra function or activity functions.
I have tried to include it as parameter of orchestra function but it gives me error
HTTP trigger
#load "../Shared/jsonObject.csx"
public static async Task<HttpResponseMessage> Run(
HttpRequestMessage req,
DurableOrchestrationClient starter,
string functionName,
ILogger log)
{
HttpContent requestContent = req.Content;
string jsonContent = requestContent.ReadAsStringAsync().Result;
jsonObject bsObj = JsonConvert.DeserializeObject<jsonObject> (jsonContent);
// Pass the function name as part of the route
string instanceId = await starter.StartNewAsync("Orchestra", bsObj);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
return starter.CreateCheckStatusResponse(req, instanceId);
}
Orchestra function
#load "../Shared/jsonObject.csx"
[FunctionName("Orchestra")]
public static async void Run(DurableOrchestrationContext context)
{
var output = await context.CallActivityAsync<int>("checkConditions", bsObj);
}
Activity functionn not implemented yet
#load "../Shared/jsonObject.csx"
public static string Run(jsonObject bsObj)
{
return "done";
}
[Error] run.csx(19,74): error CS0103: The name 'bsObj' does not exist in the current context
Please try running the below code
HTTP Trigger function.
[FunctionName("HttpTriggerCSharp")]
public static async Task<HttpResponseMessage> Run(
[HttpTrigger(AuthorizationLevel.Function, "post")]HttpRequestMessage req,
[OrchestrationClient]DurableOrchestrationClient starter,
ILogger log)`
{
HttpContent requestContent = req.Content;
string jsonContent = requestContent.ReadAsStringAsync().Result;
JObject bsObj = JsonConvert.DeserializeObject<JObject>(jsonContent);
// Pass the function name as part of the route
string instanceId = await starter.StartNewAsync("Orchestra", bsObj);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
return starter.CreateCheckStatusResponse(req, instanceId);
}
Orchestration Function
[FunctionName("Orchestra")]
public static async Task RunOrchestrator(
[OrchestrationTrigger] DurableOrchestrationContext context)
{
var bsObj = context.GetInput<JObject>();
var output = await context.CallActivityAsync<JObject>("checkConditions", bsObj);
}
Activity Function
[FunctionName("checkConditions")]
public static string SayHello([ActivityTrigger] JObject bsObj, ILogger log)
{
//Assign input value to any variable
var json = bsObj;
//Logic of the code
return "done";
}
I have referenced this doc for durable functions extention and this doc for HTTP Triggered functions.

When the ProcessEventsAsync(PartitionContext context, ienumerable<EventData> messages) method will be fired

I am currently working on Internet Of Things, in my current project I was Created the One Azure Cloud Service Project in that I Created the Worker Role, inside the worker role i have wrote below lines of code.
public class WorkerRole : RoleEntryPoint
{
private readonly CancellationTokenSource cancellationTokenSource = new CancellationTokenSource();
private readonly ManualResetEvent runCompleteEvent = new ManualResetEvent(false);
private static string connectionString;
private static string eventHubName;
public static ServiceClient iotHubServiceClient { get; private set; }
public static EventHubClient eventHubClient { get; private set; }
public override void Run()
{
Trace.TraceInformation("EventsForwarding Run()...\n");
try
{
this.RunAsync(this.cancellationTokenSource.Token).Wait();
}
finally
{
this.runCompleteEvent.Set();
}
}
public override bool OnStart()
{
// Set the maximum number of concurrent connections
ServicePointManager.DefaultConnectionLimit = 12;
// For information on handling configuration changes
// see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.
bool result = base.OnStart();
Trace.TraceInformation("EventsForwarding OnStart()...\n");
connectionString = ConfigurationManager.AppSettings["Microsoft.ServiceBus.ConnectionString"];
eventHubName = ConfigurationManager.AppSettings["Microsoft.ServiceBus.EventHubName"];
string storageAccountName = ConfigurationManager.AppSettings["AzureStorage.AccountName"];
string storageAccountKey = ConfigurationManager.AppSettings["AzureStorage.Key"];
string storageAccountString = string.Format("DefaultEndpointsProtocol=https;AccountName={0};AccountKey={1}",
storageAccountName, storageAccountKey);
string iotHubConnectionString = ConfigurationManager.AppSettings["AzureIoTHub.ConnectionString"];
iotHubServiceClient = ServiceClient.CreateFromConnectionString(iotHubConnectionString);
eventHubClient = EventHubClient.CreateFromConnectionString(connectionString, eventHubName);
var defaultConsumerGroup = eventHubClient.GetDefaultConsumerGroup();
string eventProcessorHostName = "SensorEventProcessor";
EventProcessorHost eventProcessorHost = new EventProcessorHost(eventProcessorHostName, eventHubName, defaultConsumerGroup.GroupName, connectionString, storageAccountString);
eventProcessorHost.RegisterEventProcessorAsync<SensorEventProcessor>().Wait();
Trace.TraceInformation("Receiving events...\n");
return result;
}
public override void OnStop()
{
Trace.TraceInformation("EventsForwarding is OnStop()...");
this.cancellationTokenSource.Cancel();
this.runCompleteEvent.WaitOne();
base.OnStop();
Trace.TraceInformation("EventsForwarding has stopped");
}
private async Task RunAsync(CancellationToken cancellationToken)
{
while (!cancellationToken.IsCancellationRequested)
{
//Trace.TraceInformation("EventsToCommmandsService running...\n");
await Task.Delay(1000);
}
}
}
Next I have wrote the below lines of code in SensorEventProcessor, for receiving the messages from event hub and send those messages to IoT hub.
class SensorEventProcessor : IEventProcessor
{
Stopwatch checkpointStopWatch;
PartitionContext partitionContext;
public async Task CloseAsync(PartitionContext context, CloseReason reason)
{
Trace.TraceInformation(string.Format("EventProcessor Shuting Down. Partition '{0}', Reason: '{1}'.", this.partitionContext.Lease.PartitionId, reason.ToString()));
if (reason == CloseReason.Shutdown)
{
await context.CheckpointAsync();
}
}
public Task OpenAsync(PartitionContext context)
{
Trace.TraceInformation(string.Format("Initializing EventProcessor: Partition: '{0}', Offset: '{1}'", context.Lease.PartitionId, context.Lease.Offset));
this.partitionContext = context;
this.checkpointStopWatch = new Stopwatch();
this.checkpointStopWatch.Start();
return Task.FromResult<object>(null);
}
public async Task ProcessEventsAsync(PartitionContext context, IEnumerable<EventData> messages)
{
Trace.TraceInformation("\n");
Trace.TraceInformation("........ProcessEventsAsync........");
//string commandParameterNew = "{\"Name\":\"AlarmThreshold\",\"Parameters\":{\"SensorId\":\"" + "Hello World" + "\"}}";
//await WorkerRole.iotHubServiceClient.SendAsync("astranidevice", new Microsoft.Azure.Devices.Message(Encoding.UTF8.GetBytes(commandParameterNew)));
foreach (EventData eventData in messages)
{
try
{
string jsonString = Encoding.UTF8.GetString(eventData.GetBytes());
Trace.TraceInformation(string.Format("Message received at '{0}'. Partition: '{1}'",
eventData.EnqueuedTimeUtc.ToLocalTime(), this.partitionContext.Lease.PartitionId));
Trace.TraceInformation(string.Format("-->Raw Data: '{0}'", jsonString));
SimpleTemperatureAlertData newSensorEvent = this.DeserializeEventData(jsonString);
Trace.TraceInformation(string.Format("-->Serialized Data: '{0}', '{1}', '{2}', '{3}', '{4}'",
newSensorEvent.Time, newSensorEvent.RoomTemp, newSensorEvent.RoomPressure, newSensorEvent.RoomAlt, newSensorEvent.DeviceId));
// Issuing alarm to device.
string commandParameterNew = "{\"Name\":\"AlarmThreshold\",\"Parameters\":{\"SensorId\":\"" + "Hello World" + "\"}}";
Trace.TraceInformation("Issuing alarm to device: '{0}', from sensor: '{1}'", newSensorEvent.DeviceId, newSensorEvent.RoomTemp);
Trace.TraceInformation("New Command Parameter: '{0}'", commandParameterNew);
await WorkerRole.iotHubServiceClient.SendAsync(newSensorEvent.DeviceId, new Microsoft.Azure.Devices.Message(Encoding.UTF8.GetBytes(commandParameterNew)));
}
catch (Exception ex)
{
Trace.TraceInformation("Error in ProssEventsAsync -- {0}\n", ex.Message);
}
}
await context.CheckpointAsync();
}
private SimpleTemperatureAlertData DeserializeEventData(string eventDataString)
{
return JsonConvert.DeserializeObject<SimpleTemperatureAlertData>(eventDataString);
}
}
When I was debug my code, the ProcessEventsAsync(PartitionContext context, IEnumerable messages) method will never call and just enter into OpenAsync() method then itstop the debugging.
Please tell me Where I did mistake in my project and tell me when the ProcessEventsAsync() method will call.
Regards,
Pradeep
IEventProcessor.ProcessEventsAsync is invoked when there are any unprocessed messages in the EventHub.
An Event Hub contains multiple partitions. A partition is an ordered sequence of events. Within a partition, each event includes an offset. This offset is used by consumers (IEventProcessor) to show the location in the event sequence for a given partition. When an IEventProcessor connects (EventProcessorHost.RegisterEventProcessorAsync), it passes this offset to the Event Hub to specify the location at which to start reading. When there are unprocessed messages (events with higher offset), they are delivered to the IEventProcessor. Checkpointing is used to persist the offset of processed messages (PartitionContext.CheckpointAsync).
You can find detailed information about the internals of EventHub: Azure Event Hubs overview
Have you sent any messages to the EventHub (EventHubClient.SendAsync(EventData))?

Resources