How to cancel a running trigger function within azure durable functions? - azure

I am trying the fan-out, fan-in pattern. Here's my code
[FunctionName("af_cancellation")]
public static async Task<string> RunOrchestrator(
[OrchestrationTrigger] DurableOrchestrationContext context, ILogger log)
{
var taskList = new List<Task<string>>();
var tokenSource = new CancellationTokenSource();
taskList.Add(context.CallActivityAsync<string>("af_cancellation_Hello", new { ct = tokenSource.Token, city = "Tokyo" }));
taskList.Add(context.CallActivityAsync<string>("af_cancellation_Hello", new { ct = tokenSource.Token, city = "Seattle" }));
taskList.Add(context.CallActivityAsync<string>("af_cancellation_Hello", new { ct = tokenSource.Token, city = "London" }));
try
{
await Task.WhenAll(taskList);
}
catch (FunctionException)
{
log.LogError("trigger function failed");
tokenSource.Cancel();
}
string output = "";
foreach (var t in taskList)
{
output += t.Result;
}
return output;
}
I would like to cancel all the tasks in taskList if any of them throw an exception. What i am noticing is that await Task.WhenAll finishes all the tasks before moving forward.
Here's the sample trigger function
[FunctionName("af_cancellation_Hello")]
public static string SayHello([ActivityTrigger] DurableActivityContext context, ILogger log)
{
var data = context.GetInput<dynamic>();
var name = (string)data.city;
// unable to de-serialize cancellation token here but we'll ignore that.
var ct = JsonConvert.DeserializeObject<CancellationToken>(data.ct);
if (name != "London")
{
System.Threading.Thread.Sleep(1000 * 30);
}
else
{
System.Threading.Thread.Sleep(1000 * 10);
throw new FunctionException("don't like london");
}
log.LogInformation($"Saying hello to {name}.");
return $"Hello {name}!";
}
How can i achieve this?

According to this I think it's not possible. If you need to undo the job from the other activities that succeeded, you must take a look on the Saga Pattern and launch compensating activities.
more info:
https://microservices.io/patterns/data/saga.html
https://www.enterpriseintegrationpatterns.com/patterns/conversation/CompensatingAction.html

Related

Calling the POST API call from the Timer Triggered Azure Function

I have couple of tables in the DB, one to stores the list of Holidays(HolidayWeeks) in the year and other one to store the setting(Settings) like when an application should be running. Both the tables are like below in the Azure DB
I started to build the Timer Triggered AzureFunction to check everyday at 5 AM if today doesnot fall under the Holiday list as well Today (Day of the week ) is equal to the Order day in the Settings table like below
public static class StartorStopSMAppService
{
[FunctionName("StartorStopSMAppService")]
public static void Run([TimerTrigger("0 0 5 * * *")]TimerInfo myTimer, ILogger log)
{
int holidaychk = 0;
var str = Environment.GetEnvironmentVariable("SurplusMouseDBConnection");
using (SqlConnection conn = new SqlConnection(str))
{
try
{
conn.Open();
var settingscheck = "SELECT TOP 1 [EndTime],[StartTime],[OrderDay]"+
"FROM[dbo].[Settings]"+
"where SUBSTRING(DATENAME(weekday, getdate() AT TIME ZONE 'UTC' AT TIME ZONE 'Eastern Standard Time'), 0, 4) = OrderDay";
var holidaycheck = "SELECT Count(*) FROM[dbo].[HolidayWeeks] where FORMAT(getdate(), 'yyyy-MM-dd') = [HolidateDate]";
SqlCommand cmd1 = new SqlCommand(holidaycheck, conn);
holidaychk = (Int32)cmd1.ExecuteScalar();
if (holidaychk != 0)
{
SqlCommand command = new SqlCommand(settingscheck, conn);
using (SqlDataReader reader = command.ExecuteReader())
{
reader.Read();
if (reader.HasRows)
{
var starttime = reader["StartTime"].ToString();
var endtime = reader["EndTime"].ToString();
// Not sure how to call the start API at this starttime time
}}}}
catch (Exception ex)
{
log.LogInformation(ex.Message);
}
finally
{
if (conn.State != ConnectionState.Closed)
{
conn.Close();
} }}
When the both the conditions are satisfied I need to start the web service on the starttime give on the Settings table which I am storing in the var starttime = reader["StartTime"].ToString(); I understand that I can POST call the API https://learn.microsoft.com/en-us/rest/api/appservice/web-apps/start to start Web App in the App Service. But how can start the application based on starttime variable. I am kind of stuck any help is greatly appreciated
Once your code determines that it is not a holiday and you would like to start the web app at that particular time, you can post a scheduled message to Azure Queue, see this. If you write a Azure Function Queue trigger, as and when it gets invoked, it can call the start REST API for your WebApp.
Its possible with Durable functions as well. You can implement this as below.
Timer trigger function
public static class TimerFunction
{
[FunctionName("ScheduledStart")]
public static async Task RunScheduled([TimerTrigger("0 * * * * *")] TimerInfo timerInfo,[DurableClient] IDurableClient starter,ILogger log)
{
string functionName = "OrchestractionFunction";
string time = "26/10/2021 7:00AM";
var startTime = DateTime.Parse(time).ToUniversalTime();
string instanceId = await starter.StartNewAsync(functionName, Guid.NewGuid().ToString(), startTime);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
}
}
Create a Durable function with delay
`
[FunctionName("OrchestractionFunction")]
public static async Task RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context)
{
var startTime = context.GetInput<DateTime>();
Console.WriteLine(startTime);
await context.CreateTimer(startTime, CancellationToken.None);
await context.CallActivityAsync("ActivityFunction", "Jhon");
}
`
Activity function invoking a Http request
[FunctionName("ActivityFunction")]
public static string SayHello([ActivityTrigger] string name, ILogger log){
log.LogInformation(name);
//HttpRequest here
}
you can extend the orchestration function to call multiple different APIs
The other two answers are good, but using your current code, all you need to do is check if current date is greater than start time, then, perform the request. Here's a sample using Flurl to simplify the request:
public static class StartorStopSMAppService {
[FunctionName("StartorStopSMAppService")]
public static void Run([TimerTrigger("0 0 5 * * *")] TimerInfo myTimer, ILogger log) {
int holidaychk = 0;
var str = Environment.GetEnvironmentVariable("SurplusMouseDBConnection");
using(SqlConnection conn = new SqlConnection(str)) {
try {
conn.Open();
var settingscheck = "SELECT TOP 1 [EndTime],[StartTime],[OrderDay]" +
"FROM[dbo].[Settings]" +
"where SUBSTRING(DATENAME(weekday, getdate() AT TIME ZONE 'UTC' AT TIME ZONE 'Eastern Standard Time'), 0, 4) = OrderDay";
var holidaycheck = "SELECT Count(*) FROM[dbo].[HolidayWeeks] where FORMAT(getdate(), 'yyyy-MM-dd') = [HolidateDate]";
SqlCommand cmd1 = new SqlCommand(holidaycheck, conn);
holidaychk = (Int32) cmd1.ExecuteScalar();
if (holidaychk != 0) {
SqlCommand command = new SqlCommand(settingscheck, conn);
using(SqlDataReader reader = command.ExecuteReader()) {
reader.Read();
if (reader.HasRows) {
var starttime = reader["StartTime"].ToString();
var endtime = reader["EndTime"].ToString();
string subscriptionId = ""; // enter this value
string resourceGroupName = ""; // enter this value
string webAppName = ""; // enter this value
string url = $"https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Web/sites/{webAppName}/start?api-version=2021-02-01";
DateTime dStartTime;
DateTime.TryParse(starttime, out dStartTime);
if (DateTime.UtcNow >= dStartTime){
await url.PostAsync(); // requires Flurl.Http package
}
}
}
}
} catch (Exception ex) {
log.LogInformation(ex.Message);
} finally {
if (conn.State != ConnectionState.Closed) {
conn.Close();
}
}
}

Azure Function : When Function AutoScale, function called multiple times

In Azure Queue add single entry for queueitem but function called multiple times for same queueitem, not able to understand what is wrong, We have used Async/Await in function and code is also Async, below is Azure function settings,
below is function code,
public class CreateTempVMTempTablesFunction
{
private static Container container;
private static IProcessingFunctionService _processingFunctionService;
private static IAzureFunctionFailuresRepository _azureFunctionFailuresRepository;
private static ITrackLogEventsService _trackLogEventsService;
[FunctionName("CreateTempVMTempTablesFunction")]
public static async Task Run([QueueTrigger("%environment-plan%" + AzureFunctionConstants.CreateTempTableQueue, Connection = "AzureWebJobsStorage")]string myQueueItem, ILogger log)
{
string ErrorMessage = string.Empty;
var start = DateTime.Now;
FunctionStatusEnum IsSuccess = FunctionStatusEnum.Success;
log.LogInformation($"C# Queue trigger function processed: {myQueueItem} - {start}");
Guid tempid = new Guid(myQueueItem);
TempVM tempVM = new TempVM();
try
{
container = BusinessLogic.Helpers.SimpleInjectorWebApiInitializer.InitializeSingleton();;
_processingFunctionService = container.GetInstance<IProcessingFunctionService>();
_azureFunctionFailuresRepository = container.GetInstance<IAzureFunctionFailuresRepository>();
_trackLogEventsService = container.GetInstance<ITrackLogEventsService>();
tempVM = await _processingFunctionService.GetById(tempid);
if (tempVM != null)
{
FunctionStatusEnum IsAlreadyPerformed = await _azureFunctionFailuresRepository.GetAzureFunctionFailureStatus(AzureFunctionConstants.CreateTempVMTempTablesFunction, tempVM.Id);
if (IsAlreadyPerformed != FunctionStatusEnum.Success)
{
ResponseData response = await _processingFunctionService.CreateTempVMTempTables(tempid);
}
else
{
ErrorMessage = AzureFunctionConstants.FunctionAlreadyProcessed;
}
}
else
{
ErrorMessage = AzureFunctionConstants.TempVMNotFound;
}
}
catch (Exception ex)
{
IsSuccess = FunctionStatusEnum.Failed;
ErrorMessage = ex.ToString();
}
finally
{
AzureFunctionFailures azureFunctionFailures = new AzureFunctionFailures()
{
Id = Guid.NewGuid(),
FunctionName = AzureFunctionConstants.CreateTempVMTempTablesFunction,
QueueItem = myQueueItem,
ErrorMessage = ErrorMessage,
StartTime = start,
EndTime = DateTime.Now,
FailureTypeId = tempid,
FunctionStatus = IsSuccess,
ProcessTime = (DateTime.Now - start).TotalMilliseconds,
};
await _azureFunctionFailuresRepository.Add(azureFunctionFailures);
}
log.LogInformation($"End Time : {DateTime.Now} - QueueItem {myQueueItem}");
log.LogInformation($"Total Time : {DateTime.Now - start} - QueueItem {myQueueItem}");
}
}
I have check the code where added entry in queue, but only one entry is added for one queueitem.
This issue happened when added multiple entry in same queue (i.e. load testing where I have added 24 request only) for different queue item, when single queue is run then this is not happened, our functions are in App Service Plan with autoscaling
As mentioned in comments. Just set value of WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT as 1 in application settings of your function.

Azure Table Storage not saving all object properties

I have a problem with the Azure Table Storage. What I'm trying to achieve is saving the ChangeToken of the SharePoint list in order to use the webhooks properly.
Here is the code:
public class TablesHelper
{
private static readonly string TokenTableName = "TokenTable";
public static async Task<ListChangeToken> GetChangeTokenForListAsync(string listId)
{
var retrieveOperation = TableOperation.Retrieve<ListChangeToken>("Lists", listId, new List<string>() { "ChangeToken" });
var tableReference = await GetTableReferenceAsync(TokenTableName);
var tableResult = await tableReference.ExecuteAsync(retrieveOperation);
if(tableResult.Result != null)
{
return tableResult.Result as ListChangeToken;
}
return null;
}
public static async Task SaveChangeTokenForListAsync(ListChangeToken changeToken)
{
var insertOperation = TableOperation.Insert(changeToken);
var tableReference = await GetTableReferenceAsync(TokenTableName);
var result = await tableReference.ExecuteAsync(insertOperation);
}
private static async Task<CloudTable> GetTableReferenceAsync(string tableName)
{
var storageAccount = CloudStorageAccount.Parse(ConfigurationHelper.CloudStorage);
var tableClient = storageAccount.CreateCloudTableClient();
var reference = tableClient.GetTableReference(tableName);
await reference.CreateIfNotExistsAsync();
return reference;
}
}
The ListChangeToken class:
public class ListChangeToken : TableEntity
{
public ListChangeToken(string listId, string changeToken)
{
this.PartitionKey = "Lists";
this.RowKey = listId;
this.ChangeToken = changeToken;
}
public ListChangeToken() { }
public string ChangeToken { get; set;}
}
As per request, the function calling TablesHelper:
[FunctionName("EventHandler")]
public static async Task Run([QueueTrigger("events", Connection = "CloudStorage")]string myQueueItem, TraceWriter log)
{
var notificationGroup = Newtonsoft.Json.JsonConvert.DeserializeObject<NotificationGroup>(myQueueItem);
var contextHelper = new ContextHelper();
foreach (var notification in notificationGroup.Value)
{
UriBuilder uriBuilder = new UriBuilder();
uriBuilder.Scheme = "https";
uriBuilder.Host = ConfigurationHelper.TenantDomain;
uriBuilder.Path = notification.SiteUrl;
using (var ctx = contextHelper.GetAppOnlyContext(uriBuilder.ToString()))
{
//Read change token
var currentChangeToken = await TablesHelper.GetChangeTokenForListAsync(notification.Resource);
if(currentChangeToken == null)
{
log.Error($"No change token found for list {notification.Resource}. This is a NO GO. Please use the '/api/Setup' function.");
}
var listId = Guid.Parse(notification.Resource);
var changes = await CSOMHelper.GetListItemChangesAsync(ctx, listId, currentChangeToken.ChangeToken);
if(changes.Count > 0)
{
var lastChange = changes[changes.Count - 1];
//Save the last change token
var changeTokenValue = lastChange.ChangeToken.StringValue;
await TablesHelper.SaveChangeTokenForListAsync(new ListChangeToken(
notification.Resource,
changeTokenValue
));
await HandleChanges(ctx, changes);
}
}
}
log.Info($"C# Queue trigger function processed: {myQueueItem}");
}
The problem is that always, when using the "GetChangeTokenForListAsync" the Entity is received properly, but the .ChangeToken property is always null. It is also not visible when browsing with the Azure Storage Explorer. What am I doing wrong here?
The issue is related to the Azure Storage Emulator (V. 5.7.0.0). The same code works perfectly when working with the "live" Azure.

How to schedule a task in Fabric service using multiple actors?

I am getting errors while using this code, not able to use the switch case.
Is there any alternative method for using two or more actors
protected async override Task RunAsync(CancellationToken cancellationToken)
{
await base.RunAsync(cancellationToken);
var proxy = ActorProxy.Create<T>(new ActorId(0));
switch (proxy)
{
case IInvoiceMailActor a1:
await a1.RegisterReminder();
break;
case IActor2 a2:
await a2.RegisterReminder();
break;
default:
throw new NotImplementedException($"{GetType().FullName}.{nameof(RunAsync)}");
}
}
My Actor Class
[StatePersistence(StatePersistence.None)]
internal class Actor1 : Actor, IActor1, IRemindable {
public Actor1(ActorService actorService, ActorId actorId) :
base(actorService, actorId) { }
public async Task ReceiveReminderAsync(string reminderName, byte[] state, TimeSpan dueTime, TimeSpan period) {
var location = Directory.GetCurrentDirectory();
var current = DateTime.Now;
Thread.Sleep(2 * 60 * 1000);
using (var writer = File.AppendText("actor.txt")) {
await writer.WriteLineAsync("1 :: " + current.ToString() + " --> " + DateTime.Now.ToString());
}
}
public async Task RegisterReminder() {
try {
var previousRegistration = GetReminder("Reminder1");
await UnregisterReminderAsync(previousRegistration);
} catch (ReminderNotFoundException) { }
var reminderRegistration = await RegisterReminderAsync("Reminder1",
null, TimeSpan.FromMinutes(0), TimeSpan.FromMinutes(1));
}
}
internal class ScheduledActorService<T> : ActorService where T : IActor {
public ScheduledActorService(StatefulServiceContext context, ActorTypeInformation actorType) : base(context, actorType) { }
protected async override Task RunAsync(CancellationToken cancellationToken) {
await base.RunAsync(cancellationToken);
var proxy = ActorProxy.Create<T>(new ActorId(0));
switch (proxy) {
case IActor1 a1:
await a1.RegisterReminder();
break;
case IActor2 a2:
await a2.RegisterReminder();
break;
default:
throw new NotImplementedException($"{GetType().FullName}.{nameof(RunAsync)}");
}
}
}

How to check number of jobs remaining in job collection of Azure scheculer

I have implemented Azure scheduler by following below blog
http://fabriccontroller.net/a-complete-overview-to-get-started-with-the-windows-azure-scheduler/
Now I am able to create/update/delete jobs in the scheduler but how can I check whether the jobcollection is full? Basically I want to create another collection whenever my current jobcollection is full.
Sharing my code snippet
public class AzureSchedulerStorage : ISchedulerStorage
{
private CertificateCloudCredentials credentials;
//private CloudServiceManagementClient cloudServiceClient;
private string cloudServiceName;
// private IHalseyLogger logger;
public AzureSchedulerStorage(string cloudServiceName, CertificateCloudCredentials credentials)
{
this.cloudServiceName = cloudServiceName;
this.credentials = credentials;
// this.logger = logger;
}
public SchedulerOperationStatusResponse CreateJobCollection(string jobCollectionName)
{
var schedulerServiceClient = new SchedulerManagementClient(credentials);
var jobCollectionCreateParameters = new JobCollectionCreateParameters()
{
Label = jobCollectionName,
IntrinsicSettings = new JobCollectionIntrinsicSettings()
{
Plan = JobCollectionPlan.Standard,
Quota = new JobCollectionQuota()
{
MaxJobCount = 50,
MaxRecurrence = new JobCollectionMaxRecurrence()
{
Frequency = JobCollectionRecurrenceFrequency.Minute
}
}
}
};
var result = schedulerServiceClient.JobCollections.Create(this.cloudServiceName, jobCollectionName, jobCollectionCreateParameters);
return result;
}
public JobCollectionGetResponse GetJobCollection(string jobCollectionName)
{
var schedulerServiceClient = new SchedulerManagementClient(credentials);
var result = schedulerServiceClient.JobCollections.Get(this.cloudServiceName, jobCollectionName);
return result;
}
public void CreateOrUpdate(string jobcollectionName, string jobId, DateTime startDate)
{
var schedulerClient = new SchedulerClient(this.cloudServiceName, jobcollectionName, this.credentials);
var job = new JobCreateOrUpdateParameters()
{
Action = new JobAction()
{
Type = JobActionType.Https,
Request = new JobHttpRequest()
{
Body = "customer=sandrino&command=sendnewsletter",
Headers = new Dictionary<string, string>()
{
{ "Content-Type", "application/x-www-form-urlencoded" },
{ "x-something", "value123" }
},
Method = "POST",
Uri = new Uri("http://postcatcher.in/catchers/527af9acfe325802000001cb"),
}
},
StartTime = startDate,
};
var result = schedulerClient.Jobs.CreateOrUpdate(jobId, job);
}
}
}
After looking at the Scheduler API, it seems that there is no straightforward approach to get the length of a Job Collection.
Probably you can attempt to create a Job and if there is a quota error, then you can create a new Job Collection and add that job.

Resources