Azure Function, delay - azure

I have a CRM system, when a contact is added, I want to add them to an accounting system.
I have setup a webhook in the CRM system that passes the contact to an Azure Function. The Azure function connects to the accounting system API and creates them there.
There is a little other processing I need to do before the user can be added to the accounting system.
I need about a 5 minute delay after receiving the webhook before I can add the user to the accounting system.
I would rather not add a pause or delay statement in the Azure Function as there is a timeout limit, and also It's a consumption plan so I want each function to action quickly.
I am using Powershell core.
Is a Service Bus Queue the best way to do this?

You could use a Timer in a Durable Function for this. Then you won't need an extra component like a queue. A Durable Function is all you need. For example (warning: not compiled this):
Note: Durable Functions do support powershell but I don't ;-) So the code below is to understand the concept.
[FunctionName("Orchestration_HttpStart")]
public static async Task<HttpResponseMessage> HttpStart(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")] HttpRequestMessage req,
[DurableClient] IDurableOrchestrationClient starter,
ILogger log)
{
// Function input comes from the request content.
string content = await req.Content.ReadAsStringAsync();
string instanceId = await starter.StartNewAsync("Orchestration", content);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
return starter.CreateCheckStatusResponse(req, instanceId);
}
[FunctionName("Orchestration")]
public static async Task Run(
[OrchestrationTrigger] IDurableOrchestrationContext context)
{
var requestContent = context.GetInput<string>();
DateTime waitAWhile = context.CurrentUtcDateTime.Add(TimeSpan.FromMinutes(5));
await context.CreateTimer(waitAWhile, CancellationToken.None);
await context.CallActivityAsync("ProcessEvent", requestContent);
}
[FunctionName("ProcessEvent")]
public static string ProcessEvent([ActivityTrigger] string requestContent, ILogger log)
{
// Do something here with requestContent
return "Done!";
}
I would rather not add a pause or delay statement in the Azure Function as there is a timeout limit, and also It's a consumption plan so I want each function to action quickly.
The 5 minutes delay introduced by the timer won't count as active time so you won't run out of time on the consumption plan for those minutes.

Is a Service Bus Queue the best way to do this?
You can use it, but Azure Storage Queue is cheaper for your scenario.
What you can do is create a time triggered functions (* */5 * * * *) and will check for a message in a queue. If the time between the execution and the time the message was created is greater than minutes, then you process and complete the message, otherwise, don't complete the message and it will return to the queue for the next execution.

Related

Batch insert to Table Storage via Azure function

I have a following azure storage queue trigger azure function which is binded to azure table for the output.
[FunctionName("TestFunction")]
public static async Task<IActionResult> Run(
[QueueTrigger("myqueue", Connection = "connection")]string myQueueItem,
[Table("TableXyzObject"), StorageAccount("connection")] IAsyncCollector<TableXyzObject> tableXyzObjectRecords)
{
var tableAbcObject = new TableXyzObject();
try
{
tableAbcObject.PartitionKey = DateTime.UtcNow.ToString("MMddyyyy");
tableAbcObject.RowKey = Guid.NewGuid();
tableAbcObject.RandomString = myQueueItem;
await tableXyzObjectRecords.AddAsync(tableAbcObject);
}
catch (Exception ex)
{
}
return new OkObjectResult(tableAbcObject);
}
public class TableXyzObject : TableEntity
{
public string RandomString { get; set; }
}
}
}
I am looking for a way to read 15 messages from poisonqueue which is different than myqueue (queue trigger on above azure function) and batch insert it in to dynamic table (tableXyz, tableAbc etc) based on few conditions in the queue message. Since we have different poison queues, we want to pick up messages from multiple poison queues (name of the poison queue will be provided in the myqueue message). This is done to avoid to spinning up new azure function every time we have a new poison queue.
Following is the approach I have in my mind,
--> I might have to get 15 queue messages using queueClient (create new one) method - ReceiveMessages(15) of Azure.Storage.Queue package
--> And do a batch insert using TableBatchOperation class (cannot use output binding)
Is there any better approch than this?
Unfortunately, storage queues don't have a great solution for this. If you want it to be dynamic then the idea of implementing your own clients and table outputs is probably your best option. The one thing I would suggest changing is using a timer trigger instead of a queue trigger. If you are putting a message on your trigger queue every time you add something to the poison queue it would work as is, but if not a timer trigger ensures that poisoned messages are handled in a timely fashion.
Original Answer (incorrectly relating to Service Bus queues)
Bryan is correct that creating a new queue client inside your function isn't the best way to go about this. Fortunately, the Service Bus extension does allow batching. Unfortunately the docs haven't quite caught up yet.
Just make your trigger receive an array:
[QueueTrigger("myqueue", Connection = "connection")]string myQueueItem[]
You can set your max batch size in the host.json:
"extensions": {
"serviceBus": {
"batchOptions": {
"maxMessageCount": 15
}
}
}

How can I get the count of Azure function execution programmatically in c#?

I have timer trigger azure function which runs on every minute. Is it possible to get the count of how many time the function executed in my Azure function c# code at every time function execute?
PS: I need this count to generate a sequential counter.Also with HTTP trigger function too!
Is it possible with Application Insights -> metrics with function count filter. In a way we get this count in our code ?
Maybe you could implement it like the below picture, the variable will be shared. However if you restart your function, it will reload the variable time.
In case of the function restart, suppose you could set it with a storage queue to implement it, every time add 1 to the queue number and replace the queue.
public static async void Run(
[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req,
ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(Environment.GetEnvironmentVariable("AzureWebJobsStorage"));
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
CloudQueue queue = queueClient.GetQueueReference("time");
CloudQueueMessage message = queue.GetMessage();
log.LogInformation(message.AsString);
int num = int.Parse(message.AsString);
num++;
queue.DeleteMessage(message);
queue.AddMessage(new CloudQueueMessage(num.ToString()));
}
Hope this could help you, if you still have other problem please feel free to let me know.
It is not possible to get the execution times and there is no API to do that, you need to manually maintain the count in a datastore and fetch it using custom logic.

In Azure function that returns ServiceBus message, how do I conditionally returning message?

I have an azure function which returns a service bus message. However, I want to conditionally return a service bus message, instead of being forced to return the message every time.
here is an example
[FunctionName("ServiceBusOutput")]
[return: ServiceBus("myqueue", Connection = "ServiceBusConnection")]
public static string ServiceBusOutput([HttpTrigger] dynamic input, ILogger log)
{
log.LogInformation($"C# function processed: {input.Text}");
// check condition here, abort return completely
// Otherwise, return
return input.Text;
}
Said another way, I want to return a message on a service bus when certain conditions apply within the function code block. Is this possible?
One idea that does not work is to throw an exception. However, this just results in the message being placed into the DL queue. I want to completely abort the operation of returning the message on the service bus, and avoid DL.
Another idea that does not work is to simply execute
return;
But this results in compile-time error, which is sort of expected
"An object of a type convertible to 'MyReturnType1' is required"
I can think of a hack which I dont like, which is to return null, and handle the null later in the chain. But this is sort of dirty to me.
You could just bind ServiceBus as MessageSender type, then use the SendAsync() method to send the message.
The below is my test code, if the request name equals "george", it will send the name to the message queue.
public static class Function1
{
[FunctionName("Function1")]
public static async System.Threading.Tasks.Task RunAsync(
[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req,
[ServiceBus("myqueue", Connection = "ServiceBusConnection")] MessageSender messagesQueue,
ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
string name = req.Query["name"];
string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
dynamic data = JsonConvert.DeserializeObject(requestBody);
name = name ?? data?.name;
if (name.Equals("george")) {
byte[] bytes = Encoding.ASCII.GetBytes(name);
Message m1 = new Message();
m1.Body = bytes;
await messagesQueue.SendAsync(m1);
}
}
}
Suppose this is what you want, hope this could help you, if you still have other problem please feel free to let me know.
Instead of using the output binding available in Azure Function, you can send a message to the queue from a custom queue client created inside the function.
Posting the message based on a condition is not possible with the bindings.

Masstransit not creating Error queue for Azure Function event subscriber

We followed this example (http://masstransit-project.com/MassTransit/usage/azure-functions.html) to try to set up Azure Functions as Azure Service Bus event (topic) subscribers using MassTransit (for .Net CORE 2.1, Azure Functions 2.0).
When using Azure Webjobs this is as simple as using RabbitMQ, configure the publisher, let the subscriber configure and set up its queue, and have Masstransit automatically create one topic per event, redirect to queue and to "queue_error" after all retries have failed. You do not have to setup anything manually.
But with Azure Functions we seem to manually (through Service Bus Explorer or ARM templates) have to add the subscribers to the topic (which is created by the publisher on the first event it publishes) and the queues as well (though these don't even seem to be necessary, the events are handled directly by the consuming Azure Function topic subscribers.).
Maybe we are doing something wrong, I cannot see from the docs that MT will not, as it normally does, set up the subscriber andd creating queues when using Azure Functions. But it works, except for when the consumer throws an exception and after all setup retries have been executed. We simply do not get the event in the deadletter queue and the normally MT-generated error queue does not even get generated.
So how do we get MT to create the error queues, and MOVE the failed events there?
Our code:
[FunctionName("OrderShippedConsumer")]
public static Task OrderShippedConsumer(
[ServiceBusTrigger("xyz.events.order/iordershipped", "ordershippedconsumer-queue", Connection = "AzureServiceBus")] Message message,
IBinder binder,
ILogger logger,
CancellationToken cancellationToken,
ExecutionContext context)
{
var config = CreateConfig(context);
var handler = Bus.Factory.CreateBrokeredMessageReceiver(binder, cfg =>
{
var serviceBusEndpoint = Parse.ConnectionString(config["AzureServiceBus"])["Endpoint"];
cfg.CancellationToken = cancellationToken;
cfg.SetLog(logger);
cfg.InputAddress = new Uri($"{serviceBusEndpoint}{QueueName}");
cfg.UseRetry(x => x.Intervals(TimeSpan.FromSeconds(5)));
cfg.Consumer(() => new OrderShippedConsumer(cfg.Log, config));
});
return handler.Handle(message);
}
And the Consumer code:
public OrderShippedConsumer(ILog log, IConfigurationRoot config)
{
this.config = config;
this.log = log;
}
public async Task Consume(ConsumeContext<IOrderShipped> context)
{
// Handle the event
}
}

TaskCanceledException on azure function (Service bus trigger)

I have a Service Bus Trigger Azure function, which is triggered every time a topic receives a message.
Messages arrive at regular intervals, for example every 30 minutes. Between lots, no activity.
The function does nothing special, it does an asynchronous posting of the message via HttpClient. The function is regularly stopped with a TaskCanceledException.
The HttpClient is static
public static class SampleEventTrigger
{
private static DefaultHttpWebHook webHook = new DefaultHttpWebHook(new Uri("https://nonexistent.invalid/sampleWebHook"), "/event/sampleEvent");
[FunctionName("SampleEventTrigger")]
public static async Task Run(
[ServiceBusTrigger("sampleevent", "SampleEvent.Subs", AccessRights.Manage, Connection = GlobalConfiguration.ServiceBusConnection)]BrokeredMessage message,
TraceWriter log)
{
log.Info("launch sample event subscription");
try
{
var resp = await webHook.Post(message, log);
log.Info($"{resp.StatusCode}, {resp.ReasonPhrase}");
}
catch (Exception ex)
{
log.Error($"exception in webhook: {ex.Message}", ex);
throw;
}
}
}
If I raise it again just after, this time it passes.
Where does this exception come from? How do we avoid that?
Is it related to a timeout, or to launching the function that would be too slow?
My function is in Consumption mode.
Chances are that your Http call is timing out. Awaited Http calls that time out throw TaskCanceledException . I'm not sure what your DefaultHttpWebHook class does under the covers, but it should be using PostAsync in the Post method (which itself should have the Async suffix).
To verify you could catch TaskCanceledException and examine the inner exception. If you are still struggling, convert your code to non-async during local development to get a better handle on what's happening - it'll give you back a true exception rather than bubbling it up as a TCE.

Resources