Time trigger function with CosmosDB inputs - azure

I have created a time trigger function in azure functions and Added a CosmosDB input as shown below.
Below is the .csx file
#r "Microsoft.Azure.Documents.Client"
using System;
using Microsoft.Azure.Documents;
using Microsoft.Azure.Documents.Client;
public static async Task Run(TimerInfo myTimer, string[] inputDocument, TraceWriter log)
{
log.Info($"C# Timer trigger function executed at: {DateTime.Now}");
// string [] = bindings.inputDocument;
DocumentClient client;
}
How to get the input documents from cosmosDb into this csx file?
I am not familiar with C#, in javascript we will use var Data = context.bindings.DataInput;
How to do the same in c#?

You can use it like the below snippet
public static void Run(TimerInfo myTimer, IEnumerable<dynamic> documents)
{
foreach (var doc in documents)
{
// operate on each document
}
}
More examples in documentation
Questions from comments
If we have more than one cosmos Db input do we need to add as below ?
No if even if you have more than one inputs the IEnumerable<dynamic> documents is used. And you can iterate the list.
How to add if we have a cosmosDB output ?
The out object is used in this which points to your binding.
public static void Run(string myQueueItem, out object employeeDocument, ILogger log)
{
log.LogInformation($"C# Queue trigger function processed: {myQueueItem}");
dynamic employee = JObject.Parse(myQueueItem);
employeeDocument = new {
id = employee.name + "-" + employee.employeeId,
name = employee.name,
employeeId = employee.employeeId,
address = employee.address
};
}
More information on Output

Related

Azure Functions - Dynamic blob container in Blob binding

I need to add files to an Azure Storage Account using an Azure Function App with a Queuetrigger. But the container needs to be added dynamically. How is that possible?
public static async Task Run(
[QueueTrigger("activitylogevents", Connection = "StorageConnectionAppSetting")] Log activitylogevents,
Dynamic ==> [Blob("{dynamicc container}/dev/bronze/logs.json", FileAccess.Read)] Stream streamIn,
ILogger log)
{ ... Code doing stuff ... }
Thanks
You can make use of IBinder to dynamically define your BlobAttribute:
public static void MyFunction1(
[QueueTrigger("activitylogevents", Connection = "StorageConnectionAppSetting")] Log activitylogevents,
IBinder binderIn,
ILogger log)
{
var blobInAttribute = new BlobAttribute(myUrl, FileAccess.Read) { Connection = "StorageConnectionAppSetting" };
var streamIn = binderIn.Bind<Stream>(blobInAttribute);
//other code
}

Azure Durable entity functions to get & set timestamp info using timer trigger

I am trying to store the time stamp information in durable entities and retrieve it every time a trigger fired. Here is how I am doing it. I want the timestamp value set by the current execution to be available for the next trigger. But when the control reaches "string prevTS = await context.CallEntityAsync(entityId, "Get");" to goes back to start of the function again. What am I missing here.
I want execution to be sequential between the timer triggers.
'''
***public static class GetOpenDataRealtimeFeed
{
[FunctionName("GetOpenDataOrchestrator")]
public static async Task<List<string>> RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context, Binder binder, ILogger log)
{
var outputs = new List<string>();
var entityId = new EntityId(nameof(GetPrevLastModifiedTimestamp), "entityKey2");
string prevTS = await context.CallEntityAsync<string>(entityId, "Get");
string currentTS = DateTime.Now.ToString();
outputs.Add(currentTS);
outputs.Add(prevTS);
context.SignalEntity(entityId, "Set", currentTS);
return null;
}
//Durable entity function to get & set the last modified timestamp
[FunctionName("GetPrevLastModifiedTimestamp")]
public static void GetPrevLastModifiedTimestamp([EntityTrigger] IDurableEntityContext ctx)
{
switch (ctx.OperationName.ToLowerInvariant())
{
case "set":
ctx.SetState(ctx.GetInput<string>());
break;
case "get":
ctx.Return(ctx.GetState<string>());
break;
}
}
[FunctionName("getOpenDataRealtimeFeed_Trigger")]
public static async Task Run(
[TimerTrigger("%triggerTimer%")] TimerInfo myTimer,
[DurableClient] IDurableOrchestrationClient starter,
ILogger log)
{
// Function input comes from the request content.
string instanceId = await starter.StartNewAsync("GetOpenDataOrchestrator", null);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
}
}
}***
'''
I assume you are referring to the current line while debugging. If so, this is expected.
Since Durable Functions replays functions after awaiting a durable client call, execution won't ever go through the first round. Only the final replay will be "sequential" step overs.

How to fetch events from eventhub from a timer enabled azure function?

I am working with Azure Event Hubs. My requirement is to fetch the events from Azure Event Hub, using azure function on a daily basis. Basically my azure function will be timer enabled. It should be able to fetch the data from azure event hubs. Is there a mechanism for this ?
I am aware that we can trigger a azure function whenever an event is received at event hub. This i don't want as the function will execute n number of time. I want to just fetch the events on a daily basis.
You can still create a timer triggered function and create consumer clients in your code to receive events. See sample code below. Let me know if you have any questions.
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Azure.Messaging.EventHubs.Consumer;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
namespace FunctionApp7
{
public static class Function1
{
const string EventHubsConnectionString = "your connection string";
const string EventHubName = "evethub name";
const string ConsumerGroupName = "cgname";
[FunctionName("Function1")]
public static void Run([TimerTrigger("0 */5 * * * *")]TimerInfo myTimer, ILogger log)
{
log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
// You better dissover partitions by eventhub client. I am just hardcoding them here for now.
var partitions = new List<string> { "0", "1" };
var receiveTasks = new List<Task>();
foreach(var p in partitions)
{
receiveTasks.Add(ReadEventsFromPartition(p));
}
// Wait until all reads complete.
Task.WhenAll(receiveTasks);
}
public static async Task ReadEventsFromPartition(string partitionId)
{
await using (var consumer = new EventHubConsumerClient(ConsumerGroupName, EventHubsConnectionString, EventHubName))
{
EventPosition startingPosition = EventPosition.FromOffset(CheckpointStore.ReadOffsetForPartition(partitionId));
long lastOffset = -1;
await foreach (PartitionEvent receivedEvent in consumer.ReadEventsFromPartitionAsync(partitionId, startingPosition))
{
// Process received events here.
// Break if no events left.
if (receivedEvent.Data == null)
{
break;
}
lastOffset = receivedEvent.Data.Offset;
}
// Persist last event's offset so we can continue reading from this position next time function is triggered.
if (lastOffset != -1)
{
// Write offset into some durable store.
CheckpointStore.WriteOffsetForPartition(partitionId, lastOffset);
}
}
}
}
}

Azure Durable Function: JsonSerializationException by passing complex object from trigger to orchestrator

I have an azure function with EventHubTrigger:
[FunctionName("TradesDataProcessStarterEh")]
public static async Task TradesDataProcessStarterEh([EventHubTrigger("aeehrobotronapiintegrationdev", Connection = "EventHubConnectionString", ConsumerGroup = "$Default")]
EventData eventData, PartitionContext partitionContext, [OrchestrationClient] DurableOrchestrationClient starter, ILogger log)
{
if (partitionContext.PartitionId != "1")
return;
var orchestrationId = await starter.StartNewAsync("O_ProcessTradesFromEventHub", eventData);
await partitionContext.CheckpointAsync();
}
The orchestrator function is receiving then the eventData:
[FunctionName("O_ProcessTradesFromEventHub")]
public static async Task ProcessTradesFromEventHub([OrchestrationTrigger] DurableOrchestrationContext context,
ILogger log)
{
if (!context.IsReplaying)
Console.WriteLine("O_ProcessTradesFromEventHub is triggered");
var eventData = context.GetInput<EventData>();
//do stuff...
}
But by execution of context.GetInput() I get an exception:
Function 'O_ProcessTradesFromEventHub (Orchestrator)' failed with an error. Reason: Newtonsoft.Json.JsonSerializationException: Unable to find a constructor to use for type Microsoft.Azure.EventHubs.EventData. A class should either have a default constructor, one constructor with arguments or a constructor marked with the JsonConstructor attribute. Path 'Body', line 1, position 81.
I can think of 3 possible solutions that you can try:
1 - Wrap EventData in your own class with a constructor (possibly via inheritance?).
2 - Try casting to object, doubt this will work but, but worth a try as it's a simple fix.
3 - Build your own DTO (Data Transfer Object) to transform EventData to <your class> and then pass <your class> to the orchestration.
I think (3) is the cleanest solution and you have full control over what you pass, unfortunately it is likely the least performant and most tedious.
Good luck!
Use LINQ to JSON - a year later but hopefully it'll save somebody else some time.
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
public static async Task Run(
[OrchestrationTrigger] DurableOrchestrationContext context, ILogger log) {
var eventData = context.GetInput<JObject>();
log.LogInformation ($"Executing tasks with eventData = {eventData}");
string step = (string)eventData.SelectToken("Step");
log.LogInformation ($"Step = {step}");
}

Azure Functions and DocumentDB triggers

Is it possible to specify the DocumentDB is to fire triggers when writing to DocumentDB?
I have an Azure function that pulls JSON messages off a Service Bus Queue and puts them into DocumentDB like so:
using System;
using System.Threading.Tasks;
public static string Run(string myQueueItem, TraceWriter log)
{
log.Info($"C# ServiceBus queue trigger function processed message: {myQueueItem}");
return myQueueItem;
}
This inserts new documents into the database as they are added to the service bus queue, however I need DocumentDB to process these as they are added and add attachments. This cannot be done in the present setup and I would like to tell DocumentDB to fire a trigger.
I have tried something like this:
using System;
using System.Threading.Tasks;
public static string Run(string myQueueItem, TraceWriter log)
{
log.Info($"C# ServiceBus queue trigger function processed message: {myQueueItem}");
return "x-ms-documentdb-post-trigger-include: addDocument\n" + myQueueItem;
}
It doesn't work and gives me errors like this:
Exception while executing function:
Functions.ServiceBusQueueTriggerCSharp1. Microsoft.Azure.WebJobs.Host:
Error while handling parameter _return after function returned:.
Newtonsoft.Json: Unexpected character encountered while parsing value:
x. Path '', line 0, position 0.
I like this setup because I can saturate the queue with requests to add records and they just buffer until the database can deal with it, which deals with spikes in demand, but it allows data offload from the client machine as fast as the network can carry it and then the queue/database combination gets caught up when demand drops again.
You could refer to the following code sample to create document with the trigger enabled in Azure Functions.
using System;
using System.Threading.Tasks;
using Microsoft.Azure.Documents;
using Microsoft.Azure.Documents.Client;
public static void Run(string myQueueItem, TraceWriter log)
{
string EndpointUri = "https://{documentdb account name}.documents.azure.com:443/";
string PrimaryKey = "{PrimaryKey}";
DocumentClient client = new DocumentClient(new Uri(EndpointUri), PrimaryKey);
client.CreateDocumentAsync(UriFactory.CreateDocumentCollectionUri("{databaseid}", "{collenctionid}"), new MyChunk { MyProperty = "hello" },
new RequestOptions
{
PreTriggerInclude = new List<string> { "YourTriggerName" },
}).Wait();
log.Info($"C# ServiceBus queue trigger function processed message: {myQueueItem}");
}
public class MyChunk
{
public string MyProperty { get; set; }
}
Note: for using Microsoft.Azure.DocumentDB NuGet package in a C# function, please upload a project.json file to the function's folder in the function app's file system.
project.json
{
"frameworks": {
"net46":{
"dependencies": {
"Microsoft.Azure.DocumentDB": "1.13.1"
}
}
}
}
Besides, please make sure you have created triggers in your DocumentDB, for details about creating triggers, please refer to this article.

Resources