Azure Function DocumentClient Binding - azure

I am trying to upgrade my DocumentDB nuget package from 1.13 to 1.18
I am facing issue while upgrading my azure function which is having a DocumentClient binding.
In DocumentDB 1.13 the binding sections does not take :{Id} as an binding parameter and was creating the DocumentClient object perfectly . Whereas the DocumentDB 1.18 needs {Id} as an binding parameter [ Which i dont want , as I want to iterate through entire documents in the collection ]
my host.json binding before 1.18 was
{
"frameworks": {
"net46": {
"dependencies": {
"Dynamitey": "1.0.2",
"Microsoft.Azure.DocumentDB": "1.13.0",
"Microsoft.Azure.WebJobs.Extensions.DocumentDB": "1.0.0"
}
}
}
my local.settings.json had only
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "
DefaultEndpointsProtocol=xxxxx/xxxxx==;EndpointSuffix=core.windows.net",
"AzureWebJobsDashboard": "",
"AzureWebJobsDocumentDBConnectionString":
"AccountEndpoint=xxxxx/;AccountKey=xxxx==;",
}
}
and my azure function looks like
[FunctionName("DeleteAVFeedAuditData")]
public static async Task Run([TimerTrigger("0 0/1 * * * *")]TimerInfo myTimer, [DocumentDB]DocumentClient client,
TraceWriter log)
{
var c = client;
log.Info($"C# Timer trigger function executed at: {DateTime.Now}");
var value=ConfigurationManager.AppSettings["AVAuditFlushAfterDays"];
var collectionUri = UriFactory.CreateDocumentCollectionUri("AVFeedAudit", "AuditRecords");
//var documents = client.CreateDocumentQuery(collectionUri,"Select * from c where c.EndedAt");
//foreach (Document d in documents)
//{
// await client.DeleteDocumentAsync(d.SelfLink);
//}
}
}
Now when running the azure function with updated package of documentDB 1.18 it says to bind the {Id} which will give only the single document with the specified id . Whereas my requirment is same as the previous version of DocumentDB 1.13.
Please tell how can i get the entire documents binded with my DocumentClient with the new updated package.

According to your description, I checked this issue and reproduced this issue as follows:
Please tell how can i get the entire documents binded with my DocumentClient with the new updated package.
Based on your scenario, I would recommend you construct the DocumentClient by yourself instead of using the binding to DocumentClient for a workaround to achieve your purpose.
DocumentClient client = new DocumentClient(new Uri("https://<your-account-name>.documents.azure.com:443/"), "<your-account-key>");
And you could configure the serviceEndpoint and accountKey under your local.settings.json file just as the app setting AzureWebJobsStorage. Then you could use the following code for retrieving your setting value:
ConfigurationManager.AppSettings["your-appsetting-key"];
Moreover, here is a issue about constructing the DocumentClient from the connection string, you could refer to it.
UPDATE:
For 1.18, the following code could work as expected:
[FunctionName("Function1")]
public static void Run([TimerTrigger("*/10 * * * * *")]TimerInfo myTimer, [DocumentDB("brucedb01", "brucecoll01",ConnectionStringSetting = "AzureWebJobsDocumentDBConnectionString")] IEnumerable<dynamic> documents, TraceWriter log)
{
foreach (JObject doc in documents)
{
//doc.SelectToken("_self").Value<string>();
log.Info(doc.ToString());
}
}

Related

CosmosDBTrigger not getting invoked when CosmosDB binding is also present

I'm trying to make a CosmosDBTriggered function in my precompiled C# CI/CD deployed project.
Here's the function implementation, which gets deployed with no complaints. I've tried static and instance methods.
There are no errors but also no invocations as reported by the monitoring/Insights tools even though the watched Collection has items and changes while it's deployed.
The function says it's enabled and has a Cosmosdb trigger:
I've tried adding these dependencies individually, but no changes:
<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.CosmosDB" Version="3.0.10" />
<PackageReference Include="Microsoft.Azure.WebJobs.Script.ExtensionsMetadataGenerator" Version="1.2.1" />
This function DOES NOT appear in the Triggers of any CosmosDB Collection as I might expect, but I think that's possibly for a different kind of Trigger.
What configuration step am I missing??
UPDATE
When I comment out this [CosmosDB] DocumentClient binding (and anything that relies on it), the function is invoked. So I guess it's a problem with those bindings being used together?
Are you sure you set the CosmosDbConnection in azure function app on azure?
For example, this is my function app:
Function1.cs
using System;
using System.Collections.Generic;
using Microsoft.Azure.Documents;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Extensions.Logging;
namespace FunctionApp109
{
public static class Function1
{
[FunctionName("Function1")]
public static void Run([CosmosDBTrigger(
databaseName: "testbowman",
collectionName: "testbowman",
ConnectionStringSetting = "str",
CreateLeaseCollectionIfNotExists = true,
LeaseCollectionName = "lease")]IReadOnlyList<Document> input, ILogger log)
{
if (input != null && input.Count > 0)
{
log.LogInformation("Documents modified " + input.Count);
log.LogInformation("First document Id " + input[0].Id);
}
}
}
}
local.settings.json
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "DefaultEndpointsProtocol=https;AccountName=0730bowmanwindow;AccountKey=xxxxxx;EndpointSuffix=core.windows.net",
"FUNCTIONS_WORKER_RUNTIME": "dotnet",
"str": "AccountEndpoint=https://testbowman.documents.azure.com:443/;AccountKey=xxxxxx;"
}
}
But when deploy function app to azure, the local.settings.json will not be used, you need to set the connection string here:
The function app on azure will not tell you this thing, it just doesn't work.
Based on your updated post, the issue is that the Functions runtime is not initializing your Function because of some configuration issue in your bindings.
Normally, the actual error should be in your Application Insights logs.
The Cosmos DB output binding you are using is missing the collection and database properties. Checking the official samples: https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-cosmosdb-v2-input?tabs=csharp#http-trigger-get-multiple-docs-using-documentclient-c and assuming CosmosDBConnection is a variable that points to the setting name of a setting that contains the connection string:
[CosmosDB(
databaseName: "<your-db-name>",
collectionName: "<your-collection-name>",
ConnectionStringSetting = CosmosDBConnection)] DocumentClient dbClient,

How do I pass in the storage account connection string for a CosmosDBTrigger?

I'm trying to figure out the proper way to pass in a storage account connection string to a CosmosDBTrigger. I have a function that runs when there is a change on a CosmosDB container. This function copies image blobs from one container to another. If you look at the code below, I have commented out the line where I am trying to fine the storage account that I want to connect to. This function runs when that is commented out. It does not run when I have that un-commented. Why?
public static class Function1
{
[FunctionName("ImageCopier")]
public static async Task Run([CosmosDBTrigger(
databaseName: "MyDatabase",
collectionName: "Orders",
ConnectionStringSetting = "databaseConnection",
CreateLeaseCollectionIfNotExists = true,
LeaseDatabaseName = "TriggerLeases",
LeaseCollectionName = "TriggerLeases",
LeaseCollectionPrefix = "ImageCopier")]IReadOnlyList<Document> input,
//[StorageAccount("MyStorageAccount")]string storageConnectionString,
ILogger log)
{
I have MyStorageAccount defined in my local.settings.json file and I also have it in my Azure Function Configuration settings. I copied the connection string directly from the storage account keys panel.
When you set up a CosmosDB trigger, the information that is supplied in that trigger is specific to the trigger. If you need a setting or configuration not related to the trigger in your code, you can use the Environment.GetEnvironmentVariable method.
In your local environment, you can set these variables by editing the local.settings.json file, specifically the Values array. For example:
{
"IsEncrypted": false,
"Values": {
"JobUri": "https://yourapiendpointurl.com",
"BlobStorageConnectionString" : "the connection string",
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"FUNCTIONS_WORKER_RUNTIME": "dotnet"
}
}
In your method, you may grab that value like so:
public static class Function1
{
[FunctionName("ImageCopier")]
public static async Task Run([CosmosDBTrigger(
databaseName: "MyDatabase",
...
ILogger log)
{
var connectionString =
Environment.GetEnvironmentVariable("BlobStorageConnectionString");
}
}
The local.settings.json file will not be used when it's running in Azure.
I am not sure that when you publish the function if your local.settings.json file will migrate the settings to your Azure Function app's configuration, so I would check to make sure that your settings are in there after publishing.
Side note: Be carful when committing code to repos .. you don't want "secrets" in your repositories in case someone gets in to your repo and discovers it.
While you can access raw configuration values using GetEnvironmentVariable, a more robust/idiomatic approach with .NET in particular is to leverage the built-in dependency injection of configuration.
Using this, you can accept an IConfiguration or strongly-typed IOptions through the function's constructor and use the values in your code. For example:
public class Function1
{
private readonly IConfiguration configuration;
public Function1(IConfiguration configuration)
{
this.configuration = configuration;
}
[FunctionName("ImageCopier")]
public async Task Run([CosmosDBTrigger(/* trigger params */)] IReadOnlyList<Document> input)
{
var connectionString = configuration["MyStorageAccount"];
// Use connection string
}
}
You can take this further to inject services like an "ImageBlobService" into your function that have already been configured in a common Startup Configure method just like ASP.NET Core. That way the individual functions don't need to know anything about configuration and just ask for the relevant service to use.

Azure function not recognizing cloud storage and queue classes

I am following along with this guide: https://learn.microsoft.com/en-us/azure/storage/queues/storage-dotnet-how-to-use-queues and attempting to create a simple queue in a time triggered function. It is not recognizing CloudStorageAcount, CloudConfigurationManager, CloudQueueClient, etc.
Here is my run.csx file
using Microsoft.Azure;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Queue;
using System;
public static void Run(TimerInfo myTimer, TraceWriter log) {
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Retrieve storage account from connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create the queue client.
CloudQueueClient queueClient =
storageAccount.CreateCloudQueueClient();
// Retrieve a reference to a container.
CloudQueue queue = queueClient.GetQueueReference("myqueue");
// Create the queue if it doesn't already exist
queue.CreateIfNotExists();
}
Here is my project.json file:
{
"frameworks": {
"net45":{
"dependencies": {
"Microsoft.WindowsAzure.ConfigurationManager" : "3.2.3",
"Microsoft.WindowsAzure.Storage" : "8.0.0"
},
"frameworks": {
"netcoreapp1.0": {
"dependencies": {
"Microsoft.NETCore.App": {
"type": "platform",
"version": "1.0.0"
}
},
"imports": "dnxcore50"
}
}
}
}
}
The package Microsoft.WindowsAzure.Storage is referenced by Azure Functions itself by default. Remove the whole project.json file and add this line to the top of your function:
#r "Microsoft.WindowsAzure.Storage"
But you might not even need that. Azure Functions have higher-level API to work with Storage Queues, both for sending (output bindings) and receiving (triggers). Refer to Azure Queue storage bindings for Azure Functions.
One more advice: it's preferred to use precompiled functions deployed as class libraries compiled with Visual Studio or VS Code. This way it's much easier to manage dependencies and troubleshoot.

CosmosDB Azure Function binding

I am trying to use COSMOS DB with the Azure function
My function looks like
[FunctionName("DeleteAVFeedAuditData")]
public static async Task Run([TimerTrigger("0 0/1 * * * *")]TimerInfo myTimer, [DocumentDB]DocumentClient client,
TraceWriter log)
{
var c = client;
log.Info($"C# Timer trigger function executed at: {DateTime.Now}");
var value=ConfigurationManager.AppSettings["AVAuditFlushAfterDays"];
var collectionUri = UriFactory.CreateDocumentCollectionUri("AVFeedAudit", "AuditRecords");
//var documents = client.CreateDocumentQuery(collectionUri,"Select * from c where c.EndedAt");
//foreach (Document d in documents)
//{
// await client.DeleteDocumentAsync(d.SelfLink);
//}
}
}
and local.settings.json
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "
DefaultEndpointsProtocol=xxxxx/xxxxx==;EndpointSuffix=core.windows.net",
"AzureWebJobsDashboard": "",
"AzureWebJobsDocumentDBConnectionString":
"AccountEndpoint=xxxxx/;AccountKey=xxxx==;",
}
}
I had configured the connection
"AzureWebJobsDocumentDBConnectionString" with the cosmosdb connection string which contains the #"endpointurl+key"
When trying to run the application . its saying ID of the document is required, whereas on google it says it will create the document client object based on the connection string.
Please advise me what wrong i am doing in the binding. As my object is to create the document client through which i can query the document and delete the document.
Got the answer.
install-package microsoft.azure.documentdb -version 1.13 [ Note - not version 1.17]
go to local.settings.json and add ""AzureWebJobsDocumentDBConnectionString":
go to host.json and add the assembly reference
{
"frameworks": {
"net46": {
"dependencies": {
"Dynamitey": "1.0.2",
"Microsoft.Azure.DocumentDB": "1.13.0",
"Microsoft.Azure.WebJobs.Extensions.DocumentDB": "1.0.0"
}
}
}
and it will create the DocumentClient object by which you can perform any CRUD operation on it.
Thanks

Azure Functions Update DocumentDb document with Binder

A follow-question to my previous post about Azure Functions. I need to update a document in DocumentDB using the imperative binder (Binder). I don't really understand the documentation and I can't find any examples (I more or less find one kind of example which is the TextWriter one). The documentation says I can bind to "out T" by I find no examples of this.
Say that the document looks like this before running the function:
{
child: {
value: 0
}
}
And the functions looks like this:
var document = await binder.BindAsync<dynamic>(new DocumentDBAttribute("myDB", "myCollection")
{
ConnectionStringSetting = "my_DOCUMENTDB",
Id = deviceId
});
log.Info($"C# Event Hub trigger function processed a message: document: { document }");
document.value = 100;
document.child.value = 200;
log.Info($"Updated document: { document }");
According to the second logging row, the document isn't properly updated. The child is not updated (which existed when read from the store) and value is added. Either way, nothing is persisted. I've tried adding an Output in the function.json, but the compiler complains about it and the documentation states that you shouldn't have any.
What am I missing?
Mathew's sample (using DocumentClient) works, but I wanted to clarify the other way you can do it with Binder and an output binding.
You're bumping into two issues:
The Document implementation of dynamic appears to return a new object instance every time you request a child object. This isn't related to Functions, but explains why document.child.value = 200 doesn't work. You are updating one instance of child that is not actually attached to the document. I'll try to double-check this with DocumentDb folks, but that is confusing. One way around this is to request a JObject instead of a dynamic. My code below does that.
As #mathewc pointed out, Binder does not auto-update the document. We'll track that in the issue he filed. Instead, you can use an output binding with IAsyncCollector<dynamic> to update the document. Behind-the-scenes we'll call InsertOrReplaceDocumentAsync, which will update the document.
Here's a full sample that worked for me:
Code:
#r "Microsoft.Azure.WebJobs.Extensions.DocumentDB"
#r "Newtonsoft.Json"
using System;
using Newtonsoft.Json.Linq;
public static async Task Run(string input, Binder binder, IAsyncCollector<dynamic> collector, TraceWriter log)
{
string deviceId = "0a3aa1ff-fc76-4bc9-9fe5-32871d5f451b";
dynamic document = await binder.BindAsync<JObject>(new DocumentDBAttribute("ItemDb", "ItemCollection")
{
ConnectionStringSetting = "brettsamfunc_DOCUMENTDB",
Id = deviceId
});
log.Info($"C# Event Hub trigger function processed a message: document: { document }");
document.value = 100;
document.child.value = 200;
await collector.AddAsync(document);
log.Info($"Updated document: { document }");
}
binding:
{
"type": "documentDB",
"name": "collector",
"connection": "brettsamfunc_DOCUMENTDB",
"direction": "out",
"databaseName": "ItemDb",
"collectionName": "ItemCollection",
"createIfNotExists": false
}
Yes, I do believe there is an issue here, and I've logged a bug here in our repo to track it.
To work around this until we fix it, you can bind to and use the DocumentClient directly to perform the update, e.g.:
public static async Task Run(
string input, Binder binder, DocumentClient client, TraceWriter log)
{
var docId = "c31d48aa-d74b-46a3-8ba6-0d4c6f288559";
var document = await binder.BindAsync<JObject>(
new DocumentDBAttribute("ItemDb", "ItemCollection")
{
ConnectionStringSetting = "<mydb>",
Id = docId
});
log.Info("Item before: " + document.ToString());
document["text"] = "Modified!";
var docUri = UriFactory.CreateDocumentUri("ItemDb", "ItemCollection", docId);
await client.ReplaceDocumentAsync(docUri, document);
}
However, once you're using DocumentClient directly like this, it might turn out that you can just use it directly for all of your operations, your call. For example:
public static async Task Run(
string input, DocumentClient client, TraceWriter log)
{
var docId = "c31d48aa-d74b-46a3-8ba6-0d4c6f288559";
var docUri = UriFactory.CreateDocumentUri("ItemDb", "ItemCollection", docId);
var response = await client.ReadDocumentAsync(docUri);
dynamic document = response.Resource;
log.Info("Value: " + dynamic.text);
document.text = "Modified!";
await client.ReplaceDocumentAsync(docUri, document);
}

Resources