I'm trying to make a CosmosDBTriggered function in my precompiled C# CI/CD deployed project.
Here's the function implementation, which gets deployed with no complaints. I've tried static and instance methods.
There are no errors but also no invocations as reported by the monitoring/Insights tools even though the watched Collection has items and changes while it's deployed.
The function says it's enabled and has a Cosmosdb trigger:
I've tried adding these dependencies individually, but no changes:
<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.CosmosDB" Version="3.0.10" />
<PackageReference Include="Microsoft.Azure.WebJobs.Script.ExtensionsMetadataGenerator" Version="1.2.1" />
This function DOES NOT appear in the Triggers of any CosmosDB Collection as I might expect, but I think that's possibly for a different kind of Trigger.
What configuration step am I missing??
UPDATE
When I comment out this [CosmosDB] DocumentClient binding (and anything that relies on it), the function is invoked. So I guess it's a problem with those bindings being used together?
Are you sure you set the CosmosDbConnection in azure function app on azure?
For example, this is my function app:
Function1.cs
using System;
using System.Collections.Generic;
using Microsoft.Azure.Documents;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Extensions.Logging;
namespace FunctionApp109
{
public static class Function1
{
[FunctionName("Function1")]
public static void Run([CosmosDBTrigger(
databaseName: "testbowman",
collectionName: "testbowman",
ConnectionStringSetting = "str",
CreateLeaseCollectionIfNotExists = true,
LeaseCollectionName = "lease")]IReadOnlyList<Document> input, ILogger log)
{
if (input != null && input.Count > 0)
{
log.LogInformation("Documents modified " + input.Count);
log.LogInformation("First document Id " + input[0].Id);
}
}
}
}
local.settings.json
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "DefaultEndpointsProtocol=https;AccountName=0730bowmanwindow;AccountKey=xxxxxx;EndpointSuffix=core.windows.net",
"FUNCTIONS_WORKER_RUNTIME": "dotnet",
"str": "AccountEndpoint=https://testbowman.documents.azure.com:443/;AccountKey=xxxxxx;"
}
}
But when deploy function app to azure, the local.settings.json will not be used, you need to set the connection string here:
The function app on azure will not tell you this thing, it just doesn't work.
Based on your updated post, the issue is that the Functions runtime is not initializing your Function because of some configuration issue in your bindings.
Normally, the actual error should be in your Application Insights logs.
The Cosmos DB output binding you are using is missing the collection and database properties. Checking the official samples: https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-cosmosdb-v2-input?tabs=csharp#http-trigger-get-multiple-docs-using-documentclient-c and assuming CosmosDBConnection is a variable that points to the setting name of a setting that contains the connection string:
[CosmosDB(
databaseName: "<your-db-name>",
collectionName: "<your-collection-name>",
ConnectionStringSetting = CosmosDBConnection)] DocumentClient dbClient,
Related
I'm trying to figure out the proper way to pass in a storage account connection string to a CosmosDBTrigger. I have a function that runs when there is a change on a CosmosDB container. This function copies image blobs from one container to another. If you look at the code below, I have commented out the line where I am trying to fine the storage account that I want to connect to. This function runs when that is commented out. It does not run when I have that un-commented. Why?
public static class Function1
{
[FunctionName("ImageCopier")]
public static async Task Run([CosmosDBTrigger(
databaseName: "MyDatabase",
collectionName: "Orders",
ConnectionStringSetting = "databaseConnection",
CreateLeaseCollectionIfNotExists = true,
LeaseDatabaseName = "TriggerLeases",
LeaseCollectionName = "TriggerLeases",
LeaseCollectionPrefix = "ImageCopier")]IReadOnlyList<Document> input,
//[StorageAccount("MyStorageAccount")]string storageConnectionString,
ILogger log)
{
I have MyStorageAccount defined in my local.settings.json file and I also have it in my Azure Function Configuration settings. I copied the connection string directly from the storage account keys panel.
When you set up a CosmosDB trigger, the information that is supplied in that trigger is specific to the trigger. If you need a setting or configuration not related to the trigger in your code, you can use the Environment.GetEnvironmentVariable method.
In your local environment, you can set these variables by editing the local.settings.json file, specifically the Values array. For example:
{
"IsEncrypted": false,
"Values": {
"JobUri": "https://yourapiendpointurl.com",
"BlobStorageConnectionString" : "the connection string",
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"FUNCTIONS_WORKER_RUNTIME": "dotnet"
}
}
In your method, you may grab that value like so:
public static class Function1
{
[FunctionName("ImageCopier")]
public static async Task Run([CosmosDBTrigger(
databaseName: "MyDatabase",
...
ILogger log)
{
var connectionString =
Environment.GetEnvironmentVariable("BlobStorageConnectionString");
}
}
The local.settings.json file will not be used when it's running in Azure.
I am not sure that when you publish the function if your local.settings.json file will migrate the settings to your Azure Function app's configuration, so I would check to make sure that your settings are in there after publishing.
Side note: Be carful when committing code to repos .. you don't want "secrets" in your repositories in case someone gets in to your repo and discovers it.
While you can access raw configuration values using GetEnvironmentVariable, a more robust/idiomatic approach with .NET in particular is to leverage the built-in dependency injection of configuration.
Using this, you can accept an IConfiguration or strongly-typed IOptions through the function's constructor and use the values in your code. For example:
public class Function1
{
private readonly IConfiguration configuration;
public Function1(IConfiguration configuration)
{
this.configuration = configuration;
}
[FunctionName("ImageCopier")]
public async Task Run([CosmosDBTrigger(/* trigger params */)] IReadOnlyList<Document> input)
{
var connectionString = configuration["MyStorageAccount"];
// Use connection string
}
}
You can take this further to inject services like an "ImageBlobService" into your function that have already been configured in a common Startup Configure method just like ASP.NET Core. That way the individual functions don't need to know anything about configuration and just ask for the relevant service to use.
I have an Azure WebJobs with QueueTrigger:
public void ProcessTestQueueByTrigger(
[QueueTrigger("test-queue", Connection = "MyCustomStorageConnection")] string queueMessage,
int dequeueCount,
TextWriter log)
{
var message = $"ProcessTestQueueByTrigger executed: {queueMessage}, dequeue: {dequeueCount}";
log.WriteLine(message);
}
}
Where MyCustomStorageConnection is not the same as default jobs connection (that is why I define it in the QueueTrigger attribute). I also have two local files which defines settings for dev and prod environment: appsettings.json and appsettings.Production.json.
When I start the WebJobs, I'm reading the valid configuration and settings to the config object
var configuration = new JobHostConfiguration
{
DashboardConnectionString = config.GetConnectionString("AzureWebJobsDashboard"),
StorageConnectionString = config.GetConnectionString("AzureWebJobsStorage"),
};
and the valid connection stored in config.GetConnectionString("MyCustomStorageConnection") but there is no place to set it with the host. And whenever host is started, the QueueTrigger is reading the value from the appsettings.json and totally ignores the appsettings.Production.json.
How can I force QueueTrigger to use the proper config or just define the value for the QueueTrigger connection string?
You can achieve this easily by setting environment variable.
The following are the steps, on windows 10 with visual studio 2017.
1.Create a .NET core webjob(you can follow this doc if not familiar), and the completed code as below:
Program.cs:
using System;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
namespace MySampleWebjob
{
class Program
{
static void Main(string[] args)
{
//since we have 2 .json files, we can control which .json file is to be used by setting a Environment variable. And then read the value here.
var environment = Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT");
Console.WriteLine($"the environment is: "+ environment);
var builder = new HostBuilder();
builder.ConfigureWebJobs(b =>
{
b.AddAzureStorageCoreServices();
b.AddAzureStorage();
})
.ConfigureAppConfiguration((hostContext, configApp) => {
configApp.AddJsonFile("appsettings.json", optional: true,reloadOnChange:true);
configApp.AddJsonFile($"appsettings.{environment}.json", optional: true, reloadOnChange: true);
configApp.AddEnvironmentVariables();
})
.ConfigureLogging((context, b) =>
{
b.AddConsole();
})
;
var host = builder.Build();
using (host)
{
host.Run();
}
}
}
}
Functions.cs:
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
namespace MySampleWebjob
{
public class Functions
{
public static void ProcessQueueMessage([QueueTrigger("myqueue-items")] string message, ILogger logger)
{
logger.LogInformation(message);
}
}
}
Then add 2 .json files in the project: appsettings.Production.json and appsettings.json. The structure of the 2 .json files are the same, but have different storage connectiong strings.
appsettings.json:
{
"AzureWebJobsStorage": "DefaultEndpointsProtocol=https;AccountName=xxx;AccountKey=xx;EndpointSuffix=core.windows.net"
}
appsettings.Production.json:
{
"AzureWebJobsStorage": "Another azure storage connection string"
}
Note that in visual studio, right click each of the json file -> properties -> set "Copy to Output Directory" to "Copy if newer".
At last, set environment variable via cmd or UI. The cmd command is setx ASPNETCORE_ENVIRONMENT "Production", then restart the visual studio if you want to use the environment variable.
So if you want to use the storage connection string from appsettings.json, don't need to set the environment variable. If you want to use the connection string from appsettings.Production.json, need to set the environment variable.
Note that, if you set an environment variable or remove an environment variable, remember to restart the visual studio to take effect.
I'm using the following code to connect. I can connect to other Azure Resources ok.
But for one resource I get the following error: URL and Key are correct.
{"A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond"}
The Code is as follows
_searchClient = new SearchServiceClient(searchServiceName, new
SearchCredentials(apiKey));
_httpClient.DefaultRequestHeaders.Add("api-key", apiKey);
_searchServiceEndpoint = String.Format("https://{0}.{1}",
searchServiceName, _searchClient.SearchDnsSuffix);
bool result = RunAsync().GetAwaiter().GetResult();
Any ideas? thx in advance? How can I troubleshoot this?
I will show how this is done in c#
you will need a appsettings.json
you will need this code in the program.cs file
there are a lot of other files in the example from the document
that you may need to use , learn and edit for ur usecase
When working in c# and azure, always know what is unique about the file structured your solution first. This is why we build the examples from the docs as we learn the solution. Next we must study the different blocks of code that when executed deliver one feature or functionality to the solution as a whole.
appsettings.json
{
"SearchServiceName": "[Put your search service name here]",
"SearchIndexName": "hotels",
"SearchServiceAdminApiKey": "[Put your primary or secondary Admin API key here]",
"SearchServiceQueryApiKey": "[Put your primary or secondary Query API key here]"
}
Program.cs
namespace AzureSearch.SDKHowTo
{
using System;
using System.Linq;
using System.Threading;
using Microsoft.Azure.Search;
using Microsoft.Azure.Search.Models;
using Microsoft.Extensions.Configuration;
using Microsoft.Spatial;
// This sample shows how to delete, create, upload documents and query an index
static void Main(string[] args)
{
IConfigurationBuilder builder = new ConfigurationBuilder().AddJsonFile("appsettings.json");
IConfigurationRoot configuration = builder.Build();
SearchServiceClient serviceClient = CreateSearchServiceClient(configuration);
string indexName = configuration["SearchIndexName"];
Console.WriteLine("{0}", "Deleting index...\n");
DeleteIndexIfExists(indexName, serviceClient);
Console.WriteLine("{0}", "Creating index...\n");
CreateIndex(indexName, serviceClient);
ISearchIndexClient indexClient = serviceClient.Indexes.GetClient(indexName);
Console.WriteLine("{0}", "Uploading documents...\n");
UploadDocuments(indexClient);
ISearchIndexClient indexClientForQueries = CreateSearchIndexClient(indexName, configuration);
RunQueries(indexClientForQueries);
Console.WriteLine("{0}", "Complete. Press any key to end application...\n");
Console.ReadKey();
}
private static SearchServiceClient CreateSearchServiceClient(IConfigurationRoot configuration)
{
string searchServiceName = configuration["SearchServiceName"];
string adminApiKey = configuration["SearchServiceAdminApiKey"];
SearchServiceClient serviceClient = new SearchServiceClient(searchServiceName, new SearchCredentials(adminApiKey));
return serviceClient;
}
private static SearchIndexClient CreateSearchIndexClient(string indexName, IConfigurationRoot configuration)
{
string searchServiceName = configuration["SearchServiceName"];
string queryApiKey = configuration["SearchServiceQueryApiKey"];
SearchIndexClient indexClient = new SearchIndexClient(searchServiceName, indexName, new SearchCredentials(queryApiKey));
return indexClient;
}
private static void DeleteIndexIfExists(string indexName, SearchServiceClient serviceClient)
{
if (serviceClient.Indexes.Exists(indexName))
{
serviceClient.Indexes.Delete(indexName);
}
}
private static void CreateIndex(string indexName, SearchServiceClient serviceClient)
{
var definition = new Index()
{
Name = indexName,
Fields = FieldBuilder.BuildForType<Hotel>()
};
serviceClient.Indexes.Create(definition);
}}
Azure concepts to learn
How and why we create azure clients
Why do we use appsettings.json
What is some example file structures for azure search solutions
What coding lanague do you want to use to build that solutio
do u want to use the azure sdk
How to find and create api keys
C# concepts to learn
What is an interface and how do you use it
How to import one file in the file structure into another
How the main function works
How to call variables in to a function
How to call a function with a function
How to write server side code vs client side code
How to deploy c# code to azure
What version of c# are u using What’s is asp.net and what version will u use
What is asp.net core and what version will u use
As u can see azure and c# have a high learning curve.
Luckily you have stack overflow and documentation to research all of the above questions and more:)
For how u would troubleshoot...what I do is research each block of code in the documentation example and run all of the code locally. Then I test each block of code one at a time. Ur always testing data flowing thought the block of code. So you can just console log the result of a block a code by creating a test varable and print that varable to the console.
Because each block of Code represents one feature or functionality, each test will output either a pass or fail delivery of that feature or functionality. Thus you can design functionality, implement that design and create a test for new Feature.
The user inputs the CRON expression from an interface. The function app should update the appsettings to reflect the user input.
My current approach
The TimerTrigger function with schedule appsetting
[FunctionName("Cleanup")]
public static async Task Run([TimerTrigger("%schedule%")]TimerInfo myTimer, ILogger log)
{
// Get the connection string from app settings and use it to create a connection.
var str = Environment.GetEnvironmentVariable("db_connection");
log.LogInformation($"db_connection : {str}");
}
Setting the schedule appsetting via environment variable
[FunctionName("SetConfig")]
public static async Task<HttpResponseMessage> SetConfig([HttpTrigger(AuthorizationLevel.Function, "post", Route = null)]HttpRequestMessage req)
{
HistoryLogDeleteConfigDto data = await req.Content.ReadAsAsync<HistoryLogDeleteConfigDto>();
Environment.SetEnvironmentVariable("schedule", data.Schedule);
return req.CreateResponse(HttpStatusCode.OK);
}
local.settings.json file
"Values": {
"db_connection": "Server=DESKTOP-DFJ3PBT;Database=CovalentLogger;Trusted_Connection=True;MultipleActiveResultSets=true",
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"FUNCTIONS_WORKER_RUNTIME": "dotnet",
"schedule": "*/20 * * * * *"
}
POSTMAN request body to update the schedule appsetting
{
"Schedule": "*/30 * * * * *"
}
But no luck. After sending the request from postman to update the setting, if I access the azure portal function app setting I still can see the old value.
But if I query the environmnet variable like below
Environment.GetEnvironmentVariable("schedule", EnvironmentVariableTarget.Process)
I can see the new expression. But in Azure portal function appsetting it stills the old value. So the job still executes based on the old schedule.
Where have I gone wrong?
Thank you
I don't think updating the Environment variable like that will work as the the schedule is only being read at the initialization of the Function - or when the app setting is updated on the app service. However, it should still be fairly doable. This does basically exactly what you are looking for: https://stackoverflow.com/a/50116234/1537195
Just package this in your HTTP-triggered Function (and I'd probably use Managed Identity) and you are good to go.
I am trying to upgrade my DocumentDB nuget package from 1.13 to 1.18
I am facing issue while upgrading my azure function which is having a DocumentClient binding.
In DocumentDB 1.13 the binding sections does not take :{Id} as an binding parameter and was creating the DocumentClient object perfectly . Whereas the DocumentDB 1.18 needs {Id} as an binding parameter [ Which i dont want , as I want to iterate through entire documents in the collection ]
my host.json binding before 1.18 was
{
"frameworks": {
"net46": {
"dependencies": {
"Dynamitey": "1.0.2",
"Microsoft.Azure.DocumentDB": "1.13.0",
"Microsoft.Azure.WebJobs.Extensions.DocumentDB": "1.0.0"
}
}
}
my local.settings.json had only
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "
DefaultEndpointsProtocol=xxxxx/xxxxx==;EndpointSuffix=core.windows.net",
"AzureWebJobsDashboard": "",
"AzureWebJobsDocumentDBConnectionString":
"AccountEndpoint=xxxxx/;AccountKey=xxxx==;",
}
}
and my azure function looks like
[FunctionName("DeleteAVFeedAuditData")]
public static async Task Run([TimerTrigger("0 0/1 * * * *")]TimerInfo myTimer, [DocumentDB]DocumentClient client,
TraceWriter log)
{
var c = client;
log.Info($"C# Timer trigger function executed at: {DateTime.Now}");
var value=ConfigurationManager.AppSettings["AVAuditFlushAfterDays"];
var collectionUri = UriFactory.CreateDocumentCollectionUri("AVFeedAudit", "AuditRecords");
//var documents = client.CreateDocumentQuery(collectionUri,"Select * from c where c.EndedAt");
//foreach (Document d in documents)
//{
// await client.DeleteDocumentAsync(d.SelfLink);
//}
}
}
Now when running the azure function with updated package of documentDB 1.18 it says to bind the {Id} which will give only the single document with the specified id . Whereas my requirment is same as the previous version of DocumentDB 1.13.
Please tell how can i get the entire documents binded with my DocumentClient with the new updated package.
According to your description, I checked this issue and reproduced this issue as follows:
Please tell how can i get the entire documents binded with my DocumentClient with the new updated package.
Based on your scenario, I would recommend you construct the DocumentClient by yourself instead of using the binding to DocumentClient for a workaround to achieve your purpose.
DocumentClient client = new DocumentClient(new Uri("https://<your-account-name>.documents.azure.com:443/"), "<your-account-key>");
And you could configure the serviceEndpoint and accountKey under your local.settings.json file just as the app setting AzureWebJobsStorage. Then you could use the following code for retrieving your setting value:
ConfigurationManager.AppSettings["your-appsetting-key"];
Moreover, here is a issue about constructing the DocumentClient from the connection string, you could refer to it.
UPDATE:
For 1.18, the following code could work as expected:
[FunctionName("Function1")]
public static void Run([TimerTrigger("*/10 * * * * *")]TimerInfo myTimer, [DocumentDB("brucedb01", "brucecoll01",ConnectionStringSetting = "AzureWebJobsDocumentDBConnectionString")] IEnumerable<dynamic> documents, TraceWriter log)
{
foreach (JObject doc in documents)
{
//doc.SelectToken("_self").Value<string>();
log.Info(doc.ToString());
}
}