We are using a Cosmos DB input trigger for Azure Functions and are unable to use environment variables in our function.json like other input triggers for sticky slot settings.
Has anyone else had success using environment variables in function.json using Cosmos DB trigger type?
function.json
{
"bindings": [
{
"type": "cosmosDBTrigger",
"name": "inputDocs",
"direction": "in",
"leaseDatabaseName": "leases",
"leaseCollectionName": "MyCosmosCollection-myFunction",
"connectionStringSetting": "CosmosTriggers-SourceAdapter",
"databaseName": "%cosmos-triggers-database-name%",
"collectionName": "MyCosmosCollection",
"createLeaseCollectionIfNotExists": true
}
],
"disabled": false
}
Azure Function Error
Function ($myFunction) Error: The listener for function 'Functions.myFunction' was unable to start.
Microsoft.Azure.WebJobs.Extensions.DocumentDB: Either the source collection 'MyCosmosCollection' (in database '%cosmos-triggers-database-name%') or the lease collection 'MyCosmosCollection-myFunction' (in database 'leases') does not exist. Both collections must exist before the listener starts. To automatically create the lease collection, set 'CreateLeaseCollectionIfNotExists' to 'true'. Microsoft.Azure.Documents.Client: Message: {"Errors":["Resource Not Found"]}
ActivityId: b00f7802-fccb-47eb-972d-0bd70ec896c1, Request URI: rntbd://bn6prdddc05-docdb-1.documents.azure.com:14639/apps/6628b461-75d4-4e4a-9897-ada4076dc30c/services/1b0fc27a-de15-45cf-a1b2-ebfce044d1e2/partitions/34cfee55-54aa-4e31-81f4-08cf1bfdf62f/replicas/131523094168492638s/.
Session Id: 092ccb7ce9104407bf56c26a5cc8b119
Timestamp: 2017-10-31T19:13:03.914Z
Related
I can't quite seem to get the output bindings to enable a file to be saved to blob storage. I have created an Azure Function using Python, that uses a CosmosDB Change Feed trigger. I need to save that document to blob storage.
I've set-up the function.json file as follows:
{
"scriptFile": "__init__.py",
"bindings": [
{
"type": "cosmosDBTrigger",
"name": "documents",
"direction": "in",
"leaseCollectionName": "leases",
"connectionStringSetting": "cosmos_dev",
"databaseName": "MyDatabase",
"collectionName": "MyCollection",
"createLeaseCollectionIfNotExists": "true"
},
{
"type": "blob",
"direction": "out",
"name": "outputBlob",
"path": "raw/changefeedOutput/{blobname}",
"connection": "blobStorageConnection"
}
]
}
So the trigger will get a documents like the following:
{ "id": "documentId-12345",
other sections here
"entity": "customer"
}
In the init.py file I have the base code of
def main(documents: func.DocumentList) -> func.Document:
logging.info(f"CosmosDB trigger executed!")
for doc in documents:
blobName = doc['id'] + '.json'
blobFolder= doc['entity']
blobData = doc.to_json()
I think i need to add in the def something like 'outputBlob: func.Out' but unsure how to proceed
Looking at the examples on github
https://github.com/yokawasa/azure-functions-python-samples/tree/master/v2functions/blob-trigger-watermark-blob-out-binding
it look like i have to
outputBlob.set(something)
So i'm looking for how to set up the def part and send the blob to the location that i've set from the data in the cosmosdb document.
I have tried the following:
def main(documents: func.DocumentList, outputBlob: func.Out[str] ) -> func.Document:
logging.info(f"CosmosDB trigger executed!")
for doc in documents:
blobName = doc['id'] + '.json'
outputBlob.set(blobName)
and get the result:
CosmosDB trigger executed!
Executed 'Functions.CosmosTrigger_py' (Failed, Id=XXXXX)
System.Private.CoreLib: Exception while executing function: Functions.CosmosTrigger_py. Microsoft.Azure.WebJobs.Host: No value for named parameter 'blobname'.
I could just call the connection stuff from the os.enviro, and get the connection string that way, I think and use the standard create_blob_from_text, with location, name and blob data,
block_blob_service.create_blob_from_text(blobLocation, blobName, formattedBlob)
Any pointers would be great
I user those and run Azure Functions at local environment.
Azure Functions Core Tools (2.0.3)
Function Runtime Version:2.0.12115.0
azurite#2.7.0
I try as Microsoft document says.
Here is functions.json
{
"bindings": [
{
"name": "input",
"type": "httpTrigger",
"direction": "in"
},
{
"tableName": "Person",
"connection": "MyStorageConnectionAppSetting",
"name": "tableBinding",
"type": "table",
"direction": "out"
}
],
"disabled": false
}
Here is local.settings.json
{
"IsEncrypted": false,
"Values": {
"FUNCTIONS_WORKER_RUNTIME": "node",
"MyStorageConnectionAppSetting": "UseDevelopmentStorage=true"
}
}
Here is index.js
module.exports = function (context) {
context.bindings.tableBinding = [];
for (var i = 1; i < 10; i++) {
context.bindings.tableBinding.push({
PartitionKey: "Test",
RowKey: i.toString(),
Name: "Name " + i
});
}
context.done();
};
Installed extensions with this.
$ func extensions install -p Microsoft.Azure.WebJobs.Extensions.Storage --version 3.0.0
Run functions from mac terminal, Send http request, I got this error.
System.Private.CoreLib: Exception while executing function:
Functions.test. Microsoft.Azure.WebJobs.Host: Error while handling
parameter _binder after function returned:.
Microsoft.Azure.WebJobs.Extensions.Storage: Not Implemented (HTTP
status code 501: . ). Microsoft.WindowsAzure.Storage: Not Implemented.
Error from Table Storage
POST /devstoreaccount1/$batch 501 0.980 ms - 45
Any help?
UseDevelopmentStorage=true represents function is set to use Storage emulator, which provides a local environment that emulates the Azure Storage Blob, Queue and Table services for development purposes. But unfortunately it's only available on OS Windows, so you got Not Implemented on Mac.
Workaround is to use a real world Storage account, follow the tutorial to get the Connection string, if you don't have a Storage account you may have to create one first.
One more suggestion is install latest Azure Functions Core Tools, right now it's 2.1.725.
I have a activity function that should store message in Blob storage.I can overwrite a file in blob storage but i need to store data in different name.how to do that? Azure function doesn't support dynamic binding in nodejs.
Find one workaround, see whether it's useful.
Along with blob output binding, there's an activity trigger to receive message msg, we can put self-defined blob name in msg for blob binding path to consume.
In your orchestrator function which calls Activity function
yield context.df.callActivity("YourActivity", {'body':'messagecontent','blobName':'myblob'});
Then Activity function code should be modified
context.bindings.myOutputBlob = context.bindings.msg.body;
And its function.json can use blobName as expected
{
"bindings": [
{
"name": "msg",
"type": "activityTrigger",
"direction": "in"
},
{
"name":"myOutputBlob",
"direction": "out",
"type": "blob",
"connection": "AzureWebJobsStorage",
"path": "azureblob/{blobName}"
}
],
"disabled": false
}
I am trying to run an azure function locally on my laptop using the Azure Functions Core Tools. Notice that this function is configured as a cosmosDB trigger
I was partially inspired by the instructions in this tutorial
I started by creating a function called MyFirstFunction with the following commands (and inserting the required inputs when prompted):
func init
func start
My generated javascript function is (the same the Azure portal creates for the same kind of template function):
module.exports = function (context, documents) {
if (!!documents && documents.length > 0) {
context.log('Document Id: ', documents[0].id);
}
context.done();
}
My generated function.json is:
{
"bindings":
[
{
"type": "cosmosDBTrigger",
"name": "documents",
"direction": "in",
"leaseCollectionName": "leases"
}
]
}
My generated local.settings.json is
{
"IsEncrypted": false,
"Values": {
"FUNCTIONS_WORKER_RUNTIME": "node",
"AzureWebJobsStorage": "UseDevelopmentStorage=true"
}
}
Given this setup I try to run the function by executing:
func host start
Everything runs fine in the console output until the error message:
Unable to configure binding 'documents' of type 'cosmosDBTrigger'. This may indicate invalid function.json properties. Can't figure out which ctor to call.
What I missing? I was supposed to trigger the function through an http POST to:
http://localhost:{port}/admin/functions/{function_name}
as explained in the tutorial linked above (being this function a cosmosDB trigger) but the function cannot even be loaded after this error.
What am I missing to run a cosmosDB trigger locally?
Many thanks.
The problem is your local.settings.json and function.json lack necessary configuration of cosmosdb connection string.
See cosmosdb trigger document.
function.json
{
"bindings":
[
{
"type": "cosmosDBTrigger",
"name": "documents",
"direction": "in",
"leaseCollectionName": "leases",
// Missed in your code
"connectionStringSetting": "CosmosDBConnectionString",
"databaseName": "<Get db name>",
"collectionName": "<Get coll name>",
"createLeaseCollectionIfNotExists": true
}
]
}
local.settings.json
{
"IsEncrypted": false,
"Values": {
"FUNCTIONS_WORKER_RUNTIME": "node",
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
//Missed in your code
"CosmosDBConnectionString":"<Get connection string from Azure portal>"
}
}
Note that in latest version of function core tools(2.0.1-beta.31), no need to register CosmosDB extension if you use func new to create CosmosDB Trigger, tools will install it automatically.
After this problem solved, if you met another error about Microsoft.Azure.Documents.ServiceInterop.dll
The listener for function 'Functions.CosmosDb' was unable to start.
The listener for function 'Functions.CosmosDb' was unable to start. System.Private.CoreLib: One or more errors occurred. (Unable to load DLL 'Microsoft.Azure.Documents.ServiceInterop.dll' or one of its dependencies: The specified module could not be found. (Exception from HRESULT: 0x8007007E)).
Microsoft.Azure.DocumentDB.Core: Unable to load DLL 'Microsoft.Azure.Documents.ServiceInterop.dll' or one of its dependencies: The specified module could not be found. (Exception from HRESULT: 0x8007007E).
Just install one package to fix(See this issue)
func extensions install -p Microsoft.Azure.DocumentDB.Core -v 2.0.0-preview
Then everything should work.
Is it possible to invoke cosmos db trigger in azure pipeline? Pipeline is just copy data from azrue storage to cosmos db collection and it is necessary to invoke pre trigger. How to specify trigger id for copy activity?
From what you are saying, you might solve this by using Azure Functions with a Blob Trigger and a DocumentDB output binding.
With a functions.json similar to:
{
"disabled": false,
"bindings": [
{
"name": "myBlob",
"type": "blobTrigger",
"direction": "in",
"path": "<name-of-the-folder-where-files-get-uploaded>",
"connection":"MyStorageAccount"
},
{
"name": "documentToSave",
"type": "documentDB",
"databaseName": "MyDatabase",
"collectionName": "MyCollection",
"createIfNotExists": true,
"connection": "MyAccount_COSMOSDB",
"direction": "out"
}
]
}
And the function body could be something like:
// Blob trigger binding to a CloudBlockBlob
#r "Microsoft.WindowsAzure.Storage"
using Microsoft.WindowsAzure.Storage.Blob;
public static void Run(CloudBlockBlob myBlob, out object documentToSave, TraceWriter log)
{
// some logic to read the blob and parse it
documentToSave = new {
id = "some value",
.. other properties here
};
}