Is it possible to invoke cosmos db trigger in azure pipeline? Pipeline is just copy data from azrue storage to cosmos db collection and it is necessary to invoke pre trigger. How to specify trigger id for copy activity?
From what you are saying, you might solve this by using Azure Functions with a Blob Trigger and a DocumentDB output binding.
With a functions.json similar to:
{
"disabled": false,
"bindings": [
{
"name": "myBlob",
"type": "blobTrigger",
"direction": "in",
"path": "<name-of-the-folder-where-files-get-uploaded>",
"connection":"MyStorageAccount"
},
{
"name": "documentToSave",
"type": "documentDB",
"databaseName": "MyDatabase",
"collectionName": "MyCollection",
"createIfNotExists": true,
"connection": "MyAccount_COSMOSDB",
"direction": "out"
}
]
}
And the function body could be something like:
// Blob trigger binding to a CloudBlockBlob
#r "Microsoft.WindowsAzure.Storage"
using Microsoft.WindowsAzure.Storage.Blob;
public static void Run(CloudBlockBlob myBlob, out object documentToSave, TraceWriter log)
{
// some logic to read the blob and parse it
documentToSave = new {
id = "some value",
.. other properties here
};
}
Related
We know the Connection string for event hub can be used from local.setting.json file. So for the same function app in different environments, I can add the event hub connection string setting in Application settings in the azure portal.
As the EventHubTrigger function app also expects event name and consumer group(optional) as attributes parameters, I was wondering how the event hub name and consumer group can be used from app settings?
public static void EventHubTriggerFunc([EventHubTrigger("myeventhubname", Connection = "EventHubConnectionAppSetting", ConsumerGroup = "myconsumergroupname")] EventData myEventHubMessage, DateTime enqueuedTimeUtc, Int64 sequenceNumber, string offset, ILogger log)
{
// Here EventHubConnectionAppSetting is specified in local.setting.json file
//myeventhubname & myconsumergroupname are hard coded string
}
local.settings.Json
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"FUNCTIONS_WORKER_RUNTIME": "dotnet",
"EventHubConnectionAppSetting": "Endpoint=.....",
"EventHubConsumerGroup": "myconsumergroup"
}
}
([EventHubTrigger("%myeventhubname%", Connection = "EventHubConnectionAppSetting", ConsumerGroup = "%myconsumergroupname%")]
Tried #Roman Kiss answer, and applied it to Python Azure Functions and it works.
In function.json:
{
"scriptFile": "__init__.py",
"bindings": [
{
"type": "eventHubTrigger",
"name": "events",
"direction": "in",
"eventHubName": "%EVENT_HUB_NAME%",
"connection": "EVENT_HUB_CONN_STR",
"cardinality": "many",
"consumerGroup": "$Default",
"dataType": "binary"
}
]
}
Notice that connection string does not need %
In local.settings.json:
{
...
"Values": {
...
"EVENT_HUB_NAME": "<actual name of event hub>",
"EVENT_HUB_CONN_STR": "Endpoint=sb://...;SharedAccessKeyName=...;SharedAccessKey=...",
...
},
}
I can't quite seem to get the output bindings to enable a file to be saved to blob storage. I have created an Azure Function using Python, that uses a CosmosDB Change Feed trigger. I need to save that document to blob storage.
I've set-up the function.json file as follows:
{
"scriptFile": "__init__.py",
"bindings": [
{
"type": "cosmosDBTrigger",
"name": "documents",
"direction": "in",
"leaseCollectionName": "leases",
"connectionStringSetting": "cosmos_dev",
"databaseName": "MyDatabase",
"collectionName": "MyCollection",
"createLeaseCollectionIfNotExists": "true"
},
{
"type": "blob",
"direction": "out",
"name": "outputBlob",
"path": "raw/changefeedOutput/{blobname}",
"connection": "blobStorageConnection"
}
]
}
So the trigger will get a documents like the following:
{ "id": "documentId-12345",
other sections here
"entity": "customer"
}
In the init.py file I have the base code of
def main(documents: func.DocumentList) -> func.Document:
logging.info(f"CosmosDB trigger executed!")
for doc in documents:
blobName = doc['id'] + '.json'
blobFolder= doc['entity']
blobData = doc.to_json()
I think i need to add in the def something like 'outputBlob: func.Out' but unsure how to proceed
Looking at the examples on github
https://github.com/yokawasa/azure-functions-python-samples/tree/master/v2functions/blob-trigger-watermark-blob-out-binding
it look like i have to
outputBlob.set(something)
So i'm looking for how to set up the def part and send the blob to the location that i've set from the data in the cosmosdb document.
I have tried the following:
def main(documents: func.DocumentList, outputBlob: func.Out[str] ) -> func.Document:
logging.info(f"CosmosDB trigger executed!")
for doc in documents:
blobName = doc['id'] + '.json'
outputBlob.set(blobName)
and get the result:
CosmosDB trigger executed!
Executed 'Functions.CosmosTrigger_py' (Failed, Id=XXXXX)
System.Private.CoreLib: Exception while executing function: Functions.CosmosTrigger_py. Microsoft.Azure.WebJobs.Host: No value for named parameter 'blobname'.
I could just call the connection stuff from the os.enviro, and get the connection string that way, I think and use the standard create_blob_from_text, with location, name and blob data,
block_blob_service.create_blob_from_text(blobLocation, blobName, formattedBlob)
Any pointers would be great
I have a activity function that should store message in Blob storage.I can overwrite a file in blob storage but i need to store data in different name.how to do that? Azure function doesn't support dynamic binding in nodejs.
Find one workaround, see whether it's useful.
Along with blob output binding, there's an activity trigger to receive message msg, we can put self-defined blob name in msg for blob binding path to consume.
In your orchestrator function which calls Activity function
yield context.df.callActivity("YourActivity", {'body':'messagecontent','blobName':'myblob'});
Then Activity function code should be modified
context.bindings.myOutputBlob = context.bindings.msg.body;
And its function.json can use blobName as expected
{
"bindings": [
{
"name": "msg",
"type": "activityTrigger",
"direction": "in"
},
{
"name":"myOutputBlob",
"direction": "out",
"type": "blob",
"connection": "AzureWebJobsStorage",
"path": "azureblob/{blobName}"
}
],
"disabled": false
}
This question already has an answer here:
Azure Function blob binding
(1 answer)
Closed 5 years ago.
Trying to remake the Azure Grid Image Resize example in c# using visual studio but having issues making the azure function trigger be triggered by the event grid and bind to the blob storage.
Current Code:
using Microsoft.Azure.WebJobs.Extensions.EventGrid;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Azure.WebJobs;
using Microsoft.WindowsAzure.Storage.Blob;
namespace FunctionApp
{
public static class CreateIndex
{
[FunctionName("CreateIndex")]
[StorageAccount("backup_STORAGE")]
public static void Run(
[EventGridTrigger()] EventGridEvent myEvent,
[Blob("{data.url}")] CloudBlockBlob inputBlob,
TraceWriter log)
{
log.Info(myEvent.ToString());
log.Info(inputBlob.ToString());
}
}
}
Generated function.json:
{
"generatedBy": "Microsoft.NET.Sdk.Functions.Generator-1.0.6",
"configurationSource": "attributes",
"bindings": [
{
"type": "eventGridTrigger",
"name": "myEvent"
}
],
"disabled": false,
"scriptFile": "../bin/FunctionApp.dll",
"entryPoint": "FunctionApp.CreateIndex.Run"
}
The binding is working for the event grid trigger but not the Blob input.
Expected function.json:
{
"bindings": [
{
"type": "EventGridTrigger",
"name": "myEvent",
"direction": "in"
},
{
"type": "blob",
"name": "inputBlob",
"path": "{data.url}",
"connection": "myblobstorage_STORAGE",
"direction": "in"
}
],
"disabled": false
}
Precompiled functions generate function.json for you, but they only put trigger binding inside of it. It's OK that your blob binding is not in this file.
The input Blob binding will still work: runtime will pick it up based on your attributes.
Trying to figure out how to use SendGrid from an azure function. Not much docs to find, but this is what I've tried:
#r "SendGrid"
using SendGrid.Helpers.Mail;
using System;
public static void Run(string myQueueItem, out Mail message, TraceWriter log)
{
log.Info($"C# Queue trigger function processed: {myQueueItem}");
message=new Mail();
}
I've hardcoded to, from, subject and body, together with api key in the integrate output section. This is my function.json:
{
"bindings": [
{
"name": "myQueueItem",
"type": "queueTrigger",
"direction": "in",
"queueName": "send-email-request",
"connection": "myStorage"
},
{
"type": "sendGrid",
"name": "message",
"apiKey": "SG.xxxxxxxxxxx",
"direction": "out",
"to": "me#gmail.com",
"from": "me#gmail.no",
"subject": "hardcoded",
"text": "test213"
}
],
"disabled": false
}
I get the following error:
Function ($SendEmailOnQueueTriggered) Error: Microsoft.Azure.WebJobs.Host: Error indexing method 'Functions.SendEmailOnQueueTriggered'. Microsoft.Azure.WebJobs.Host: 'SG.xxxxxx' does not resolve to a value.
Session Id: 9426f2d7e4374c7ba7e0612ea5dc1814
Timestamp: 2017-01-07T12:18:01.930Z
I've granted the Apikey full access in SendGrid. Any ideas what I've missed?
Larsi
The ApiKey field is not the actual ApiKey, it should instead be the name of an AppSettings key defined in the "functions app settings".
Adding "AzureWebJobsSendGridApiKey" in your AppSettings via local.settings.json and in Configuration -> Application Setting via Azure Portal will work.