Azure Function: How to add row to cloud table on blob creating? - azure

Question

I'm developing an Azure Function which should add new line to an Azure table when new a new blob is added. The application has many containers in Blob Storage, and my Azure Function should process all blobs from all containers.
I tried to implement event getting with EventGrid, but it gives an error.
My Azure function:
#r "D:\home\site\wwwroot\BlobCreatedFunction\Microsoft.Azure.EventGrid.dll"
#r"D:\home\site\wwwroot\BlobCreatedFunction\Microsoft.WindowsAzure.Storage.dll"
using Microsoft.Azure.EventGrid.Models;
using Microsoft.WindowsAzure.Storage.Table;
using System;
public class TemporaryBlobEntity : TableEntity
{
public TemporaryBlobEntity(string partitionKey, string rowKey)
{
this.PartitionKey = partitionKey;
this.RowKey = rowKey;
}
public string BlobUrl { get; set; }
public DateTime BlobUploaded { get; set; }
}
public static TemporaryBlobEntity Run(EventGridEvent eventGridEvent, ILogger log)
{
if (eventGridEvent.Data is StorageBlobCreatedEventData eventData)
{
log.LogInformation(eventData.Url);
log.LogInformation(eventGridEvent.Data.ToString());
var temporaryBlob = new TemporaryBlobEntity("blobs", eventData.Url)
{
BlobUrl = eventData.Url,
BlobUploaded = DateTime.UtcNow
};
return temporaryBlob;
}
return null;
}
Here is my integration JSON:
{
"bindings": [
{
"type": "eventGridTrigger",
"name": "eventGridEvent",
"direction": "in"
},
{
"type": "table",
"name": "$return",
"tableName": "temporaryBlobs",
"connection": "AzureWebJobsStorage",
"direction": "out"
}
]
}
In my Azure Function settings, I added the value for AzureWebJobsStorage.
When I press Run in the test section, logs show:
2019-07-08T13:52:16.756 [Information] Executed 'Functions.BlobCreatedFunction' (Succeeded, Id=6012daf1-9b98-4892-9560-932d05857c3e)
Looks good, but there is no new item in cloud table. Why?
Then I tried to connect my function with EventGrid topic. I filled new subscription form, selected "Web Hook" as endpoint type, and set subscriber endpoint at: https://<azure-function-service>.azurewebsites.net/runtime/webhooks/EventGrid?functionName=<my-function-name>. Then I got the following error message:
Deployment has failed with the following error: {"code":"Url validation","message":"The attempt to validate the provided endpoint https://####.azurewebsites.net/runtime/webhooks/EventGrid failed. For more details, visit https://aka.ms/esvalidation."}
As far as I can understand, the application needs some kind of request validation. Do I really need to implement validation in each of my azure functions? Or shoudl I use another endpoint type?

Answer 1

When you enter a webhook into Event Grid it sends out a request to verify that you actually have permissions on that endpoint. The easiest way to connect a Function to Event Grid is to create the subscription from the Functions app instead of the Event Grid blade.
Opening up the Function in the portal you should find a link at the top to "Add Event Grid subscription". Even if the Functions app was created locally and published to Azure so the code isn't viewable the link will be available.
This will open up the screen for creating an Event Grid subscription. The difference is that instead of the Event Grid topic info being prefilled, the web hook info is prepopulated for you. Fill in the info about the Event Grid topic to finish creating the subscription.
If you decide you want to implement the validation response for whatever reason, it is possible to do this by checking the type of the message.
// Validate whether EventType is of "Microsoft.EventGrid.SubscriptionValidationEvent"
if (eventGridEvent.EventType == "Microsoft.EventGrid.SubscriptionValidationEvent")
{
var eventData = (SubscriptionValidationEventData)eventGridEvent.Data;
// Do any additional validation (as required) such as validating that the Azure resource ID of the topic matches
// the expected topic and then return back the below response
var responseData = new SubscriptionValidationResponse()
{
ValidationResponse = eventData.ValidationCode
};
if (responseData.ValidationResponse != null)
{
return Ok(responseData);
}
}
else
{
//Your code here
}
There is also an option to validate the link manually by getting the validation link out of the validation message and navigating to it in your browser. This method is primarily for 3rd party services where you can't add the validation code.

Answer 2

The following are changes in your EventGridTrigger function:
#r "Microsoft.WindowsAzure.Storage"
#r "Microsoft.Azure.EventGrid"
#r "Newtonsoft.Json"
using System;
using Newtonsoft.Json.Linq;
using Microsoft.Azure.EventGrid.Models;
using Microsoft.WindowsAzure.Storage.Table;
public static TemporaryBlobEntity Run(EventGridEvent eventGridEvent, ILogger log)
{
log.LogInformation(eventGridEvent.Data.ToString());
var eventData = (eventGridEvent.Data as JObject)?.ToObject<StorageBlobCreatedEventData>();
if(eventData?.Api == "PutBlob")
{
log.LogInformation(eventData.Url);
return new TemporaryBlobEntity("blobs", eventData.Sequencer)
{
BlobUrl = eventData.Url,
BlobUploaded = DateTime.UtcNow
};
}
return null;
}
public class TemporaryBlobEntity : TableEntity
{
public TemporaryBlobEntity(string partitionKey, string rowKey)
{
this.PartitionKey = partitionKey;
this.RowKey = rowKey;
}
public string BlobUrl { get; set; }
public DateTime BlobUploaded { get; set; }
}
Notes:
You don't need to validate an EventGridTrigger function for AEG subscription webhook endpoint. This validation is built-in the preprocessing of the EventGridTrigger function.
The eventGridEvent.Data property is a JObject and must be converted (deserialized) to the StorageBlobCreatedEventData object, see here.
For RowKey (and PartitionKey) see the restriction characters in here, so I changed it to the Sequencer value in this example.
The AEG subscription webhook endpoint for the EventGridTrigger function has the following format:
https://{azure-function-service}.azurewebsites.net/runtime/webhooks/EventGrid?functionName={my-function-name}&code={systemkey}