Creating Eventgrid subscription using ARM on Function endpoint - azure

I'm trying to create an Eventgrid subscription on an Azure Storage Account using an ARM template. Manually creating it in the Portal and going to the advanced settings yielded me the template below. I further added the required template items such as schema to it, but it keeps yielding me errors. I've tried looking online for similar templates, but can't seem to find any using the "endpointType": "AzureFunction". Also within the Resource Explorer there's no mention of the deployment to further help me along.
Anybody can help me out what is wrong?
The template as generated during creation from the portal:
{
"name": "test123",
"properties": {
"topic": "/subscriptions/<guid>/resourceGroups/<myGroup>/providers/Microsoft.Storage/storageAccounts/<myStorageAccount>",
"destination": {
"endpointType": "AzureFunction",
"properties": {
"resourceId": "/subscriptions/<guid>/resourceGroups/<myGroup>/providers/Microsoft.Web/sites/<myFunctionsApp>/functions/<myFunction>",
"maxEventsPerBatch": 1,
"preferredBatchSizeInKilobytes": 64
}
},
"filter": {
"includedEventTypes": [
"Microsoft.Storage.BlobCreated"
],
"advancedFilters": [
{
"operatorType": "StringContains",
"key": "Subject",
"values": [
"-original"
]
}
]
},
"labels": [],
"eventDeliverySchema": "EventGridSchema"
}
}
The full template:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
},
"resources": [
{
"name": "test123",
"type": "Microsoft.EventGrid/eventSubscriptions",
"apiVersion": "2020-01-01-preview",
"location": "westeurope",
"properties": {
"topic": "/subscriptions/<guid>/resourceGroups/<myGroup>/providers/Microsoft.Storage/storageAccounts/<myStorageAccount>",
"destination": {
"endpointType": "AzureFunction",
"properties": {
"resourceId": "/subscriptions/<guid>/resourceGroups/<myGroup>/providers/Microsoft.Web/sites/<myFunctionsApp>/functions/<myFunction>",
"maxEventsPerBatch": 1,
"preferredBatchSizeInKilobytes": 64
}
},
"filter": {
"includedEventTypes": [
"Microsoft.Storage.BlobCreated"
],
"advancedFilters": [
{
"operatorType": "StringContains",
"key": "Subject",
"values": [
"-original"
]
}
]
},
"labels": [
],
"eventDeliverySchema": "EventGridSchema"
}
}
]
}
The error:
The specified topic property does not match the expected topic from the event subscription scope

I've been trying to do the exact same thing by any option in the Azure tool chain (ARM Template/CLI/REST). I looked at the Portal's calls and found it is using the 2020-01-01-preview EventGrid API that you show.
After some testing I can confirm the new API allows deploying a subscription with an EndpointType of AzureFunction like so:
{
"name": "[concat(variables('eventDomainName'), '/Microsoft.EventGrid/', variables('subscriptionName'))]",
"type": "Microsoft.EventGrid/domains/providers/eventSubscriptions",
"location": "[variables('location')]",
"apiVersion": "2020-01-01-preview",
"properties": {
"destination": {
"endpointType": "AzureFunction",
"properties": {
"resourceId": "[resourceId('Microsoft.Web/sites/functions/', parameters('functionAppName'), parameters('functionName'))]"
}
},
"filter": "[parameters('subscriptionProperties').filter]"
}
}
It seems that your problem is unrelated to trying to target the AzureFunction and you're using the right API version so it doesnt seem to be that.
I think the problem is your "Type" value. I think it should be in this format: //providers/eventSubscriptions
So it would be Microsoft.Storage/storageAccounts/providers/eventSubscriptions.

I don't believe there is a separate endpointType of AzureFunction as documented. It is simply a special case of a webhook handler.
This GitHub Repo contains a sample ARM Template that you can refer to. Here is the exact snippet that you would need
...
"destination": {
"endpointType": "WebHook",
"properties": {
"endpointUrl": "[concat(variables('functionUrl'), listKeys(resourceId('Microsoft.Web/sites/host/', variables('functionAppName'), 'default'),'2016-08-01').systemkeys.eventgrid_extension)]"
}
}
...

Related

How to create Event Subscription for Event Grid Domain Topic using arm template

I want to use the arm template to create the Event Subscription for the Event Grid Domain Topic. It's ok when I create Event Grid Domain and Event Grid Domain Topic but when I try to create the Event Subscription to listen to messages from Event Grid Domain Topic. It always fails. I think I defined the wrong "scrope" or "dependsOn".
Actually, I can't find the document or tutorial to create the Event Subscription for Event Grid Domain Topic. Almost document guide the way to create the Event Subscription for Event Grid Topic.
Thanks for support
This is my arm template
{
"type": "Microsoft.EventGrid/eventSubscriptions",
"apiVersion": "2021-06-01-preview",
"name": "[parameters('eventSubscription')]",
"scope": "[format('Microsoft.EventGrid/domains/topics/{0}', concat(variables('eventGridDomainName'), '/',parameters('topic')))]",
"properties": {
"deadletterdestination": {
"endpointType": "StorageBlob",
"properties": {
"blobContainerName": "parameters('containerName')",
"resourceId": "/subscriptions/{subscriptions}/resourceGroups/{resourceGroupName}/providers/Microsoft.Storage/storageAccounts/{containerName}"
}
},
"destination": {
"endpointType": "WebHook",
"properties": {
"endpointUrl": "string"
}
},
"eventDeliverySchema": "EventGridSchema",
"filter": {
"advancedFilters": [],
"enableAdvancedFilteringOnArrays": true
},
"labels": []
},
"dependsOn": [
"[resourceId('Microsoft.EventGrid/domains/topics',variables('eventGridDomainName'),parameters('topic')]"
]
}
I had to modify the scope a bit to get it working, take a look at the snippet below.
"scope": "[concat('Microsoft.EventGrid/domains', '/', parameters('domains_azdomaineg_name'), '/', 'topics', '/', parameters('topicName'))]"
/
{
"type": "Microsoft.EventGrid/eventSubscriptions",
"apiVersion": "2020-06-01",
"name": "[concat(parameters('domains_azdomaineg_name'), 'topic-dommain-subscription')]",
"dependsOn": [
"[resourceId('Microsoft.EventGrid/domains', parameters('domains_azdomaineg_name'))]",
"[resourceId('Microsoft.EventHub/namespaces/eventHubs', parameters('eventHubNamespace'), parameters('eventHubName'))]"
],
"properties": {
"destination": {
"endpointType": "EventHub",
"properties": {
"resourceId": "[resourceId('Microsoft.EventHub/namespaces/eventhubs', parameters('eventHubNamespace'), parameters('eventHubName'))]"
}
},
"filter": {
"includedEventTypes": [
"first, last"
],
"advancedFilters": [
{
"key": "dataversion",
"operatorType": "StringIn",
"values": [
"test"
]
}
]
}
},
"scope": "[concat('Microsoft.EventGrid/domains', '/', parameters('domains_azdomaineg_name'), '/', 'topics', '/', parameters('topicName'))]"
}
]
}

Event subscription of type WEBHOOK with static Header using ARM template

Problem area:
I tried creating a new event grid topic subscription using an ARM Template following the official documentation.
Script ran fine within PowerShell terminal but I couldn't find the event subscription being generated under the specified topic in the azure portal.
Sample JSON Template:
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"type": "Microsoft.EventGrid/eventSubscriptions",
"apiVersion": "2021-06-01-preview",
"name": "Subscription_1",
"properties": {
"destination": {
"topic": "/subscriptions/{Subscription Id})/resourceGroups/{Resource group name}/providers/Microsoft.EventGrid/topics/{Topic name}}",
"endpointType": "WebHook",
"properties": {
"endpointUrl": "{Endpoint URL}",
"deliveryAttributeMappings": [
{
"name": "test",
"type": "Static",
"properties": {
"value": "test"
}
}
]
}
},
"eventDeliverySchema": "EventGridSchema",
"filter": {
"advancedFilters": [],
"enableAdvancedFilteringOnArrays": true
},
"labels": []
}
}
]
}
Solution:
Got this working by changing the 'type' and the 'naming convention' in the arm-template as below:
"type": "Microsoft.EventGrid/topics/providers/eventSubscriptions",
"apiVersion": "2021-06-01-preview",
"name": "{Topic Name}/ Microsoft.EventGrid/ {Subsctription Name}",

Deploying Azure Firewall IP Group changes fails with conflict

I am attempting to deploy an Azure Firewall with a Policy, a Rule and a set of IPGroups. When I deploy the ARM templates to start everything works.. Later If I want to change something in one of the IPGroups, and I try to deploy that IPGroup change, the Azure Deployment fails with a Status: Conflict with message:
{
"status": "Failed",
"error": {
"code": "ResourceDeploymentFailure",
"message": "The resource operation completed with terminal provisioning state 'Failed'."
}
}
I've attempted to both manage the IPGroups distinctly in their own ARM Template, and place them in with the Azure Policy Rule Collection ARM Template with a DependsOn to see if deploying them all together would help, but either way we just get "Conflict".. I Guess I am wondering what is the appropriate way to update an IPGroup that is a part of a Firewall Network rule? If I can't simply update the IPGroup?
Here is an example of my full ARM Template for my Policy with the IPGroups..
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"firewallPolicyName": {
"defaultValue": "[concat('onelucki-fw-parent-policy', uniqueString(resourceGroup().id))]",
"type": "String"
},
"DevSubnets": {
"defaultValue": "DevSubnets",
"type": "String"
},
"AzureSubnets": {
"defaultValue": "AzureSubnets",
"type": "String"
}
},
"variables": {
"fwPolicyName": "[parameters('firewallPolicyName')]"
},
"resources": [
{
"type": "Microsoft.Network/ipGroups",
"apiVersion": "2020-05-01",
"name": "AzureSubnets",
"location": "centralus",
"tags": { "Zone": "MixedZones" },
"properties": {
"ipAddresses": [
"10.99.1.1"
]
}
},
{
"type": "Microsoft.Network/ipGroups",
"apiVersion": "2020-05-01",
"name": "DevSubnets",
"location": "centralus",
"tags": { "Zone": "Dev" },
"properties": {
"ipAddresses": [
"10.99.2.2"
]
}
},
{
"type": "Microsoft.Network/firewallPolicies",
"apiVersion": "2020-11-01",
"name": "[parameters('firewallPolicyName')]",
"location": "centralus",
"properties": {
"sku": {
"tier": "Standard"
},
"threatIntelMode": "Alert"
}
},
{
"type": "Microsoft.Network/firewallPolicies/ruleCollectionGroups",
"apiVersion": "2020-11-01",
"name": "[concat(parameters('firewallPolicyName'), '/DefaultNetworkRuleCollectionGroup')]",
"location": "westus",
"dependsOn": [
"[resourceId('Microsoft.Network/ipGroups', parameters('AzureSubnets'))]",
"[resourceId('Microsoft.Network/ipGroups', parameters('DevSubnets'))]",
"[resourceId('Microsoft.Network/firewallPolicies', parameters('firewallPolicyName'))]"
],
"properties": {
"priority": 200,
"ruleCollections": [
{
"ruleCollectionType": "FirewallPolicyFilterRuleCollection",
"action": {
"type": "Allow"
},
"rules": [
{
"ruleType": "NetworkRule",
"name": "DemoRule",
"ipProtocols": [
"TCP"
],
"sourceAddresses": [],
"sourceIpGroups": [
"/subscriptions/<subscriptionIDHere>/resourceGroups/onelucki-fw/providers/Microsoft.Network/ipGroups/DevSubnets"
],
"destinationAddresses": [],
"destinationIpGroups": [
"/subscriptions/<subscriptionIDHere>/resourceGroups/onelucki-fw/providers/Microsoft.Network/ipGroups/AzureSubnets"
],
"destinationFqdns": [],
"destinationPorts": [
"135",
"445"
]
}
],
"name": "DemoDeployRuleCollection",
"priority": 1300
}
]
}
}
]
}
IP groups need to be deployed one at a time. Also the firewall policy needs a depends on the IP groups being used despite it not having them listed.
The deploy of the IP groups seems to do some validation/update on the firewall policy during deploy.
Deploy nested resources in Azure using DependsOn

ARM template for storage account blob create event grid subscription

I created an event grid subscription on storage account for a blob created event. I want to create the same thing using an ARM template but having trouble.
Observations-
running the below template creates an event grid subscription on the resource group and not on the storage account.
As per the Issue #563 and #455, providers should be used as the type. But 'providers' type is not valid in apiversion "2020-04-01-preview".
{
"name": "[parameters('blobcreate_eventsubscription_name')]",
"apiVersion": "2020-04-01-preview",
"type": "Microsoft.EventGrid/eventSubscriptions",
"dependsOn": [
"[variables('storageAccountResourceId')]" //,
//"[variables('functionAppResourceId')]"
],
"properties": {
// "topic": "[variables('storageAccountResourceId')]",
"destination": {
"endpointType": "AzureFunction",
"properties": {
"resourceId": "[variables('azureFunctionResourceId')]",
"maxEventsPerBatch": 1,
"preferredBatchSizeInKilobytes": 64
}
},
"filter": {
"subjectBeginsWith": "[concat('/blobServices/default/containers', parameters('storageAccounts_blobname'))]",
"subjectEndsWith": ".xml",
"includedEventTypes": [
"Microsoft.Storage.BlobCreated"
],
"advancedFilters": []
},
"labels": [],
"eventDeliverySchema": "EventGridSchema",
"retryPolicy": {
"maxDeliveryAttempts": "[parameters('eventgrid_maxDeliveryAttemps')]",
"eventTimeToLiveInMinutes": "[parameters('eventgrid_eventTimeToLiveInMinutes')]"
},
"deadLetterDestination": {
"endpointType": "StorageBlob",
"properties": {
"resourceId": "[variables('storageAccountResourceId')]",
"blobContainerName": "[parameters('storageAccounts_deadletterblob_name')]"
}
}
}
}
here's an official example which you can use a base:
{
"type": "Microsoft.Storage/storageAccounts/providers/eventSubscriptions",
"name": "[concat(parameters('storageName'), '/Microsoft.EventGrid/', parameters('eventSubName'))]",
"apiVersion": "2018-01-01",
"dependsOn": [
"[parameters('storageName')]"
],
"properties": {
"destination": {
"endpointType": "WebHook",
"properties": {
"endpointUrl": "[parameters('endpoint')]"
}
},
"filter": {
"subjectBeginsWith": "",
"subjectEndsWith": "",
"isSubjectCaseSensitive": false,
"includedEventTypes": [
"All"
]
}
}
}
notice the resourceType and name of the resource.
https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/101-event-grid-subscription-and-storage/azuredeploy.json

Event subscription by ARM template for topic with EndpointType as AzureFunction

I am trying to create an event grid topic subscription with "endpointType": "AzureFunction". It is giving following error: -
"error": {
"code": "InvalidRequest",
"message": "Invalid event subscription request: Supplied URL is invalid. It cannot be null or empty and should be a proper HTTPS URL
like https://www.example.com." }
My ARM template is given below: -
{
"name": "[concat(variables('eventGridTopicName'), '/Microsoft.EventGrid/', variables('myFuncName'))]",
"type": "Microsoft.EventGrid/topics/providers/eventSubscriptions",
"apiVersion": "2019-01-01",
"location": "[parameters('location')]",
"properties": {
"topic": "[concat('/subscriptions/', parameters('subscriptionId'),'/resourcegroups/', parameters('resourceGroupName'), '/providers/Microsoft.EventGrid/topics/', variables('eventGridTopicName'))]",
"destination": {
"endpointType": "AzureFunction",
"properties": {
"resourceId": "[resourceId('Microsoft.Web/sites/functions/', variables('funcAppName'), variables('myFuncName'))]",
"maxEventsPerBatch": 1,
"preferredBatchSizeInKilobytes": 64
}
},
"filter": {
"advancedFilters": [
{
"operatorType": "StringIn",
"key": "eventType",
"values": [
"xyzEvent"
]
},
{
"operatorType": "StringIn",
"key": "subject",
"values": [
"xyzEventReceived"
]
}
]
},
"labels": [],
"eventDeliverySchema": "EventGridSchema"
},
"dependsOn": [
"[variables('eventGridTopicName')]"
]
}
Earlier, I was using EndpointType as a webhook since new event handlers like Azure Function, storage Queues, etc. were not available (https://learn.microsoft.com/en-us/azure/event-grid/event-handlers). I used the generated arm template from Azure portal as shown below: -
Has anyone faced this issue?
Yes ! found this when I had same issue! ..
Update! found an example that uses another API version and it seems to work beter, now my issue is that there is no code on it when deploying first time, so I need to break the template into two and deploy content in btween (or deploy content via template ofc).
"apiVersion": "2020-01-01-preview",
https://blog.brooksjc.com/2019/07/19/arm-template-for-event-grid-integration-with-a-new-azure-function/
Update 2, after adding the content and rerunning the template, it work fine!
here is my full code for my storage trigger
{
"name": "[concat(variables('storageAccountName'), '/Microsoft.EventGrid/coreCostManagementExport')]",
"type": "Microsoft.Storage/storageAccounts/providers/eventSubscriptions",
"apiVersion": "2020-01-01-preview",
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts',variables('storageAccountName'))]",
"[resourceId('Microsoft.Web/sites',parameters('functionAppName'))]"
],
"properties": {
"topic": "[resourceId('Microsoft.Storage/storageAccounts',variables('storageAccountName'))]",
"destination": {
"endpointType": "AzureFunction",
"properties": {
"resourceId": "[resourceId('Microsoft.Web/sites/functions/', parameters('functionAppName'), 'QueueUsageOnExport')]",
"maxEventsPerBatch": 1,
"preferredBatchSizeInKilobytes": 64
}
},
"filter": {
"subjectBeginsWith": "/blobServices/default/containers/usage",
"subjectEndsWith": ".csv",
"includedEventTypes": [
"Microsoft.Storage.BlobCreated"
],
"advancedFilters": [
]
},
"labels": [
],
"eventDeliverySchema": "EventGridSchema"
}
}
Jakob's suggestion for changing api version worked for me with change in resourceId. Here is my modified working template: -
{
"name": "[concat(variables('eventGridTopicName'), '/Microsoft.EventGrid/', variables('myFuncName'))]",
"type": "Microsoft.EventGrid/topics/providers/eventSubscriptions",
"apiVersion": "2020-01-01-preview",
"location": "[parameters('location')]",
"properties": {
"topic": "[concat('/subscriptions/', parameters('subscriptionId'),'/resourcegroups/', parameters('resourceGroupName'), '/providers/Microsoft.EventGrid/topics/', variables('eventGridTopicName'))]",
"destination": {
"endpointType": "AzureFunction",
"properties": {
"resourceId": "[concat('/subscriptions/', parameters('subscriptionId'),'/resourcegroups/', parameters('resourceGroupName'), '/providers/Microsoft.Web/sites/', variables('funcAppName'), '/functions/' , variables('myFuncName'))]",
"maxEventsPerBatch": 1,
"preferredBatchSizeInKilobytes": 64
}
},
"filter": {
"advancedFilters": [
{
"operatorType": "StringIn",
"key": "eventType",
"values": [
"xyzEvent"
]
},
{
"operatorType": "StringIn",
"key": "subject",
"values": [
"xyzEventReceived"
]
}
]
},
"labels": [],
"eventDeliverySchema": "EventGridSchema"
},
"dependsOn": [
"[variables('eventGridTopicName')]"
]
}
In my scenario, I was trying to add a function app Subscription to an event grid topic using "AzureFunctionEventSubscriptionDestination" as the destination. My issue was I missed adding the /functions/{targetFunctionName} to the resource id.
"resourceId": "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Web/sites/{functionAppName}/functions/{targetFunctionName}"

Resources