I want to create 2 edge nodes in HDI cluster using ARM template. I need to have Edge node 2 is depends on Edge node 1
Here is my ARM template for Edge node:-
{
"name": "[concat(parameters('clusterName'),'/', parameters('TestEdgenodeName1'))]",
"type": "Microsoft.HDInsight/clusters/applications",
"apiVersion": "2015-03-01-preview",
"dependsOn": [
"[concat('Microsoft.HDInsight/clusters/', parameters('clusterName'))]"
],
"properties": {
"marketPlaceIdentifier": "TestEdgeNode1",
"computeProfile": {
"roles": [{
"name": "edgenode",
"targetInstanceCount": 1,
"hardwareProfile": {
"vmSize": "[parameters('EdgenodeSize1')]"
}
}]
},
"installScriptActions": [{
"name": "[concat('node','-' ,uniquestring(parameters('TestEdgenodeName1')))]",
"uri": "[variables('InstallationScript1')]",
"parameters": "",
"roles": ["edgenode"]
}],
"uninstallScriptActions": [],
"httpsEndpoints": [],
"applicationType": "CustomApplication"
}
}
"name": "[concat(parameters('clusterName'),'/', parameters('TestEdgenodeName2'))]",
"type": "Microsoft.HDInsight/clusters/applications",
"apiVersion": "2015-03-01-preview",
"dependsOn": [
"[concat('Microsoft.HDInsight/clusters/applications/', parameters('clusterName'), '/', parameters('TestEdgenodeName1'))]"
],
"properties": {
"marketPlaceIdentifier": "TestEdgeNode2",
"computeProfile": {
"roles": [{
"name": "edgenode",
"targetInstanceCount": 1,
"hardwareProfile": {
"vmSize": "[parameters('EdgenodeSize2')]"
}
}]
},
"installScriptActions": [{
"name": "[concat('node','-' ,uniquestring(parameters('TestEdgenodeName2')))]",
"uri": "[variables('installationScript2')]",
"parameters": "",
"roles": ["edgenode"]
}],
"uninstallScriptActions": [],
"httpsEndpoints": [],
"applicationType": "CustomApplication"
}
}
I am getting the below error when deploying ARM template.
InvalidTemplate : Deployment template validation failed: 'The resource 'Microsoft.HDInsight/clusters/applications/test-cluster/test-edgenode' is not defined in the template.
HDInsight supports single edge node at this time.
I am able to solve the issue by using the below on second edge node.
"dependsOn": [
"[resourceId('Microsoft.HDInsight/clusters/applications', parameters('clusterName'), parameters('TestEdgenodeName1'))]"
],
Related
I am trying to create a serverless account with Cosmosdb sql api and i have not found any samples given here
I have tried with the following ARM template and it's not creating a serverless account
"resources" : [
{
"type": "Microsoft.DocumentDB/databaseAccounts",
"apiVersion": "2020-04-01",
"kind": "Serverless",
"name": "[parameters('accountName')]",
"location": "[parameters('location')]",
"properties": {
"enableFreeTier": false,
"databaseAccountOfferType": "Standard",
"consistencyPolicy": {
"defaultConsistencyLevel": "Session"
},
"locations": [
{
"locationName": "[parameters('location')]"
}
]
}
},
{
"type": "Microsoft.DocumentDB/databaseAccounts/sqlDatabases",
"apiVersion": "2020-04-01",
"name": "[format('{0}/{1}', parameters('accountName'), parameters('databaseName'))]",
"properties": {
"resource": {
"id": "[parameters('databaseName')]"
},
"options": {}
},
"dependsOn": [
"[resourceId('Microsoft.DocumentDB/databaseAccounts', parameters('accountName'))]"
]
}
]
throwing an error "
"message": "Resource kind Serverless is unknown\r\nActivityId: 0c86f162-3386-49e1-b354-57ba309bb44f, Microsoft.Azure.Documents.Common/2.14.0""
The error is valid, below are the possible values available for the databaseAccount kind
'GlobalDocumentDB'
'MongoDB'
'Parse'
To create a serverless account, you need to pass the capabilities parameter as below under properties
"properties": {
"enableFreeTier": false,
"capabilities": [
{
"name": "EnableServerless"
}
],
"databaseAccountOfferType": "Standard",
"consistencyPolicy": {
"defaultConsistencyLevel": "Session"
},
"locations": [
{
"locationName": "[parameters('location')]"
}
]
}
I am attempting to deploy an Azure Firewall with a Policy, a Rule and a set of IPGroups. When I deploy the ARM templates to start everything works.. Later If I want to change something in one of the IPGroups, and I try to deploy that IPGroup change, the Azure Deployment fails with a Status: Conflict with message:
{
"status": "Failed",
"error": {
"code": "ResourceDeploymentFailure",
"message": "The resource operation completed with terminal provisioning state 'Failed'."
}
}
I've attempted to both manage the IPGroups distinctly in their own ARM Template, and place them in with the Azure Policy Rule Collection ARM Template with a DependsOn to see if deploying them all together would help, but either way we just get "Conflict".. I Guess I am wondering what is the appropriate way to update an IPGroup that is a part of a Firewall Network rule? If I can't simply update the IPGroup?
Here is an example of my full ARM Template for my Policy with the IPGroups..
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"firewallPolicyName": {
"defaultValue": "[concat('onelucki-fw-parent-policy', uniqueString(resourceGroup().id))]",
"type": "String"
},
"DevSubnets": {
"defaultValue": "DevSubnets",
"type": "String"
},
"AzureSubnets": {
"defaultValue": "AzureSubnets",
"type": "String"
}
},
"variables": {
"fwPolicyName": "[parameters('firewallPolicyName')]"
},
"resources": [
{
"type": "Microsoft.Network/ipGroups",
"apiVersion": "2020-05-01",
"name": "AzureSubnets",
"location": "centralus",
"tags": { "Zone": "MixedZones" },
"properties": {
"ipAddresses": [
"10.99.1.1"
]
}
},
{
"type": "Microsoft.Network/ipGroups",
"apiVersion": "2020-05-01",
"name": "DevSubnets",
"location": "centralus",
"tags": { "Zone": "Dev" },
"properties": {
"ipAddresses": [
"10.99.2.2"
]
}
},
{
"type": "Microsoft.Network/firewallPolicies",
"apiVersion": "2020-11-01",
"name": "[parameters('firewallPolicyName')]",
"location": "centralus",
"properties": {
"sku": {
"tier": "Standard"
},
"threatIntelMode": "Alert"
}
},
{
"type": "Microsoft.Network/firewallPolicies/ruleCollectionGroups",
"apiVersion": "2020-11-01",
"name": "[concat(parameters('firewallPolicyName'), '/DefaultNetworkRuleCollectionGroup')]",
"location": "westus",
"dependsOn": [
"[resourceId('Microsoft.Network/ipGroups', parameters('AzureSubnets'))]",
"[resourceId('Microsoft.Network/ipGroups', parameters('DevSubnets'))]",
"[resourceId('Microsoft.Network/firewallPolicies', parameters('firewallPolicyName'))]"
],
"properties": {
"priority": 200,
"ruleCollections": [
{
"ruleCollectionType": "FirewallPolicyFilterRuleCollection",
"action": {
"type": "Allow"
},
"rules": [
{
"ruleType": "NetworkRule",
"name": "DemoRule",
"ipProtocols": [
"TCP"
],
"sourceAddresses": [],
"sourceIpGroups": [
"/subscriptions/<subscriptionIDHere>/resourceGroups/onelucki-fw/providers/Microsoft.Network/ipGroups/DevSubnets"
],
"destinationAddresses": [],
"destinationIpGroups": [
"/subscriptions/<subscriptionIDHere>/resourceGroups/onelucki-fw/providers/Microsoft.Network/ipGroups/AzureSubnets"
],
"destinationFqdns": [],
"destinationPorts": [
"135",
"445"
]
}
],
"name": "DemoDeployRuleCollection",
"priority": 1300
}
]
}
}
]
}
IP groups need to be deployed one at a time. Also the firewall policy needs a depends on the IP groups being used despite it not having them listed.
The deploy of the IP groups seems to do some validation/update on the firewall policy during deploy.
Deploy nested resources in Azure using DependsOn
I started Infrastructure as Code with ARM Template and previously all my deployment was made with Powershell. Hope you can help me to fix this issue.
I would like to deploy {2 app services + Azure FrontDoor]. In FrontDoor-Backendpool I want to define the 2 appservices. Below my code:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"location": {
"type": "array",
"metadata": {
"description": "array of region"
},
"defaultValue": [
"centralus",
"eastus"
]
},
"Stage": {
"type": "string",
"metadata": {
"description": "Stage dev, prod"
},
"allowedValues": [
"Dev",
"Prod"
],
"defaultValue": "Dev"
}
},
"functions": [],
"variables": {
"appServicePlanName": "[concat('AppServicePlan-', parameters('Stage'),'-')]",
"appServiceName": "[concat('AppService-', parameters('Stage'), '-')]",
"frontDoorName": "[concat('FrontDoor-', parameters('Stage'), uniqueString(resourceGroup().id))]"
},
"resources": [
{ // App Service Plan
"type": "Microsoft.Web/serverfarms",
"name": "[concat(variables('appServicePlanName'),parameters('location')[copyIndex()])]",
"apiVersion": "2018-02-01",
"copy": {
"count": "[length(parameters('location'))]",
"name": "copy multiple"
},
"location": "[parameters('location')[copyIndex()]]",
"sku": {
"name": "F1",
"capacity": 1
},
"tags": {
"cost": "[parameters('Stage')]"
},
"properties": {
"name": "[concat(variables('appServicePlanName'),parameters('location')[copyIndex()])]"
}
},
{ // App Services
"type": "Microsoft.Web/sites",
"name": "[concat(variables('appServiceName'), parameters('location')[copyIndex()])]",
"apiVersion": "2018-11-01",
"copy": {
"name": "Copy website",
"count": "[length(parameters('location'))]"
},
"location": "[parameters('location')[copyIndex()]]",
"tags": {
"cost": "[parameters('Stage')]"
},
"dependsOn": [
"[resourceId('Microsoft.Web/serverfarms', concat(variables('appServicePlanName'),parameters('location')[copyIndex()]))]"
],
"properties": {
"name": "[concat(variables('appServiceName'), parameters('location')[copyIndex()])]",
"serverFarmId": "[resourceId('Microsoft.Web/serverfarms', concat(variables('appServicePlanName'),parameters('location')[copyIndex()]))]"
}
},
{ // Front Door
"type": "Microsoft.Network/frontDoors",
"apiVersion": "2020-05-01",
"name": "[variables('frontDoorName')]",
"location": "global",
"properties": {
"routingRules": [
{
"name": "routingRule1",
"properties": {
"frontendEndpoints": [
{
"id": "[resourceId('Microsoft.Network/frontDoors/frontendEndpoints', variables('frontDoorName'), 'frontendEndpoint1')]"
}
],
"acceptedProtocols": [
"Http",
"Https"
],
"patternsToMatch": [
"/*"
],
"routeConfiguration": {
"#odata.type": "#Microsoft.Azure.FrontDoor.Models.FrontdoorForwardingConfiguration",
"forwardingProtocol": "MatchRequest",
"backendPool": {
"id": "[resourceId('Microsoft.Network/frontDoors/backendPools', variables('frontDoorName'), 'backendPool1')]"
}
},
"enabledState": "Enabled"
}
}
],
"healthProbeSettings": [
{
"name": "healthProbeSettings1",
"properties": {
"path": "/",
"protocol": "Http",
"intervalInSeconds": 120
}
}
],
"loadBalancingSettings": [
{
"name": "loadBalancingSettings1",
"properties": {
"sampleSize": 4,
"successfulSamplesRequired": 2
}
}
],
"backendPools": [
{
"id": "backendPool1",
"name": "backendPool1",
"properties": {
"copy": [
{
"name": "backends",
"count": "[length(parameters('location'))]",
"input": {
"address": "[concat(variables('appServiceName'), parameters('location')[copyIndex()], '.azurewebsites.net') ]",
"httpPort": 80,
"httpsPort": 443,
"weight": 50,
"priority": 1,
"enabledState": "Enabled"
}
}
],
"loadBalancingSettings": {
"id": "[resourceId('Microsoft.Network/frontDoors/loadBalancingSettings', variables('frontDoorName'), 'loadBalancingSettings1')]"
},
"healthProbeSettings": {
"id": "[resourceId('Microsoft.Network/frontDoors/healthProbeSettings', variables('frontDoorName'), 'healthProbeSettings1')]"
}
}
}
],
"frontendEndpoints": [
{
"name": "frontendEndpoint1",
"properties": {
"hostName": "[concat(variables('frontDoorName'), '.azurefd.net')]",
"sessionAffinityEnabledState": "Enabled"
}
}
],
"enabledState": "Enabled"
}
}
],
"outputs": {}
}
As you can see i iterate on paramater location to create my AppService Plan and AppService and it worked well. So I thought to do same for BackEndpool.
Here part of code which break my head
address": "[concat(variables('appServiceName'), parameters('location')[copyIndex()], '.azurewebsites.net') ]",
Something is wrong inside but I have no idea why.
Error retuned is:
Error: Code=InvalidTemplate; Message=Deployment template language expression evaluation
failed: 'The template language function 'copyIndex' has an invalid argument. The provided copy name '' doesn't exist in the resource.
Please see https://aka.ms/arm-copy for usage details.'. Please see https://aka.ms/arm-template-expressions for usage details.
I take my inspiration from official MS documentation link from MS
Any idea on how I can fix it ?
Thx
You need to include the copy name property in the call to copyIndex in the backendPools part. That is why is says "The provided copy name '' doesn't exist". The property copy is treated a little differently than the resource copy.
"The loopName property enables you to specify whether copyIndex is referring to a resource iteration or property iteration. If no value is provided for loopName, the current resource type iteration is used. Provide a value for loopName when iterating on a property."
Source: https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/template-functions-numeric#copyindex
parameters('location')[copyIndex('backends')]
I am trying to create an event grid topic subscription with "endpointType": "AzureFunction". It is giving following error: -
"error": {
"code": "InvalidRequest",
"message": "Invalid event subscription request: Supplied URL is invalid. It cannot be null or empty and should be a proper HTTPS URL
like https://www.example.com." }
My ARM template is given below: -
{
"name": "[concat(variables('eventGridTopicName'), '/Microsoft.EventGrid/', variables('myFuncName'))]",
"type": "Microsoft.EventGrid/topics/providers/eventSubscriptions",
"apiVersion": "2019-01-01",
"location": "[parameters('location')]",
"properties": {
"topic": "[concat('/subscriptions/', parameters('subscriptionId'),'/resourcegroups/', parameters('resourceGroupName'), '/providers/Microsoft.EventGrid/topics/', variables('eventGridTopicName'))]",
"destination": {
"endpointType": "AzureFunction",
"properties": {
"resourceId": "[resourceId('Microsoft.Web/sites/functions/', variables('funcAppName'), variables('myFuncName'))]",
"maxEventsPerBatch": 1,
"preferredBatchSizeInKilobytes": 64
}
},
"filter": {
"advancedFilters": [
{
"operatorType": "StringIn",
"key": "eventType",
"values": [
"xyzEvent"
]
},
{
"operatorType": "StringIn",
"key": "subject",
"values": [
"xyzEventReceived"
]
}
]
},
"labels": [],
"eventDeliverySchema": "EventGridSchema"
},
"dependsOn": [
"[variables('eventGridTopicName')]"
]
}
Earlier, I was using EndpointType as a webhook since new event handlers like Azure Function, storage Queues, etc. were not available (https://learn.microsoft.com/en-us/azure/event-grid/event-handlers). I used the generated arm template from Azure portal as shown below: -
Has anyone faced this issue?
Yes ! found this when I had same issue! ..
Update! found an example that uses another API version and it seems to work beter, now my issue is that there is no code on it when deploying first time, so I need to break the template into two and deploy content in btween (or deploy content via template ofc).
"apiVersion": "2020-01-01-preview",
https://blog.brooksjc.com/2019/07/19/arm-template-for-event-grid-integration-with-a-new-azure-function/
Update 2, after adding the content and rerunning the template, it work fine!
here is my full code for my storage trigger
{
"name": "[concat(variables('storageAccountName'), '/Microsoft.EventGrid/coreCostManagementExport')]",
"type": "Microsoft.Storage/storageAccounts/providers/eventSubscriptions",
"apiVersion": "2020-01-01-preview",
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts',variables('storageAccountName'))]",
"[resourceId('Microsoft.Web/sites',parameters('functionAppName'))]"
],
"properties": {
"topic": "[resourceId('Microsoft.Storage/storageAccounts',variables('storageAccountName'))]",
"destination": {
"endpointType": "AzureFunction",
"properties": {
"resourceId": "[resourceId('Microsoft.Web/sites/functions/', parameters('functionAppName'), 'QueueUsageOnExport')]",
"maxEventsPerBatch": 1,
"preferredBatchSizeInKilobytes": 64
}
},
"filter": {
"subjectBeginsWith": "/blobServices/default/containers/usage",
"subjectEndsWith": ".csv",
"includedEventTypes": [
"Microsoft.Storage.BlobCreated"
],
"advancedFilters": [
]
},
"labels": [
],
"eventDeliverySchema": "EventGridSchema"
}
}
Jakob's suggestion for changing api version worked for me with change in resourceId. Here is my modified working template: -
{
"name": "[concat(variables('eventGridTopicName'), '/Microsoft.EventGrid/', variables('myFuncName'))]",
"type": "Microsoft.EventGrid/topics/providers/eventSubscriptions",
"apiVersion": "2020-01-01-preview",
"location": "[parameters('location')]",
"properties": {
"topic": "[concat('/subscriptions/', parameters('subscriptionId'),'/resourcegroups/', parameters('resourceGroupName'), '/providers/Microsoft.EventGrid/topics/', variables('eventGridTopicName'))]",
"destination": {
"endpointType": "AzureFunction",
"properties": {
"resourceId": "[concat('/subscriptions/', parameters('subscriptionId'),'/resourcegroups/', parameters('resourceGroupName'), '/providers/Microsoft.Web/sites/', variables('funcAppName'), '/functions/' , variables('myFuncName'))]",
"maxEventsPerBatch": 1,
"preferredBatchSizeInKilobytes": 64
}
},
"filter": {
"advancedFilters": [
{
"operatorType": "StringIn",
"key": "eventType",
"values": [
"xyzEvent"
]
},
{
"operatorType": "StringIn",
"key": "subject",
"values": [
"xyzEventReceived"
]
}
]
},
"labels": [],
"eventDeliverySchema": "EventGridSchema"
},
"dependsOn": [
"[variables('eventGridTopicName')]"
]
}
In my scenario, I was trying to add a function app Subscription to an event grid topic using "AzureFunctionEventSubscriptionDestination" as the destination. My issue was I missed adding the /functions/{targetFunctionName} to the resource id.
"resourceId": "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Web/sites/{functionAppName}/functions/{targetFunctionName}"
My goal is to deploy a streaming analytics who contain an eventhub as input. To do this, I need to get the shareAcessPolicyKey. After some search, I found the ListKeys function but still not working for my case.
{
"error": {
"code": "ResourceNotFound",
"message": "The Resource 'Microsoft.ServiceBus/namespaces/tbiNamespace' under resource group 'devOps' was not found."
}
.
EDIT - Solution
"sharedAccessPolicyKey": "[listKeys(resourceId('Microsoft.Eventhub/namespaces/authorizationRules',parameters('namespaces'), parameters('AuthorizationRules_name')),'2017-04-01').primaryKey]"
Create the namespaces rules
{
"type": "Microsoft.EventHub/namespaces/AuthorizationRules",
"name": "[concat(parameters('namespaces_tornosbi_name'), '/', parameters('AuthorizationRules_RootManageSharedAccessKey_name'))]",
"apiVersion": "2017-04-01",
"location": "North Europe",
"scale": null,
"properties": {
"rights": [
"Listen",
"Manage",
"Send"
]
},
"dependsOn": [
"[resourceId('Microsoft.EventHub/namespaces', parameters('namespaces_tornosbi_name'))]"
]
},
create the resource streaming jobs input
"resources": [{
"type": "Microsoft.StreamAnalytics/streamingjobs/inputs",
"name": "[concat(parameters('streamingjobs_tornosbi_name'), '/', parameters('inputs_eh_input_name'))]",
"apiVersion": "2016-03-01",
"scale": null,
"properties": {
"type": "Stream",
"datasource": {
"type": "Microsoft.ServiceBus/EventHub",
"properties": {
"eventHubName": "[parameters('eventhubs_tornosbi_hub_name')]",
"serviceBusNamespace": "[parameters('namespaces_tornosbi_name')]",
"sharedAccessPolicyName": "[parameters('AuthorizationRules_RootManageSharedAccessKey_name')]",
"sharedAccessPolicyKey": "[listKeys(resourceId(concat('Microsoft.ServiceBus/namespaces/','eventhub','/authorizationRules'),parameters('namespaces_tornosbi_name'),parameters('eventhubs_tornosbi_hub_name'),parameters('AuthorizationRules_RootManageSharedAccessKey_name')),'2016-03-01').primaryKey]"
}
},
"compression": {
"type": "None"
},
"serialization": {
"type": "Json",
"properties": {
"encoding": "UTF8"
}
}
},
"dependsOn": [
"[resourceId('Microsoft.StreamAnalytics/streamingjobs', parameters('streamingjobs_tornosbi_name'))]",
"[resourceId('Microsoft.EventHub/namespaces', parameters('namespaces_tornosbi_name'))]",
"[resourceId('Microsoft.EventHub/namespaces/eventhubs', parameters('namespaces_tornosbi_name'), parameters('eventhubs_tornosbi_hub_name'))]"
]
},
the error clearly states there is no such resource in the resource group. Impossible to help you without knowing where the resource is, but resourceId() function accepts resource group and subscription as arguments:
resourceId(subscription, resourcegroup, 'Microsoft.ServiceBus/namespaces/eventhub/authorizationRules',
namespace, eventhub, rule)
ps. you dont need to do concat('Microsoft.ServiceBus/namespaces/','eventhub','/authorizationRules'), just use a string