Related
When I deploy this template via Terraform and Azure Devops, I get an Invalid template error while the template deploys normally on the portal. This is the error:
'The template resource '' of type 'microsoft.insights/workbooks' at
line '1' and column '1512' is not valid. The name property cannot be
null or empty. Please see https://aka.ms/arm-template/#resources for
usage details.'."
AdditionalInfo=[{"info":{"lineNumber":1,"linePosition":1512,"path":"properties.template.resources[0]"},"type":"TemplateViolation"}]
What modification should I make to deploy via Terraform?
{
"contentVersion": "1.0.0.0",
"parameters": {
"workbookDisplayName": {
"type": "string",
"defaultValue": "Azure Firewall Workbook",
"metadata": {
"description": "The friendly name for the workbook that is used in the Gallery or Saved List. This name must be unique within a resource group."
}
},
"workbookType": {
"type": "string",
"allowedValues": [
"workbook",
"sentinel"
],
"defaultValue": "workbook",
"metadata": {
"description": "The gallery that the workbook will been shown under. Supported values include workbook, tsg, etc. Usually, this is 'workbook'"
}
},
"DiagnosticsWorkspaceName": {
"type": "string",
"defaultValue": "WorkspaceName",
"metadata": {
"description": "Provide the workspace name for your Network Diagnostic logs"
}
},
"DiagnosticsWorkspaceSubscription": {
"type": "string",
"defaultValue": "WorkspaceSubscriptionID",
"metadata": {
"description": "Provide the workspace subscription GUID for your Network Diagnostic logs"
}
},
"DiagnosticsWorkspaceResourceGroup": {
"type": "string",
"defaultValue": "ResourceGroupName",
"metadata": {
"description": "Provide the workspace resourcegroupname for your Network Diagnostic logs"
}
},
"workbookId": {
"type": "string",
"defaultValue": "[newGuid()]",
"metadata": {
"description": "The unique guid for this workbook instance"
}
}
},
"variables": {
"workbookSourceId": "[concat('/subscriptions/',parameters('DiagnosticsWorkspaceSubscription'),'/resourcegroups/', parameters('DiagnosticsWorkspaceResourceGroup'), '/providers/Microsoft.OperationalInsights/workspaces/',parameters('DiagnosticsWorkspaceName'))]"
},
"resources": [
{
"name": "[parameters('workbookId')]",
"type": "microsoft.insights/workbooks",
"location": "[resourceGroup().location]",
"apiVersion": "2018-06-17-preview",
"dependsOn": [],
"kind": "shared",
"properties": {
"displayName": "[parameters('workbookDisplayName')]"}",
"version": "1.0",
"sourceId": "[variables('workbookSourceId')]",
"category": "[parameters('workbookType')]"
}
}
],
"outputs": {
"workbookId": {
"type": "string",
"value": "[resourceId( 'microsoft.insights/workbooks', parameters('workbookId'))]"
}
},
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#"
}
I don't know ARM templates but I have really good experience on Terraform AWS & Terraform Azure providers.
First of all, you better take a look Terraform resource page which is here. That would be helpful in understanding resource needs and outcomes.
I might be wrong, because your Terraform script is not visible in the question section. Nevertheless, I guess you might have an issue in Terraform side. As I understood, you are getting error from ARM templates. It is complaining about missing name parameter which is mandatory. You may forget passing parameter names from Terraform to ARM template. I might be wrong, this is just a suggestion, the correct way would be reviewing Terraform azurerm_template_deployment resource.
// ARM Template part
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"storageAccountType": {
"type": "string",
"defaultValue": "Standard_LRS",
"allowedValues": [
"Standard_LRS",
"Standard_GRS",
"Standard_ZRS"
],
"metadata": {
"description": "Storage Account type"
}
}
},
// Terraform resource provisioning
parameters = {
"storageAccountType" = "Standard_GRS"
}
I am trying to deploy workflows in Azure using workflows Rest API's(https://learn.microsoft.com/en-us/rest/api/logic/workflows ). I want to enable Log Analytics Workspace while deploying this workflow in Logic App. This is straight forward in portal while creating logic app. But I didn't find any document or information to pass log analytics workspace while creating workflow definition(JSON File). Please let me know, if there is a way to pass it
In a comment above you mentioned "If it's possible in ARM Template, then I am fine with it and I will use ARM Template deployment API's to deploy it."
Here's an example, feel free to amend it so that it suits your needs:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"logicapp-name": {
"type": "string",
"defaultValue": "testLogicApp",
"metadata": {
"description": "Name of the Logic App"
}
},
"loganalytics-workspace-resourceid": {
"type": "string",
"defaultValue": "/subscriptions/11111111-2222-3333-4444-555555555555/resourcegroups/testResourceGroup/providers/microsoft.operationalinsights/workspaces/testLogAnalyticsWorkspace",
"metadata": {
"description": "Resource ID of the Log Analytics workspace"
}
}
},
"variables": {
"logAnalyticsSettingName": "testDiagnosticSettingName"
},
"resources": [
{
"type": "Microsoft.Logic/workflows/providers/diagnosticSettings",
"name": "[concat(parameters('logicapp-name'),'/Microsoft.Insights/', variables('logAnalyticsSettingName'))]",
"apiVersion": "2017-05-01-preview",
"properties": {
"workspaceId": "[parameters('loganalytics-workspace-resourceid')]",
"logs": [
{
"category": "WorkflowRuntime",
"enabled": true,
"retentionPolicy": {
"days": 0,
"enabled": false
}
}
],
"metrics": [
{
"timeGrain": "PT1M",
"enabled": true,
"retentionPolicy": {
"enabled": false,
"days": 0
}
}
]
}
}
]
}
I am trying to deploy a key vault with secretName and secretValue and I have created a variable group in azure devops with all the secrets and I am using the below parameters in parameter file, but when this gets deployed the secret value gets stored as $(secret) and not the password actually stored in the task group in Azure DevOps.
"secretsObject": {
"value": {
"secrets": [
{
"secretName": "App012",
"secretValue": "$(mysecret)"
},
and this is what I got in the key vault template:
{
"type": "Microsoft.KeyVault/vaults/secrets",
"name": "[concat(parameters('keyVaultName'), '/', parameters('secretsObject').secrets[copyIndex()].secretName)]",
"apiVersion": "2018-02-14",
"dependsOn": [
"[concat('Microsoft.KeyVault/vaults/', parameters('keyVaultName'))]"
],
"copy": {
"name": "secretsCopy",
"count": "[length(parameters('secretsObject').secrets)]"
},
"properties": {
"value": "[parameters('secretsObject').secrets[copyIndex()].secretValue]"
}
}
]
}
Any idea how to pass the "secretvalue" as a variable?
I believe your asking how to leverage your secrets that are stored as a variable group to be deployed securely with your ARM template via Azure DevOps. If that is the case look at using Override Template Parameters in your release task.
This would be in the format of -NameOfARMParameter $(NameofDevOpsVariable)
In your case it would be -mysecret $(NameOfDevOpsVariable)
The deployment .json should look like this for parameter declaration:
"secretValue": {
"type": "string",
"metadata": {
"description": "This is for receiving a value from DevOps releases of the secret to be stored in the key vault"
}
},
"secretName": {
"type": "string",
"metadata": {
"description": "Name of the Secret"
}
},
For the actual deployment
{
"type": "Microsoft.KeyVault/vaults/secrets",
"name": "[concat(variables('keyVaultName'),'/',parameters('secretName'))]",
"apiVersion": "2018-02-14",
"properties": {
"contentType": "text/plain",
"value": "[parameters('secretValue')]"
},
"dependsOn": [
"[resourceId('Microsoft.KeyVault/vaults', variables('keyVaultName'))]"
]
},
And the parameters file doesn't need to have anything in it if these values will be fed from Dev Ops
You need to create a parameter file with the secret / link to Key Vault.
Here's a sample of it:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"adminLogin": {
"value": "exampleadmin"
},
"adminPassword": {
"reference": {
"keyVault": {
"id": "/subscriptions/<subscription-id>/resourceGroups/<rg-name>/providers/Microsoft.KeyVault/vaults/<vault-name>"
},
"secretName": "ExamplePassword"
}
},
"sqlServerName": {
"value": "<your-server-name>"
}
}
}
More info:
https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/parameter-files
and https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/add-template-to-azure-pipelines
I have an ACS Kubernetes cluster that was created with an agent count of 1. I went to the portal to increase the agent count to 2 and received a generic error saying the provisioning of resource(s) for container service failed.
Looking at the activity logs, there is a bit more information.
Write ContainerServices - PreconditionFailed - Provisioning of resource(s) for container service 'xxxxxxx' in
resource group 'xxxxxxxx' failed.
Validate - InvalidTemplate - Deployment template validation failed: 'The resource 'Microsoft.Network/networkSecurityGroups/k8s-master-3E4D5818-nsg' is not defined in the template. Please see https://aka.ms/arm-template for usage details.'.
Trying to change it via the Azure CLI 2.0 also returns the same error.
Update: The cluster was stood up using an ARM template with a single container service resource based on the sample in the quickstart templates repo.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"dnsNamePrefix": {
"type": "string",
"metadata": {
"description": "Sets the Domain name prefix for the cluster. The concatenation of the domain name and the regionalized DNS zone make up the fully qualified domain name associated with the public IP address."
}
},
"agentCount": {
"type": "int",
"defaultValue": 1,
"metadata": {
"description": "The number of agents for the cluster. This value can be from 1 to 100 (note, for Kubernetes clusters you will also get 1 or 2 public agents in addition to these seleted masters)"
},
"minValue":1,
"maxValue":100
},
"agentVMSize": {
"type": "string",
"defaultValue": "Standard_D2_v2",
"allowedValues": [
"Standard_A0", "Standard_A1", "Standard_A2", "Standard_A3", "Standard_A4", "Standard_A5",
"Standard_A6", "Standard_A7", "Standard_A8", "Standard_A9", "Standard_A10", "Standard_A11",
"Standard_D1", "Standard_D2", "Standard_D3", "Standard_D4",
"Standard_D11", "Standard_D12", "Standard_D13", "Standard_D14",
"Standard_D1_v2", "Standard_D2_v2", "Standard_D3_v2", "Standard_D4_v2", "Standard_D5_v2",
"Standard_D11_v2", "Standard_D12_v2", "Standard_D13_v2", "Standard_D14_v2",
"Standard_G1", "Standard_G2", "Standard_G3", "Standard_G4", "Standard_G5",
"Standard_DS1", "Standard_DS2", "Standard_DS3", "Standard_DS4",
"Standard_DS11", "Standard_DS12", "Standard_DS13", "Standard_DS14",
"Standard_GS1", "Standard_GS2", "Standard_GS3", "Standard_GS4", "Standard_GS5"
],
"metadata": {
"description": "The size of the Virtual Machine."
}
},
"linuxAdminUsername": {
"type": "string",
"defaultValue": "azureuser",
"metadata": {
"description": "User name for the Linux Virtual Machines."
}
},
"orchestratorType": {
"type": "string",
"defaultValue": "Kubernetes",
"allowedValues": [
"Kubernetes",
"DCOS",
"Swarm"
],
"metadata": {
"description": "The type of orchestrator used to manage the applications on the cluster."
}
},
"masterCount": {
"type": "int",
"defaultValue": 1,
"allowedValues": [
1
],
"metadata": {
"description": "The number of Kubernetes masters for the cluster."
}
},
"sshRSAPublicKey": {
"type": "string",
"metadata": {
"description": "Configure all linux machines with the SSH RSA public key string. Your key should include three parts, for example 'ssh-rsa AAAAB...snip...UcyupgH azureuser#linuxvm'"
}
},
"servicePrincipalClientId": {
"metadata": {
"description": "Client ID (used by cloudprovider)"
},
"type": "securestring",
"defaultValue": "n/a"
},
"servicePrincipalClientSecret": {
"metadata": {
"description": "The Service Principal Client Secret."
},
"type": "securestring",
"defaultValue": "n/a"
}
},
"variables": {
"adminUsername":"[parameters('linuxAdminUsername')]",
"agentCount":"[parameters('agentCount')]",
"agentsEndpointDNSNamePrefix":"[concat(parameters('dnsNamePrefix'),'agents')]",
"agentVMSize":"[parameters('agentVMSize')]",
"masterCount":"[parameters('masterCount')]",
"mastersEndpointDNSNamePrefix":"[concat(parameters('dnsNamePrefix'),'mgmt')]",
"orchestratorType":"[parameters('orchestratorType')]",
"sshRSAPublicKey":"[parameters('sshRSAPublicKey')]",
"servicePrincipalClientId": "[parameters('servicePrincipalClientId')]",
"servicePrincipalClientSecret": "[parameters('servicePrincipalClientSecret')]",
"useServicePrincipalDictionary": {
"DCOS": 0,
"Swarm": 0,
"Kubernetes": 1
},
"useServicePrincipal": "[variables('useServicePrincipalDictionary')[variables('orchestratorType')]]",
"servicePrincipalFields": [
null,
{
"ClientId": "[parameters('servicePrincipalClientId')]",
"Secret": "[parameters('servicePrincipalClientSecret')]"
}
]
},
"resources": [
{
"apiVersion": "2016-09-30",
"type": "Microsoft.ContainerService/containerServices",
"location": "[resourceGroup().location]",
"name":"[resourceGroup().name]",
"properties": {
"orchestratorProfile": {
"orchestratorType": "[variables('orchestratorType')]"
},
"masterProfile": {
"count": "[variables('masterCount')]",
"dnsPrefix": "[variables('mastersEndpointDNSNamePrefix')]"
},
"agentPoolProfiles": [
{
"name": "agentpools",
"count": "[variables('agentCount')]",
"vmSize": "[variables('agentVMSize')]",
"dnsPrefix": "[variables('agentsEndpointDNSNamePrefix')]"
}
],
"linuxProfile": {
"adminUsername": "[variables('adminUsername')]",
"ssh": {
"publicKeys": [
{
"keyData": "[variables('sshRSAPublicKey')]"
}
]
}
},
"servicePrincipalProfile": "[variables('servicePrincipalFields')[variables('useServicePrincipal')]]"
}
}
],
"outputs": {
"masterFQDN": {
"type": "string",
"value": "[reference(concat('Microsoft.ContainerService/containerServices/', resourceGroup().name)).masterProfile.fqdn]"
},
"sshMaster0": {
"type": "string",
"value": "[concat('ssh ', variables('adminUsername'), '#', reference(concat('Microsoft.ContainerService/containerServices/', resourceGroup().name)).masterProfile.fqdn, ' -A -p 22')]"
},
"agentFQDN": {
"type": "string",
"value": "[reference(concat('Microsoft.ContainerService/containerServices/', resourceGroup().name)).agentPoolProfiles[0].fqdn]"
}
}
}
This is a known service issue for old clusters. A fix is currently rolling out and is being tracked in this github issue, https://github.com/Azure/ACS/issues/16
Jack (a dev on the ACS team)
I had test in my lab with this template, but I can't reproduce your error.
please try to use azure resource explorer to edit the count of agent pool:
I have an existing Service Bus with one queue and event hub deployed using Azure Resource Manager.
Now I am interested to retrieve the primary key and connection string using Azure PowerShell wiithout using the ServiceBus.dll. Is it possible??
As a workaround I have created an ARM template which does not deploy anything but just query the existing resource and retrieve the information I need. The below template retrieves the connection string and primary key of an event hub/queue for a specific service bus namespace
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"serviceBusNamespace": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "The name of the service bus namespace to create."
}
},
"resourceName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "The name of the resource to be retreived."
}
},
"resourceType": {
"type": "string",
"minLength": 1,
"allowedValues": [
"queues",
"eventhubs"
],
"metadata": {
"description": "The type of the resource"
}
},
"policy": {
"type": "string",
"minLength": 1,
"defaultValue": "ManagePolicy",
"allowedValues": [
"ManagePolicy",
"SendPolicy",
"ListenPolicy"
],
"metadata": {
"description": "The type of the resource"
}
}
},
"variables": {
},
"resources": [ ],
"outputs": {
"connectionString": {
"type": "string",
"value": "[listKeys(resourceId(concat('Microsoft.ServiceBus/namespaces/',parameters('resourceType'),'/authorizationRules'),parameters('serviceBusNamespace'),parameters('resourceName'),parameters('policy')),'2015-08-01').primaryConnectionString]"
},
"primaryKey": {
"type": "string",
"value": "[listKeys(resourceId(concat('Microsoft.ServiceBus/namespaces/',parameters('resourceType'),'/authorizationRules'),parameters('serviceBusNamespace'),parameters('resourceName'),parameters('policy')),'2015-08-01').primaryKey]"
}
}
}
Is it abusing to use ARM template to query for a resource and not actually deploy anything?
EDIT
To capture the output of the ARM template within PowerShell use the below code
$ep = New-AzureRmResourceGroupDeployment -Name "getEventHub" -ResourceGroupName myResourceGroup -Mode Incremental -TemplateFile getEventHub.json -TemplateParameterFile getEventHub.param.json
$RuleConnString = $ep.Outputs.connectionString.value
$RulePrimaryKey = $ep.Outputs.primaryKey.value
Note that the property names connectionString and primaryKey are same as defined in my template file
EDIT 2
If I re-run the ARM template to deploy the event hub second time I get the below error.
I din't find any option other than to use the ARM template to query the details.
I don’t see what’s wrong with what you’re doing. In my view Resource Manager templates in their nature are incremental. So you could author a template to create your existing service bus with the same resources. If the properties are the same then it will leave the existing resources intact and return you the connection string and primary key of the relevant resource.
I have a need to automate the creation of a service bus and queue and separate send/listen shared access policies. You can retrieve the connection string on the service bus itself using PowerShell natively without using the .Net ServiceBus.dll assembly by using Get-AzureSBAuthorizationRule but due to a still current bug this doesn’t work at the queue level.
I tried using the ServiceBus.dll to create the shared access policies but sometimes it would randomly fail but subsequently work if you ran it a second time immediately afterwards. I also tried Resource Manager templates but previously you had to pass in the keys you’d generated yourself. Now I see Microsoft generate those for you but you’re still left trying to get the key in an automated fashion so I like your solution.
One question though, can you capture the Resource Manager template outputs and pass them back to a PowerShell script, do you know?
Cheers
Rob
{ "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": {
"servicebusNamespace": {
"type": "string",
"metadata": {
"description": "The service bus namespace"
}
},
"notificationssmsqueue": {
"type": "string",
"metadata": {
"description": "Notifications SMS queue"
}
} }, "variables": {
"location": "[resourceGroup().location]", }, "resources": [
{
"apiVersion": "2015-08-01",
"name": "[parameters('servicebusNamespace')]",
"type": "Microsoft.ServiceBus/namespaces",
"location": "[variables('location')]",
"properties": {
"messagingSku": 2
},
"resources": [
{
"apiVersion": "2015-08-01",
"name": "[parameters('notificationssmsqueue')]",
"type": "Queues",
"dependsOn": [
"[concat('Microsoft.ServiceBus/namespaces/', parameters('servicebusNamespace'))]"
],
"properties": {
"path": "[parameters('notificationssmsqueue')]"
},
"resources": [
{
"apiVersion": "2015-08-01",
"name": "[concat(parameters('notificationssmsqueue'),'.listen')]",
"type": "AuthorizationRules",
"dependsOn": [
"[parameters('notificationssmsqueue')]"
],
"properties": {
"keyName": "[concat(parameters('notificationssmsqueue'),'.listen')]",
"claimType": "SharedAccessKey",
"claimValue": "None",
"rights": [ "Listen" ],
"revision": -1
}
},
{
"apiVersion": "2015-08-01",
"name": "[concat(parameters('notificationssmsqueue'),'.send')]",
"type": "AuthorizationRules",
"dependsOn": [
"[parameters('notificationssmsqueue')]"
],
"properties": {
"keyName": "[concat(parameters('notificationssmsqueue'),'.send')]",
"claimType": "SharedAccessKey",
"claimValue": "None",
"rights": [ "Send" ],
"revision": -1
}
}
]
}
]
} ], "outputs": {
"connectionString": {
"type": "string",
"value": "[listKeys(resourceId(concat('Microsoft.ServiceBus/namespaces/AuthorizationRules'),parameters('serviceBusNamespace'),'RootManageSharedAccessKey'),'2015-08-01').primaryConnectionString]"
},
"smsSendPrimaryKey": {
"type": "string",
"value": "[listKeys(resourceId(concat('Microsoft.ServiceBus/namespaces/Queues/AuthorizationRules'),parameters('serviceBusNamespace'),parameters('notificationssmsqueue'),concat(parameters('notificationssmsqueue'),'.send')),'2015-08-01').PrimaryKey]"
},
"smsListenPrimaryKey": {
"type": "string",
"value": "[listKeys(resourceId(concat('Microsoft.ServiceBus/namespaces/Queues/AuthorizationRules'),parameters('serviceBusNamespace'),parameters('notificationssmsqueue'),concat(parameters('notificationssmsqueue'),'.listen')),'2015-08-01').PrimaryKey]"
} } }
But I call my templates like this:
New-AzureRMResourceGroupDeployment -ResourceGroupName $ResourceGroupName -TemplateFile "$scripts_folder$SB_create_script" -TemplateParameterObject `
#{ servicebusNamespace = $servicebusNamespace;
notificationssmsqueue = $NotificationSMSqueue }
This is the correct way to get the information you are seeking. The Resource Manager provides a common interface to interact with all the services. It is how the Portal access the services, and each of the language SDKs are just wrappers for similar requests to the one you have created.
I usually use the Python or java SDKs, but I have been told that NodeJS is a very easy way to wrap the Web APIs that ARM calls to construct similar calls like the one you made, if you are looking for a none ARM way to do this.