I would like to enable diagnostics for Azure Datafactory using ARM-Template.
I have found this post and then I try to test my code:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": { },
"variables": {},
"resources": [
{
"name": "test-adf2-we",
"type": "Microsoft.DataFactory/factories",
"apiVersion": "2018-06-01",
"location": "West Europe",
"identity": {
"type": "SystemAssigned"
}
},
{
"name": "test-ala-we",
"type": "Microsoft.OperationalInsights/workspaces",
"apiVersion": "2015-11-01-preview",
"location": "West Europe"
},
{
"type": "microsoft.datafactory/factories/providers/diagnosticsettings",
"name": "[concat('test-adf2-we','/Microsoft.Insights/diagnostics')]",
"location": "West Europe",
"apiVersion": "2017-05-01-preview",
"properties": {
"name": "diagnostics",
"storageAccountId": null,
"eventHubAuthorizationRuleId": null,
"eventHubName": null,
"workspaceId": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/test-arm/providers/Microsoft.OperationalInsights/test-ala-we"
"logs": [
{
"category": "PipelineRuns",
"enabled": true,
"retentionPolicy": {
"enabled": false,
"days": 0
}
},
{
"category": "TriggerRuns",
"enabled": true,
"retentionPolicy": {
"enabled": false,
"days": 0
}
},
{
"category": "ActivityRuns",
"enabled": true,
"retentionPolicy": {
"enabled": false,
"days": 0
}
}
],
"metrics": [
{
"category": "AllMetrics",
"timeGrain": "PT1M",
"enabled": true,
"retentionPolicy": {
"enabled": false,
"days": 0
}
}
]
}
}
}
After running this code. I get this error:
Template deployment returned the following errors:
Resource microsoft.datafactory/factories/providers/diagnosticsettings 'test-adf2-we/Microsoft.Insights/diagnostics' failed with message '{
"code": "BadRequest",
"message": "\"Resource type 'microsoft.operationalinsights/test-ala-we' is invalid for property 'properties.workspaceId'. Expected types are 'microsoft.operationalinsights/workspaces'\""
}'
could you help me to fix this problem?
Be sure to add
"logAnalyticsDestinationType": "Dedicated"
Otherwise Log Analytics will write to the Default AzureDiagnostic Tables. This is Microsoft Best Practice when using Log Analytics since there is a hard stop at 500 columns in the AzureDiagnostics table. Once the 500 column threshold is met records won't be inserted into the table.
your resourceId is wrong, fix it like so:
"/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/test-arm/providers/Microsoft.OperationalInsights/workspaces/test-ala-we"
you left out workspaces (and this is what the error is telling you).
Related
I have created an Event Hub namespace with event hub and trying to add it into application gateway diagnostic settings but it is not working. (Note: Using nested template)
Here is a working arm template for event-hub:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"hapiEnvironmentName": {
"type": "string"
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]"
},
"eventHubSku": {
"type": "string",
"defaultValue": "Standard",
"allowedValues": [ "Basic", "Standard" ]
}
},
"variables": {
"eventHubNamespaceName": "[concat(parameters('hapiEnvironmentName'), 'GatewayWafLogs')]",
"eventHubName": "[concat(parameters('hapiEnvironmentName'), 'AppGWafLogs')]"
},
"resources": [
{
"type": "Microsoft.EventHub/namespaces",
"apiVersion": "2017-04-01",
"name": "[variables('eventHubNamespaceName')]",
"location": "[parameters('location')]",
"sku": {
"name": "[parameters('eventHubSku')]",
"tier": "[parameters('eventHubSku')]",
"capacity": 5
},
"properties": {
"isAutoInflateEnabled": true,
"maximumThroughputUnits": 10
}
},
{
"type": "Microsoft.EventHub/namespaces/eventhubs",
"apiVersion": "2017-04-01",
"name": "[concat(variables('eventHubNamespaceName'), '/', variables('eventHubName'))]",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.EventHub/namespaces', variables('eventHubNamespaceName'))]"
],
"properties": {
"messageRetentionInDays": 5,
"partitionCount": 1
}
}
]
}
And here is a piece of application gateway arm template through which I am trying to add Event hub as diagnostic setting.
{
"apiVersion": "2017-05-01-preview",
"type": "Microsoft.Network/applicationGateways/providers/diagnosticSettings",
"name": "[variables('diagnosticSettingsNameForWafLogs')]",
"properties": {
"storageAccountId": null,
"workspaceId": null,
"eventHubAuthorizationRuleId": "/subscriptions/xxx-xxxx-xxxx-xxxxxxx/resourceGroups/xxxxxxxx/providers/Microsoft.EventHub/namespaces/xxxxxxxxx/authorizationrules/RootManageSharedAccessKey",
"eventHubName": "param[('eventhubname')]",
"metrics": [
{
"category": "AllMetrics",
"enabled": false,
"retentionPolicy": {
"enabled": false,
"days": 0
}
}
],
"logs": [
{
"category": "ApplicationGatewayAccessLog",
"enabled": false,
"retentionPolicy": {
"enabled": false,
"days": 0
}
},
{
"category": "ApplicationGatewayFirewallLog",
"enabled": true,
"retentionPolicy": {
"enabled": false,
"days": 0
}
},
{
"category": "ApplicationGatewayPerformanceLog",
"enabled": false,
"retentionPolicy": {
"enabled": false,
"days": 0
}
}
]
},
"dependsOn": [
"[resourceId('Microsoft.Network/applicationGateways', variables('agwafv2Name'))]"
]
}
The error I am getting is:
{
"code": "DeploymentFailed",
"message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
"details": [
{
"code": "NoRegisteredProviderFound",
"message": "No registered resource provider found for location 'westcentralus' and API version '2017-05-01-preview' for type 'applicationGateways'. The supported api-versions are '2014-12-01-preview, 2015-05-01-preview, 2015-06-15, 2016-03-30, 2016-06-01, 2016-07-01, 2016-08-01, 2016-09-01, 2016-10-01, 2016-11-01, 2016-12-01, 2017-03-01, 2017-04-01, 2017-06-01, 2017-08-01, 2017-09-01, 2017-10-01, 2017-11-01, 2018-01-01, 2018-02-01, 2018-03-01, 2018-04-01, 2018-05-01, 2018-06-01, 2018-07-01, 2018-08-01, 2018-10-01, 2018-11-01, 2018-12-01, 2019-02-01, 2019-04-01, 2019-06-01, 2019-07-01, 2019-08-01, 2019-09-01, 2019-11-01, 2019-12-01, 2020-01-01, 2020-03-01, 2020-04-01, 2020-05-01, 2020-06-01, 2020-07-01, 2020-08-01, 2020-11-01, 2021-01-01, 2021-02-01, 2021-03-01'. The supported locations are 'westus, eastus, northeurope, westeurope, eastasia, southeastasia, northcentralus, southcentralus, centralus, eastus2, japaneast, japanwest, brazilsouth, australiaeast, australiasoutheast, centralindia, southindia, westindia, canadacentral, canadaeast, westcentralus, westus2, ukwest, uksouth, koreacentral, koreasouth, francecentral, australiacentral, southafricanorth, uaenorth, switzerlandnorth, germanywestcentral, norwayeast, westus3, jioindiawest'."
}
]
}
But the same is working for Log Analytics.
Your template use 2017-05-01-preview api version. In the error message states that this API version is not supported. Does it help if you update api to 2021-03-01.
When I am enabling the diagonstic setting fromt the azure portal for ADF & Azuresql, in the ARM template I am not able to find anything in ARM with respect to diagonstic setting.Similar way for keyvault and sql I need the ARM template for enabling the diagonstic setting.
I tried from my side for ADF since I new to ARM template I am not able to find the method for enabling the diagonstic setting.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"factoryName": {
"type": "string",
"metadata": {
"description": "The name of the Data Factory"
}
}
},
"resources": [
{
"type": "Microsoft.DataFactory/factories",
"apiVersion": "2018-06-01",
"name": "[parameters('factoryName')]",
"location": "[resourceGroup().location]",
"identity": {
"type": "SystemAssigned"
},
"properties": {
},
"resources": [
{
"type": "Microsoft.DataFactory/factories/providers/diagnosticSettings",
"apiVersion": "2017-05-01-preview",
"name": "[concat(parameters('factoryName'),'/microsoft.insights/', parameters('settingName'))]",
"location": "[resourceGroup().location]",
"dependsOn": [
"[concat('Microsoft.DataFactory/factories/', parameters('factoryName'))]"
],
"properties": {
"name": "[parameters('DS03')]",
"workspaceId": "[/subscriptions/3xxxxx-xxxxx-x-xxxx--xx/resourceGroups/BDAZxfdfG01]"
}
}
]
}
]
}
The ARM template above is creating the diagnostic settings; however it is not actually configuring the logging of anything. Add the following for all Data Factory metrics after your workspaceID property.
"logAnalyticsDestinationType": "Dedicated",
"logs": [
{
"category": "PipelineRuns",
"enabled": true,
"retentionPolicy": {
"enabled": false,
"days": 0
}
},
{
"category": "TriggerRuns",
"enabled": true,
"retentionPolicy": {
"enabled": false,
"days": 0
}
},
{
"category": "ActivityRuns",
"enabled": true,
"retentionPolicy": {
"enabled": false,
"days": 0
}
}
],
"metrics": [
{
"category": "AllMetrics",
"timeGrain": "PT1M",
"enabled": true,
"retentionPolicy": {
"enabled": false,
"days": 0
}
}
]
Besides configuring the diagnostic settings what metrics and diagnostics must be select to send to log analytics. These fields align to those on the diagnostic blade:
The "logAnalyticsDestinationType": "Dedicated" is to ensure the logs go to their own table as opposed to the default AzureDiagnostic table. There is documented limitation in the original table.
When i try to deploy/enable diagnostic settings on Azure Firewall through ARM template. Though i get error.
I'm following this link
code snippet which i'm using:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {},
"variables": {
"workspaceId": "[Concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', resourceGroup().name, '/providers/Microsoft.OperationalInsights/workspaces/', 'Fw-LA')]"
},
"resources": [
{
"type": "Microsoft.Network/azureFirewalls/diagnosticSettings",
"apiVersion": "2017-05-01-preview",
"name": "[concat('FWHM/','Microsoft.Insights/', 'dignostic')]",
"properties": {
"name": "diagnostic",
"workspaceId": "[variables('workspaceId')]",
"logs": [
{
"category": "AzureFirewallApplicationRule",
"enabled": true,
"retentionPolicy": {
"days": 10,
"enabled": false
}
}
],
"metrics": [
{
"category": "AllMetrics",
"enabled": true,
"retentionPolicy": {
"enabled": false,
"days": 0
}
}
]
}
}
]
}
this:
"type": "Microsoft.Network/azureFirewalls/diagnosticSettings",
should be this instead:
"type": "Microsoft.Network/azureFirewalls/providers/diagnosticSettings",
read the article you linked carefully
I'm attempting to deploy to a new resource group containing an existing app service plan in Azure using an ARM script. If I run the deployment through the Azure Portal UI, it is successful. The issue happens when I try to download the template ARM script for the deployment and use that.
I'm attempting to create a Web app and associated application insights instance.
Here is my template.json
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"subscriptionId": {
"type": "string"
},
"name": {
"type": "string"
},
"location": {
"type": "string"
},
"hostingEnvironment": {
"type": "string"
},
"hostingPlanName": {
"type": "string"
},
"serverFarmResourceGroup": {
"type": "string"
},
"alwaysOn": {
"type": "bool"
},
"currentStack": {
"type": "string"
}
},
"resources": [
{
"apiVersion": "2018-02-01",
"name": "[parameters('name')]",
"type": "Microsoft.Web/sites",
"location": "[parameters('location')]",
"tags": {},
"dependsOn": [
"microsoft.insights/components/LicensingService-API"
],
"properties": {
"name": "[parameters('name')]",
"siteConfig": {
"appSettings": [
{
"name": "APPINSIGHTS_INSTRUMENTATIONKEY",
"value": "[reference('microsoft.insights/components/LicensingService-API', '2015-05-01').InstrumentationKey]"
},
{
"name": "ApplicationInsightsAgent_EXTENSION_VERSION",
"value": "~2"
},
{
"name": "XDT_MicrosoftApplicationInsights_Mode",
"value": "default"
},
{
"name": "DiagnosticServices_EXTENSION_VERSION",
"value": "disabled"
},
{
"name": "APPINSIGHTS_PROFILERFEATURE_VERSION",
"value": "disabled"
},
{
"name": "APPINSIGHTS_SNAPSHOTFEATURE_VERSION",
"value": "disabled"
},
{
"name": "InstrumentationEngine_EXTENSION_VERSION",
"value": "disabled"
},
{
"name": "SnapshotDebugger_EXTENSION_VERSION",
"value": "disabled"
},
{
"name": "XDT_MicrosoftApplicationInsights_BaseExtensions",
"value": "disabled"
}
],
"metadata": [
{
"name": "CURRENT_STACK",
"value": "[parameters('currentStack')]"
}
],
"alwaysOn": "[parameters('alwaysOn')]"
},
"serverFarmId": "[concat('/subscriptions/', parameters('subscriptionId'),'/resourcegroups/', parameters('serverFarmResourceGroup'), '/providers/Microsoft.Web/serverfarms/', parameters('hostingPlanName'))]",
"hostingEnvironment": "[parameters('hostingEnvironment')]",
"clientAffinityEnabled": true
}
},
{
"apiVersion": "2015-05-01",
"name": "LicensingService-API",
"type": "microsoft.insights/components",
"location": "westus2",
"tags": {},
"properties": {
"ApplicationId": "[parameters('name')]",
"Request_Source": "IbizaWebAppExtensionCreate"
}
}
]
}
And my parameters.json
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"subscriptionId": {
"value": "REMOVED"
},
"name": {
"value": "LicensingService-API"
},
"location": {
"value": "West US 2"
},
"hostingEnvironment": {
"value": ""
},
"hostingPlanName": {
"value": "LicensingServiceProductionAppServicePlan"
},
"serverFarmResourceGroup": {
"value": "LicensingServicePROD"
},
"alwaysOn": {
"value": true
},
"currentStack": {
"value": "dotnetcore"
}
}
}
There is one particular parameter that I'm having issues with. It is the "hostingEnvironment" parameter. I am unable to determine what should be placed in that field, as the default template provided by Azure leaves this blank. If I enter a value here (LicensingServiceProductionAppServicePlan for example), I get an error on the deployment of the web app that reads:
{
"Code": "NotFound",
"Message": "Cannot find Stamp with name LicensingServiceProductionAppServicePlan.",
"Target": null,
"Details": [
{
"Message": "Cannot find Stamp with name LicensingServiceProductionAppServicePlan."
},
{
"Code": "NotFound"
},
{
"ErrorEntity": {
"ExtendedCode": "51004",
"MessageTemplate": "Cannot find {0} with name {1}.",
"Parameters": [
"Stamp",
"LicensingServiceProductionAppServicePlan"
],
"Code": "NotFound",
"Message": "Cannot find Stamp with name LicensingServiceProductionAppServicePlan."
}
}
],
"Innererror": null
}
If I instead remove the parameter from both the template and the parameters files, as suggested in this answer, I get a BadRequest error that reads:
{
"error": {
"code": "InvalidTemplate",
"message": "Unable to process template language expressions for resource '/subscriptions/REMOVED/resourceGroups/LicensingServicePROD/providers/Microsoft.Web/serverfarms/LicensingServicePROD' at line '151' and column '9'. 'The template parameter 'hostingEnvironment' is not found. Please see https://aka.ms/arm-template/#parameters for usage details.'",
"additionalInfo": [
{
"type": "TemplateViolation",
"info": {
"lineNumber": 151,
"linePosition": 9,
"path": ""
}
}
]
}
}
Likely this is because I can see that the "hostingEnvironment" parameter is used in the template script.
So I'm left wondering why this works when done through the Azure UI but not from the script generated from the UI. My final question that I'm looking to solve is what is the value that should be provided for the "hostingEnvironment" parameter?
First hostingEnvironment is not required. It is required if you have an App Service Environment and you want to deploy the site on it.
You can leave it empty the value or remove it from the template.
See the details from the template reference site Web Site template reference
The solution is to make the following changes:
template.json:
"hostingEnvironment": {
"type": "string",
"defaultValue": ""
},
parameters.json
"hostingEnvironment": {
"value": ""
},
I'm using Azure RM Template deployments with a Visual Studio 2017 Resource Group project to deploy IoTHub instance with diagnostics settings in Log Analytics.
Deploying IoTHub alone is successful, the problem is with deployment of the Diagnostics Settings template.
I'm following the instructions for deploying Diagnostics Settings as Non-Compute resource template
The strange error that I receive is the following:
Error: Code=InvalidTemplate; Message=Deployment template validation
failed: 'The template resource 'Microsoft.Insights/BasicDiagnostics'
for type 'providers/diagnosticSettings' at line '69' and column '9'
has incorrect segment lengths. A nested resource type must have
identical number of segments as its resource name. A root resource
type must have segment length one greater than its resource name.
Why does it fail like this, even though I follow the documentation with the provided example?
Here are my template definitions:
"resources": [
{
"type": "Microsoft.Devices/IotHubs",
"sku": {
"name": "[parameters('sku.name')]",
"capacity": "[parameters('sku.units')]"
},
"name": "[parameters('iothubname')]",
"apiVersion": "2018-04-01",
"location": "[resourceGroup().location]",
"properties": {
"eventHubEndpoints": {
"events": {
"retentionTimeInDays": "[parameters('endpoints.events.retention')]",
"partitionCount": "[parameters('endpoints.events.partitions')]"
},
"operationsMonitoringEvents": {
"retentionTimeInDays": "[parameters('endpoints.operationsMonitoringEvents.retention')]",
"partitionCount": "[parameters('endpoints.operationsMonitoringEvents.partitions')]"
}
},
"features": "[parameters('features')]"
}
},
{
"type": "providers/diagnosticSettings",
"name": "[concat('Microsoft.Insights/', parameters('iotHub.diagnostics.settingName'))]",
"dependsOn": [
"[resourceId('Microsoft.Devices/IoTHubs', parameters('iothubname'))]"
],
"apiVersion": "2017-05-01-preview",
"properties": {
"name": "[parameters('iotHub.diagnostics.settingName')]",
"workspaceId": "[parameters('iotHub.diagnostics.workspaceId')]",
"logs": [
{
"category": "Connections",
"enabled": true,
"retentionPolicy": {
"days": 0,
"enabled": false
}
},
{
"category": "Configurations",
"enabled": true,
"retentionPolicy": {
"days": 0,
"enabled": false
}
},
{
"category": "D2CTwinOperations",
"enabled": true,
"retentionPolicy": {
"days": 0,
"enabled": false
}
},
{
"category": "C2DTwinOperations",
"enabled": true,
"retentionPolicy": {
"days": 0,
"enabled": false
}
}
],
"metrics": [
{
"category": "AllMetrics",
"enabled": true,
"retentionPolicy": {
"days": 0,
"enabled": false
}
}
]
}
}
]
Any help much appreciated!
this needs to be a sub resource of the IOT hub, not a separate resource.
{
"type": "Microsoft.Devices/IotHubs",
"sku": {
"name": "[parameters('sku.name')]",
"capacity": "[parameters('sku.units')]"
},
"name": "[parameters('iothubname')]",
"apiVersion": "2018-04-01",
"location": "[resourceGroup().location]",
"properties": {
xxx
},
"features": "[parameters('features')]",
"resources": [
{
"type": "providers/diagnosticsSettings",
xxx
}
]
}
},