Enabling auditing settings on Azure SQL database via ARM Template - azure

I have been working on a template to deploy SQL/XSS injection detection. All is well except for enabling the auditing settings. In the docs I see the following:
{
"name": "default",
"type": "Microsoft.Sql/servers/databases/auditingSettings",
"apiVersion": "2017-03-01-preview",
"properties": {
"state": "string",
"storageEndpoint": "string",
"storageAccountAccessKey": "string",
"retentionDays": "integer",
"auditActionsAndGroups": [
"string"
],
"storageAccountSubscriptionId": "string",
"isStorageSecondaryKeyInUse": boolean
}
}
I believe I've followed this structure. See my full code here or the snippet here:
- apiVersion: 2017-03-01-preview
type: Microsoft.Sql/servers/auditingSettings
name: "[concat(parameters('sqlServerName'), '/auditing-default')]"
dependsOn:
- "[resourceId('Microsoft.Sql/servers', parameters('sqlServerName'))]"
properties:
state: Enabled
storageEndpoint: "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')),
'2018-03-01-preview').PrimaryEndpoints.Blob]"
storageAccountAccessKey: "[listKeys(resourceId('Microsoft.Storage/storageAccounts',
parameters('storageAccountName')), '2018-03-01-preview').keys[0].value]"
retentionDays: 0
storageAccountSubscriptionId: "[subscription().subscriptionId]"
isStorageSecondaryKeyInUse: false'
I am seeing that there is a discrepancy between the servers/databases and just /servers for the type, but I actually borrowed this code from the Azure Quick Starts and the specific file here where the code is the following:
{
"apiVersion": "2017-03-01-preview",
"type": "Microsoft.Sql/servers/auditingSettings",
"name": "[concat(parameters('sqlServerName'), '/', 'default')]",
"properties": {
"state": "Enabled",
"storageEndpoint": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2018-03-01-preview').PrimaryEndpoints.Blob]",
"storageAccountAccessKey": "[listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName')), '2018-03-01-preview').keys[0].value]",
"retentionDays": 0,
"auditActionsAndGroups": null,
"storageAccountSubscriptionId": "[subscription().subscriptionId]",
"isStorageSecondaryKeyInUse": false
}
}
The official docs don't seem to have info on adding the auditingSettings on a server level, but then here the type is directly under server, so I'm a bit lost. I haven't looked into the schema yet, but any help/guidance as to what might be going on here would be much appreciated!

We recently published a template that shows how to deploy an Azure SQL Server with server auditing enabled.
The full example is here: https://github.com/Azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.sql/sql-auditing-server-policy-to-blob-storage

As the other answers are returning 404s here's a full list of instructions to get the basics working in ARM for auditing at the SQL Server level. This will therefore audit all databases within the SQL Server.
Firstly, create a parameter for the name of your SQL Server and storage account:
"sqlServerName": {
"type": "string"
},
"auditingStorageAccountName": {
"type": "string"
}
Then in your resources section create a storage account to store your audit records, this example will replicate the audit blobs to the paired region (RA-GRS). It was necessary to add network ACLs explicitly as shown so that Azure can write the audit logs. This example also uses storage account assigned keys but managed identities are also possible:
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2019-06-01",
"name": "[parameters('auditingStorageAccountName')]",
"location": "[resourceGroup().location]",
"sku": {
"name": "Standard_RAGRS",
"tier": "Standard"
},
"kind": "StorageV2",
"properties": {
"networkAcls": {
"bypass": "AzureServices",
"virtualNetworkRules": [],
"ipRules": [],
"defaultAction": "Allow"
},
"supportsHttpsTrafficOnly": true,
"allowBlobPublicAccess": false,
"encryption": {
"services": {
"blob": {
"keyType": "Account",
"enabled": true
}
},
"keySource": "Microsoft.Storage"
},
"accessTier": "Hot"
}
},
...
Finally add the auditing settings themselves - this example is for a resource added at the root (i.e. directly within "resources": {}), to add it as sub-resource to the SQL Server itself the type needs to be just "auditingSettings". A retention days of zero means audit records will be kept indefinitely. It was necessary to add the subscription ID explicitly otherwise the settings do not appear correctly when viewed in the portal:
{
"type": "Microsoft.Sql/servers/auditingSettings",
"name": "default",
"apiVersion": "2020-11-01-preview",
"dependsOn": [
"[resourceId('Microsoft.Sql/servers/', parameters('sqlServerName'))]",
"[resourceId('Microsoft.Storage/storageAccounts', parameters('auditingStorageAccountName'))]"
],
"properties": {
"retentionDays": 0,
"state": "Enabled",
"storageEndpoint": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('auditingStorageAccountName'))).primaryEndpoints.blob]",
"storageAccountAccessKey": "[listKeys(parameters('auditingStorageAccountName'), '2019-06-01').keys[0].value]",
"storageAccountSubscriptionId": "[subscription().subscriptionId]"
}
},
...

For those looking for guidance on enabling server level auditing to a Log Analytics workspace, I found this github link

Related

Functionapp With Custom Runtime Image, Use ACR Admin Credentials in ARM Template

I'm in the process of migrating our functionapps to custom runtime containers. I'm doing this through ARM templates.
I've got to the point where I can do this, however, in order to get it to work, I have to manually open the Deployment Center and hit save after provisioning, otherwise the functionapp cannot pull down from the ACR (and the logs say there's an auth error).
2022-10-10T22:25:29.055Z INFO - Recycling container because of AppSettingsChange and isMainSite = True
2022-10-10T22:25:32.116Z ERROR - DockerApiException: Docker API responded with status code=InternalServerError, response={"message":"Get https://redacted.azurecr.io/v2/redacted/manifests/preview: unauthorized: authentication required, visit https://aka.ms/acr/authorization for more information."}
As soon as I click save (I don't even change anything) it pulls down and deploys correctly.
Whilst I don't need to reprovision often, this manual step is a pain and I want to fix it, what do I need to add to my ARM template to facilitate this?
The relevent section of the ARM template is:
{
"type": "Microsoft.Web/sites",
"apiVersion": "2022-03-01",
"name": "[parameters('functionAppName')]",
"location": "[parameters('location')]",
"kind": "functionapp,linux,container",
"identity": {
"type": "SystemAssigned"
},
"dependsOn": [
"[variables('appServicePlanResourceId')]",
"[variables('deploymentStorageAccountId')]",
"[variables('networkResourceId')]",
"[resourceId('microsoft.insights/components', parameters('functionAppName'))]"
],
"tags": {
"Product": "[variables('productTag')]",
"Environment": "[parameters('environmentTag')]"
},
"properties": {
"ftpsState": "FtpsOnly",
"httpsOnly": true,
"reserved": true,
"serverFarmId": "[variables('appServicePlanResourceId')]",
"siteConfig": {
"appSettings": [
{
"name": "AzureWebJobsStorage",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=', parameters('deploymentStorageAccountName'), ';EndpointSuffix=', environment().suffixes.storage, ';AccountKey=',listKeys(variables('deploymentStorageAccountId'), '2019-06-01').keys[0].value)]"
},
{
"name": "WEBSITE_CONTENTAZUREFILECONNECTIONSTRING",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=', parameters('deploymentStorageAccountName'), ';EndpointSuffix=', environment().suffixes.storage, ';AccountKey=',listKeys(variables('deploymentStorageAccountId'), '2019-06-01').keys[0].value)]"
},
{
"name": "WEBSITE_CONTENTSHARE",
"value": "[toLower(parameters('functionAppName'))]"
},
{
"name": "FUNCTIONS_EXTENSION_VERSION",
"value": "~3"
},
{
"name": "APPLICATIONINSIGHTS_CONNECTION_STRING",
"value": "[concat('InstrumentationKey=', reference(resourceId('Microsoft.Insights/components', parameters('functionAppName')), '2020-02-02-preview').instrumentationKey)]"
},
{
"name": "FUNCTIONS_WORKER_RUNTIME",
"value": "dotnet"
},
{
"name": "EventGridTopicEndpoint",
"value": "[reference(variables('eventGridTopicId')).endpoint]"
},
{
"name": "EventGridTopicAccessKey",
"value": "[listKeys(variables('eventGridTopicId'), '2020-06-01').key1]"
},
{
"name": "WEBSITE_DNS_SERVER",
"value": "redacted"
},
{
"name": "WEBSITE_VNET_ROUTE_ALL",
"value": 1
},
{
"name": "WEBSITES_ENABLE_APP_SERVICE_STORAGE",
"value": "false"
}
],
"linuxFxVersion": "[parameters('linuxFxVersion')]",
"acrUseManagedIdentityCreds": false
}
},
"resources": [
{
"type": "networkConfig",
"apiVersion": "2019-08-01",
"name": "virtualNetwork",
"dependsOn": [ "[variables('functionAppResourceId')]" ],
"properties": {
"subnetResourceId": "[variables('subnetResourceId')]",
"isSwift": true
}
}
]
}
[parameters('linuxFxVersion')] evaluates to DOCKER|redacted.azurecr.io/redacted:preview
Every answer that I've found so far requires either adding config options with docker usernames and passwords, or using a managed identity, neither of which is what we want.
You need to add an RBAC assignment to your ACR instance granting the system-assigned identity of your function app the AcrPull role.
The alternative is using admin credentials.
When you hit "Save" in the deployment center, it's using one of those two methods -- it's retrieving the admin credentials from the ACR and applying them to the app service. It's not doing anything special, it's doing exactly what you can do yourself.
I recommend using managed identities instead. You can even create a single user-assigned identity and share it across multiple function apps, if you really want to.
Reference a secret in a key vault:
"adminPassword": {
"reference": {
"keyVault": {
"id": "/subscriptions/<SubscriptionID>/resourceGroups/mykeyvaultdeploymentrg/providers/Microsoft.KeyVault/vaults/<KeyVaultName>"
},
"secretName": "vmAdminPassword"
}
}
So with hints taken from the other two answers and from here, I've devised two solutions.
Using Service Principal Role
Add "acrUseManagedIdentityCreds": true to the siteConfig in my ARM template
Assign the AcrPull role to the service principal of the functionapp (I've not tested this snippet because perms weren't set-up quite right and it's too late for me to ask someone to change them)
"resources": [
{
"type": "Microsoft.Authorization/roleAssignments",
"apiVersion": "2018-09-01-preview",
"name": "[guid(resourceGroup().id)]",
"dependsOn": [
"[parameters('functionAppName')]"
],
"properties": {
"roleDefinitionId": "[concat('/subscriptions/', subscription().subscriptionId, '/providers/Microsoft.Authorization/roleDefinitions/', '7f951dda-4ed3-4680-a7ca-43fe172d538d')]",
"principalId": "[reference(resourceId('Microsoft.Web/sites', parameters('functionAppName')), '2022-03-01').identity.principalId]"
}
}
]
Getting Admin Creds with Reference
Add these variables to my template:
"registryName": "containerRegName",
"registrySubscriptionId": "container-reg-sub-id",
"registryResourceGroup": "container-reg-rg",
"registryResourceId": "[resourceId(variables('registrySubscriptionId'), variables('registryResourceGroup'), 'Microsoft.ContainerRegistry/registries', variables('registryName'))]"
},
Then add these configuration options to my appsettings:
{
"name": "DOCKER_REGISTRY_SERVER_URL",
"value": "[reference(variables('registryResourceId'), '2019-05-01').loginServer]"
},
{
"name": "DOCKER_REGISTRY_SERVER_USERNAME",
"value": "[listCredentials(variables('registryResourceId'), '2019-05-01').username]"
},
{
"name": "DOCKER_REGISTRY_SERVER_PASSWORD",
"value": "[listCredentials(variables('registryResourceId'), '2019-05-01').passwords[0].value]"
}

Unable to create KB in QnA Maker when deployed via ARM (No Endpoint keys found)

I am having an issue where I cannot create KBs in QnA Maker for services which I have deployed via ARM template/DevOps. There are a number of issues here and on Github, but the main suggestions (create all the resources in the same region, don't put anything else on the app service plan, delete and redeploy) have not worked for me. As noted the resources HAVE been created and deleted multiple times with the same names, so I don't know if that's part of the issue. The resources create just fine (cognitive service, app service, app service plan, azure search, and app insights), all in WestUS, but then I am unable to create a knowledge base either through the API or directly at qnamaker.ai. In both cases I get the error message:
No Endpoint keys found.
I can get the keys through Azure CLI, plus they are showing in the portal, so that's not the issue. It may perhaps be an issue with the Authorization EndpointKey which is generated/shown after publishing a new KB, but as I cannot create or publish one, I cannot find this key. Not sure if that is the key the error message is referring to.
Here is the ARM template I am using the set up the resources.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"sites_etn_qnamaker_name": {
"defaultValue": "etn-qnamaker",
"type": "string"
},
"serverfarms_etn_qnamaker_name": {
"defaultValue": "etn-qnamaker",
"type": "string"
},
"components_etn_qnamaker_ai_name": {
"defaultValue": "etn-qnamaker-ai",
"type": "string"
},
"accounts_etn_qnamaker_name": {
"defaultValue": "etn-qnamaker",
"type": "string"
},
"searchServices_etnqnamaker_azsearch_name": {
"defaultValue": "etnqnamaker-azsearch",
"type": "string"
},
"smartdetectoralertrules_failure_anomalies___etn_qnamaker_ai_name": {
"defaultValue": "failure anomalies - etn-qnamaker-ai",
"type": "string"
},
"actiongroups_application_20insights_20smart_20detection_externalid": {
"defaultValue": "/subscriptions/REDACTED/resourceGroups/avcnc-chatbot-rg/providers/microsoft.insights/actiongroups/application%20insights%20smart%20detection",
"type": "string"
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.CognitiveServices/accounts",
"apiVersion": "2017-04-18",
"name": "[parameters('accounts_etn_qnamaker_name')]",
"location": "westus",
"sku": {
"name": "S0"
},
"kind": "QnAMaker",
"properties": {
"apiProperties": {
"qnaRuntimeEndpoint": "[concat('https://', parameters('accounts_etn_qnamaker_name'), '.azurewebsites.net')]"
},
"customSubDomainName": "[parameters('accounts_etn_qnamaker_name')]"
}
},
{
"type": "Microsoft.Insights/components",
"apiVersion": "2015-05-01",
"name": "[parameters('components_etn_qnamaker_ai_name')]",
"location": "westus",
"tags": {
"hidden-link:/subscriptions/REDACTED/resourceGroups/ENTP-Chatbot-rg/providers/Microsoft.Web/sites/etn-qnamaker": "Resource"
},
"kind": "web",
"properties": {
"Application_Type": "web"
}
},
{
"type": "Microsoft.Search/searchServices",
"apiVersion": "2015-08-19",
"name": "[parameters('searchServices_etnqnamaker_azsearch_name')]",
"location": "West US",
"sku": {
"name": "basic"
},
"properties": {
"replicaCount": 1,
"partitionCount": 1,
"hostingMode": "default"
}
},
{
"type": "Microsoft.Web/serverfarms",
"apiVersion": "2018-02-01",
"name": "[parameters('serverfarms_etn_qnamaker_name')]",
"location": "West US",
"sku": {
"name": "S1",
"tier": "Standard",
"size": "S1",
"family": "S",
"capacity": 1
},
"kind": "app",
"properties": {
"perSiteScaling": false,
"maximumElasticWorkerCount": 1,
"isSpot": false,
"reserved": false,
"isXenon": false,
"hyperV": false,
"targetWorkerCount": 0,
"targetWorkerSizeId": 0
}
},
{
"type": "microsoft.alertsmanagement/smartdetectoralertrules",
"apiVersion": "2019-06-01",
"name": "[parameters('smartdetectoralertrules_failure_anomalies___etn_qnamaker_ai_name')]",
"location": "global",
"dependsOn": [
"[resourceId('microsoft.insights/components', parameters('components_etn_qnamaker_ai_name'))]"
],
"properties": {
"description": "Failure Anomalies notifies you of an unusual rise in the rate of failed HTTP requests or dependency calls.",
"state": "Enabled",
"severity": "Sev3",
"frequency": "PT1M",
"detector": {
"id": "FailureAnomaliesDetector",
"name": "Failure Anomalies",
"description": "Detects if your application experiences an abnormal rise in the rate of HTTP requests or dependency calls that are reported as failed. The anomaly detection uses machine learning algorithms and occurs in near real time, therefore there's no need to define a frequency for this signal.<br/></br/>To help you triage and diagnose the problem, an analysis of the characteristics of the failures and related telemetry is provided with the detection. This feature works for any app, hosted in the cloud or on your own servers, that generates request or dependency telemetry - for example, if you have a worker role that calls <a class=\"ext-smartDetecor-link\" href=\\\"https://learn.microsoft.com/en-us/azure/application-insights/app-insights-api-custom-events-metrics#trackrequest\\\" target=\\\"_blank\\\">TrackRequest()</a> or <a class=\"ext-smartDetecor-link\" href=\\\"https://learn.microsoft.com/en-us/azure/application-insights/app-insights-api-custom-events-metrics#trackdependency\\\" target=\\\"_blank\\\">TrackDependency()</a>.",
"supportedResourceTypes": [
"ApplicationInsights"
],
"imagePaths": [
"https://globalsmartdetectors.blob.core.windows.net/detectors/FailureAnomaliesDetector/v0.18/FailureAnomaly.png"
]
},
"scope": [
"[resourceId('microsoft.insights/components', parameters('components_etn_qnamaker_ai_name'))]"
],
"actionGroups": {
"groupIds": [
"[parameters('actiongroups_application_20insights_20smart_20detection_externalid')]"
]
}
}
},
{
"type": "Microsoft.Web/sites",
"apiVersion": "2018-11-01",
"name": "[parameters('sites_etn_qnamaker_name')]",
"location": "West US",
"dependsOn": [
"[resourceId('Microsoft.Web/serverfarms', parameters('serverfarms_etn_qnamaker_name'))]"
],
"tags": {
"hidden-related:/subscriptions/REDACTED/resourcegroups/ENTP-Chatbot-rg/providers/Microsoft.Web/serverfarms/etn-qnamaker": "empty"
},
"kind": "app",
"properties": {
"enabled": true,
"hostNameSslStates": [
{
"name": "[concat(parameters('sites_etn_qnamaker_name'), '.azurewebsites.net')]",
"sslState": "Disabled",
"hostType": "Standard"
},
{
"name": "[concat(parameters('sites_etn_qnamaker_name'), '.scm.azurewebsites.net')]",
"sslState": "Disabled",
"hostType": "Repository"
}
],
"serverFarmId": "[resourceId('Microsoft.Web/serverfarms', parameters('serverfarms_etn_qnamaker_name'))]",
"reserved": false,
"isXenon": false,
"hyperV": false,
"scmSiteAlsoStopped": false,
"clientAffinityEnabled": true,
"clientCertEnabled": false,
"hostNamesDisabled": false,
"containerSize": 0,
"dailyMemoryTimeQuota": 0,
"httpsOnly": false,
"redundancyMode": "None"
}
},
{
"type": "Microsoft.Web/sites/config",
"apiVersion": "2018-11-01",
"name": "[concat(parameters('sites_etn_qnamaker_name'), '/web')]",
"location": "West US",
"dependsOn": [
"[resourceId('Microsoft.Web/sites', parameters('sites_etn_qnamaker_name'))]"
],
"tags": {
"hidden-related:/subscriptions/REDACTED/resourcegroups/ENTP-Chatbot-rg/providers/Microsoft.Web/serverfarms/etn-qnamaker": "empty"
},
"properties": {
"numberOfWorkers": 1,
"defaultDocuments": [
"Default.htm",
"Default.html",
"Default.asp",
"index.htm",
"index.html",
"iisstart.htm",
"default.aspx",
"index.php",
"hostingstart.html"
],
"netFrameworkVersion": "v4.0",
"phpVersion": "5.6",
"requestTracingEnabled": false,
"remoteDebuggingEnabled": false,
"httpLoggingEnabled": false,
"logsDirectorySizeLimit": 35,
"detailedErrorLoggingEnabled": false,
"publishingUsername": "[concat('$',parameters('sites_etn_qnamaker_name'))]",
"scmType": "None",
"use32BitWorkerProcess": true,
"webSocketsEnabled": false,
"alwaysOn": false,
"managedPipelineMode": "Integrated",
"virtualApplications": [
{
"virtualPath": "/",
"physicalPath": "site\\wwwroot",
"preloadEnabled": false
}
],
"loadBalancing": "LeastRequests",
"experiments": {
"rampUpRules": []
},
"autoHealEnabled": false,
"cors": {
"allowedOrigins": [
"*"
],
"supportCredentials": false
},
"localMySqlEnabled": false,
"ipSecurityRestrictions": [
{
"ipAddress": "Any",
"action": "Allow",
"priority": 1,
"name": "Allow all",
"description": "Allow all access"
}
],
"scmIpSecurityRestrictions": [
{
"ipAddress": "Any",
"action": "Allow",
"priority": 1,
"name": "Allow all",
"description": "Allow all access"
}
],
"scmIpSecurityRestrictionsUseMain": false,
"http20Enabled": false,
"minTlsVersion": "1.2",
"ftpsState": "AllAllowed",
"reservedInstanceCount": 0
}
},
{
"type": "Microsoft.Web/sites/hostNameBindings",
"apiVersion": "2018-11-01",
"name": "[concat(parameters('sites_etn_qnamaker_name'), '/', parameters('sites_etn_qnamaker_name'), '.azurewebsites.net')]",
"location": "West US",
"dependsOn": [
"[resourceId('Microsoft.Web/sites', parameters('sites_etn_qnamaker_name'))]"
],
"properties": {
"siteName": "[parameters('sites_etn_qnamaker_name')]",
"hostNameType": "Verified"
}
}
]
}
Here are just a few of the sites I checked
https://github.com/MicrosoftDocs/azure-docs/issues/44719
https://github.com/MicrosoftDocs/azure-docs/issues/40089
Unable to create knowledgebase for azure cognitive service (Error: "No Endpoint keys found.")
EDIT: KB creation fails both through qnamaker.ai and via API. On qnamaker.ai, I get this message when trying to create a KB:
And here is the PowerShell script I was using to try and create it programmatically:
$body = Get-Content '$(System.DefaultWorkingDirectory)/_AveryCreek_OEM_CSC_Bot/models/qnamaker/Avery_Creek_Commercial_QnA.json' | Out-String
$header = #{
"Content-Type"="application/json"
"Ocp-Apim-Subscription-Key"="$(QNA_KEY)"
}
Invoke-RestMethod -Uri "https://westus.api.cognitive.microsoft.com/qnamaker/v4.0/knowledgebases/create" -Method 'Post' -Body $body -Headers $header
Searching for issues with endpoint keys and qnamaker turns up a fair few results.
I've just closed a case with Azure support for the same issue, and here are some of the steps we checked on the way to fixing this, hopefully one of these will be useful for anyone having this issue in the future as the error message doesn't give you much to go on:
First, check the troubleshooting FAQ https://learn.microsoft.com/en-us/azure/cognitive-services/qnamaker/troubleshooting. There's nothing on the endpoint keys issue, but when you hit something else it's a good starting point.
All services - check your naming. For example, for me my search service was named differently than the rest of my config was expecting, and also my cognitive services runtime endpoint in the api-properties was incorrect. Still deployed though - you won't always get an error on the service itself if you provide incorrect names to later created services, you'll just fail at the point of creating your KBs.
All services - check your SKUs. While there's no problem that I could find being on free/basic, you can only have 1 qna cognitive service on a free subscription, so you'll need to tear down and recreate or update as you go.
QnA cognitive service - config settings (keys and values) are case-sensitive.
Qna web app and web app plan - check your quotas haven't been hit, particularly memory and CPU.
QnA Web App - You should be able to go the https://{endpoint}/qnamaker/corehealthstatus and see a positive json response like this (or if there's an initException, you've at least got another error to go on):
{"processId":4920,"runtimeVersion":"5.46.0","initException":"","startupTime":"10/28/2020 2:44:39 PM"}
Qna Web App - You should also be able to go the https://{endpoint}/qnamaker/proxyhealthstatus and see a positive json response like this.:
{
"coreVersion": "5.46.0",
"coreProcessId": 4920,
"coreUrl": "http://localhost:50061"
}
Qna Web App - Don't try to create a KB, whether through the qnamaker portal or dynamically, if your app doesn't show similar successes on those two check endpoints - build a wait if need be. You'll almost certainly see the endpoint errors via the API if you hit it immediately.
For the check endpoints above, the endpoint is visible in the overview section of your web app in the portal, and usually is the name of your app e.g. https://example-app-qnamaker-webapp.azurewebsites.net/qnamaker/corehealthstatus if the app was called example-app-qnamaker-webapp. In my own creation scripts, I checked against coreProcessId > 0 and startupTime is a valid date to indicate service readiness before creating a KB.
EDIT: I'd also add that if it takes a long time to deploy, part of your config is probably wrong. Every time I've had things work correctly, it's been a rapid deployment (and that goes for the services, knowledge bases, and calls to both az cli and the qnamaker REST api).
I suspect you may have been downvoted because this looks an awful lot more like a bug report than a Stack Overflow question. From the first issue you linked:
We will go ahead and close this issue as this is a service level issue and the best way to report it if it occurs again is through the QnA portal from "General Enquiry through uservoice" option from the top right corner.
I'll try to answer you anyway. You say you've tried creating all the resources in the same region, but remember that resource groups have locations too. You should make sure the resource group is also in the same region according to the answer to the Stack Overflow question you linked to: Unable to create knowledgebase for azure cognitive service (Error: "No Endpoint keys found.")
It seems that there is sometimes the problem that the endpoint keys can only be found, if the Resource Group holding all resources for the QnA Maker Service (like App Service, Application Insights, Search Service and the Application Service Plan) is hosted in the same region as the QnA Maker Service itself.
I also see that you've tried not putting anything else on the app service plan, and you've tried deleting and redeploying. But you might also try just waiting a while, or retrying more persistently. From another GitHub issue:
These failures are intermittent, If I persistently retry a failure, the knowledgebase will often eventually get created.
And from this issue:
According to the QnA Maker team, this error is shown when the QnA Maker service has not finished provisioning. There appear to be service issues QnA Maker right now that are causing the provisioning process to take even longer than the time we wait in the script.
If you would like to raise an issue through UserVoice, I highly recommend posting it on the forum so that other people can see the problem and upvote it.

How to make AzureRM not to drop previously configured properties of `Microsoft.Web/sites/config` at next deployment?

This is current configuration, https://management.azure.com/subscriptions/<subid>/resourceGroups/<groupid>/providers/Microsoft.Web/sites/<sitename>/config/logs?api-version=2018-02-01 :
{
"id": "/subscriptions/<subid>/resourceGroups/<group>/providers/Microsoft.Web/sites/<sitename>/config/logs",
"name": "logs",
"type": "Microsoft.Web/sites/config",
"location": "West Europe",
"tags": {
"displayName": "Website",
},
"properties": {
"applicationLogs": {
"fileSystem": {
"level": "Off"
},
"azureTableStorage": {
"level": "Off",
"sasUrl": null
},
"azureBlobStorage": {
"level": "Verbose",
"sasUrl": "<here is fully specified sasUrl>",
"retentionInDays": 0
}
},
// ...
This is done in order to configure that SasUrl there which is done only after we have SA, and using Powershell. When New-AzureRmResourceGroupDeployment run with this template, those properties are lost (sasUrl and other).
{
"apiVersion": "2016-08-01",
"name": "[variables('prodWebAppName')]",
"type": "Microsoft.Web/sites",
"location": "[parameters('location')]",
"dependsOn": [
"[variables('appServicePlanName')]"
],
"tags": {
"displayName": "Website"
},
"properties": {
"serverFarmId": "[resourceId('Microsoft.Web/serverfarms', variables('appServicePlanName'))]",
"hostNames": [
// ...
],
"enabledHostNames": [
// ..
],
"defaultHostName": "[concat(variables('prodWebAppName'), '.azurewebsites.net')]",
"siteConfig": {
"netFrameworkVersion": "v4.6",
"phpVersion": "Off",
"alwaysOn": true,
"webSocketsEnabled": true,
"appSettings": [
// ...
],
"connectionStrings": [
// ...
]
}
},
// .. and then slots configuration
That seems to happen because those netFrameworkVersion and like settings are under Microsoft.Web/sites/<sitename>/config also. Probably it should be put in a separate json template to run for this site or slot specifically. The template is used to provision all environments (AAT, QA, Prod) at once which makes things more complex.
Also, do you think it is right to make QA and AAT in App Service slots like MS advertise at https://learn.microsoft.com/en-us/azure/app-service/web-sites-staged-publishing? To my mind this does not suit well for script/template based continuous deployment/delivery for not simple systems.
ARM is meant to apply final state to your resources so it will apply the values in your ARM template (blanks/nulls) to the settings in your resources.
Instead of using PS after the fact to create the storage resource and update the setting, have you considered deploying the storage account in the ARM template and using a reference to it's key using listKeys instead?

Azure - Set WebSocket On from ARM json template

I'm trying to turn WebSockets On for an Azure WebApp from an Azure ARM json template that deploys my whole infrastructure.
Here is an extract with regards to the Azure Web App. It doesn't work, i.e the WebSockets are still Off. I unsuccessfully tried different spelling: webSocketsEnabled or WebSockets.
"resources":[
{
"name": "[variables('MyApp')]",
"type": "Microsoft.Web/sites",
"location": "Brazil South",
"apiVersion": "2016-08-01",
"dependsOn": [
"[resourceId('Microsoft.Web/serverfarms', variables('MyAppPlanBrazil'))]"
],
"tags": {
"[concat('hidden-related:', resourceId('Microsoft.Web/serverfarms', variables('MyAppPlanBrazil')))]": "Resource",
"displayName": "MyAppAppBrazil"
},
"properties": {
"name": "[variables('MyAppPlanBrazil')]",
"serverFarmId": "[resourceId('Microsoft.Web/serverfarms', variables('MyAppPlanBrazil'))]",
"siteConfig": {
"AlwaysOn": true,
"webSocketsEnabled": true,
"connectionStrings": [
{
...
},
{
...
},
]
}
}
]
UPDATE
As suggested in answer below I updated the apiVersion to "2016-08-01" but this still doesn't work.
Also note that while my schema is the one described here, apiVersion is squiggled in VS and it says the authorized value is "2015-08-01" only.
UPDATE2
I tried the solutions below. They work for their authors but not for me. I guess the problem is elsewhere. My infrastructure is already deployed and I try to update it with webSocketsEnabled. Whereas in the solution below I imagine the authors directly create the web app with webSocketsEnabled.
Also, I coupled webSocketsEnabled with alwaysOn whereas the pricing tier of my webapp doesn't allow "AlwaysOn" (as it says in the portal I need to upgrade to use that feature) so I'll try without alwaysOn.
UPDATE3
At the end, the above template worked when I removed AlwaysOn.
Thank you to those who tried to help me.
Set your api version to this: "2016-08-01"
Use
"webSocketsEnabled": true
This is from the Microsoft.Web/sites template reference:
https://learn.microsoft.com/en-us/azure/templates/microsoft.web/sites
The api version you are using (2015-08-01) from:
https://github.com/Azure/azure-resource-manager-schemas/blob/master/schemas/2015-08-01/Microsoft.Web.json
Doesn't have web sockets in it, but the later one:
https://github.com/Azure/azure-resource-manager-schemas/blob/master/schemas/2016-08-01/Microsoft.Web.json
Does have webSocketsEnabled.
Please have a try to use the following code. It works correctly on my side.
Updated: add whole test arm template and you could have a try to use the following code with your service plan name and resource group name
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"serverFarmName": {
"type": "string",
"defaultValue": "YourPlan"
},
"serverFarmResourceGroup": {
"type": "string",
"defaultValue": "ResourceGroupName"
}},
"variables": {
"ARMtemplateTestName": "[concat('ARMtemplateTest', uniqueString(resourceGroup().id))]"},
"resources": [
{
"name": "[variables('ARMtemplateTestName')]",
"type": "Microsoft.Web/sites",
"location": "southcentralus",
"apiVersion": "2015-08-01",
"dependsOn": [ ],
"tags": {
"[concat('hidden-related:', resourceId(parameters('serverFarmResourceGroup'), 'Microsoft.Web/serverFarms', parameters('serverFarmName')))]": "Resource",
"displayName": "ARMtemplateTest"
},
"properties": {
"name": "[variables('ARMtemplateTestName')]",
"serverFarmId": "[resourceId(parameters('serverFarmResourceGroup'), 'Microsoft.Web/serverFarms', parameters('serverFarmName'))]"
},
"resources": [
{
"name": "web",
"type": "config",
"apiVersion": "2015-08-01",
"dependsOn": [
"[resourceId('Microsoft.Web/sites', variables('ARMtemplateTestName'))]"
],
"tags": {
"displayName": "enableWebSocket"
},
"properties": {
"webSocketsEnabled": true,
"alwaysOn": true
}
},
{
"apiVersion": "2015-08-01",
"name": "connectionstrings",
"type": "config",
"dependsOn": [
"[resourceId('Microsoft.Web/Sites', variables('ARMtemplateTestName'))]"
],
"properties": {
"ConnString1": {
"value": "My custom connection string",
"type": "custom"
},
"ConnString2": {
"value": "My SQL connection string",
"type": "SQLAzure"
}
}
},
{
"name": "appsettings",
"type": "config",
"apiVersion": "2015-08-01",
"dependsOn": [
"[resourceId('Microsoft.Web/sites', variables('ARMtemplateTestName'))]"
],
"tags": {
"displayName": "Appsetting"
},
"properties": {
"key1": "value1",
"key2": "value2"
}
}
]
}],
"outputs": {}
}
Test Result:
All the above solution should work.
My initial snippet worked as well ... as soon as I removed alwaysOn.
Indeed, I was using a free tiers App Service Plan for which alwaysOn is not available. While there was no errors or anything else indicating something wrong, I could not set webSocketEnabled and alwaysOn at the same time in that case.

Generate or Specify Azure Service Bus Namespace SAS Token with ARM Template

I'm using Azure ARM templates in order to make sure that I can repeatedly deploy uniform infrastructure and services. My ARM template consists of an App Service, Web App, Service Bus Queue, and Azure SQL database. On top of this, I'm setting up continuous deployment through VSTS.
Everything is working well EXCEPT I am not sure how to set a Service Bus SAS token at the Namespace level. I don't see a way in the Service Bus ARM template to specify one, so I cannot pre-generate a token and place it in my web.config file. I also don't see a way to have one generated on my behalf, then pull the values back to my web.config file. Any suggestions would be greatly appreciated.
I believe you have two options:
1) Get generated key from the output:
"outputs": {
"eh:Endpoint": {
"value": "[listKeys(resourceId('Microsoft.EventHub/namespaces/authorizationRules', variables('eventHubNamespaceName'), 'SendOnlyKey'),'2015-08-01').primaryKey]",
"type": "string"
},
}
And incorporate it in your build/release process.
2) Try to push a key with a template:
{
"apiVersion": "[parameters('eventHubVersion')]",
"name": "[variables('eventHubNamespaceName')]",
"type": "Microsoft.EventHub/namespaces",
"location": "[resourceGroup().location]",
"resources": [
{
"apiVersion": "2014-09-01",
"name": "[variables('eventHubName')]",
"type": "eventHubs",
"dependsOn": [
"[concat('Microsoft.EventHub/namespaces/', variables('eventHubNamespaceName'))]"
],
"properties": {
"path": "[variables('eventHubName')]",
"MessageRetentionInDays": "[parameters('messageRetentionInDays')]",
"PartitionCount": "[parameters('partitionCount')]"
},
"resources": [
{
"apiVersion": "[parameters('eventHubVersion')]",
"name": "StorageRetention",
"type": "consumergroups",
"dependsOn": [
"[variables('eventHubName')]",
"[concat('Microsoft.EventHub/namespaces/', variables('eventHubNamespaceName'))]"
],
"tags": {
"displayName": "eh"
}
}
]
},
{
"apiVersion": "[parameters('eventHubVersion')]",
"name": "[concat(variables('eventHubNamespaceName'),'/SendOnlyKey')]",
"type": "Microsoft.EventHub/namespaces/authorizationRules",
"dependsOn": [
"[concat('Microsoft.EventHub/namespaces/', variables('eventHubNamespaceName'))]"
],
"location": "[resourceGroup().location]",
"properties": {
"KeyName": "SendOnlyKey",
"ClaimType": "SendSharedAccessKey",
"ClaimValue": "None",
"PrimaryKey": "[parameters('eventHubSendPrimaryKey')]",
"SecondaryKey": "your_key",
"Rights": [ "Send" ],
"Revision": -1
}
},
{
"apiVersion": "[parameters('eventHubVersion')]",
"name": "[concat(variables('eventHubNamespaceName'),'/ListenOnlyKey')]",
"type": "Microsoft.EventHub/namespaces/authorizationRules",
"dependsOn": [
"[concat('Microsoft.EventHub/namespaces/', variables('eventHubNamespaceName'))]"
],
"location": "[resourceGroup().location]",
"properties": {
"KeyName": "ListenOnlyKey",
"ClaimType": "ReceiveSharedAccessKey",
"ClaimValue": "None",
"PrimaryKey": "your_key",
"SecondaryKey": "your_key",
"Rights": [ "Listen" ],
"Revision": -1
}
}
]
}
However note that the second solutions works only for an older version of API and sooner or later will be deprecated. Additionally I tested it only for pushing keys for a hub, not a namespace.
This might help others arriving at this answer
In EastUS using API_VERSION = 2017-04-01
The following will work to obtain references to primarykey and related fields
- connectionString: "[concat('',listKeys(resourceId('Microsoft.EventHub/namespaces/eventhubs/authorizationRules','{{ eh_namespace }}', '{{ eventhub_name }}','fw'), '2017-04-01').primaryConnectionString,'')]"

Resources