I'm finding absolutely 0 documentation about how to output from Stream Analytics to Azure Synapse Analytics as an output... I've got it configured in the portal but when I export the template all of the details about this output are lost besides the name.
I tried building it from scratch and there's no documentation at all. How do I write this into an ARM template? I have the following...
"outputs": [
{
"name": "synapse-output",
"properties": {
"datasource": {
"type": "",
"properties": {
}
}
}
}
]
And there are no details about how to fill it in... what is the Type for this type of output and how do I even fill in the properties with 0 documentation?
There is a reference for all template resources in microsoft documentation see the Microsoft.StreamAnalytics/streamingjobs/outputs template reference
The code to create stream analytics outputs but there is no reference for Azure Synapse Analytics output
{
"name": "string",
"type": "Microsoft.StreamAnalytics/streamingjobs/outputs",
"apiVersion": "2016-03-01",
"properties": {
"datasource": {
"type": "string",
"properties": {
}
},
"serialization": {
"type": "string",
"properties": {
}
}
}
}
You can use refernce function to output the output details
"output": {
"type": "object",
"value": "[reference(resourceId('Microsoft.StreamAnalytics/streamingjobs/outputs', 'JobName', 'outputName'), '2016-03-01', 'Full')]"
}
Related
There is an option to create Managed Identity from terraform for Stream analytics job (azurerm_stream_analytics_job, using identity block).
And it is possible to use Managed Identity to connect to databases (as explained here)
But I could not find how to use managed identity to create input using azurerm_stream_analytics_reference_input_mssql
UPDATE:
To be clear, thats what I am after:
And then
As Per July 2022
It does not look like terraform is supporting it (see documentation).
With this arm template, I was able to deploy ("authenticationMode": "Msi"):
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"streamAnalyticsJobName": {
"type": "string"
},
"streamAnalyticsJobNameInputName": {
"type": "string"
},
"sqlServerName": {
"type": "string"
},
"databaseName": {
"type": "string"
}
},
"resources": [
{
"type": "Microsoft.StreamAnalytics/streamingjobs/inputs",
"apiVersion": "2017-04-01-preview",
"name": "[format('{0}/{1}', parameters('streamAnalyticsJobName'), parameters('streamAnalyticsJobNameInputName'))]",
"properties": {
"type": "Reference",
"datasource": {
"type": "Microsoft.Sql/Server/Database",
"properties": {
"authenticationMode": "Msi",
"server": "[parameters('sqlServerName')]",
"database": "[parameters('databaseName')]",
"refreshType": "Static",
"fullSnapshotQuery": "SELECT Id, Name, FullName\nFrom dbo.Device\nFOR SYSTEM_TIME AS OF #snapshotTime --Optional, available if table Device is temporal"
}
}
}
}
]
}
So you could always use azurerm_template_deployment resource to deploy using terraform.
To create an API Connection to Azure Service Bus using Managed Identity I'm using the following template:
"resources": [
{
"type": "MICROSOFT.WEB/CONNECTIONS",
"apiVersion": "2018-07-01-preview",
"name": "[parameters('connections_servicebus_name')]",
"location": "[parameters('connections_servicebus_location')]",
"kind": "V1",
"properties": {
"alternativeParameterValues": {},
"displayName": "[parameters('connections_servicebus_displayname')]",
"api": {
"name": "[parameters('connections_servicebus_name')]",
"displayName": "[parameters('connections_servicebus_displayname')]",
"id": "[concat(subscription().id, '/providers/Microsoft.Web/locations/', parameters('connections_servicebus_location'), '/managedApis/', 'servicebus')]",
"type": "Microsoft.Web/locations/managedApis"
},
"customParameterValues": {},
"parameterValueSet": {
"name": "managedIdentityAuth",
"values": {}
}
}
}
]
that is actually working except for the fact that the 'NamespaceEndpoint' information (like: sb://mySBNS.servicebus.windows.net) is not provided anywhere and so the field appear empty on Azure portal:
After adding it manually, the connection and the LogicApp that is using it, start to work.
What is the json field to provide that information??
You can refer to this question for full details:
"parameterValueSet": {
"name": "managedIdentityAuth",
"values": {
"namespaceEndpoint": {
"value": "sb://<servicebus-namespace-name>.servicebus.windows.net/"
}
}
}
Trying to deploy Azure Dashboard through ARM template and getting an issue after deployment (see screenshot below).
The following document was used to construct template: Azure Dashboard
ARM template looks like:
"log-analytics-workspace-id": {
"type": "string",
"defaultValue": "/subscriptions/xxxx-xxxx-xxxx-xxxx-xxxx/resourcegroups/rg-ProjectX-dev-infra",
"allowedValues": [
"/subscriptions/xxxx-xxxx-xxxx-xxxx-xxxx/resourcegroups/rg-ProjectX-dev-infra"
],
"metadata": {
"description": "The resource ID for an existing Log Analytics workspace."
}
}
"variables": {
"la-name": "[concat('la', '-', parameters('base-name'), '-', 'workspace')]",
{
"name": "Scope",
"value": {
"resourceIds": "[resourceId('Microsoft.Operationalinsights/workspaces', parameters('log-analytics-workspace-id'), variables('la-name'))]"
},
"isOptional": true
}
resourceIds value defined like:
"name": "Scope",
"value": {
"resourceIds": "[parameters('log-analytics-workspace-id')]"
},
"isOptional": true
},
...and paremeter defined log-analytics-workspace-id:
"log-analytics-workspace-id": {
"type": "string",
"defaultValue": "/subscriptions/xxxx-xxxx-xxxx-xxxx-xxxx/resourcegroups/rg-ProjectX-dev-infra/providers/microsoft.operationalinsights/workspaces/la-ProjectX-dev-workspace",
"allowedValues": [
"/subscriptions/xxxx-xxxx-xxxx-xxxx-xxxx/resourcegroups/rg-ProjectX-dev-infra/providers/microsoft.operationalinsights/workspaces/la-ProjectX-dev-workspace"
],
"metadata": {
"description": "The resource ID for an existing Log Analytics workspace."
}
}
Azure Dashboard has deployed successfully, however, the dashboard is not operational yet:
Azure dashboard view
Fixed an issue by combining the value in array:
{
"name": "Scope",
"value": {
"resourceIds": [
"[parameters('log-analytics-workspace-id')]"
]
}
},
I need to dynamically call an Azure Function from inside my ADF pipeline.
Currently i'm able to parameterize the functionName through the Azure Function Activity, but i'm not able to parameterize nor the functionKey nor the URL.
The URL is no problem since I can store all the functions below the same URL but the functionKey is really a must for this.
Do you now any option to do that?
What I've tried
Parameter inside the json as with DataStoreLinkedServices:
{
"properties": {
"type": "AzureFunction",
"annotations": [],
"parameters": {
"functionSecret": {
"type": "String"
}
},
"typeProperties": {
"functionAppUrl": "https://<myurl>.azurewebsites.net",
"functionKey": {
"type": "AzureKeyVaultSecret",
"store": {
"referenceName": "KeyVaultLinkedService",
"type": "LinkedServiceReference"
},
"secretName": "#{linkedService().functionSecret}"
}
}
}
}
ErrorMsg:
"code":"BadRequest","message":"No value provided for Parameter 'functionSecret'"
Is there a way to achieve this? It seems not obvious, and I didn't found anything surfing the web. The most similar was this
I'll answer myself just in case someone have the same problem, what we do to manage this was parameterize the needed information from the pipeline itself.
So we have a pipeline that just call a generic Azure Function. In the caller pipeline, there is a process to obtain the desired parameters from the KeyVault and pass them to the AF pipeline.
The LS remain as follows:
{
"properties": {
"annotations": [],
"type": "AzureFunction",
"typeProperties": {
"functionAppUrl": "https://#{linkedService().functionAppUrl}.azurewebsites.net",
"functionKey": {
"type": "AzureKeyVaultSecret",
"store": {
"referenceName": "KeyVaultLinkedService",
"type": "LinkedServiceReference"
},
"secretName": "#{linkedService().functionKey}"
}
},
"parameters": {
"functionAppUrl": {
"type": "String",
"defaultValue": "#pipeline().parameters.functionAppUrl"
},
"functionKey": {
"type": "String",
"defaultValue": "#pipeline().parameters.functionKey"
}
}
}
}
I'm creating a logic app which will do some operations on a blob storage, thus it needs a Connector to a specific blob storage. I'm able to define which Connector should be used (providing its name and other properties), however if it doesn't exist yet, the template fails to deploy. I know we can create these connectors via logic app designer, but i would very much like to automate that process. Hence the question:
Is it possible to deploy/create this connector using an ARM template or a script?
You can check this post related to Logic App connector.
Here is an ARM Template that create an API connection to blob storage:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"azureBlobConnectionAPIName": {
"type": "string",
"metadata": {
"description": "The name of the connection api to access the azure blob storage."
}
},
"storageAccountName": {
"type": "string",
"metadata": {
"description": "The Storage Account Name."
}
}
},
"variables": {
"storageAccountId": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]"
},
"resources": [
{
"type": "Microsoft.Web/connections",
"name": "[parameters('azureBlobConnectionAPIName')]",
"apiVersion": "2016-06-01",
"location": "[resourceGroup().location]",
"scale": null,
"properties": {
"displayName": "[parameters('azureBlobConnectionAPIName')]",
"parameterValues": {
"accountName": "[parameters('storageAccountName')]",
"accessKey": "[listKeys(variables('storageAccountId'),'2015-05-01-preview').key1]"
},
"api": {
"id": "[concat('subscriptions/', subscription().subscriptionId, '/providers/Microsoft.Web/locations/', parameters('defaultResourceLocation'), '/managedApis/azureblob')]"
}
},
"dependsOn": []
}
]
}