Azure - Data Factory - New Pipeline Created - azure

is there any way to receive a mail (some kind of alert) when someone creates a new Pipeline in a Specific Data Factory?
Something like "user XYZ created a new Pipeline"
Thanks for your inputs,
Marcelo

To specify an alert definition, you create a JSON file describing the operations that you want to be alerted on.
Following example creates an alert for Run Completion.
Below JSON will help you to create similar for update alert.
{
"contentVersion": "1.0.0.0",
"$schema": "http://schema.management.azure.com/schemas/2014-04-01-preview/deploymentTemplate.json#",
"parameters": {},
"resources": [
{
"name": "ADFAlertsSlice",
"type": "microsoft.insights/alertrules",
"apiVersion": "2014-04-01",
"location": "East US",
"properties": {
"name": "ADFAlertsSlice",
"description": "One or more of the data slices for the Azure Data Factory has failed processing.",
"isEnabled": true,
"condition": {
"odata.type": "Microsoft.Azure.Management.Insights.Models.ManagementEventRuleCondition",
"dataSource": {
"odata.type": "Microsoft.Azure.Management.Insights.Models.RuleManagementEventDataSource",
"operationName": "RunFinished",
"status": "Failed",
"subStatus": "FailedExecution"
}
},
"action": {
"odata.type": "Microsoft.Azure.Management.Insights.Models.RuleEmailAction",
"customEmails": [
"#contoso.com"
]
}
}
}

Related

Is it possible to create alerting in Azure App Insight automatically by script or other ways?

There are a lot of manual steps to create alert on App Insight. Hence, I would like to have the script to configure alerting to save reduce effort when I would like to deploy new or even update it in future
Yes this is possible, for example using an ARM or Bicep template. Or you can leverage the azure cli to create alerts
For Bicep/ARM the template format can be found in the docs. The nice thing about this approach is that you can deploy the alerts using a ci/cd pipeline, for example using Azure DevOps.
Here is an alert based on an Application Insights log search result:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"scheduledqueryrules_alert_exception_name": {
"defaultValue": "alert-fexception",
"type": "String"
},
"components_ai_prod_externalid": {
"defaultValue": "/subscriptions/xxx/resourceGroups/xxx/providers/microsoft.insights/components/xxx",
"type": "String"
},
"actiongroups_team_blue": {
"defaultValue": "/subscriptions/xxx/resourceGroups/xxx/providers/microsoft.insights/actiongroups/xxx",
"type": "String"
}
},
"variables": {},
"resources": [
{
"type": "microsoft.insights/scheduledqueryrules",
"apiVersion": "2021-08-01",
"name": "[parameters('scheduledqueryrules_alert_exception_name')]",
"location": "westeurope",
"properties": {
"displayName": "[parameters('scheduledqueryrules_alert_exception_name')]",
"severity": 1,
"enabled": true,
"evaluationFrequency": "PT5M",
"scopes": [
"[parameters('components_ai__externalid')]"
],
"targetResourceTypes": [
"microsoft.insights/components"
],
"windowSize": "PT5M",
"criteria": {
"allOf": [
{
"query": "exceptions\n| extend requestId = tostring(customDimensions.SpanId)\n| join kind=leftouter (requests) on $left.requestId == $right.['id']\n| project timestamp, operation_Id, problemId, cloud_RoleName, url, outerMessage, application_Version\n",
"timeAggregation": "Count",
"dimensions": [],
"operator": "GreaterThan",
"threshold": 0,
"failingPeriods": {
"numberOfEvaluationPeriods": 1,
"minFailingPeriodsToAlert": 1
}
}
]
},
"autoMitigate": false,
"actions": {
"actionGroups": [
"[parameters('actiongroups_team_blue')]"
]
}
}
}
]
}

Azure ARM role assignment for System Assigned Managed Identity fails the first run

My goal is to deploy a logic app with a system managed identity and a role assignment for that identity. Preferably, this is done in one ARM template.
I have a setup that fails the first run, but succeeds successive runs. Correct me if I'm wrong, but I think that the reason for this is that the deployment of the role assignment happens before the managed identity of the logic app is "ready", hence the following error I get the first time that I deploy the template. I don't get this error the second time I deploy the template, probably because the Identity already exists at that time.
{
"code": "DeploymentFailed",
"message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
"details": [
{
"code": "PrincipalNotFound",
"message": "Principal *** does not exist in the directory ***."
}
]
}
My template (removed logic app definition for brevity). In this case the identity of the logic app requires access to a storage account which is located in another resource group.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"logicAppName": {
"type": "string"
},
"storageAccountResourceGroup": {
"type": "String"
},
"storageAccountName": {
"type": "String"
}
},
"variables": {
"logicAppResourceId": "[resourceId('Microsoft.Logic/workflows', parameters('logicAppName'))]",
"Storage Blob Data Contributor": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'ba92f5b4-2d11-453d-a403-e96b0029c9fe')]"
},
"resources": [
{
"type": "Microsoft.Logic/workflows",
"apiVersion": "2017-07-01",
"name": "[parameters('logicAppName')]",
"location": "[resourceGroup().location]",
"identity": {
"type": "SystemAssigned"
},
"properties": {
"state": "Enabled",
"definition": {
...
}
}
},
{
"type": "Microsoft.Resources/deployments",
"name": "[concat('RoleAssignment-', parameters('logicAppName'))]",
"apiVersion": "2020-06-01",
"resourceGroup": "[parameters('storageAccountResourceGroup')]",
"subscriptionId": "[subscription().subscriptionId]",
"dependsOn": [
"[parameters('logicAppName')]"
],
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"apiVersion": "2018-09-01-preview",
"type": "Microsoft.Storage/storageAccounts/providers/roleAssignments",
"name": "[concat(parameters('storageAccountName'), '/Microsoft.Authorization/', guid(subscription().subscriptionId, parameters('logicAppName')))]",
"properties": {
"roleDefinitionId": "[variables('Storage Blob Data Contributor')]",
"principalId": "[reference(variables('logicAppResourceId'), '2019-05-01', 'Full').identity.principalId]"
}
}
]
}
}
}
]
}
As you can see in the template, I added a dependsOn on the logic app itself. However that doesn't seem to be sufficient.
Does anyone have a solution for this?
Thank you!
I found the answer here: https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-template#new-service-principal
Deployment works consistently after adding "principalType": "ServicePrincipal"

Update operations in Azure Api Managment with swagger link using ARM template

I would like to update operation of an API with swagger link using ARM Template. When I run the below ARM Template I get the error
"error": { "code": "InvalidRequestContent", "message": "The request
content was invalid and could not be deserialized: 'Could not find
member 'dependsOn' on object of type 'DeploymentPropertiesDefinition'.
Path 'properties.dependsOn', line 1, position 734.'."
https://learn.microsoft.com/en-us/azure/templates/microsoft.apimanagement/2018-06-01-preview/service/apis
{
"$schema":
"https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"ApimServiceName": {
"type": "string"
},
"swaggerjson": {"type": "string"}
},
"variables": {
},
"resources": [
{
"type": "Microsoft.ApiManagement/service/apis",
"name": "[concat(parameters('ApimServiceName'), '/animalAPI4')]",
"apiVersion": "2018-02-01-preview",
"scale": null,
"properties": {
"displayName": "HTTP animal API",
"apiRevision": "1",
"description": "API Management facade for a very handy and free online HTTP toolsds",
"serviceUrl": "https://animailhttpbin.org",
"path": "animals4",
"contentFormat": "swagger-link-json",
"contentValue": "[parameters('swaggerjson')]",
"apiVersionSet": {
"id": "[concat(resourceId('Microsoft.ApiManagement/service', parameters('ApimServiceName')), '/api-version-sets/versionset-animal-api4')]"
},
"protocols": [
"https"
],
"authenticationSettings": null,
"subscriptionKeyParameterNames": null,
"apiVersion": "v1"
},
"dependsOn": [
]
}
]
}
Figured it out the issue was with the linked template from where I was calling the bracket around properties also include dependsOn. fixed the bracket issue

Azure Alert not triggered WebHook for Azure WebJobs

I'm trying to find what is wrong in my Template Deployment for Azure Alert. I want to Send Email and triggered a WebHook in Azure WebJobs with Basic Authentification.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
},
"resources": [
{
"type": "Microsoft.Insights/alertRules",
"name": "tesTalert",
"location": "westeurope",
"apiVersion": "2014-04-01",
"properties": {
"name": "test",
"isEnabled": true,
"condition": {
"odata.type": "Microsoft.Azure.Management.Insights.Models.ManagementEventRuleCondition",
"dataSource": {
"odata.type": "Microsoft.Azure.Management.Insights.Models.RuleManagementEventDataSource",
"operationName": "RunFinished",
"status": "Failed",
"subStatus": "FailedExecution"
}
},
"action": {
"odata.type": "Microsoft.Azure.Management.Insights.Models.RuleWebhookAction",
"serviceUri": "https://$myUserName:MyPassWord#MyWebsite.scm.azurewebsites.net/api/triggeredwebjobs/WebJobName/run",
"properties": {}
}
}
}
]
}
i have also try with this template with email and it's working but not for the webhook. Maybe the problem is from the api version , so i try different version. Nothing happend.
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": { },
"resources": [
{
"type": "microsoft.insights/alertrules",
"name": "ADFAlertsSlice",
"apiVersion": "2015-01-01",
"location": "westeurope",
"properties": {
"name": "ADFAlertsSlice",
"description": "One or more of the data slices for the Azure Data Factory has failed processing.",
"isEnabled": true,
"condition": {
"odata.type": "Microsoft.Azure.Management.Insights.Models.ManagementEventRuleCondition",
"dataSource": {
"odata.type": "Microsoft.Azure.Management.Insights.Models.RuleManagementEventDataSource",
"operationName": "RunFinished",
"status": "Failed",
"subStatus": "FailedExecution"
}
},
"actions": [
{
"odata.type": "Microsoft.Azure.Management.Insights.Models.RuleWebhookAction",
"serviceUri": "https://$myUserName:MyPassWord#MyWebsite.scm.azurewebsites.net/api/triggeredwebjobs/WebJobName/run",
"properties": {}
},
{
"odata.type": "Microsoft.Azure.Management.Insights.Models.RuleEmailAction",
"customEmails": [
"MyEmail#email.com"
]
}
]
}
}
]
}
I can't find what is wrong. thanks for your help.
According to Microsoft docs, webhooks from metric alert rules can only use token authentication.
Authenticating the webhook
The webhook can authenticate using token-based authorization. The webhook URI is saved with a token ID, eg.
https://mysamplealert/webcallback?tokenid=sometokenid&someparameter=somevalue
https://learn.microsoft.com/en-us/azure/monitoring-and-diagnostics/insights-webhooks-alerts
And if you scroll down to the comments other ahve the same issue as you do, they even started a feature request with MS.
https://feedback.azure.com/forums/602299-azure-alert-management/suggestions/19711606-enable-user-password-authentication-to-webhooks

Resource Manager - JSON having multiple alerts fails

I use below JSON to create AI alert, which works fine. However when I try to add add multiple alerts in this JSON file, it creates few alerts, sometimes one and other times two or three and finally returns provisioning state as Failed. It never creates all the alerts in the JSON (six).
{
"$schema": "http://schema.management.azure.com/schemas/2014-04-01-preview/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"AlertName": {
"type": "string"
},
"Location": {
"type": "string",
"defaultValue": "East US"
},
"ResourceGroup": {
"type": "string"
},
"ResourceName": {
"type": "string"
}
},
"resources": [{
"apiVersion": "2014-04-01",
"name": "[parameters('AlertName')]",
"type": "microsoft.insights/alertrules",
"location": "[parameters('Location')]",
"tags": {
"displayName": "AppInsightsAlert"
},
"properties": {
"name": "[concat('AppInsightsAlert ', parameters('AlertName'))]",
"description": "[concat('App Insights Alert ', parameters('AlertName'))]",
"isEnabled": true,
"condition": {
"odata.type": "Microsoft.Azure.Management.Insights.Models.ThresholdRuleCondition",
"dataSource": {
"odata.type": "Microsoft.Azure.Management.Insights.Models.RuleMetricDataSource",
"resourceUri": "[concat('/subscriptions/<subscription id>/resourcegroups/', parameters('ResourceGroup'), '/providers/microsoft.insights/components/', parameters('ResourceName'))]",
"metricName": "view.count"
},
"threshold": 1,
"windowSize": "PT5M"
},
"action": {
"odata.type": "Microsoft.Azure.Management.Insights.Models.RuleEmailAction",
"sendToServiceOwners": true,
"customEmails": []
}
}
}]
}
Are you using PowerShell for deployment - if so, add the -debug switch to the New-AzureResourceGroup* command and sift through the output. The raw error from the platform is in there and it's usually pretty good. If that doesn't help, can you post the entire json template for the deployments that are failing?

Resources