I have been trying to add an Azure monitor alert to slack. The resource is being provisioned via ARM template. However I couldn't get the alert to go to slack.
This is my actions section in the template
"actions": [
{
"actionGroupId": "[resourceId('Microsoft.Insights/actionGroups/', 'SlackWebhook')]",
"webHookProperties": {
"text": "Alert: memory usage exceeded threshold"
}
}
]
I have set up a dummy webhook to inspect the payload being sent. This is what I found
{
"schemaId": "AzureMonitorMetricAlert",
"data": {
"version": "2.0",
"properties": {
"text": "Dropbox connector Memory usage exceeded the threshold of 1GB"
},
"status": "Activated",
"context": {...}
}
}
The property goes inside data.properties. Slack expects the text property to be at the top level of request body. How to make this work?
Related
Our team wants to import the log of Azure Function by Azure Event Hub and Filebeat into Elastic Search. We followed this references to set up an Event Hub for Azure function. But we faced an issue with the wrong format of the log stream.
Firstly, Let me show what's the correct format we expect. Take Azure PostgresSQL's log from Event Hub for example:
[
{
"records": [
{
"time": "2023-01-04T03:45:31.1040000Z",
"properties": {
"timestamp": "2023-01-04 03:45:31.104 UTC",
"processId": 8909,
"errorLevel": "LOG",
"sqlerrcode": "00000",
"message": "2023-01-04 03:45:31 UTC-63b4f65b.22cd-LOG: connection received: host=<host> port=<port>"
},
"resourceId": "/SUBSCRIPTIONS/<my subscription id>/RESOURCEGROUPS/<my resource group>/PROVIDERS/MICROSOFT.DBFORPOSTGRESQL/FLEXIBLESERVERS/<postgres server>",
"category": "PostgreSQLLogs",
"operationName": "LogEvent"
}
]
}
]
Notice that the properties is a flattened json so that it can be consumed by Filebeat. We want this kind of properties. But the Azure Function's log looks like the following, which is a string rather than flattened json:
[
{
"records": [
{
"level": "Informational",
"resourceId": "/SUBSCRIPTIONS/<my subscription id>/RESOURCEGROUPS/<my resource group>/PROVIDERS/MICROSOFT.WEB/SITES/<my azure function>"
"operationName": "Microsoft.Web/sites/functions/log",
"category": "FunctionAppLogs",
"time": "01/04/2023 01:55:00",
"properties": "{'appName':'<my azure function>','roleInstance':'<id>','message':'Host Status: {\\n \\'id\\': \\'<function app id>\\',\\n \\'state\\': \\'Running\\',\\n \\'version\\': \\'4.13.0.0\\',\\n \\'versionDetails\\': \\'4.13.0+da9a765ed67be48c79440526f78fa1b5c6efdeea\\',\\n \\'platformVersion\\': \\'99.0.10.764\\',\\n \\'instanceId\\': \\'<instance id>\\',\\n \\'computerName\\': \\'<computer name>\\',\\n \\'processUptime\\': 69254486,\\n \\'functionAppContentEditingState\\': \\'Unknown\\'\\n}','category':'Host.Controllers.Host','hostVersion':'4.13.0.0','hostInstanceId':'<host id>','level':'Information','levelId':2,'processId':1}",
"EventStampType": "Stamp",
"EventPrimaryStampName": "waws-prod-ty1-081",
"EventStampName": "waws-prod-ty1-081",
"Host": "<host name>",
"EventIpAddress": "<ip address>"
},
The string value of properties can't be processed by decode_json_fields of Filebeat either because the format is not json (the format is 'key': value rather than "key": value). Is there any way to correct the format of properties before it is consumed by Filebeat? By the way, our Azure Function is deployed using a container.
I've created an Alert in my App Service that sends an Alert to a logic app, the logic app is then posting a message to Microsoft Teams.
https://learn.microsoft.com/en-us/azure/azure-monitor/platform/action-groups-logic-app
Everything is working as expected accept that i can get the data out of the individual into my Message.
I've used the following in schema in my logic App
{
"schemaId": "azureMonitorCommonAlertSchema",
"data": {
"essentials": {
"alertId": "/subscriptions/MyAlert",
"alertRule": "Web - Test teams",
"severity": "Sev1",
"signalType": "Metric",
"monitorCondition": "Fired",
"monitoringService": "Platform",
"alertTargetIDs": [
"/subscriptions/MySub"
],
"originAlertId": "bd40051b-35fa-,
"firedDateTime": "2020-06-03T14:53:34.0942607Z",
"description": "",
"essentialsVersion": "1.0",
"alertContextVersion": "1.0"
},
"alertContext": {
"properties": null,
"conditionType": "SingleResourceMultipleMetricCriteria",
"condition": {
"windowSize": "PT5M",
"allOf": [
{
"metricName": "Http2xx",
"metricNamespace": "Microsoft.Web/sites",
"operator": "GreaterThan",
"threshold": "5",
"timeAggregation": "Total",
"dimensions": [
{
"name": "ResourceId",
"value": "MyWebs.com"
}
],
"metricValue": 24,
"webTestName": null
}
],
"windowStartTime": "2020-06-03T14:45:23.095Z",
"windowEndTime": "2020-06-03T14:50:23.095Z"
}
}
}
}
Then in the designer added the fields
Here is the details from the Logic code view for the message body
"content": "Your Azure Monitor alert was triggered\nAzure monitor alert rule Web - Test teams was triggered at #{triggerBody()?['body']?['data']?['alertContext']?['condition']?['windowEndTime']}\n\nRule: #{triggerBody()?['body']?['data']?['essentials']?['alertRule']}\nBody:#{triggerBody()}\nHeader:#{triggerOutputs()['headers']}\nheaders:#{triggerBody()?['headers']}\nessentials:#{triggerBody()?['body']?['data']?['essentials']}\ndata:#{triggerBody()?['body']?['data']}\nbody:#{triggerBody()?['body']}"
The only field that gets populated is the body and none of the specific fields
Your Azure Monitor alert was triggered
Azure monitor alert rule Web - Test teams was triggered at
Rule:
Body:{"schemaId":"azureMonitorCommonAlertSchema","data":{"essentials":{"alertId":"/subscriptions/bresourceGroups/Microsoft.AlertsManagement/alerts","alertRule":"Web - Test Alert","severity":"Sev0","signalType":"Metric","monitorCondition":"Fired","monitoringService":"Platform","alertTargetIDs":[""],"originAlertId":"":"2020-06-03T15:49:20.1712118Z","description":"","essentialsVersion":"1.0","alertContextVersion":"1.0"},"alertContext":{"properties":null,"conditionType":"SingleResourceMultipleMetricCriteria","condition":{"windowSize":"PT5M","allOf":[{"metricName":"Http2xx","metricNamespace":"Microsoft.Web/sites","operator":"GreaterThan","threshold":"3","timeAggregation":"Count","dimensions":[{"name":"ResourceId","value":""}],"metricValue":7.0,"webTestName":null}],"windowStartTime":"2020-06-03T15:41:05.994Z","windowEndTime":"2020-06-03T15:46:05.994Z"}}}}
Header:{"Connection":"Keep-Alive","Expect":"100-continue","Host":"prod-06.uksouth.logic.azure.com","User-Agent":"IcMBroadcaster/1.0","X-CorrelationContext":"RkkKACgAAAACAAAAEABEgMLahbH0Sqw1EVoRy7Y8AQAQANlpmHhZlSRMkU6bLTb+DSk=","Content-Length":"1254","Content-Type":"application/json; charset=utf-8"}
headers:
essentials:
data:
body:
I had to manually update the Logic Code and remove the additional body tag
original
#{triggerBody()?['body']?['data']?['alertContext']?['condition']?['windowEndTime']}
to this
#{triggerBody()?['data']?['alertContext']?['condition']?['windowEndTime']}
I would like to be able to get custom output from an "Execute Pipeline Activity". During the execution of the invoked pipeline, I capture some information in a variable using the "Set Variable" activity. I would like to be able to use that value in the master pipeline.
I know that the master pipeline can read the invoked pipeline's name and runId using "#activity('InvokedPipeline').output," but those are the only properties available.
I have the invokable pipeline because it's configurable to be used by multiple other pipelines, assuming we can get the output from it. It currently consists of 8 activities; I would hate to have to duplicate them all across multiple pipelines just because we can't get the output from an invoked pipeline.
Reference: Execute Pipeline Activity
[
{
"name": "MasterPipeline",
"type": "Microsoft.DataFactory/factories/pipelines"
"properties": {
"description": "Uses the results of the invoked pipeline to do some further processing",
"activities": [
{
"name": "ExecuteChildPipeline",
"description": "Executes the child pipeline to get some value.",
"type": "ExecutePipeline",
"dependsOn": [],
"userProperties": [],
"typeProperties": {
"pipeline": {
"referenceName": "InvokedPipeline",
"type": "PipelineReference"
},
"waitOnCompletion": true
}
},
{
"name": "UseVariableFromInvokedPipeline",
"description": "Uses the variable returned from the invoked pipeline.",
"type": "Copy",
"dependsOn": [
{
"activity": "ExecuteChildPipeline",
"dependencyConditions": [
"Succeeded"
]
}
]
}
],
"parameters": {},
"variables": {}
}
},
{
"name": "InvokedPipeline",
"type": "Microsoft.DataFactory/factories/pipelines"
"properties": {
"description": "The child pipeline that makes some HTTP calls, gets some metadata, and sets a variable.",
"activities": [
{
"name": "SetMyVariable",
"description": "Sets a variable after some processing from other activities.",
"type": "SetVariable",
"dependsOn": [
{
"activity": "ProcessingActivity",
"dependencyConditions": [
"Succeeded"
]
}
],
"userProperties": [],
"typeProperties": {
"variableName": "MyVariable",
"value": {
"value": "#activity('ProcessingActivity').output",
"type": "Expression"
}
}
}
],
"parameters": {},
"variables": {
"MyVariable": {
"type": "String"
}
}
}
}
]
Hello Heather and thank you for your inquiry. Custom outputs are not an inbuilt feature at this time. You can request/upvote for the feature in the Azure feedback forum. For now, I do have two workarounds.
Utilizing the invoked pipeline's runID, we can query the REST API (using Web Activity) for the activity run logs, and from there, the activity outputs. However, before making the query, it is necessary to authenticate.
REST call to get the activities of a pipeline
For authentication I reccomend using the Web Activity to get an oauth2 token. The URL would be https://login.microsoftonline.com/tenantid/oauth2/token. Headers "Content-Type": "application/x-www-form-urlencoded" and body "grant_type=client_credentials&client_id=xxxx&client_secret=xxxx&resource=https://management.azure.com/". Since this request is to get credentials, the Authentication setting for this request is type 'None'. These credentials correspond to an App you create via Azure Active Directory>App Registrations. Do not forget to assign the app RBAC in Data FActory Access Control (IAM).
Another workaround, has the child pipeline write its output. It can write to a database table, or it can write to a blob (I passed the Data Factory variable to a Logic App which wrote to blob storage), or to something else of your choice. Since you are planning to use the child pipeline for many different parent pipelines, I would recommend passing the child pipeline a parameter that it uses to identify the output to the parent. That could mean a blob name, or writing the parent runID to a SQL table. This way the parent pipeline knows where to look to get the output.
just had a chat with ADF team, and the response
[10:11 PM] Mark Kromer
Brajesh Jaishwal: any plans on custom output from execute pipeline activity?
Yes, this work is on the engineering work plan
Hi have an ARM Template that deploys some custom connectors. I have a connector called Start in that when i try to use this below piece of code which sets message body and header seperately it works fine. But when i do the same on a following connector , Assue Connector-Start as A and its followed by connector B. The input is not showing in different fields. Instead its not even relevant. can anyone help.
In the connector parameters here's the first block
{
"name": "Body",
"in": "body",
"schema": {
"type": "object",
"properties": {
"MessageBody": {
"type": "object",
"description": "Message body passed to the http trigger"
},
"MessageHeader": {
"type": "object",
"description": "Message header passed to the http trigger"
}
},
"required": [
"MessageBody",
"MessageHeader"
]
},
"description": "Message body to get properties from the message payload.",
"required": true
},
Here's the input shown in portal for connector-A
It seems, after deployment the old connector is cached. Removing it and adding it again worked.
I'm attempting to deploy an Azure Logic App that includes an action to Send a message on a Service Bus using an ARM template.
In addition to deploying the Logic App, the ARM template deploys a Service Bus Namespace, a Queue and two AuthorizationRule (one for sending and one for listening).
I want to dynamically set the connection information for the Send Service Bus Message action to use the Connection string generated for the AuthorizationRule that supports sending.
When I create this in the portal editor (specifying the connection string for sending), I noticed the following is generated in code view...
"Send_message.": {
"conditions": [
{
"dependsOn": "<previous action>"
}
],
"inputs": {
"body": {
"ContentData": "#{encodeBase64(triggerBody())}"
},
"host": {
"api": {
"runtimeUrl": "https://logic-apis-westus.azure-apim.net/apim/servicebus"
},
"connection": {
"name": "#parameters('$connections')['servicebus']['connectionId']"
}
},
"method": "post",
"path": "/#{encodeURIComponent(string('<queuename>'))}/messages"
},
"type": "apiconnection"
}
},
I assume that the connection information is somehow buried in #parameters('$connections')['servicebus']['connectionId']"
I then used resources.azure.com to navigate to the logic app to see if I could get more details as to how #parameters('$connections')['servicebus']['connectionId']" is defined.
I found this:
"parameters": {
"$connections": {
"value": {
"servicebus": {
"connectionId": "/subscriptions/<subguid>/resourceGroups/<rgname>/providers/Microsoft.Web/connections/servicebus",
"connectionName": "servicebus",
"id": "/subscriptions/<subguid>/providers/Microsoft.Web/locations/westus/managedApis/servicebus"
}
}
}
}
But I still don't see where the connection string is set.
Where can I set the connection string for the service bus action in an ARM template using something like the following?
[listkeys(variables('sendAuthRuleResourceId'), variables('sbVersion')).primaryConnectionString]
EDIT: Also, I've referred to was seems to be a promising Azure quick start on github (based on the title), but I can't make any sense of it. It appears to use an older schema 2014-12-01-preview, and the "queueconnector" references an Api Gateway. If there is a newer example out there for this scenario, I'd love to see it.
I've recently worked on an ARM Template for the deployment of logic apps and service bus connection. Here is the sample template for configuring service bus connection string within the type "Microsoft.Web/connections". Hope it helps.
{
"type": "Microsoft.Web/connections",
"apiVersion": "2016-06-01",
"name": "[parameters('connections_servicebus_name')]",
"location": "centralus",
"dependsOn": [
"[resourceId('Microsoft.ServiceBus/namespaces/AuthorizationRules', parameters('ServiceBusNamespace'), 'RootManageSharedAccessKey')]"
],
"properties": {
"displayName": "ServiceBusConnection",
"customParameterValues": {},
"api": {
"id": "[concat(subscription().id, '/providers/Microsoft.Web/locations/centralus/managedApis/servicebus')]"
},
"parameterValues": {
"connectionString": "[listKeys(resourceId('Microsoft.ServiceBus/namespaces/authorizationRules', parameters('ServiceBusNamespace'), 'RootManageSharedAccessKey'), '2017-04-01').primaryConnectionString]"
}
}
}
As you know connections is a resource so it needs to be created first did you refer this https://blogs.msdn.microsoft.com/logicapps/2016/02/23/deploying-in-the-logic-apps-preview-refresh/. Quick start link you are referring is for older schema.