I am working on creating a workbook that provides an umbrella view over multiple app insight instances. Our solution has many microservices (Azure functions) each having its own app insight instance. Aim of this workbook is to provide a health status for the whole app by surfacing up errors across app insight instances in to a single view.
I have used the "Failure Analysis" template to set this up. User is able to select different app insight instances at the top and the views will filter based on that. There's a view that shows exception counts with trends like this. Each error may belong to different App Insight instances.
When you click on a line item all instances of that error will be shown in a following view like this
I use the following query to load it
let row = dynamic({Row});
let req = requests
| where '{Row}' == '{}' or (row.Kind == 'Application' and row.Id
== appName) or (row.Kind == 'Request' and row.Id ==
strcat(appName, "::", name))
| where success == "False";
let errors = exceptions
| where appName == appName
| where timestamp between({TimeRange:start}..{TimeRange:end});
errors
| join req on operation_Id
| project operation_Id, itemId, timestamp,requestName=name,
exception=type, method, outerMessage, innermostMessage,
details, appName
As mentioned in this question Get exception details from a Azure Monitor Workbook, the itemId is available and I try to link it to the "Exception Details" view. Please note that these errors can come from one of many App Insight instances based on what gets selected at the previous view.
I have configured the item Id and appName columns as follows using Link renderer and Automatic renderer.
However the link always directs to one specific app insight instance (Not the one associated with the error) and hence the error won't get loaded. Is it possible to load the "Detail Views" across App Insight instances using this technique? If not what could be other avenues?
When the application insights is workspace based, each telemetry item has a field _ResourceId that contains a link to the resource, for example
/subscriptions/c8vfbeab-a5a67-4272-aa6e-4c9f4142e962/resourcegroups/rg-my-resource-group/providers/microsoft.insights/components/my-ai-resource
You can use this a part of the url to create a deep link to the details page of the Application Insights resource of the telemetry with the specified item id. Take this query for example:
exceptions
| take 1
| extend portalUrl = strcat("https://portal.azure.com/#blade/AppInsightsExtension/BladeRedirect/BladeName/searchV1/ResourceId/", url_encode(_ResourceId), "/BladeInputs/%7B%22tables%22%3A%5B%22availabilityResults%22%2C%22requests%22%2C%22exceptions%22%2C%22pageViews%22%2C%22traces%22%2C%22customEvents%22%2C%22dependencies%22%5D%2C%22timeContextWhereClause%22%3A%22%7C%20where%20timestamp%20%3E%20datetime(%5C%222022-02-12T12%3A55%3A02.739Z%5C%22)%20and%20timestamp%20%3C%20datetime(%5C%222022-03-14T12%3A55%3A02.739Z%5C%22)%22%2C%22filterWhereClause%22%3A%22%7C%20where%20*%20has%20%5C%22a1a20ad1a12ff348a852288a4d9953a5%5C%22%7C%20order%20by%20timestamp%20desc%22%2C%22originalParams%22%3A%7B%22eventTypes%22%3A%5B%7B%22value%22%3A%22availabilityResult%22%2C%22tableName%22%3A%22availabilityResults%22%2C%22label%22%3A%22Availability%22%7D%2C%7B%22value%22%3A%22request%22%2C%22tableName%22%3A%22requests%22%2C%22label%22%3A%22Request%22%7D%2C%7B%22value%22%3A%22exception%22%2C%22tableName%22%3A%22exceptions%22%2C%22label%22%3A%22Exception%22%7D%2C%7B%22value%22%3A%22pageView%22%2C%22tableName%22%3A%22pageViews%22%2C%22label%22%3A%22Page%20View%22%7D%2C%7B%22value%22%3A%22trace%22%2C%22tableName%22%3A%22traces%22%2C%22label%22%3A%22Trace%22%7D%2C%7B%22value%22%3A%22customEvent%22%2C%22tableName%22%3A%22customEvents%22%2C%22label%22%3A%22Custom%20Event%22%7D%2C%7B%22value%22%3A%22dependency%22%2C%22tableName%22%3A%22dependencies%22%2C%22label%22%3A%22Dependency%22%7D%5D%2C%22timeContext%22%3A%7B%22durationMs%22%3A2592000000%7D%2C%22filter%22%3A%5B%5D%2C%22searchPhrase%22%3A%7B%22originalPhrase%22%3A%22", itemId,"%22%2C%22_tokens%22%3A%5B%7B%22conjunction%22%3A%22and%22%2C%22value%22%3A%22a1a20ad1a12ff348a852288a4d9953a5%22%2C%22isNot%22%3Afalse%2C%22kql%22%3A%22%20*%20has%20%5C%22a1a20ad1a12ff348a852288a4d9953a5%5C%22%22%7D%5D%7D%2C%22sort%22%3A%22desc%22%7D%7D")
| project timestamp, problemId, itemId, portalUrl
If you create a workbook and render a table based on the query above you have to modify the itemId column to be a link like this:
Clicking the itemId column it should open the correct Application Insights resource details page:
Now, I hope this give you enough clues to extend your own queries by including the deep link url in your query output.
For completeness, this is the full workbook Gallery Template:
{
"version": "Notebook/1.0",
"items": [
{
"type": 3,
"content": {
"version": "KqlItem/1.0",
"query": "exceptions\n| take 1\n| extend portalUrl = strcat(\"https://portal.azure.com/#blade/AppInsightsExtension/BladeRedirect/BladeName/searchV1/ResourceId/\", url_encode(_ResourceId), \"/BladeInputs/%7B%22tables%22%3A%5B%22availabilityResults%22%2C%22requests%22%2C%22exceptions%22%2C%22pageViews%22%2C%22traces%22%2C%22customEvents%22%2C%22dependencies%22%5D%2C%22timeContextWhereClause%22%3A%22%7C%20where%20timestamp%20%3E%20datetime(%5C%222022-02-12T12%3A55%3A02.739Z%5C%22)%20and%20timestamp%20%3C%20datetime(%5C%222022-03-14T12%3A55%3A02.739Z%5C%22)%22%2C%22filterWhereClause%22%3A%22%7C%20where%20*%20has%20%5C%22a1a20ad1a12ff348a852288a4d9953a5%5C%22%7C%20order%20by%20timestamp%20desc%22%2C%22originalParams%22%3A%7B%22eventTypes%22%3A%5B%7B%22value%22%3A%22availabilityResult%22%2C%22tableName%22%3A%22availabilityResults%22%2C%22label%22%3A%22Availability%22%7D%2C%7B%22value%22%3A%22request%22%2C%22tableName%22%3A%22requests%22%2C%22label%22%3A%22Request%22%7D%2C%7B%22value%22%3A%22exception%22%2C%22tableName%22%3A%22exceptions%22%2C%22label%22%3A%22Exception%22%7D%2C%7B%22value%22%3A%22pageView%22%2C%22tableName%22%3A%22pageViews%22%2C%22label%22%3A%22Page%20View%22%7D%2C%7B%22value%22%3A%22trace%22%2C%22tableName%22%3A%22traces%22%2C%22label%22%3A%22Trace%22%7D%2C%7B%22value%22%3A%22customEvent%22%2C%22tableName%22%3A%22customEvents%22%2C%22label%22%3A%22Custom%20Event%22%7D%2C%7B%22value%22%3A%22dependency%22%2C%22tableName%22%3A%22dependencies%22%2C%22label%22%3A%22Dependency%22%7D%5D%2C%22timeContext%22%3A%7B%22durationMs%22%3A2592000000%7D%2C%22filter%22%3A%5B%5D%2C%22searchPhrase%22%3A%7B%22originalPhrase%22%3A%22\", itemId,\"%22%2C%22_tokens%22%3A%5B%7B%22conjunction%22%3A%22and%22%2C%22value%22%3A%22a1a20ad1a12ff348a852288a4d9953a5%22%2C%22isNot%22%3Afalse%2C%22kql%22%3A%22%20*%20has%20%5C%22a1a20ad1a12ff348a852288a4d9953a5%5C%22%22%7D%5D%7D%2C%22sort%22%3A%22desc%22%7D%7D\")\n| project timestamp, problemId, itemId, portalUrl",
"size": 1,
"timeContext": {
"durationMs": 86400000
},
"queryType": 0,
"resourceType": "microsoft.insights/components",
"gridSettings": {
"formatters": [
{
"columnMatch": "problemId",
"formatter": 1,
"formatOptions": {
"linkColumn": "portalUrl",
"linkTarget": "Url"
}
}
]
}
},
"name": "query - 2"
}
],
"fallbackResourceIds": [
"/subscriptions/4547474-1a67-4272-aa6e-4c9f4142e269/resourceGroups/rg-resourcegroup-prod/providers/microsoft.insights/components/appi-demo-prod"
],
"$schema": "https://github.com/Microsoft/Application-Insights-Workbooks/blob/master/schema/workbook.json"
}
It looks like from a quick perusal of the code, that if you also have an appName column in the row where itemId is, it will try to look in the resource list used in the query to try to find a resource with that same name, and if not, just take the first one it can find?
but i see that you have appName there, but i'm not sure the rest of the configuration from your step? are you also using all of the resources in the query?
Related
Please let me know how to include metadata in the insight query for QnAMaker logs
something as this, here the meatadata field comes empty
| where url endswith "generateAnswer"
| project timestamp, id, url, resultCode, duration, performanceBucket
| parse kind = regex url with *"(?i)knowledgebases/"KbId"/generateAnswer"
| join kind= inner (
traces | extend id = operation_ParentId
) on id
| extend question = tostring(customDimensions['traces'])
| extend answer = tostring(customDimensions['Answer'])
| extend score = tostring(customDimensions['Score'])
| extend metadata = tostring(customDimensions['qnaMakerOptions'])
| project answer,['metadata']
To help the other community Members posting the suggestion from the GITHUB where the issue is discussed.
If you look within the customEvents table, you can find an event with the name 'QnAMessage' - this is an event that is produced by the underlying QnAMaker service within the Bot Framework SDK, and produces more information beyond the trace event you identified. The corresponding class is Microsoft.Bot.Builder.AI.QnA.QnAMaker, located within the microsoft/botbuilder-dotnet repo.
With this class, it is possible to supply custom event properties and metrics that can be captured within Application Insights. To do this, the recommendation would be to create a custom dialog that inherits from QnAMakerDialog, and add logic as appropriate to supply any additional telemetry properties and metrics.
There is a sample custom dialog, the MultiplyDialog in the samples repo that shows how to write a custom dialog.
The general approach you would take is to write a custom dialog extending from QnAMakerDialog, probably overriding the protected getQnaMakerClient(...) method to return a custom QnAMaker implementation that includes the metadata you care about in the on QnaResults method. Below is some pseudocode that illustrates the example
class CustomQnAMakerDialog extends QnAMakerDialog {
async getQnaMakerClient(dialogContext) {
const client = await super.getQnAMakerClient(dialogContext);
const onQnaResults = client.onQnaResults.bind(client);
client.onQnaResults = (qnaResults, turnContext, properties = {}, metrics = {}) => {
properties = { ...properties, custom: 'property' };
metrics = { ...metrics, custom: 1.0 };
return onQnaResults(qnaResults, turnContext, properties, metrics);
}
}
}
For more information about logging metadata into app insights for qna maker service refer the below git hub link with complete code
QnaMakerDialog & QnaMaker
What is the query format for Terraform New Relic Workload resource? I have the below resource specified in my main.tf file and I want to be able to get all the entities that have a specific label.
...
resource "newrelic_workload" "prod-app" {
name = "Prod App"
account_id = <account id>
entity_search_query {
query = "label.STAGE = prod"
}
}
However, it seems to throw this error:
Error: A downstream error occurred., [{"extensions":{},"message":"Search query \u003clabel.STAGE = 'prod'\u003e is invalid, response message is \u003cProvided filter: label.STAGE = 'prod' is not a valid NRQL query (You can't use field: label.STAGE, that is not defined in schema.)\u003e"}]
There doesn't seem to be much info on the documentation other that it needs to be a valid NRQL query. The example seems to use "name like 'Example application'"
https://registry.terraform.io/providers/newrelic/newrelic/latest/docs/resources/workload
Ok, I figured it out. Turns out that I was just referring to a key that doesn't exist.
It should be tags.<tag key> so tags.label.STAGE.
I noticed my mistake when I looked at the API behind the scenes. It uses the NerdGraph API and the Query entities section describes how to structure your query.
I have a logic app , which needs to pickup records from azure table storage based on a condition where event_time (one of the field in table) >= current time +24 (hrs)
I have tried to implement above logic as
I am getting following error
{
"odata.error": {
"code": "InvalidInput",
"message": {
"lang": "en-US",
"value": "')' or operator expected at position 24 in '(event_date_time gt 2020-10-06T21:52:30.2795067Z)'.
}
}
}
Is there any way to implement above logic in azure logic app
For this problem, you can refer to my logic app below:
The expression of addToTime(...) is same with yours, and please notice there is a pair of '' on both sides of addToTime(...). And please also pay attention to the case of your field, I test with field TESTDATE in "Filter Query" but it doesn't work. So if exists upper case and lower case in your event_date_time, please use the original letter in "Filter Query".
I am new to Azure Data Factory and i am trying to copy data from JIRA to Azure Storage using Data Factory.
I am trying the JIRA REST API to fetch the data, but unfortunately i am struggling with pagination. Can Someone help me how to use the pagination in this case.
Request URL:
https://jira.abc.com/rest/api/3/search?jql=&startAt=0&maxResult=10
Sample response from JIRA:
{
"startAt" : 0,
"maxResults" : 10,
"total": 200,
"isLast": false,
"values": [
{ /* result 0 */ },
{ /* result 1 */ },
{ /* result 2 */ }
]
}
Click on the Copy Data activity (the block) in your pipeline
Click on
the Source tab
Scroll down to Pagination rules and click the plus
sign to add a rule
As highlighted below, put AbsoluteURL in the first box
and then, for JIRA it appears (according to their docs) that you need to put $._links.next instead of $.next_page in my screenshot. (This is the reference to the part of the JSON doc returned by JIRA containing the URL of the next page.)
Please comment if it works for you.
Refs:
MS data factory docs
JIRA API docs
I am having a very specific issue with a gateway plugin I am trying to finish.
I am trying to navigate to a different plugin using
'buildfire.pluginInstance.get($scope.deepLinnk,function (err, plugin) {
if (err) {
$scope.status = 'error!';
}
else {
console.log(plugin);
$scope.navigateSignIn(plugin);}
});
$scope.navigateSignIn = function (plugin) {
buildfire.navigation.navigateTo({
pluginId: plugin.token,
instanceId: plugin.instanceId,
title: plugin.title,
folderName: plugin.pluginTypeId
});
};
The navigateTo object is the only way I can get buildfire.navigate.navigateTo to work for buildfire made plugins.
However, when I try to navigate to plugins that I have created, the debugger shows and an alert saying "cannot load config file" then the entire platform crashes and makes me sign in again.
How can I navigate to plugins that I have created?
How are you getting the pluginId, instanceId and folderName? You cant simply save them or hard code them in. You need to initiate a dynamic data lookup see https://github.com/BuildFire/sdk/wiki/How-to-use-the-Datastore-Dynamic-Data
also you can look at an example such as the folder plugin https://github.com/BuildFire/folderPlugin/blob/d84551feb06cfc304c325480ca96d87795a66929/widget/widget.controller.js#L163
Basically every time a plugin is updated the plugin identifiers like folderName or title may change. So you need to keep your reference data fresh using dynamic data.
Here is a simple example that may draw a better picture. If you are referencing a plugin titled "Holiday Sales" so you save to your datastore collection {title: "Holiday Sales"} and hence forth refer to it by that title. This may work for a short period of time. However, if the app owner changes the title to "Summer Sale" now your copy is out-of-date. In traditional databases you would have 2 tables one with the source of truth and the other would have a foreign key referencing the first table. This way you join and always display the latest data.
Dynamic data is sort of an assisted lookup for you. You simply give it a key and what that key references. Then at run time when you make the call it will make the lookup you need server side and return to you the latest data you are looking for.
sample:
buildfire.datastore.save("MyData",{
_buildfire: { /// key identifier
myPluginsToNavTo: {
data:["123123-123123","asdasda-asdasd"] /// plugin instances
,dataType: "pluginInstance"
}
}
});
======
buildfire.datastore.getWithDynamicData("MyData",function(err,data){
// data would be:
/*
_buildfire: { /// key identifier
myPluginsToNavTo: {
data:["55f71347d06b61b4010351dc","asdasda-asdasd"]
,dataType: "pluginInstance"
,result: [ /// <=============new property added dynamically
{
"id": "55f71347d06b61b4010351dc",
"data": {
"pluginTypeId": 3212,
"token": "6372b101-addf-45da-bb0a-9208a09e7b6b",
"title": "YouTube Plugin",
"iconUrl": "http://s3-us-west-2.amazonaws.com/pluginserver/plugins/6372b101-addf-45da-bb0a-9208a09e7b6b/resources/image.png",
}
,{
"id": "asdasda-asdasd",
"data": {
"pluginTypeId": 123123,
"token": "1223123123-addf-45da-bb0a-9208a09e7b6b",
"title": "Plugin 2",
"iconUrl": "...",
}
}
}
]
}
}
*/
});
hope this helps