I'm quite new to azure's application insights analytics query.
I'm trying to make some reports out of the data I have.
In the table customEvents, there are rows that represents the start and the return (aka Start & Success) of a request (aka Event), but I cannot figure out how to combine the Start and Success for calculating an average/making a report.
let table1 = customEvents | extend Start=timestamp | where customDimensions.Action == "Start" and customDimensions.Event == "A" | project Start, operation_Id;
let table2 = customEvents | extend Success=timestamp | where customDimensions.Action == "Success" and customDimensions.Event == "A" | project Success, operation_Id;
union table*
Disappointingly, I can only get the following result:
Start Success operation_Id
___________________________________________________________________
2016-12-12T07:09:23.466Z null EktA4
2016-12-12T07:09:32.479Z null EktA4
2016-12-12T07:09:37.392Z null EktA4
2016-12-12T09:09:27.645Z null YpgOq
null 2016-12-12T07:09:26.551Z EktA4
null 2016-12-12T07:09:33.848Z EktA4
null 2016-12-12T07:09:38.265Z EktA4
null 2016-12-12T09:09:29.927Z YpgOq
You need a join for that, not a union. It took me a while to create joins in Application Insights but try this.
let startEvents = customEvents
| where customDimensions.Action == "Start" and customDimensions.Event == "A"
| extend Start = timestamp
| project operation_Id, Start;
customEvents
| where customDimensions.Action == "Succes" and customDimensions.Event == "A"
| extend Success = timestamp
| join kind=leftouter startEvents on operation_Id
| project operation_Id, Start, Success
Edit:
You might run into a problem though. You have multiple starts and success for the same operation. How should those be matched correctly? You should have a unique value that relates to 1 Start/Success combination.
Related
Have a question about how could I show success rate on Azure Dashboard.
If I have single temeletry event that indicates success or failure - it's quite simple:
customEvents
| where name == "ResponseEvent" and customDimensions.Condition == "test"
| summarize count() by tostring(customDimensions.State) //State could be Success|Failure
| render piechart
But in my case - I have 2 events: RequestEvent, SuccessResponseEvent and from those two I want to get success rate, something like: successRate = 100*successCount/requestCount.
I end up with this join:
customEvents
| where name == "RequestEvent" and customDimensions.Condition == "test"
| summarize requestCount = count()
| extend joinField = "1"
| join ( customEvents
| where name == "SuccessResponseEvent" and customDimensions.Condition == "test"
| summarize successCount = count()
| extend joinField = "1")
on joinField
| extend successRate = (100 * successCount / requestCount)
//////| extend failureRate = 100 - successRate
| project successRate
| render table
I got the value I need, but I only manage to display it as table, while I need a piechart.
I thought about adding union:
let success = view () { print x=toint(80) };
let failure = view () { print x=toint(20) };
union withsource=TableName success, failure
| render piechart
But I don't see how to do this in my request.
Or create variables using let statement and try to calculate everything and join using materialize(createRequestRecieved), but it causes quite a lot of errors and I hope some simple way exists.
Question is: maybe somebody could point me to how I could achieve this: calculate one value, maybe display it as two values (success and 100-success) and arrange them in format valid for "render piechart" operator?
And second question, not so important: could I join them by some existing field? Whey I'm trying to use joinField = tostring(customDimensions.MappingField) I'm getting an error: Ensure that expression: customDimensions.MappingField is indeed a simple name
If you are going for a piechart, it would require a string legend field and a value on each row for that legens, so union of two results should work:
requests
| summarize Success = sumif(itemCount, success == true)
| project Legend = "Success", Value = Success
| union
(requests
| summarize Failed = sumif(itemCount, success == false)
| project Legend = "Failed", Value = Failed )
| render piechart
Going for a barchart would allow to use both summarize clauses in one query without join/union and may speed up performance:
requests
| summarize Success = sumif(itemCount, success == true), Failed = sumif(itemCount, success == false)
| project Legend = "Status", Success, Failed
| render barchart
Similarly, to calculate the rate in the same query:
requests
| summarize Success = sumif(itemCount, success == true), Failed = sumif(itemCount, success == false)
| extend SuccessRate = Success * 1.0 / (Success + Failed)
I'm quite sure it's not the best option an I'm mising something in this query language capabilities, but I could put my request in variable, apply some caching and repeat it twice, I suppose:
let dataSource = customEvents
| where name == "RequestEvent" and customDimensions.Condition == "test"
| summarize requestCount = count()
| extend joinField = "1"
| join ( customEvents
| where name == "SuccessResponseEvent" and customDimensions.Condition == "test"
| summarize successCount = count()
| extend joinField = "1")
on joinField
| extend successRate = (100 * successCount / requestCount)
| extend failureRate = 100 - successRate;
let cacheddataSource = materialize(dataSource);
cacheddataSource
| project Legend = "Success", Value = successRate
| union (
dataSource
|project Legend = "Failure", Value = failureRate
)
| render piechart
So, let and materialize more or less helps, maybe some tweaks will be necessary to display actual amount of successes and failures.
I am trying to simply run a subquery in Azure application insights, using Kusto, so that I can get some information from two tables displayed as one.
The query I'm trying is
table1
| extend progressLog = toscalar(
table2
| where common_Id == table1.common_Id // errors saying Ensure that expression: table1.common_Id is indeed a simple name
| summarize makelist(stringColumn)
)
I have attempted to alias this id, and even join the two tables, as such:
requests
| extend aliased_id = common_Id
| join traces on operation_Id, $left.operation_Id == $right.operation_Id
| extend test_id = operation_Id
| extend progressLog = toscalar(
traces
| where operation_Id == aliased_id // Failed to resolve column or scalar expression named 'aliased_id'
| summarize makelist(message)
)
Failed to resolve column or scalar expression named 'aliased_id'.
I am simply trying to do the equivalent of the T-SQL query:
SELECT
... ,
STRING_AGG(table2.stringColumn, ',')
FROM
table1
INNER JOIN
table2
ON table1.common_Id = table2.common_Id
GROUP BY
table.<props>
My main question is - how do I reference "common_Id" in the kusto language inside a subquery
Please see if the next query provides what you're looking for. If not, please share sample input using datatable, as I did below, and expected output:
let requests = datatable(common_Id:string, operation_Id:string)
[
"A", "X",
"B", "Y",
"C", "Z"
];
let traces = datatable(operation_Id:string, message:string)
[
"X", "m1",
"X", "m2",
"Y", "m3"
];
let messagesByOperationId = traces | summarize makelist(message) by operation_Id;
requests
| join kind=leftouter messagesByOperationId on operation_Id
| project common_Id, operation_Id, progressLog = list_message
we collect custom events in application insights for each message a user sends to a chatbot. The event is called user_message.
We use a custom dimension field customDimensions.conversationid to know which message is related to which conversation.
I want to see the first message of each conversation so basically the "oldest" timestamp of each event based on the conversation id.
I tried to work with arg_max but I didn't figure out how it works.
customEvents
| extend itemType = iif(itemType == 'customEvent',itemType,"")
| where (itemType == 'customEvent')
| where name == 'User_Message'
i was able to show all user messages ordert by the conversationID however it shows me multiple lines and i only need the first message by conversation.
Datamodel:
timestamp [UTC] 2019-04-05T13:24:10.359Z
name User_Message
itemType customEvent
customDimensions
confidence N/A
conversationId BNu0SqC5RfA1S0lZmdxxxxx
intent N/A
userMessage user text
operation_Name POST /api/messages
operation_Id xxxxxxxa5d422eadebfebb2
operation_ParentId xxxxx545a5d422eadebfebb2.99811380_13.f033f887_
application_Version 1.0.0
client_Type PC
client_OS Windows_NT 10.0.14393
client_IP 0.0.0.0
client_City Amsterdam
client_StateOrProvince North Holland
client_CountryOrRegion Netherlands
cloud_RoleName Web
cloud_RoleInstance XXXXXXXFF74D594
appId ccccccc-8b24-41bb-a02a-1cb101da84e5
appName bot-XXXXX
iKey XXXXXX
sdkVersion node:XX
itemId XXXXXXXX-57a6-11e9-a5a7-ebc91e7cf64e
itemCount 1
SOLUION
customEvents
| extend itemType = iif(itemType == 'customEvent',itemType,"")
| where (itemType == 'customEvent')
| where (name=='User_Message')
| summarize list=makeset(customDimensions.userMessage) by
tostring(customDimensions.conversationId)
| mv-expand firstMessage=list[0]
Update:
customEvents
| where name == "User_Message"
| summarize timestamp=min(timestamp) by myconid=tostring(customDimensions.[conversationID])
| join kind= inner (
customEvents
| where name == "User_Message"
| extend myconid = tostring(customDimensions.[conversationID])
) on myconid,timestamp
You can use inner join to do that.
I don't have your data, so in your case, the code looks like below(maybe you need to make a little changes):
customEvents
| summarize timestamp=min(timestamp) by conversationID
| join kind= inner (
customEvents
) on conversationID,timestamp
| project-away conversationID1,timestamp1
Please let me know if you have more issues.
I'm trying to create a custom metric alert based on some metrics in my Application Insights logs. Below is the query I'm using;
let start = customEvents
| where customDimensions.configName == "configName"
| where name == "name"
| extend timestamp, correlationId = tostring(customDimensions.correlationId), configName = tostring(customDimensions.configName);
let ending = customEvents
| where customDimensions.configName == configName"
| where name == "anotherName"
| where customDimensions.taskName == "taskName"
| extend timestamp, correlationId = tostring(customDimensions.correlationId), configName = tostring(customDimensions.configName), name= name, nameTimeStamp= timestamp ;
let timeDiffs = start
| join (ending) on correlationId
| extend timeDiff = nameTimeStamp- timestamp
| project timeDiff, timestamp, nameTimeStamp, name, anotherName, correlationId;
timeDiffs
| summarize AggregatedValue=avg(timeDiff) by bin(timestamp, 1m)
When I run this query in Analytics page, I get results, however when I try to create a custom metric alert, I got the error Search Query should contain 'AggregatedValue' and 'bin(timestamp, [roundTo])' for Metric alert type
The only response I found was adding AggregatedValue which I already have, I'm not sure why custom metric alert page is giving me this error.
I found what was wrong with my query. Essentially, aggregated value needs to be numeric, however AggregatedValue=avg(timeDiff) produces time value, but it was in seconds, so it was a bit hard to notice. Converting it to int solves the problem,
I have just updated last bit as follows
timeDiffs
| summarize AggregatedValue=toint(avg(timeDiff)/time(1ms)) by bin(timestamp, 5m)
This brings another challenge on Aggregate On while creating the alert as AggregatedValue is not part of the grouping that is coming after by statement.
I want to perform a subselect on a related set of data. That subdata needs to be filtered using data from the main query:
customEvents
| extend envId = tostring(customDimensions.EnvironmentId)
| extend organisation = tostring(customDimensions.OrganisationName)
| extend version = tostring(customDimensions.Version)
| extend app = tostring(customDimensions.Appname)
| where customDimensions.EventName contains "ApiSessionStartStart"
| extend dbInfo = toscalar(
customEvents
| extend dbInfo = tostring(customDimensions.dbInfo)
| extend serverEnvId = tostring(customDimensions.EnvironmentId)
| where customDimensions.EventName == "ServiceSessionStart" or customDimensions.EventName == "ServiceSessionContinuation"
| where serverEnvId = envId // This gives and error
| project dbInfo
| take 1)
| order by timestamp desc
| project timestamp, customDimensions.OrganisationName, customDimensions.Version, customDimensions.onBehalfOf, customDimensions.userId, customDimensions.Appname, customDimensions.apiKey, customDimensions.remoteIp, session_Id , dbInfo, envId
The above query results in an error:
Failed to resolve entity 'envId'
How can I filter the data in the subselect based on the field envId in the main query?
i believe you'd need to use join instead, where you'd join to get that value from the second query
docs for join: https://docs.loganalytics.io/docs/Language-Reference/Tabular-operators/join-operator
the left hand side of the join is your "outer" query, and the right hand side of the join would be that "inner" query, though instead of doing take 1, you'd probably do a simpler query that just gets distinct values of serverEnvId, dbInfo