Pretty new to AI queries so any help will be much appreciated.
We have a host of custom events for particular actions for example, booking appointments, ordering products, setting an address. I would like to run a query to look at users who performed both ordering a product and setting their address in the same session. I can get a count and dcount of either events happening but struggling to specify that both happen in the same session We capture the User_authID as well session id with the custom events. Any ideas?
Thanks,
Chris
There's a few ways to do this. I find using the in operator and a subquery to be the easiest to read. Here's an example that does it:
let timeRange = ago(1d);
let sessionsWithBothEvents = customEvents
| where timestamp > timeRange
| summarize CountEvent1=countif(name == "event1"), CountEvent2=countif(name == "event2") by session_Id
| where CountEvent1 > 0 and CountEvent2 > 0
| project session_Id;
customEvents
| where timestamp > timeRange
| where session_Id in (sessionsWithBothEvents)
// Here you have all the events in sessions that contained at least one instance of each event
// From here you can dcount users, etc.
It is important to note that this approach will only work for up to 1 million session ids that match the criteria. This is due to the limitations of the in operator. See https://docs.loganalytics.io/docs/Language-Reference/Scalar-operators/in_!in-operators for more information.
Related
So I am trying to write a query for Azure Application Insights logs.
Until now, I logged custom events, so all of the properties which I wanted to show could be found in the event's customDimensions. This was easy to query, it looked something like this:
customEvents |
project
name,
Endpoint = customDimensions.Endpoint,
Context = customDimensions.Context,
...
Response = customDimensions.Response
This was fine, but now there are cases where the customDimensions.Response's value is longer than 8192 characters, which is the limit of these custom properties. For this reason, I removed the Response property, and added an EventId property instead, which is a unique Id representing each event.
The responses are now stored as traces, because the trace message limit is 32k instead of 8.
In order to be able to identify which response belongs to which event, I added an EventId property to these traces too, giving it the same value as for it's custom event.
Now I am trying to write a query which could retrieve these, project the same fields it did before from customEvents, and also the Response (message) from traces, joining them on the EventId property stored in customDimensions.
Please point me in the right direction.
So you want to join data from customEvents with the traces? Just use the join operator like this:
customEvents | project
name,
Endpoint = customDimensions.Endpoint,
Context = customDimensions.Context,
eventId = tostring(customDimensions.EventId)
| join kind=leftouter
(traces | project message, eventId = tostring(customDimensions.EventId)) on eventId
I am reporting some custom events to Azure, within the custom event is a value being held under the customMeasurements object named 'totalTime'.
The event itself looks like this:
loading-time: {
customMeasurements : {
totalTime: 123
}
}
I'm trying to create a graph of the average total time of all the events reported to azure per hour. So I need to be able to collect and average the values within the events.
I can't seem to figure out how to access the customMeasurements values from within the Azure AppInsights Analytics. Here is some of the code that Azure provided.
union customEvents
| where timestamp between(datetime("2019-11-10T16:00:00.000Z")..datetime("2019-11-11T16:00:00.000Z"))
| where name == "loading-time"
| summarize Ocurrences=count() by bin(timestamp, 1h)
| order by timestamp asc
| render barchart
This code simply counts the number of reported events within the last 24 hours and displays them per hour.
I have tried to access the customMeasurements object held in the event by doing
summarize Occurrences=avg(customMeasurements["totalTime"])
But Azure doesn't like that, so I'm doing it wrong. How can I access the values I require? I can't seem to find any documentation either.
It can be useful to project the data from the customDimensions / customMeasurements property collecton into a new variable that you'll use for further aggregation. You'll normally need to cast the dimensions data to the expected type, using one of the todecimal, toint, tostring functions.
For example, I have some extra measurements on dependency telemetry, so I can do something like so
dependencies
| project ["ResponseCompletionTime"] = todecimal(customMeasurements.ResponseToCompletion), timestamp
| summarize avg(ResponseCompletionTime) by bin(timestamp, 1h)
Your query might look something like,
customEvents
| where timestamp between(datetime("2019-11-10T16:00:00.000Z")..datetime("2019-11-11T16:00:00.000Z"))
| where name == "loading-time"
| project ["TotalTime"] = toint(customMeasurements.totalTime), timestamp
| summarize avg(TotalTime) by bin(timestamp, 1h)
| render barchart
We have a requirement to get status of windows service when it is started and stopped do that I have returned one query, but I am facing issue when joining 2 tables to get output.
I have tried using inner and left outer joins but still getting duplicates
Event
| where EventLog == "System" and EventID == 7036 and Source == "Service Control Manager"
| parse kind=relaxed EventData with * '<Data Name="param1">' Windows_Service_Name '</Data><Data Name="param2">' Windows_Service_State '</Data>' *
| where Windows_Service_State == "running" and Windows_Service_Name == "Microsoft Monitoring Agent Azure VM Extension Heartbeat Service"
| extend startedtime = TimeGenerated
| join (
Event
| where EventLog == "System" and EventID == 7036 and Source == "Service Control Manager"
| parse kind=relaxed EventData with * '<Data Name="param1">' Windows_Service_Name '</Data><Data Name="param2">' Windows_Service_State '</Data>' *
| where Windows_Service_State == "stopped" and Windows_Service_Name == "Microsoft Monitoring Agent Azure VM Extension Heartbeat Service"
| extend stoppedtime = TimeGenerated
) on Computer
| extend downtime = startedtime - stoppedtime
| project Computer, Windows_Service_Name,stoppedtime , startedtime ,downtime
| top 10 by Windows_Service_Name desc
we want to get no of times that service started and stopped if the service restarted multiple times in a day we are getting duplicate timings in starttime when joining please have a look on link (https://ibb.co/JzqxjC0)
I am not sure I fully understand what is going on, since I don't have access to the data. But. I can see you are using the default join flavor.
The default is inner unique:
The inner-join function is like the standard inner-join from the SQL world. An output record is produced whenever a record on the left side has the same join key as the record on the right side.
Which means a new line in the result is created on every match between the left and the right side. Therefore. let's assume you have a computer that was restarted twice, so it has 2 lines of stopped, and 2 lines of running. That will produce 4 rows in Kusto answer.
Looking at your picture, it makes sense to me because you have lines with negative downtime. I guess that is not possible.
What I would do, is look for an identifier that is unique on every Computer run. Then you can join on that, and stay safe not to generate data that you don't want.
I have a question about azure log analytics alerts, in that I don't quite understand how the time frame works within the context of setting up an alert based on an aggregated value.
I have the code below:
Event | where Source == "EventLog" and EventID == 6008 | project TimeGenerated, Computer | summarize AggregatedValue = count(TimeGenerated) by Computer, bin_at(TimeGenerated,24h, datetime(now()))
For time window : 24/03/2019, 09:46:29 - 25/03/2019, 09:46:29
In the above the alert configuration interface insights on adding the bin_at(TimeGenerated,24h, datetime(now())) so I add the function, passing the arguments for a 24h time period. If you are already adding this then what is the point of the time frame.
Basically the result I am looking for is capturing this event over a 24 hour period and alerting when the event count is over 2. I don't understand why a time window is also necessary on top of this because I just want to run the code every five minutes and alert if it detects more than two instances of this event.
Can anyone help with this?
AFAIK you may use the query something like shown below to accomplish your requirement of capturing the required event over a time period of 24 hour.
Event
| where Source == "EventLog" and EventID == 6008
| where TimeGenerated > ago(24h)
| summarize AggregatedValue= any(EventID) by Computer, bin(TimeGenerated, 1s)
The '1s' in this sample query is the time frame with which we are aggregating and getting the output from Log Analytics workspace repository. For more information, refer https://learn.microsoft.com/en-us/azure/kusto/query/summarizeoperator
And to create an alert, you may have to go to Azure portal -> YOURLOGANALYTICSWORKSPACE -> Monitoring tile -> Alerts -> Manager alert rules -> New alert rule -> Add condition -> Custom log search -> Paste any of the above queries under 'Search query' section -> Type '2' under 'Threshold value' parameter of 'Alert logic' section -> Click 'Done' -> Under 'Action Groups' section, select existing action group or create a new one as explained in the below mentioned article -> Update 'Alert Details' -> Click on 'Create alert rule'.
https://learn.microsoft.com/en-us/azure/azure-monitor/platform/action-groups
Hope this helps!! Cheers!! :)
To answer your question in the comments part, yes the alert insists on adding the bin function and that's the reason I have provided relevant query along with bin function by having '1s' and tried to explain about it in my previous answer.
If you put '1s' in bin function then you would fetch output from Log Analytics by aggregating value of any EventID in the timespan of 1second. So output would look something like shown below where aaaaaaa is considered as a VM name, x is considered as a particular time.
If you put '24h' instead of '1s' in bin function then you would fetch output from Log Analytics by aggregating value of any EventID in the timespan of 24hours. So output would look something like shown below where aaaaaaa is considered as a VM name, x is considered as a particular time.
So in this case, we should not be using '24h' in bin function along with 'any' aggregation because if we use it then we would see only one occurrence of output in 24hours of timespan and that doesn't help you to find out event occurrence count using the above provided query having 'any' for aggregation. Instead you may use 'count' aggregation instead of 'any' if you want to have '24h' in bin function. Then this query would look something like shown below.
Event
| where Source == "EventLog" and EventID == 6008
| where TimeGenerated > ago(24h)
| summarize AggregatedValue= count(EventID) by Computer, bin(TimeGenerated, 24h)
The output of this query would look something like shown below where aaaaaaa is considered as a VM name, x is considered as a particular time, y and z are considered as some numbers.
One other note is, all the above mentioned queries and outputs are in the context of setting up an alert based on an aggregated value i.e., setting up an alert when opting 'metric measurement' under alert logic based on section. In other words, aggregatedvalue column is expected in alert query when you opt 'metric measurement' under alert logic based on section. But when you say 'you get a count of the events' that means If i am not wrong, may be you are opting 'number of results' under alert logic based on section, which would not required any aggregation column in the query.
Hope this clarifies!! Cheers!!
First at all, I'm sorry if I'm re-posting, but I couldn't find anything related.
I'm working on a application that handle very sensitive data. I want to filter this data by user's role.
I've done this (in another job) using Doctrine Filters, but I can't find any information about how to do this using Sequelize (over PostgreSQL).
Eg:
sensitive_information:
| user_id | sensible_value |
+---------+----------------+
| 1 | something |
| 2 | something_else |
I need this:
SELECT *
FROM sensitive_information
WHERE user_id = 1; /** I need this to be added automatically in all
queries to sensitive_information */
So, user 1 never will see information of another user. That's the goal.
Is it possible? I'm open to suggestions.
Thanks in advance.
This can be addressed by using Sequelize scopes.
Since scopes can be functions:
Scopes are defined in the model definition and can be finder objects, or functions returning finder objects
it's easy to define a scope that filters information according to the user that is logged in.
I haven't implemented this exactly scope yet but I have a good idea on how to address it if anyone needs helps with it.