I am trying to trigger an alert when the columns in the AzureDiagnostic Table in Log Analytics is >400 since there is a 500 column limit to the table where records will start dropping.
The issue is Alerts expects and AggregatedValue and a TimeGenerated. Since this is a schema there is not a true Time Generated. I've tried a "time" metric and renaming the column to be "TimeGenerated" but get the following error:
Search Query should contain 'AggregatedValue' and 'bin(TimeGenerated,
[roundTo])' for Metric alert type
This is the alert query I have:
AzureDiagnostics
| getschema
| summarize AggregatedValue = count(ColumnName) by bin(1d, 5m)
|project AggregatedValue, TimeGenerated=Column1
And I get these results:
I changed my logic to return a record or not. It will return a record only if the threshold has been met of 400 columns and then set my alert Threshold value to > 0.
AzureDiagnostics
| getschema
| summarize count(ColumnName)
| where count_ColumnName >400
Alert:
I'm from the Azure Monitor Log Analytics team. We are actively working in Azure Log Analytics to avoid it all together. We are working now to have dedicated tables for most of Azure resource so it wouldn't overpopulate the AzureDiagnostics table. Some Azure resource like Azure Data Factory have options to control whether it would use the dedicated tables or AzureDiagnistcs. See #4 here: https://learn.microsoft.com/en-us/azure/data-factory/monitor-using-azure-monitor#monitor-data-factory-metrics-with-azure-monitor
Related
I created an Azure Alert using a Query (KQL - Kusto Query Language) reading from the Log. That is, it's an Log Alert.
After a few minutes, the alert was triggered (as I expected based on my condition).
My condition checks if there are Pods of a specific name in Failed state:
KubePodInventory
| where TimeGenerated between (now(-24h) .. now())
| where ClusterName == 'mycluster'
| where Namespace == 'mynamespace'
| where Name startswith "myjobname"
| where PodStatus == 'Failed'
| where ContainerStatusReason == 'Completed' //or 'Error' or doesn't matter? (there are duplicated entries, one with Completed and one with Error)
| order by TimeGenerated desc
These errors stay in the log, and I only want to catch (alert about them) once per day (that is, I check if there is at least one entry in the log (threshold), then fire the alert).
Is the log query evaluated every time there is a new entry in the log, or is it evaluated in a set frequency?I could not find in Azure Portal a frequency specified to check Alerts, so maybe it evaluates the Alert(s) condition(s) every time there is something new in the Log?
I can run the 2 queries below to view the logs for a certain time, separately.
AppServiceConsoleLogs | where TimeGenerated >= datetime('2021-04-10 14:00')
AppServiceHTTPLogs | where TimeGenerated >= datetime('2021-04-10 14:00')
How do I combine these into a single query to view the logs together?
The union operator does the job to show all records from the specified tables.
I used the query below and no the problems you mentioned:
union requests, traces
| where timestamp > ago(1d)
The screenshot of the query result:
If you still have the problem, please share us the screenshot and more detailed info.
There is a blob trigger function failing, which some are caused by a System.NullReferenceException. Is it possible to get the list of blob names that are causing this exception? What would that log query look like? Here is the query I have so far, but not sure where to go from here to get the list of blob names causing the Null Reference Exceptions.
requests
| where success == false
First, you should query from the dependencies table instead of requests table.
Then to get failed blob list, you can use the query like below:
dependencies
| where timestamp >ago(7d) // here the time range is set to the latest 7 days, you can change it.
| where success == "false"
| where type == "Azure blob"
| project customDimensions.Blob
I am reporting some custom events to Azure, within the custom event is a value being held under the customMeasurements object named 'totalTime'.
The event itself looks like this:
loading-time: {
customMeasurements : {
totalTime: 123
}
}
I'm trying to create a graph of the average total time of all the events reported to azure per hour. So I need to be able to collect and average the values within the events.
I can't seem to figure out how to access the customMeasurements values from within the Azure AppInsights Analytics. Here is some of the code that Azure provided.
union customEvents
| where timestamp between(datetime("2019-11-10T16:00:00.000Z")..datetime("2019-11-11T16:00:00.000Z"))
| where name == "loading-time"
| summarize Ocurrences=count() by bin(timestamp, 1h)
| order by timestamp asc
| render barchart
This code simply counts the number of reported events within the last 24 hours and displays them per hour.
I have tried to access the customMeasurements object held in the event by doing
summarize Occurrences=avg(customMeasurements["totalTime"])
But Azure doesn't like that, so I'm doing it wrong. How can I access the values I require? I can't seem to find any documentation either.
It can be useful to project the data from the customDimensions / customMeasurements property collecton into a new variable that you'll use for further aggregation. You'll normally need to cast the dimensions data to the expected type, using one of the todecimal, toint, tostring functions.
For example, I have some extra measurements on dependency telemetry, so I can do something like so
dependencies
| project ["ResponseCompletionTime"] = todecimal(customMeasurements.ResponseToCompletion), timestamp
| summarize avg(ResponseCompletionTime) by bin(timestamp, 1h)
Your query might look something like,
customEvents
| where timestamp between(datetime("2019-11-10T16:00:00.000Z")..datetime("2019-11-11T16:00:00.000Z"))
| where name == "loading-time"
| project ["TotalTime"] = toint(customMeasurements.totalTime), timestamp
| summarize avg(TotalTime) by bin(timestamp, 1h)
| render barchart
I have data in azure Insights saved in custom events formats.
Now I need to create a dashboard page in my website that will pull data from insights and will show graphs on that data.
Questions is that how I can filter data from the customEvents based on data saved there. like based on custom events or custom data.
Provide me any resource from where I can see that how $filer, $search,$query works?
I am here https://dev.applicationinsights.io/quickstart but not looks like enough.
I tried to add filter like
startswith(customEvent/name, 'BotMessageReceived')
in https://dev.applicationinsights.io/apiexplorer/events
but it not working. is says "Something went wrong while running the query",
I have customEvents which name start with BotMessageReceived
Thanks
Dalvir
update:
There is no like operator, if you wanna use timestamp as a filter, you should use one of the three methods below:
customEvents
| where timestamp >= datetime('2018-11-23T00:00:00.000') and timestamp <=
datetime('2018-11-23T23:59:00.000')
customEvents
| where tostring(timestamp) contains "2018-12-11"
customEvents
| where timestamp between(datetime('2018-11-23T00:00:00.000') ..
datetime('2018-11-23T23:59:00.000') )
Please use this:
customEvents
| where name startswith "BotMessageReceived"
And if you use the api you metioned above, you can use:
https://api.applicationinsights.io/v1/apps/Your_application_id/query?
query=customEvents | where name startswith "BotMessageReceived"
It works at my side.