For a Query in Microsoft Defender Advanced Hunting I want to use Data from an external Table in Sharepoint but I think I need authentication because I get an error message when i try this query:
let IOC = externaldata(column:string)
[
h#"URL"
]
with(format="txt")
| parse column with * '"' ip '"' *
| project ip;
DeviceNetworkEvents
| where Timestamp > ago(30d)
| where RemoteIP in (IOC);
How can i send Authentication-data (Username+Password?) with then query?
EDIT: This is the Errormessage:
An unexpected error occurred during query execution. Please try again in a few minutes.
Related
I am trying to debug an issue with an Azure Alert not firing. This alert should run every 30 minutes and find any devices that have not emitted a heartbeat in the last 30 minutes up to the hour. In addition, an alert should only be fired once for each device until it becomes healthy again.
The kusto query is:
let missedHeartbeatsFrom30MinsAgo = traces
| where message == “Heartbeat”
| summarize arg_max(timestamp, *) by tostring(customDimensions.id)
| project Id = customDimensions_id, LastHeartbeat = timestamp
| where LastHeartbeat < ago(30m);
let missedHeartbeatsFrom1HourAgo = traces
| where message == "Heartbeat"
| summarize arg_max(timestamp, *) by tostring(customDimensions.id)
| project Id = customDimensions_id, LastHeartbeat = timestamp
| where LastHeartbeat <= ago(1h);
let unhealthyIds = missedHeartbeatsFrom30MinsAgo
| join kind=leftanti missedHeartbeatsFrom1HourAgo on Id;
let deviceDetails = customEvents
| where name == "Heartbeat"
| distinct tostring(customDimensions.deviceId), tostring(customDimensions.fullName)
| project Id = customDimensions_deviceId, FullName = customDimensions_fullName;
unhealthyIds |
join kind=leftouter deviceDetails on Id
| project Id, FullName, LastHeartbeat
| order by FullName asc
The rules for this alert are:
When I pull the plug on a device, wait ~30 minutes, and run the query manually in App Insights, I see the device in the results data set. However, no alert gets generated (nothing shows up in the Alerts history page and no one in the Action Group gets notified). Any help in this matter would be greatly appreciated!
I can see your KQL Query take several times to execute, and it consume more resource usage to run the query.
Optimize your query to avoid more resource utilization and quick response of your query result.
Make sure your alert processing rule Status should be Enabled like below
Once it is done make sure your query result should be Greater than or equal to 1. So that the alert processing rule will check the threshold if it matches the condition the alert will fire.
Still, you get the issue alert not firing try to delete the alert and run your query in a Query Editor and try to create a New alert rule.
I am using azure log analytics workspaces and are trying to write a simple query to get the exception message when a azure function fails.
This is the query I am using
union AppTraces
| union AppExceptions
| union AppRequests
| where AppRoleName has "-NEU"
| where TimeGenerated > ago(1d)
//| where Success == "false"
| order by TimeGenerated asc
| project
Success,
TimeGenerated,
AppRoleName,
message = iff(Message != '', Message, iff(InnermostMessage != '', InnermostMessage, Properties.['prop__{OriginalFormat}'])),
logLevel = Properties.['LogLevel']
| where logLevel != "Information"
The problem is that Success property is always empty and I expect it to be either true or false, I am using the Success property in other queries and it works just fine, for example as follows:
AppRequests
| project TimeGenerated, OperationName, Success, ResultCode, DurationMs, AppRoleName
| where AppRoleName has "NEU"
| where OperationName != "MinimumAppVersionHead" and OperationName != "QueueManagerHead"
| where Success != "true"
| order by TimeGenerated desc
| take 20
In the above case the Success where clause works as expected
Why is it not working in the first query?
Please check the below workaround it may help , we have tried with a simple query to check whether the success property is working or not. It works successfully using below query in logs with true and false.
As its works with second query it means you have added log analytics workspace for your function app successfully.
It seems there is an issue with "" , //| where Success == "false" instead of that try to remove the comment(//) and remove "" and use as below sample format in your query.
requests
| where success == false
| summarize failedCount=sum(itemCount), impactedUsers=dcount(user_Id) by operation_Name
| order by failedCount desc
We have tried with different output with success property as yours in the first query and getting no results . And by removing "" it works at our end.
Here are the below sample screenshots of output:-
OUTPUT OF THE GIVEN QUERY FOR FUNCTION APP FAILURE:
For more information please refer the below links:-
MS DOC| View and query your Function app logs
BLOG| Alerts on Azure Function failures
I have this query that works in Azure logs when i set the scope to the specific application insights I want to use
let usg_events = dynamic(["*"]);
let mainTable = union pageViews, customEvents, requests
| where timestamp > ago(1d)
| where isempty(operation_SyntheticSource)
| extend name =replace("\n", "", name)
| where '*' in (usg_events) or name in (usg_events)
;
let queryTable = mainTable;
let cohortedTable = queryTable
| extend dimension =tostring(client_CountryOrRegion)
| extend dimension = iif(isempty(dimension), "<undefined>", dimension)
| summarize hll = hll(user_Id) by tostring(dimension)
| extend Users = dcount_hll(hll)
| order by Users desc
| serialize rank = row_number()
| extend dimension = iff(rank > 5, 'Other', dimension)
| summarize merged = hll_merge(hll) by tostring(dimension)
| project ["Country or region"] = dimension, Counts = dcount_hll(merged);
cohortedTable
but trying to use the same in grafana just gives an error.
"'union' operator: Failed to resolve table expression named 'pageViews'"
Which is the same i get in azure logs if i dont set the scope to the specific application insights resource. So my question is. how do i make it so grafana targets this specific scope inside the logs? The query jsut gets the countries of the users that log in
As far as I know, Currently, there is no option/feature to add Scope in Grafana.
The Scope is available only in the Azure Log Analytics Workspace.
If you want the Feature/Resolution, please raise a ticket in Grafana Community where all the issues are officially addressed.
I'm trying to get my head around writing queries in Azure Application Insights which is capturing interactions with a bot built using Azure Bot Framework.
I have a table with headings such as timestamp, name, customDimensions, customDimensions and within customDimensions are objects such as
{
"conversationData": "{}",
"privateConversationData": "{\"nameForm\":{\"NAME\":\"foo\",\"ICCID\":\"12121212121212121212\"}}",
"userData": "{}",
"conversationId": "878fhiee1k33j5ci",
"userId": "default-user",
"metrics": "92.25833"
}
I can write queries easily to select items by name for example
customEvents
| where name contains "Activity"
but how do I select based on keys within objects such as those within privateConversationData above?
For example "privateConversationData": "{\"nameForm\":{\"NAME\":\"foo\",\"ICCID\":\"12121212121212121212\"}}", refers to one dialog called nameForm, how would I write a query to show the number of times the nameForm was used? Or a query that included the other kinds of dialog (e.g. not just nameForm, but fooForm, barForm) and a count of the times they were used?
Many thanks for any help!
The 'customDimensions' property is a dynamic type and therefore can be treated as a JSON document.
For example - to get the number of times nameForm was used in the last day:
customEvents
| extend conversationData = customDimensions["privateConversationData"]
| where timestamp > ago(1d) and isnotempty(conversationData) and conversationData contains "{\\\"nameForm\\\""
| count
Getting the different dialogs count will be trickier, but possible by parsing the customDimensions JSON document using the parse operator:
customEvents
| where timestamp > ago(1d)
| parse customDimensions with * "privateConversationData\": \"{\\\"" dialogKind "\\\":{\\\"NAME\\\"" *
| where isnotempty(dialogKind) and isnotnull(dialogKind)
| summarize count() by dialogKind
You can read the Analytics Reference to learn more about the language.
I imported an unknown number of events into my PIO eventserver and now I want to know that number (in order to measure and compare recommendation engines). I could not find an API for that, so I had a look at the MySQL database my server uses. I found two tables:
mysql> select count(*) from pio_event_1;
+----------+
| count(*) |
+----------+
| 6371759 |
+----------+
1 row in set (8.39 sec)
mysql> select count(*) from pio_event_2;
+----------+
| count(*) |
+----------+
| 2018200 |
+----------+
1 row in set (9.79 sec)
Both tables look very similar, so I am still unsure.
Which table is relevant? What is the difference between pio_event_1 and pio_event_2?
Is there a command or REST API where I can look up the number of stored events?
You could go through the spark shell, described in the troubleshooting docs
Launch the shell with
pio-shell --with-spark
Then find all events for your app and count them
import io.prediction.data.store.PEventStore
PEventStore.find(appName="MyApp1")(sc).count
You could also filter to find different subsets of events by passing more parameters to find. See the api docs for more details. The LEventStore is also an option
Connect to your database
\c db_name
List tables
\dt;
Run query
select count(*) from pio_event_1;
PHP
<?php
$dbconn = pg_connect("host=localhost port=5432 dbname=db_name user=postgres");
$result = pg_query($dbconn, "select count(*) from pio_event_1");
if (!$result) {
echo "An error occurred.\n";
exit;
}
// Not the best way, but output the total number of events.
while ($row = pg_fetch_row($result)) {
echo '<P><center>'.number_format($row[0]) .' Events</center></P>';
} ?>