I need to get the category name from category id using kusto query.
First i have got the most searched url from the website using the below kusto query and ran it in app insights logs
Requests
| where resultCode==200
| where url contains "bCatID"
| summarize count=sum(itemCount) by url
| sort by count
| take 1
From the above query i got the result like
https://www.test.com/item.aspx?idItem=123456789&bCatID=1282
So for corresponding categoryid=1282 i need to get the category name using kusto
you can use the parse operator.
for example:
print input = 'https://www.test.com/item.aspx?idItem=123456789&bCatID=1282'
| parse input with 'https://www.test.com/item.aspx?idItem=123456789&bCatID='category_id:long
parse_urlquery
print url ="https://www.test.com/item.aspx?idItem=123456789&bCatID=1282"
| extend tolong(parse_urlquery(url)["Query Parameters"]["bCatID"])
url
Query Parameters_bCatID
https://www.test.com/item.aspx?idItem=123456789&bCatID=1282
1282
Fiddle
Feedback to the OP query
Not an answer
KQL is case sensitive. The name of the table in Azure Application Insights is requests (and not Requests).
resultCode is of type string (and not integer/long) and should be compared to "200" (and not 200)
bCatID is a token and therefore can be searched using has or even has_cs, which should be preferred over contains due to performance reasons.
URLs can be used with different parameters. It might make more sense to summarize only by the Host & Path parts + the bCatID query parameter.
count is a reserved word. It can be used as alias only if qualified: ["count"] or ['count'] (or better not used at all).
sort followed by take 1 can be replaced with the more elegant top 1 by ...
requests
| where resultCode == "200"
| project url = parse_url(url), itemCount
| summarize sum(itemCount) by tostring(url.Host), tostring(url.Path), tolong(url["Query Parameters"]["bCatID"])
| top 1 by sum_itemCount
Related
I'm working on a project to create a Stealthwatch (Cisco) Log Analytics query from a canned workbook query available from the Azure Sentinel portal (Data Connectors > Cisco Stealthwatch). I've added the function to my LA query section and it will run (depending on the time parameters - more on that later).
When I run the query within the Log Query window, it runs great. However, when trying to use the Analytics Rule Wizard, the same query errors out with the following error:
'extend' operator: Failed to resolve scalar expression named 'start'.
Included here is the simple query I'm running:
StealthwatchEvent
| where EventSeverity == "Critical"
And here is the complete function as supplied by the Data Connector from the Azure Sentinel workbook for Cisco Stealthwatch:
Syslog
| where SyslogMessage has 'Stealthwatch'
| extend EventVendor = 'Cisco'
| extend EventProduct = 'Stealthwatch'
| extend EventProductVersion = extract(#'Stealthwatch\|([0-9\.]+)\|', 1, SyslogMessage)
| mv-apply ExtractedFields = extract_all(#'\|(?P<key>[a-zA-Z0-9-_]+)=(?P<value>[a-zA-Z0-9-_:/#.,#{}>< ]+)', dynamic(["key","value"]), SyslogMessage) on (
project packed = pack(tostring(ExtractedFields[0]), tostring(ExtractedFields[1]))
| summarize bag = make_bag(packed)
)
| evaluate bag_unpack(bag)
| extend EventStartTime=todatetime(start)
| extend EventEndTime=todatetime(end)
| project-rename EventOriginalUid=alarmID
, EventSeverity=alarmSev
, EventStatus=alarmStatus
, EventType=cat
, SrcDvcDomain=domain
, DstIpAddr=dst
, DstPortNumber=dstPort
, EventMessage=msg
, Protocol=proto
, SrcGeoCountry=sourceHG
, SrcHostSnapshot=sourceHostSnapshot
, SrcIpAddr=src
, DvcIpAddr=flowCollectorIP
, DvcHostname=flowCollectorName
, SrcUserName=sourceUser
, DstUserName=targetUser
, DstGeoCountry=argetHG
, DstDvcHostname=targetHostname
, DstHostSnapshot=targetHostSnapshot
| project-away start
, end
, SyslogMessage
I've narrowed it down to the "start" value in the one extend statement. Note when I adjust the time on the Rule wizard for the query period (I run the query every 15 minutes and lookup data from the last 15 minutes) it should pass that to the query time parameters - but I'm just not sure why it's failing. Sometimes when I adjust the time, it fails on the "end" value instead as reported by the error. I have also tried removing the two extend lines related to time altogether, but then the query dies on AlarmID in the next section.
(And yes, I am fairly new to this - I'm learning Kusto/Sentinel and Stealthwatch tuning and reporting on the fly here)
:)
Any help or pointers would be greatly appreciated.
I want to be able to toggle a filter on my query via a parameter in dashboard. How can I turn the "where" operator off?
E.g. the parameter in the dashboard is "_toggle"
let _filter = dynamic(["A", "B"]);
Table
| where id in (_filter) // execute this line only if _toggle == true
| project id
I already tried creating a second list containing all the ids and toggle between the small an the complete list via iff() but this is too resource intensive.
you could try something like this:
let _filter = dynamic(["A", "B"]);
Table
| where _toggle == false or id in (_filter)
| project id
Json text isn't parsing in KQL correctly. I tried using parse_json as well but that didn't work either. I did confirm the extend AllProperties is holding the correct data.
DeviceInfo
| where RegistryDeviceTag == "Standard"
| extend AllProperties = todynamic(LoggedOnUsers)
| project DeviceName, Users = AllProperties["Username"]
Output gives me the correct DeviceName but doesn't give any data in the Username field.
(based on the sample input you provided in the comment)
if the array that is "LoggedOnUsers" includes exactly one entry, you can do this:
print input = '[{"UserName":"TheUserName","DomainName":"TheDomainName","Sid":"TheSID#"}]'
| project UserName = parse_json(input)[0].UserName
otherwise, you can use mv-expand or mv-apply:
print input = '[{"UserName":"TheUserName","DomainName":"TheDomainName","Sid":"TheSID#"}]'
| project parse_json(input)
| mv-apply input on (
project UserName = input.UserName
)
I'm trying to whitelist a bunch of domains from Azure sentinel rules based on the !hassuffix string operator.
Im trying to do something like this:
AzureDiagnostics
| where destinationDomain !hassuffix ".google.com" and destinationDomain !hassuffix ".azure.com"
But because there is going to be a lot of whitelisted domains and subdomains would like to store the root domain/subdomains in a list which will be in blob storage like:
let whitelist = dyanmic([".google.com", ".azure.com" .........])
Does anyone know the syntax to iterate through each of these and check whether the destinationDomain !hassuffix to each of the dynamic array elements? Or is the only way to have a wall of and's? Thanks
There's no such functionality. You should use matches regex instead.
This is perhaps not the most efficient way to do it, but you could do a cross product type join across the whitelist, perform the !hassuffix check on each, then see the count of how many passed (failed I guess?) the check. For a smallish whitelist and table it should do OK and would be easier to modify / maintain.
let AzureDiagnostics = datatable(destinationDomain: string)
[
"test.google.com",
"test.notallowed.com",
"other.azure.com",
"alsonotallowed.azure2.com"
];
let Whitelist = datatable(allowedSuffix: string, dummy: long)
[
".google.com", 1,
".azure.com", 1,
];
AzureDiagnostics
| extend dummy=1 // add a dummy column for cross product join
| lookup Whitelist on dummy // do cross product (lookup used assuming Whitelist is small)
| where destinationDomain !hassuffix(allowedSuffix) // perform the suffix check
| summarize count() by destinationDomain // since the list was broken up, get the count of passes
| where count_ == toscalar(Whitelist | count) // if the !hassuffix was true for all (the count) keep the result
| project destinationDomain // get rid of the count column
I am trying to write an App Insights query that will report back the timespan between two known events, specifically circuit breaker open and close events. The assumption is that these events always occur in pairs, so we need to know the time between the two for every occurrence in a time period.
My first attempt was to use an inline function. Simplified version below.
let timeOpened = (timeClosed:datetime)
{
let result = customEvents
| where name == 'CircuitBreakerStatusChange'
| where customDimensions['State'] == 'Open'
| where timestamp < timeClosed
| order by timestamp desc
| take 1
| project timestamp;
let scalar = toscalar(result);
scalar
};
customEvents
| where timestamp > ago(4h)
| where name == 'CircuitBreakerStatusChange'
| where customDimensions['State'] == 'Closed'
| extend timeOpen = timestamp - timeOpened(timestamp)
There may be a better way to do this. If so your ideas are welcome! But in this particular attempt the only feedback I get from Azure when running this is "Syntax error". However, I don't believe there's a syntax error here because if I just change the return value of the function from scalar to now() it runs successfully. Also I can run the body of the function in isolation successfully. Any idea what's wrong here?
I think you are getting syntax error because query language does not allow possibly recursive constructs. Now() worked because it was statically (not dynamically) retrieved at the query time.
I think you may achieve the desired outcome with serialize and prev() operators:
Table | order by timestamp asc | serialize
| extend previousTime = prev(timestamp,1)
| extend Diff = iff(customDimensions['State'] == 'Closed', timestamp - previousTime, 0)
| where Diff > 0
Note: I haven't tested the example above and it may need some additional thought to make it work (e.g. making sure that the previous record is actually "Opened" before doing previousTime calculation).