I'm querying azure log analytics using Kusto, and extracting fields with the parse operator, then keeping only the records which parsed correctly:
traces
| parse message with "Search found " people " people in " groupCount " groups"
| where people != "" and groupCount != ""
| order by n desc
Is there a more terse way of parsing and dropping non-matching rows? If I am parsing out a lot of columns from a set of logs, maybe containing partial matches, this connascence between the parse and where gets fiddly.
By comparison, in SumoLogic, the parse operator automatically drops all rows which don't match a parsed pattern, which makes for really tidy pipelines:
*
| parse "Search found * people in * groups" as people, groupCount
| order by n desc
In Kusto: 'parse' operator does not auto-filter rows that does not match the provided pattern, and operator works as in mode of 'extend' - adding more columns.
If you would like to filter specific row - the recommendation is to use 'where' operator before the 'parse': this will also improve performance as 'parse' will have fewer rows to scan.
traces
| where message startswith 'Search found'
| parse message with "Search found " people " people in " groupCount " groups"
...
There's now a built in operator that will do this: parse-where
https://learn.microsoft.com/en-us/azure/kusto/query/parsewhereoperator
It has syntax just like parse, but will omit from its output any records which didn't match the parse pattern.
So the query:
traces
| parse message with "Search found " people " people in " groupCount " groups"
| where people != "" and groupCount != ""
| order by n desc
becomes:
traces
| parse-where message with "Search found " people " people in " groupCount " groups"
| order by n desc
Related
I want to extend the query result with specific values, but I do not know how to get only a fragment of information, the one that is in the screen, that is, for example, from the "rendereddescription" section, I only need information about "server_principal_name" and assign it to some value, e.g. "user" and this I know this needs to be resolved | extend "variable name" = i here i do not know what the syntax is.enter image description here
you can use the parse operator: https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/parseoperator
for example:
print RenderDescription = #"... 0000000000 session_server_principal_name:ABB\HPAM-TCS-DB10 server_principal_sid:01050000000 ...."
| parse RenderDescription with * "session_server_principal_name:" session_server_principal_name " " *
RenderDescription
session_server_principal_name
... 0000000000 session_server_principal_name:ABB\HPAM-TCS-DB10 server_principal_sid:01050000000 ....
ABB\HPAM-TCS-DB10
I am writing a macro send multiple emails that are customized based on the data required.
For example:
Subject: <name>, Thank you for visiting <store>
At the moment, I can only pull , but I have no idea how to also add so 'Content' will both of them.
Content = Replace(msg.Subject, "<name>", first_name + " " + last_Name)
Replace(msg.Subject, "<store>", store_name)
^ essentially I want to combine both of these so the macro will pull both.
You can do it in multiple steps:
Content = Replace(msg.Subject, "<name>", first_name + " " + last_Name)
Content = Replace(Content, "<store>", store_name)
Or you could nest your Replace statements:
Content = Replace(Replace(msg.Subject, "<name>", first_name + " " + last_Name), "<store>", store_name)
If you ended up with a large number of tokens to replace (or an ever growing dynamic number of them) then you could get fancy and iterate through the string to identify tokens and swap them out inside the loop, but that would be overkill here.
Given the following request URLs:
https://example.com/api/foos/123/bars/456
https://example.com/api/foos/123/bars/456
https://example.com/api/foos/123/bars/456/details
Common structure: https://example.com/api/foos/{foo-id}/bars/{bar-id}
I wish to get separate columns for the values of {foo-id} and {bar-id}
What I tried
requests
| where timestamp > ago(1d)
| extend parsed_url=parse_url(url)
| extend path = tostring(parsed_url["Path"])
| extend: foo = "value of foo-id"
| extend: bar = "value of bar-id"
This gives me /api/foos/{foo-id}/bars/{bar-id} as a new path column.
Can I solve this question without using regular expressions?
Related, but not the same question:
Application Insights: Analytics - how to extract string at specific position
Splitting on the '/' character will give you an array and then you can extract the elements you are looking for as long as the path stays consistent. Using parse_url() is optional- you could use substring() or just adjust the indexes you retrieve.
requests
| extend path = parse_url(url)
| extend elements = split(substring(path.Path, 1), "/") //gets rid of the leading slash
| extend foo=tostring(elements[2]), bar=tostring(elements[4])
| summarize count() by foo, bar
I have a string variable response:
where where where is it
I'm going there
where where did you say
sometimes it is where you think
i think its where where you go
its everywhere where you are
i am planning on going where where where i want to
As you can see, the word "where" is repeated quite often. I want to replace strings "where where" and "where where where" (or even "where where where where") with "where".
However, I don't want to replace "everywhere where" with "where".
I know I can do this manually, but I was hoping to condense the code into as few lines as possible.
This is what I have been trying so far:
gen temp = regexr(response, " (where)+ where ", " where ")
replace temp = regexr(response, "^(where)+ where ", "where ")
These are my results after running the code above:
where where is it
I'm going there
where did you say
sometimes it is where you think
i think its where where you go
its everywhere where you are
i am planning on going where where where i want to
Instead, I want the final data to look like this:
where is it
I'm going there
where did you say
sometimes it is where you think
i think its where you go
its everywhere where you are
i am planning on going where i want to
I have been using "(where)+" to capture both "where where" and "where where where" but it doesn't seem to work. I also split the code into two commands, one begins with "^(where)" and the other with " (where)" in order to avoid capturing the 'where' in "everywhere" but it seems as if the code does not capture "where where" when it occurs in the middle of the sentence.
A quick fix using Stata's string functions is the following:
clear
input str50 string1
"where where where is it"
"I'm going there"
"where where did you say"
"sometimes it is where you think"
"i think its where where you go"
"its everywhere where you are"
"i am planning on going where where where i want to"
end
generate tag1 = !strmatch(string1, "*everywhere where*")
generate tag2 = ( length(string1) - length(subinstr(string1, "where", "", .)) ) / 5
generate string2 = cond(tag1 == 1, stritrim(subinstr(string1, "where", "", tag2-1)), string1)
list string2, separator(0)
+----------------------------------------+
| string2 |
|----------------------------------------|
1. | where is it |
2. | I'm going there |
3. | where did you say |
4. | sometimes it is where you think |
5. | i think its where you go |
6. | its everywhere where you are |
7. | i am planning on going where i want to |
+----------------------------------------+
I want a single search query for below splunk query.
First search will give me a dynamic field myorderid
index=mylog "trigger.rule: Id - * : Unexpected System Error" | rex field=_raw "Id -""(?[^:]*)" | table myorderid
I want to pass the above myorderid in below search criteria
index=mylog API=Order orderid=myorderid
Can anyone please help me to create a single query using subsearch in splunk.
Have you tried the obvious?
index=mylog API=Order orderid=
[ search index=mylog "trigger.rule: Id - * : Unexpected System Error"
| rex "Id - (?<myorderid>[^:]*)" | fields myorderid ]