azure app insights get a part of a string from url and display it in the name column - azure

I am using azure app insights and i want to parse a part of the string from url and show that part in a name column
requests
| where user_AuthenticatedId != ""
and url contains "reports" and user_AuthenticatedId == "xxx"
| project timestamp, user_AuthenticatedId, client_CountryOrRegion, client_OS, url,name
| order by timestamp asc nulls last
for example i am getting url as https://localhost:80/api/external-reports/blob/39/test 01b/false so i want to take the test 01b from this and show it in the name column.
i am not sure on how to do this.

There are some functions that might be helpful.
First of all, you can get the different url parts using parse_url() method. For example, given the url https://localhost:80/api/external-reports/blob/39/test 01b/false :
requests
| project parse_url(url)
output:
{"Scheme":"https","Host":"localhost","Port":"80","Path":"/api/external-reports/blob/39/test 01b/false","Username":"","Password":"","Query Parameters":{},"Fragment":""}
You can split the result even further using the split() method:
requests
| project split(parse_url(url).Path, "/")
output:
["","api","external-reports","blob","39","test 01b","false"]
To get the part you want you can use the index:
request
| project mycolumn = split(parse_url(test).Path, "/")[5]
output:
test 01b
When an index is used that is greater than the number of parts an empty result is returned. You can replace it with a value of your own using the coalesce function:
requests
| project mycolumn = coalesce(split(parse_url(test).Path, "/")[5], "unknown")
it shows unknown when the index is out of range or the part is empty.

Related

Kusto query language split # character and take last item

If I have a string for example:
"this.is.a.string.and.I.need.the.last.part"
I am trying to get the last part of the string after the last ".", which in this case is "part"
How to I achieve this?
One way I tried was to split the string on ".", I get a array back, but then I don't know how to retrieve the last item in the array.
| extend ToSplitstring = split("this.is.a.string.and.I.need.the.last.part", ".")
gives me:
["this", "is","a","string","and","I","need","the","last", "part"]
and a second try I have tried this:
| extend ToSubstring = substring(myString, lastindexof(myString, ".")+1)
but Kusto do not have a function of lastindexof.
Anyone with tips?
you can access the last member of the array using a negative index -1.
e.g. this:
print split("this.is.a.string.and.I.need.the.last.part", ".")[-1]
returns a single table, with a single column and a single record, with the value part
You can try the code below, and feel free to change it to meet your need:
let lastIndexof = (input:string, lookup: string) {
indexof(input, lookup, 0, -1, countof(input,lookup))
};
your_table_name
| extend ToSubstring = substring("this.is.a.string.and.I.need.the.last.part", lastIndexof("this.is.a.string.and.I.need.the.last.part", ".")+1)

How to extract specific sub directory names from URL

Given the following request URLs:
https://example.com/api/foos/123/bars/456
https://example.com/api/foos/123/bars/456
https://example.com/api/foos/123/bars/456/details
Common structure: https://example.com/api/foos/{foo-id}/bars/{bar-id}
I wish to get separate columns for the values of {foo-id} and {bar-id}
What I tried
requests
| where timestamp > ago(1d)
| extend parsed_url=parse_url(url)
| extend path = tostring(parsed_url["Path"])
| extend: foo = "value of foo-id"
| extend: bar = "value of bar-id"
This gives me /api/foos/{foo-id}/bars/{bar-id} as a new path column.
Can I solve this question without using regular expressions?
Related, but not the same question:
Application Insights: Analytics - how to extract string at specific position
Splitting on the '/' character will give you an array and then you can extract the elements you are looking for as long as the path stays consistent. Using parse_url() is optional- you could use substring() or just adjust the indexes you retrieve.
requests
| extend path = parse_url(url)
| extend elements = split(substring(path.Path, 1), "/") //gets rid of the leading slash
| extend foo=tostring(elements[2]), bar=tostring(elements[4])
| summarize count() by foo, bar

Using Match in a sqlite fts5 query but need more control over ranking?

I have a virtual table created using fts5:
import sqlite3
# create a db in memory
con = sqlite3.connect(':memory:')
con.execute('create virtual table operators using fts5(family, operator, label, summary, tokenize=porter)')
# some sample data
samples = {'insideTOP':
{'label':'Inside',
'family':'TOP',
'summary':'The Inside TOP places Input1 inside Input2.'
},
'inTOP':
{'label':'In',
'family':'TOP',
'summary':'The In TOP is used to create a TOP input.'
},
'fileinSOP':
{'label':'File In',
'family':'SOP',
'summary':'The File In SOP allows you to read a file'
}
}
# fill db with those values
for operator in samples.keys():
opDescr = samples[operator]
con.executescript("insert into operators (family, operator, label, summary) values ('{0}','{1}','{2}','{3}');".format(opDescr['family'],operator,opDescr['label'],opDescr['summary']))
with following columns
+--------+-----------+------------+----------------------------------------------+
| family | operator | label | summary |
+--------+-----------+------------+----------------------------------------------+
| TOP | insideTOP | Inside | The Inside TOP places Input1 inside Input2.|
| TOP | inTOP | In | The In TOP is used to create a TOP input. |
| SOP | fileinSOP | File In | The File In SOP allows you to read a file |
+--------+-----------+------------+----------------------------------------------+
an example query is:
# query the db
query = "select operator from operators where operators match 'operator:In*' or operators match 'label:In*' order by family, bm25(operators)"
result = con.execute(query)
for row in result:
print(row)
And as a result I get
fileinSOP
insideTOP
inTOP
For this particular case though, I'd actually like the 'inTOP' to appear before the 'insideTOP' as the label is a perfect match.
What would be a good technique to be able to massage these results the way I'd like them?
Thank you very much
Markus
maybe you can put your order rule in the question.
If you use bm25 to order your results, you can't achieve the result you want
I suggest you that you can use your custom rank function, like below sql:
query = "select operator from operators where operators match 'operator:In*' or operators match 'label:In*' order by myrank(family, operators)"
define a custom rank function is very easy in fts5, you can follow the guide in the fts5 website.
if you also want bm25 result as a rank score, you can get the score in the rank method can calculate your final score.

Generate hyperlink in Sharepoint

I have to generate an hyperlink in sharepoint based on sql table like this:
+----+----------------------------+
| ID | path |
+----+----------------------------+
| 1 | file://test/9932323.pdf |
+----+----------------------------+
| 2 | file://test/1653156423.pdf |
+----+----------------------------+
Actually there is this code to generate html link:
<asp:Label runat="server" id="ff1{$ID}" text="{$thisNode/#PATH}" />
I cannot modify SQL table (dinamically generated) but I have to substitute:
/test/ with /test.abc.local/
and
displayed text with filename only ("path" field substring after last '/')
How can I to that without creating new view or calculated fields?
I tried with:
<a href="{REPLACE($thisNode/#PATH),12,1,'.abc.local/')}"><asp:L ...
but with no success. (I'm really newbie in Sharepoint)
thanks
I'm not going to remove the previous answer because it still valid and it is pretty handy if somebody comes across the same issue, so the RegEx() expression you will have something like this:
/(file\:\/\/)/g
Using the expression above you can find the string you want, so the example below gives you all the LABELS on a page, and from there you can use the following:
.replace( new RegExp("/(file\:\/\/test)","gm"),"/test.abc.local")
Using the expression above you can find the string you want, so the example below gives you all the LABELS on a page, and from there you can use the following:
$('input[type=text]').each(
function(index)
{
console.log(' text: ' + $(this).val() + ' replace: ' + $(this).val().replace( new RegExp("(file\:\/\/test)","gm"),"/test.abc.local") );
}
);
If I understood your question correctly, I would strongly suggest you to use RegEx(), it is possibly the most handy thing ever created to handle string find/replacement.
You can put a JavaScript function to perform the RegEx() substitution on the page inside of a <script language="javascript"></script> , then on your <asp:label> you will replace the output for calling this function by passing the string #thisNode/#PATH to the JavaScript function you wrote, which will find and replace the substring for the desired output

HiveQL string function questions

I'm using HiveQL to run the below query.
The intention is that the case statement removes the last XX characters from the end of the domain, dependent on the suffix (.com, .co.uk).
This doesn't seem to work as there is no change to the strings in the 'domainnew' column in the output.
Can anyone advise how I would make this work?
I also then need to take the output of 'domainnew' and take only the characters to the right of the first '.' when reading from the right handside.
domain = mobile.domain.facebook.com
domainnew = mobile.domain.facebook
newcalc = facebook
Any advice on this would be brilliant!!
Thank you
select domain, catid, apnid, sum(optimisedsize) as bytes,
CASE domain
WHEN instr(domain, '.co.uk') THEN substr(domain,LENGTH(domain)-6)
WHEN instr(domain, '.com') THEN substr(domain,LENGTH(domain)-6)
ELSE domain
END as domainnew
from udsapp.web
where dt = 20170330 and hour = 04 and loc = 'FAR1' and catid <> "0:0" group by domain, catid, apnid sort by bytes desc;
with t as (select 'mobile.domain.facebook.com' as domain)
select regexp_extract(domain,'(.*?)(\\.com|\\.co\\.uk|)$',1) as domainnew
,regexp_extract(domain,'.*?([^.]+)(\\.com|\\.co\\.uk|)$',1) as new_calc
from t
;
+------------------------+----------+
| domainnew | new_calc |
+------------------------+----------+
| mobile.domain.facebook | facebook |
+------------------------+----------+

Resources