I have the following app insights query :
let someResult=
customEvents | where name in ('SomeAction')
| parse customDimensions.someId with someId
| parse customDimensions.sometaskId with someTaskId
| parse user_AuthenticatedId with user
| summarize max(timestamp) by user, someId , someTaskId
| join (
customEvents | where name in ('someAction')
| parse customDimensions.action with someAction
| parse customDimensions.someId with someId
| project someAction,someId
) on someId
| join (
customEvents
| where name in ('someResult')
| parse customDimensions.someId with someId
| parse customDimensions.someIdsWithSomething with sometaskIds
| parse array_length(split(customDimensions.someIdsWithSomething ,',')) with someTaskCount
| distinct someId , sometaskIds,someTaskCount
| where sometaskIds<> ''
) on someId
| summarize sumif(todouble(someTaskCount),someAction=="accept")/sum(todouble(someTaskCount));
How can i divide someResult by something here . For example i want the final result to be someResult/10 . Thank you for the help.
Try this:
let someResult=
customEvents | where name in ('SomeAction')
| parse customDimensions.someId with someId
| parse customDimensions.sometaskId with someTaskId
| parse user_AuthenticatedId with user
| summarize max(timestamp) by user, someId , someTaskId
| join (
customEvents | where name in ('someAction')
| parse customDimensions.action with someAction
| parse customDimensions.someId with someId
| project someAction,someId
) on someId
| join (
customEvents
| where name in ('someResult')
| parse customDimensions.someId with someId
| parse customDimensions.someIdsWithSomething with sometaskIds
| parse array_length(split(customDimensions.someIdsWithSomething ,',')) with someTaskCount
| distinct someId , sometaskIds,someTaskCount
| where sometaskIds<> ''
) on someId
| summarize summarized = sumif(todouble(someTaskCount),someAction=="accept")/sum(todouble(someTaskCount));
someResult
| project summarized / 10
I could not test it since I do not have those custom dimensions but it is based from this working/tested example:
let someResult = requests
| summarize summarized = count();
someResult
| project summarized / 10
I hope this helps someone, as this SO question shows up as the top result but doesn't answer the question. I determined something like this from this answer:
let authRequests = requests | where operation_Name contains "Auth";
let countNon500s = toscalar(authRequests | where resultCode !startswith "5" | count);
let countAll = toscalar(authRequests | count);
let percent = 100*todouble(todouble(countNon500s) / todouble(countAll));
print todecimal(percent)
The three important parts are:
you need to convert the values to scalar values (otherwise they are tables).
you need to convert the resulting percentage to doubles (otherwise they are 1 or 0)
you need to use 'print' to output to a tabular form (or extend a column)
Related
I have products and providers. Each product has an uuid and each provider has a list of uuid of products that they can provide.
How do I select all the products that a given (i.e. by provider uuid) provider can offer?
Products:
+------+------+------+
| uuid | date | name |
+------+------+------+
| 0 | - | - |
| 1 | - | - |
| 2 | - | - |
+------+------+------+
Providers:
+------+----------------+
| uuid | array_products |
+------+----------------+
| 0 | [...] |
| 1 | [...] |
| 2 | [...] |
+------+----------------+
select p.name, u.product_uuid
from products p
join
(
select unnest(array_products) as product_uuid
from providers where uuid = :target_provider_uuid
) u on p.uuid = u.product_uuid;
Please note however that your data design is not efficient and much harder to work with than a normalized one.
I tried to do group by in SparkSQL which works good but most of the rows went missing.
spark.sql(
"""
| SELECT
| website_session_id,
| MIN(website_pageview_id) as min_pv_id
|
| FROM website_pageviews
| GROUP BY website_session_id
| ORDER BY website_session_id
|
|
|""".stripMargin).show(10,truncate = false)
I am getting output like this :
+------------------+---------+
|website_session_id|min_pv_id|
+------------------+---------+
|1 |1 |
|10 |15 |
|100 |168 |
|1000 |1910 |
|10000 |20022 |
|100000 |227964 |
|100001 |227966 |
|100002 |227967 |
|100003 |227970 |
|100004 |227973 |
+------------------+---------+
Same query in MySQL gives the desired result like this :
What is the best way to do ,so that all rows are fetched in my Query.
Please note I already checked other answers related to this, like joining to get all rows etc, but I want to know if there is any other way by with we can get the result like we get in MySQL ?
It looks like it is ordered by alphabetically, in which case 10 comes before 2.
You might want to check that the columns type is a number, not string.
What datatypes do the columns have (printSchema())?
I think website_session_id is of string type. Cast it to an integer type and see what you get:
spark.sql(
"""
| SELECT
| CAST(website_session_id AS int) as website_session_id,
| MIN(website_pageview_id) as min_pv_id
|
| FROM website_pageviews
| GROUP BY website_session_id
| ORDER BY website_session_id
|
|
|""".stripMargin).show(10,truncate = false)
Here is what I want to do:
current table:
+----+-------------+
| id | data |
+----+-------------+
| 1 | max |
| 2 | linda |
| 3 | sam |
| 4 | henry |
+----+-------------+
I have a id_str=1,3,4
Mystery Query - something like:
UPDATE table SET data = 'jen' where id in (id_str)
resulting table:
+----+-------------+
| id | data |
+----+-------------+
| 1 | jen |
| 2 | lindaa |
| 3 | jen |
| 4 | jen |
+----+-------------+
Starting from a list of ids given as a CSV string, say :id_str, you can do:
update mytable
set data = 'jen'
where ',' || :id_str || ',' like ',%' || id || ',%'
An alternative is a regex functions:
where regexp_like(:id_str, '(^|,)' || id || '(,|$)')
Both solutions work, but are rather inefficient. A much better solution would be not to pass the serch parameters as a proper list of values rather than a CSV string.
I have data coming into ADX from a single device that looks like this:
+-----------------------------+----------+-----------+-------------+
| dateTime | latitude | longitude | temperature |
+-----------------------------+----------+-----------+-------------+
| 2020-07-17T20:55:00.824313Z | 47.5783 | -78.1692 | |
+-----------------------------+----------+-----------+-------------+
| 2020-07-17T20:55:00.824311Z | | | 60 |
+-----------------------------+----------+-----------+-------------+
| 2020-07-17T20:54:01.000258Z | 47.5653 | -78.2692 | |
+-----------------------------+----------+-----------+-------------+
| 2020-07-17T20:53:00.877956Z | | | 62 |
+-----------------------------+----------+-----------+-------------+
Every other update from this source either contains the lat/long or the device's temperature.
If I want to get a snapshot of the device's complete state in a given time window, how do I flatten this data? For instance, if I wanted the latest device state, I'd like to get back a single row that contains:
+-----------------------------+----------+-----------+-------------+
| dateTime | latitude | longitude | temperature |
+-----------------------------+----------+-----------+-------------+
| 2020-07-17T20:55:00.824313Z | 47.5783 | -78.1692 | 60 |
+-----------------------------+----------+-----------+-------------+
one option would be to "split" the table into 2 (one with the lat-lon, and one with the temperature), then join both parts:
let T = datatable(id:string, dateTime:datetime, latitude:double, longitude:double, temperature:double)
[
"a", datetime(2020-07-17T20:55:00.824313Z), 47.5783, double(-78.1692), double(null),
"a", datetime(2020-07-17T20:55:00.824311Z), double(null), double(null), 60,
"a", datetime(2020-07-17T20:54:01.000258Z), 47.5653, double(-78.2692), double(null),
"a", datetime(2020-07-17T20:53:00.877956Z), double(null), double(null), 62
];
T
| where isnotnull(temperature)
| summarize arg_max(dateTime, temperature) by id
| join (
T
| where isnotnull(latitude)
| summarize arg_max(dateTime, latitude, longitude) by id
) on id
| project id, dateTime, latitude, longitude, temperature
I'd like to do,
Extracting "query" strings where param=1 as follows in "2."
Getting pageViews in Analytics with table as "3."
1. Actual urls included in pageView
https://example.com/dir01/?query=apple¶m=1
https://example.com/dir01/?query=apple¶m=1
https://example.com/dir01/?query=lemon+juice¶m=1
https://example.com/dir01/?query=lemon+juice¶m=0
https://example.com/dir01/?query=tasteful+grape+wine¶m=1
2. Value expected to extract
apple
lemon+juice
tasteful+grape+wine
3. Expected output in AI Analytics
Query Parameters | Count
apple | 2
lemon+juice | 1
tasteful+grape+wine | 1
Tried to do
https://learn.microsoft.com/en-us/azure/application-insights/app-insights-analytics-reference#parseurl
https://aka.ms/AIAnalyticsDemo
I think extract or parseurl(url) should be useful. I tried the latter parseurl(url) but don't know how to extract "Query Parameters" as one column.
pageViews
| where timestamp > ago(1d)
| extend parsed_url=parseurl(url)
| summarize count() by tostring(parsed_url)
| render barchart
url
http://aiconnect2.cloudapp.net/FabrikamProd/
parsed_url
{"Scheme":"http","Host":"aiconnect2.cloudapp.net","Port":"","Path":"/FabrikamProd/","Username":"","Password":"","Query Parameters":{},"Fragment":""}
Yes, parseurl is the way to do it. It results in a dynamic value which you can use as a json.
To get the "query" value of the query parameters:
pageViews
| where timestamp > ago(1d)
| extend parsed_url=parseurl(url)
| extend query = tostring(parsed_url["Query Parameters"]["query"])
and to summarize by the param value:
pageViews
| where timestamp > ago(1d)
| extend parsed_url=parseurl(url)
| extend query = tostring(parsed_url["Query Parameters"]["query"])
| extend param = toint(parsed["Query Parameters"]["param"])
| summarize sum(param) by query
You can see how it works on your example values in the demo portal:
let vals = datatable(url:string)["https://example.com/dir01/?
query=apple¶m=1", "https://example.com/dir01/?query=apple¶m=1",
"https://example.com/dir01/?query=lemon+juice¶m=1",
"https://example.com/dir01/?query=lemon+juice¶m=0",
"https://example.com/dir01/?query=tasteful+grape+wine¶m=1"];
vals
| extend parsed = parseurl(url)
| extend query = tostring(parsed["Query Parameters"]["query"])
| extend param = toint(parsed["Query Parameters"]["param"])
| summarize sum(param) by query
Hope this helps,
Asaf