DocumentDB multiple filter query on array - azure

Using the DocumentDB query playground, I am working on a filter type of query. I have a set of attributes in my data that are set up to allow the user to search by the specific attribute. Each attribute type becomes and OR statement if multiple items are selected from the name in the name/value collection. If attributes are selected that differ (i.e. color and size) this becomes an AND statement.
SELECT food.id,
food.description,
food.tags,
food.foodGroup
FROM food
JOIN tag1 IN food.tags
JOIN tag2 IN food.tags
WHERE (tag1.name = "snacks" OR tag1.name = "granola bars")
AND (tag2.name = "microwave")
This query works beautifully in the playground.
The main issue is that I have up to 12 attributes, and maybe more. Once I hit 5 joins, that is my maximum allowed number of joins, so the query below doesn't work. (note that this isn't playground data, but a sample of my own)
SELECT s.StyleID FROM StyleSearch s
JOIN a0 in s.Attributes
JOIN a1 in s.Attributes
JOIN a2 in s.Attributes
JOIN a3 in s.Attributes
JOIN a4 in s.Attributes
JOIN a5 in s.Attributes
WHERE (a0 = "color-finish|Grey" OR a0 = "color-finish|Brown" OR a0 = "color-finish|Beige")
AND (a1 = "fabric-type|Polyester" OR a1 = "fabric-type|Faux Leather")
AND (a2 = "design-features|Standard" OR a2 = "design-features|Reclining")
AND (a3 = "style_parent|Contemporary" OR a3 = "style_parent|Modern" OR a3 = "style_parent|Transitional")
AND (a4 = "price_buckets|$1500 - $2000" OR a4 = "price_buckets|$2000 and Up")
AND (a5 = "dimension_width|84 in +")
I am not 100% sure I am using the proper query to perform this, but a simple where clause per below which works in SQL brings back anything matching in the or statements so I end up with items from each "AND statement.
SELECT s.StyleID FROM StyleSearch s
JOIN a in s.Attributes
WHERE (a = "color-finish|Grey" OR a = "color-finish|Brown" OR a = "color-finish|Beige")
AND (a = "fabric-type|Polyester" OR a = "fabric-type|Faux Leather")
AND (a = "design-features|Standard" OR a = "design-features|Reclining")
AND (a = "style_parent|Contemporary" OR a = "style_parent|Modern" OR a = "style_parent|Transitional")
AND (a = "price_buckets|$1500 - $2000" OR a = "price_buckets|$2000 and Up")
AND (a = "dimension_width|84 in +")
Here is an example of the data:
{
"StyleID": "chf_12345-bmc",
"Attributes": [
"brand|chf",
"color|red",
"color|yellow",
"dimension_depth|30 in +",
"dimension_height|counter height",
"materials_parent|wood",
"price_buckets|$500 - $1000",
"style_parent|rustic",
"dimension_width|55 in +"
]
}
I am looking for the proper way to handle this. Thanks in advance.

Is it possible for you to change the structure of your document to add filter attributes specifically for your query on e.g.
{
"StyleID": "chf_12345-bmc",
"Attributes": [
"brand|chf",
"color|red",
"color|yellow",
"dimension_depth|30 in +",
"dimension_height|counter height",
"materials_parent|wood",
"price_buckets|$500 - $1000",
"style_parent|rustic",
"dimension_width|55 in +"
],
"filter_color": "red,yellow",
"filter_fabric_type":"Polyester,leather"
}
This would eliminate the join restriction because now your query looks something like this:
SELECT s.StyleID FROM StyleSearch s
WHERE (CONTAINS(s.filter_color, "Grey") OR CONTAINS(s.filter_color, "Red"))
AND (CONTAINS(s.filter_fabric_type, "Polyester") OR CONTAINS(s.filter_fabric_type, "Leather"))
Of course this does mean that you have additional fields to maintain.
You might also consider writing a stored proc for this and using javascript to loop through your collection and filtering that way: DocumentDB stored procedure tutorial

Related

Having() count of id's on joined table

I am trying to make this query in OrmLite:
select b.* from blog b
join blog_to_blog_category btbc on b.id = btbc.blog_id
join blog_category bc on btbc.blog_category_id = bc.id
where b.url like '%.com' and bc.id in (100, 583)
group by b.id
having count(distinct bc.id ) = 1
I can't figure out how to get the Having() method structured. I can see there is a Sql.CountDistinct() method but I can't figure out how to use it with Having().
I figure I need to do something along the lines of:
var q = db
.From<Blog>()
.LeftJoin<BlogToBlogCategory>()
.Join<BlogToBlogCategory, BlogCategory>()
.Where<BlogCategory>(x => Sql.In(x.Id, 100, 583))
.GroupBy<Blog>(bc => bc.Id)
.Having(x => Sql.CountDistinct("blog_category.id") == "2")
This gives error:
42883: operator does not exist: bigint = text
I can't see how to type it to take a table column name and return a number for comparison.
Is this query possible?
EDIT
I got around it by setting having expression explicitly
q.HavingExpression = $"having count(distinct {q.Table<BlogCategory>()}.{q.Column<BlogCategory>(bc => bc.Id)}) = 2";
I am still curious though if it is possible to do this with fluent api.
I've just added multiple typed table overloads for Having() in the latest v5.11.1 that's now available on MyGet which will allow you to reference a joined table properties in a typed expression, e.g:
var q = db
.From<Blog>()
.LeftJoin<BlogToBlogCategory>()
.Join<BlogToBlogCategory, BlogCategory>()
.Where<BlogCategory>(x => Sql.In(x.Id, 100, 583))
.GroupBy<Blog>(bc => bc.Id)
.Having<BlogCategory>(x => Sql.CountDistinct(x.Id) == 2)

ATHENA/PRESTO complex query with multiple unnested tables

i have i would like to create a join over several tables.
table login : I would like to retrieve all the data from login
table logging : calculating the Nb_of_sessions for each db & for each a specific event type by user
table meeting : calculating the Nb_of_meetings for each db & for each user
table live : calculating the Nb_of_live for each db & for each user
I have those queries with the right results :
SELECT db.id,_id as userid,firstname,lastname
FROM "logins"."login",
UNNEST(dbs) AS a1 (db)
SELECT dbid,userid,count(distinct(sessionid)) as no_of_visits,
array_join(array_agg(value.from_url),',') as from_url
FROM "loggings"."logging"
where event='url_event'
group by db.id,userid;
SELECT dbid,userid AS userid,count(*) as nb_interviews,
array_join(array_agg(interviewer),',') as interviewer
FROM "meetings"."meeting"
group by dbid,userid;
SELECT dbid,r1.user._id AS userid,count(_id) as nb_chat
FROM "lives"."live",
UNNEST(users) AS r1 (user)
group by dbid,r1.user._id;
But when i begin to try put it all together, it seems i retrieve bad data (i have only on db retrieved) and it seems not efficient.
select a1.db.id,a._id as userid,a.firstname,a.lastname,count(rl._id) as nb_chat
FROM
"logins"."login" a,
"loggings"."logging" b,
"meetings"."meeting" c,
"lives"."live" d,
UNNEST(dbs) AS a1 (db),
UNNEST(users) AS r1 (user)
where a._id = b.userid AND a._id = c.userid AND a._id = r1.user._id
group by 1,2,3,4
Do you have an idea ?
Regards.
The easiest way is to work with with to structure the subquery and then reference them.
with parameter reference:
You can use WITH to flatten nested queries, or to simplify subqueries.
The WITH clause precedes the SELECT list in a query and defines one or
more subqueries for use within the SELECT query.
Each subquery defines a temporary table, similar to a view definition,
which you can reference in the FROM clause. The tables are used only
when the query runs.
Since you already have working sub queries, the following should work:
with logins as
(
SELECT db.id,_id as userid,firstname,lastname
FROM "logins"."login",
UNNEST(dbs) AS a1 (db)
)
,visits as
(
SELECT dbid,userid,count(distinct(sessionid)) as no_of_visits,
array_join(array_agg(value.from_url),',') as from_url
FROM "loggings"."logging"
where event='url_event'
group by db.id,userid
)
,meetings as
(
SELECT dbid,userid AS userid,count(*) as nb_interviews,
array_join(array_agg(interviewer),',') as interviewer
FROM "meetings"."meeting"
group by dbid,userid
)
,chats as
(
SELECT dbid,r1.user._id AS userid,count(_id) as nb_chat
FROM "lives"."live",
UNNEST(users) AS r1 (user)
group by dbid,r1.user._id
)
select *
from logins l
left join visits v
on l.dbid = v.dbid
and l.userid = v.userid
left join meetings m
on l.dbid = m.dbid
and l.userid = m.userid
left join chats c
on l.dbid = c.dbid
and l.userid = c.userid;

Generic Inquiry Parent Child Relation

I have been trying to pull up the data from Contract, ContractDetail, ContractItem and InventoryItem in Generic Inquiry. So, after adding up all 4 tables I created relationship as below.
The relationship between Contract and ContractDetail is fine. And now since ContractDetail uses ContractItemID, I have set the relationship as below. But it throws me error like this.
And, upon looking into the stack trace, I find the generated query looks this:
DECLARE #P0 SmallDateTime(4) SET #P0='9/27/2018 12:00:00 AM' DECLARE #P1 nvarchar(MAX) SET #P1='0' SELECT COUNT(*)
FROM (
[Contract] InnerQuery_Contract
INNER JOIN (
[ContractDetail] InnerQuery_ContractDetail_ContractDetail
INNER JOIN [Contract] InnerQuery_ContractDetail_Contract ON [InnerQuery_ContractDetail_Contract].CompanyID IN (1, 3) AND 32 = SUBSTRING([InnerQuery_ContractDetail_Contract].CompanyMask, 1, 1) & 32 AND [InnerQuery_ContractDetail_Contract].[DeletedDatabaseRecord] = 0 AND [InnerQuery_ContractDetail_ContractDetail].[ContractID] = [InnerQuery_ContractDetail_Contract].[ContractID] AND [InnerQuery_ContractDetail_ContractDetail].[RevID] = [InnerQuery_ContractDetail_Contract].[RevID]
LEFT JOIN
[ContractDetail] InnerQuery_ContractDetail_ContractDetailExt_ContractDetailExt
ON [InnerQuery_ContractDetail_ContractDetailExt_ContractDetailExt].[ContractID] = [InnerQuery_ContractDetail_ContractDetail].[ContractID] AND [InnerQuery_ContractDetail_ContractDetailExt_ContractDetailExt].[LineNbr] = [InnerQuery_ContractDetail_ContractDetail].[LineNbr] AND [InnerQuery_ContractDetail_ContractDetailExt_ContractDetailExt].[RevID] = [InnerQuery_ContractDetail_Contract].[LastActiveRevID] AND [InnerQuery_ContractDetail_ContractDetailExt_ContractDetailExt].CompanyID = 3
) ON [InnerQuery_Contract].[ContractID] = [InnerQuery_ContractDetail_ContractDetail].[ContractID] AND [InnerQuery_ContractItem].[ContractItemID] = [InnerQuery_ContractDetail_ContractDetail].[ContractItemID] AND [InnerQuery_ContractDetail_ContractDetail].CompanyID = 3
INNER JOIN [ContractItem] InnerQuery_ContractItem ON [InnerQuery_ContractItem].CompanyID = 3 AND [InventoryItem].[inventoryID] = [InnerQuery_ContractItem].[RecurringItemID]
)
WHERE [InnerQuery_Contract].CompanyID IN (1, 3) AND 32 = SUBSTRING([InnerQuery_Contract].CompanyMask, 1, 1) & 32 AND [InnerQuery_Contract].[DeletedDatabaseRecord] = 0 AND [InnerQuery_Contract].[UsrLockDate] < #P0 AND (CASE WHEN ([InnerQuery_Contract].[BaseType] = 'T' OR [InnerQuery_Contract].[BaseType] = 'R') THEN CONVERT (BIT, 1) ELSE CONVERT (BIT, 0) END) = #P1 OPTION(OPTIMIZE FOR UNKNOWN)
So, there is problem with Alias name for table in the query. So, how do I fix this issue?
Thank you.
Joins in Acumatica works exactly like in SQL. First build the query in SQL and get the result as to what you need and then try to construct the same in Generic Inquiry.
Apart from than, I do not see any relations between Contracts and Inventory. Please explain what you are trying to get.
Here is the way you should join your tables in Generic Inquiry

Spotfire - advanced row level security

I'm working on row level security in Spotfire (6.5) report.
It should be implemented on 3 levels, lets call it L1, L2 and L3. There is additional mapping table that contains Userlogins and specified values on all levels where user has access. Additionaly if user is not in mapping table he is some kind of Root user so he has access to everything.
On DB side it looks like that:
CREATE TABLE SECURITY
(
USER_ID VARCHAR2(100 BYTE)
, L1 VARCHAR2(100 BYTE)
, L2 VARCHAR2(100 BYTE)
, L3 VARCHAR2(100 BYTE)
--, L1L2L3 VARCHAR2(100 BYTE) -- option there could be one column that contains lowest possible level
);
INSERT INTO SECURITY (USER_ID, L1) VALUES ('UNAME1','A');
INSERT INTO SECURITY (USER_ID, L2) VALUES ('UNAME2','BB');
INSERT INTO SECURITY (USER_ID, L3) VALUES ('UNAME3','CCC');
CREATE TABLE SECURED_DATA
(
L1 VARCHAR2(100 BYTE)
, L2 VARCHAR2(100 BYTE)
, L3 VARCHAR2(100 BYTE)
, V1 NUMBER
);
INSERT INTO SECURED_DATA (L1, V1) VALUES ('A',1);
INSERT INTO SECURED_DATA (L1, L2, V1) VALUES ('B','BB',2);
INSERT INTO SECURED_DATA (L1, L2, L3, V1) VALUES ('C','CC','CCC',3);
Finally I've made Information Link and then I've changed its' sql code to something like that:
SELECT
M.*
FROM
SECURITY S
INNER JOIN SECURED_DATA M
ON
(
M.L1 = S.L1
AND S.USER_ID = (%CURRENT_USER%)
)
UNION ALL
SELECT
M.*
FROM
SECURITY S
INNER JOIN SECURED_DATA M
ON
(
M.L2 = S.L2
AND S.USER_ID = (%CURRENT_USER%)
)
UNION ALL
SELECT
M.*
FROM
SECURITY S
INNER JOIN SECURED_DATA M
ON
(
M.L3 = S.L3
AND S.USER_ID = (%CURRENT_USER%)
)
UNION ALL
SELECT
M.*
FROM
SECURED_DATA M
WHERE
(
SELECT
COUNT(1)
FROM
SECURITY S
WHERE S.USER_ID = (%CURRENT_USER%)
)
=0
It works fine, but I'm worndering if there is more smart and more Spotfire way to get it?
Many thanks and regards,
Maciej
My guess on "more smart and more Spotfire way" is that you want to be able to cache a single data set and use it for multiple users, limiting it in the analytic rather than in the data pull. There is some danger to this, if we're doing it for security's sake, because the data will technically be in the analytic, and if they have permission to edit and add visualizations, you no longer control what they can and cannot see. If there's any authoring allowed in Web Player for the specific analytic, I recommend all securities be done DataBase-side.
If you want to do it in Spotfire anyways, here is my recommendation:
Have an Information Link (for example case, named IL_SecurityCheck) which is Select * from SECURITY WHERE S.USER_ID = (%CURRENT_USER%).
If they move from a cover page to the page with the data in it, you can put the code in the script to change pages; if not, you can use a method I explained here: Spotfire Current Date in input field with calendar popup to fire off a script on open.
Button Script required:
from Spotfire.Dxp.Data import *
crossSource = Document.Data.Tables["IL_SecurityCheck"]
rowCount = crossSource.RowCount
rowIndexSet = IndexSet(rowCount, True)
print rowCount
#rowCount = Document.Data.Tables["Managed Care UpDownStream"].RowCount
colCurs = DataValueCursor.CreateFormatted(crossSource.Columns["L1"])
colCurs2 = DataValueCursor.CreateFormatted(crossSource.Columns["L2"])
colCurs3 = DataValueCursor.CreateFormatted(crossSource.Columns["L3"])
x = ""
if rowIndexSet.IsEmpty != True:
for row in crossSource.GetRows(rowIndexSet, colCurs):
if colCurs.CurrentValue is not None:
x += "[L1] = '" + colCurs.CurrentValue + "' and "
for row in crossSource.GetRows(rowIndexSet, colCurs2):
if colCurs2.CurrentValue is not None:
x += "[L2] = '" + colCurs2.CurrentValue + "' and "
for row in crossSource.GetRows(rowIndexSet, colCurs3):
if colCurs3.CurrentValue is not None:
x += "[L3] = '" + colCurs3.CurrentValue + "' and "
x = x[:len(x) - 4]
else:
x = "1=1"
Document.Properties["SecurityLimits"] = x
Visualization Data Limited by Expression: ${SecurityLimits}

querydsl using parent query entitypath in subquery

I am trying to represent the following query in querydsl:
select a1.agreement_id, ad1.agreement_detail_id
from agreement a1
inner join agreement_details ad1 on a1.agreement_id=ad1.agreement_id
where ad1.transaction_id in (
select max(ad2.transaction_id)
from agreement_details ad2
inner join agreement a2 on ad2.agreement_id=a2.agreement_id
where
ad2.transaction_id<=1234
and a2.agreement_id=a1.agreement_id
and a2.entity_split_id=5678
)
order by a1.agreement_id asc
However, I'm having trouble getting querydsl to use a table from the parent query in the where clause of the subquery. For example, if I write code like the following:
QAgreement a1 = QAgreement.agreement;
QAgreementDetails ad1 = QAgreementDetails.agreementDetails;
QAgreement a2 = QAgreement.agreement;
QAgreementDetails ad2 = QAgreementDetails.agreementDetails;
HibernateQuery query = getHibernateQuery();
query.from(a1)
.innerJoin(a1.agreementDetailsesByAgreementId, ad1)
.fetch()
.where(ad1.transactionId.in(
new JPASubQuery().from(ad2)
.innerJoin(ad2.agreementByAgreementId, a2)
.where(ad2.transactionId.loe(maxTransactionId))
.where(a1.agreementId.eq(a2.agreementId))
.where(a2.entitySplitByEntitySplitId.entitySplitId.eq(entitySplitId))
.list(ad2.transactionId.max())))
.orderBy(a1.agreementId.asc());
and execute it, the resulting SQL looks like
select -- Some columns removed for brevity
agreement0_.agreement_id as agreemen1_0_0_,
agreementd1_.agreement_detail_id as agreemen1_11_1_
from
agreement agreement0_
inner join
agreement_details agreementd1_
on agreement0_.agreement_id=agreementd1_.agreement_id
where
agreementd1_.transaction_id in (
select
max(agreementd2_.transaction_id)
from
agreement_details agreementd2_
inner join
agreement agreement3_
on agreementd2_.agreement_id=agreement3_.agreement_id
where
agreementd2_.transaction_id<=1234
and agreement3_.agreement_id=agreement3_.agreement_id
and agreement3_.entity_split_id=5678
)
order by
agreement0_.agreement_id asc
You can see that the alias of the Agreement EntityPath a1 is not being used in the subquery and is being replaced by the alias of the Agreement EntityPath a2. Is there something else I have to do in querydsl to be able to produce this query?
The problem is that you don't use unique variables
QAgreement a1 = QAgreement.agreement;
QAgreementDetails ad1 = QAgreementDetails.agreementDetails;
QAgreement a2 = QAgreement.agreement;
QAgreementDetails ad2 = QAgreementDetails.agreementDetails;
In this case a1 and a2 as well as ad1 and ad2 refer to the same path.
The variable names in the Java code don't matter, you need to use them for example like this
QAgreement a1 = new QAgreement("a1");
QAgreementDetails ad1 = new QAgreementDetails("ad1");
QAgreement a2 = new QAgreement("a2");
QAgreementDetails ad2 = new QAgreementDetails("ad2");

Resources