If my search term is not found add to array - python-3.x

Looping through a list of terms and searching database for each term. Is there a way to return a list of search terms that didn't find results in the database?
This is my SELECT query:
SELECT x_tbl.x_str
FROM x_tbl
LEFT JOIN p_tbl ON p_tbl.p_id = x_tbl.p_id
LEFT JOIN t_tbl ON t_tbl.t_id = x_tbl.t_id
WHERE t_tbl.t = %(uw)s
OR t_tbl.ta = %(uw)s
OR %(uw)s = ANY (t_tbl.tb)
OR t_tbl.tc = %(uw)s
OR p_tbl.pa = %(uw)s
OR %(uw)s = ANY (p_tbl.pb)
I want to return uw if it does not find any results in the database. Is this possible?

I want to return uw if it does not find any results in the database.
For one search term
The two LEFT [OUTER] JOIN between x_tbl, p_tbl and t_tbl do not eliminate any rows by themselves. Two times NOT EXISTS returns a list of search terms, that cannot find anything:
SELECT %(uw)s -- old-style Python placeholder
WHERE NOT EXISTS (
SELECT FROM p_tbl
WHERE pa = %(uw)s
OR %(uw)s = ANY (p.pb)
)
AND NOT EXISTS (
SELECT FROM t_tbl
WHERE %(uw)s IN (t, ta, tc)
OR %(uw)s = ANY (tb)
);
If there can be orphans in t_tbl and/or p_tbl (not linked to any row in x_tbl), the set may be bigger, and the query gets more expensive:
SELECT %(uw)s -- old-style Python placeholder
WHERE NOT EXISTS NOT EXISTS (
SELECT FROM x_tbl JOIN p_tbl p USING (p_id)
WHERE p.pa = %(uw)s
OR %(uw)s = ANY (p.pb)
)
AND (
SELECT FROM x_tbl JOIN t_tbl t USING (t_id)
WHERE %(uw)s IN (t.t, t.ta, t.tc)
OR %(uw)s = ANY (t.tb)
);
This is dealing with one search term at a time, like your original query. You mentioned a list. Running a single query for all of them might be (much) cheaper ...
One query to rule them all
Pass the list as array (Postgres array literal) - which may require an explicit cast (::text[]) - unnest() and attach the same WHERE conditions as above:
SELECT uw
FROM unnest(%(my_list_as_array)s::text[]) q(uw)
WHERE NOT EXISTS (
SELECT FROM p_tbl
WHERE pa = q.uw
OR q.uw = ANY (p.pb)
)
AND NOT EXISTS (
SELECT FROM t_tbl
WHERE q.uw IN (t, ta, tc)
OR q.uw = ANY (tb)
);
Or, including the join to tbl_x, same as above:
SELECT uw
FROM unnest(%(my_list_as_array)s::text[]) q(uw)
WHERE NOT EXISTS (
SELECT FROM x_tbl JOIN p_tbl p USING (p_id)
WHERE p.pa = q.uw
OR q.uw = ANY (p.pb)
)
AND NOT EXISTS (
SELECT FROM x_tbl JOIN t_tbl t USING (t_id)
WHERE q.uw IN (t.t, t.ta, t.tc)
OR q.uw = ANY (t.tb)
);
Basics:
Select rows which are not present in other table
You may want to keep array elements in original order, or even attach an ordinal position. See:
PostgreSQL unnest() with element number
Aside, your original query can multiply rows - if there can be more than one row on the right side of each join. See:
Two SQL LEFT JOINS produce incorrect result

Related

Databricks AnalysisException: Column 'l' does not exist

I have a very strange occurrence with my code.
I keep on getting the error
AnalysisException: Column 'homepage_url' does not exist
However, when I do a select with cross Joins the column does actually exist.
Can someone take a look at my cross joins and let me know if that is where the problem is
SELECT DISTINCT
account.xpd_relationshipstatus AS CRM_xpd_relationshipstatus
,REPLACE(owneridname,'Data.Import #','') AS MontaguOwner
,account.ts_montaguoffice AS Montagu_Office
,CAST(account.ts_reminderdatesetto AS DATE) AS CRM_ts_reminderdatesetto
,CAST(account.ts_lastdatestatuschanged AS DATE) AS YearofCRMtslastdatestatuschanged
,organizations.name AS nameCB
,organizations.homepage_url
,iff(e like 'www.%', e, 'www.' + e) AS website
,left(category_list,charindex(',',category_list +',' )-1) AS category_CB
-- ,case when charindex(',',category_list,0) > 0 then left(category_list,charindex(',',category_list)-1) else category_list end as category_CB
,organizations.category_groups_list AS category_groups_CB
FROM basecrmcbreport.account
LEFT OUTER JOIN basecrmcbreport.CRM2CBURL_Lookup
ON account.Id = CRM2CBURL_Lookup.Key
LEFT OUTER JOIN basecrmcbreport.organizations
ON CRM2CBURL_Lookup.CB_URL_KEY = organizations.cb_url
cross Join (values (charindex('://', homepage_url))) a(a)
cross Join (values (iff(a = 0, 1, a + 3))) b(b)
cross Join (values (charindex('/', homepage_url, b))) c(c)
cross Join (values (iff(c = 0, length(homepage_url) + 1, c))) d(d)
cross Join (values (substring(homepage_url, b, d - b))) e(e)
Without the cross Joins
The main reason for cross join (or any join) to recognize the column when you select not when using table valued functions is that joins are used on tables only.
To use table valued functions, one must use cross apply or outer apply. But these are not supported in Databricks sql.
The following is the demo data I am using:
I tried using inner join on a table valued function using the following query and got the same error:
select d1.*,a from demo1 inner join (values(if(d1.team = 'OG',2,1))) a;
Instead, using the select query, the joins work as that is how they function:
select d1.*,a.no_of_wins from demo1 d1 inner join (select id,case team when 'OG' then 2 when 'TS' then 1 end as no_of_wins from demo1) a on d1.id=a.id;
So, the remedy for this problem is to replace all the table valued functions on which you are using joins with SELECT statements.

Cosmos DB Left Join

All of the documentation for Cosmos DB and it looks like it only supports the JOINkeyword, which seems to be a sort of INNER JOIN.
I have the following query:
SELECT * FROM
(
SELECT
DISTINCT(c.id),
c.OtherCollection,
FROM c
JOIN s IN c.OtherCollection
)
AS c order by c.id
This works fine and returns the data of documents that have OtherCollection populated. But It obviously does not return any documents that do not have it populated.
The reason for the join is that sometimes I execute the following query (queries are built up from user input)
SELECT * FROM
(
SELECT
DISTINCT(c.id),
c.OtherCollection,
FROM c
JOIN s IN c.OtherCollection
WHERE s.PropertyName = 'SomeValue'
)
AS c order by c.id
The question is how can I have a sort of LEFT JOIN operator in this scenario?
CosmosDB JOIN operation is limited to the scope of a single document. What possible is you can join parent object with child objects under same document.
It is totally different from SQL Join query which supports across two/many tables.
You can simulate LEFT JOIN with the EXISTS sentence.
Eg:
SELECT VALUE c
FROM c
WHERE (
(c.OtherCollection = null) OR EXISTS (--Like a "Left Join"
SELECT null
FROM s IN c.OtherCollection
WHERE s.PropertyName = 'SomeValue'
)
)
--AND/OR Some other c Node conditions
order by c.id

SQL Oracle Sub-query

I am having a issue getting this Sub-query to run. I am using Toad Data Point -Oracle. I get syntax error. I have tried several different ways with no luck. I am knew to sub-query's
Select *
from FINC.VNDR_ITEM_M as M
where M.ACCT_DOC_NBR = A.ACCT_DOC_NBR
(SELECT A.CLIENT_ID,
A.SRC_SYS_ID,
A.CO_CD,
A.ACCT_NBR,
A.CLR_DT,
A.ASGN_NBR,
A.FISCAL_YR,
A.ACCT_DOC_NBR,
A.LINE_ITEM_NBR,
A.MFR_PART_NBR,
A.POST_DT,
A.DRCR_IND,
A.DOC_CRNCY_AMT,
A.CRNCY_CD,
A.BSL_DT
FROM FINC.VNDR_ITEM_F A
WHERE A.CLR_DT IN (SELECT MAX(B.CLR_DT)
FROM FINC.VNDR_ITEM_F AS B
where (B.ACCT_DOC_NBR = A.ACCT_DOC_NBR and B.FISCAL_YR=A.FISCAL_YR and B.LINE_ITEM_NBR = A.LINE_ITEM_NBR and B.SRC_SYS_ID =A.SRC_SYS_ID and B.POST_DT=A.POST_DT and B.CO_CD=A.CO_CD)
and (B.CO_CD >='1000' and B.CO_CD <= '3000' or B.CO_CD ='7090') and (B.POST_DT Between to_date ('08/01/2018','mm/dd/yyyy')
AND to_date ('08/31/2018', 'mm/dd/yyyy')) and (B.SRC_SYS_ID ='15399') and (B.FISCAL_YR ='2018'))
GROUP BY
A.CLIENT_ID,
A.SRC_SYS_ID,
A.CO_CD,
A.ACCT_NBR,
A.CLR_DT,
A.ASGN_NBR,
A.FISCAL_YR,
A.ACCT_DOC_NBR,
A.LINE_ITEM_NBR,
A.MFR_PART_NBR,
A.POST_DT,
A.DRCR_IND,
A.DOC_CRNCY_AMT,
A.CRNCY_CD,
A.BSL_DT)
Your syntax is broken, you put subquery just at the end. Now it looks like:
select *
from dual as m
where a.dummy = m.dummy
(select dummy from dual)
It is in incorrect place, not joined, not aliased. What you should probably do is:
select *
from dual m
join (select dummy from dual) a on a.dummy = m.dummy
You also have some redundant, unnecessary brackets, but that's minor flaw. Full code (I cannot test it without data access):
select *
from FINC.VNDR_ITEM_M M
join (SELECT A.CLIENT_ID, A.SRC_SYS_ID, A.CO_CD, A.ACCT_NBR, A.CLR_DT, A.ASGN_NBR,
A.FISCAL_YR, A.ACCT_DOC_NBR, A.LINE_ITEM_NBR, A.MFR_PART_NBR, A.POST_DT,
A.DRCR_IND, A.DOC_CRNCY_AMT, A.CRNCY_CD, A.BSL_DT
FROM FINC.VNDR_ITEM_F A
WHERE A.CLR_DT IN (SELECT MAX(B.CLR_DT)
FROM FINC.VNDR_ITEM_F AS B
where B.ACCT_DOC_NBR = A.ACCT_DOC_NBR
and B.FISCAL_YR=A.FISCAL_YR
and B.LINE_ITEM_NBR = A.LINE_ITEM_NBR
and B.SRC_SYS_ID =A.SRC_SYS_ID
and B.POST_DT=A.POST_DT
and B.CO_CD=A.CO_CD
and (('1000'<=B.CO_CD and B.CO_CD<='3000') or B.CO_CD='7090')
and B.POST_DT Between to_date ('08/01/2018', 'mm/dd/yyyy')
AND to_date ('08/31/2018', 'mm/dd/yyyy')
and B.SRC_SYS_ID ='15399' and B.FISCAL_YR ='2018')
GROUP BY A.CLIENT_ID, A.SRC_SYS_ID, A.CO_CD, A.ACCT_NBR, A.CLR_DT, A.ASGN_NBR,
A.FISCAL_YR, A.ACCT_DOC_NBR, A.LINE_ITEM_NBR, A.MFR_PART_NBR, A.POST_DT,
A.DRCR_IND, A.DOC_CRNCY_AMT, A.CRNCY_CD, A.BSL_DT) A
on M.ACCT_DOC_NBR = A.ACCT_DOC_NBR and M.CO_CD=A.CO_CD;
You need to add an alias to the SubSelect (or Derived Table in Standard SQL):
select *
from
( select .......
) AS dt
join ....

MSSQL: Use the result of nested sub-queries

The following works and results in the output shown in the image below.
SELECT
SU_Internal_ID,
NQ_QuestionText,
NA_AnswerText,
NoOfTimesChoosen
FROM
(SELECT
U.SU_Internal_ID,
NQ.NQ_QuestionText,
NA.NA_AnswerText,
COUNT(PC.UserID) AS NoOfTimesChoosen
FROM [dbo].[ParticipantNSChoices] PC
INNER JOIN [dbo].[KnowledgeSurveyAnswers] NA
on PC.NA_Internal_ID = NA.NA_Internal_ID
INNER JOIN [dbo].[KnowledgeSurveyQuestions] NQ
on PC.NQ_Internal_ID = NQ.NQ_Internal_ID
INNER JOIN [dbo].[AspNetUsers] U
on PC.UserID = U.Id
WHERE
U.SU_Internal_ID=1
and NQ.NQ_QuestionText LIKE '%Do you feel comfortable working with computers%'
GROUP
BY U.SU_Internal_ID,
NQ.NQ_QuestionText,
NA.NA_AnswerText ) as A
I want to add a column to show the percent for the two answers 'No' and 'Yes': so next to 'No' I want '20' and next to 'Yes' '80', but I'm pretty new at this and am stuck; I would appreciate any help. Thanks.
Result of working script
You don't need the outer SELECT.
SELECT
U.SU_Internal_ID,
NQ.NQ_QuestionText,
NA.NA_AnswerText,
COUNT(PC.UserID) AS NoOfTimesChoosen,
(cast(COUNT(PC.UserID) as float) /
cast(
(select count(*) from [dbo].[ParticipantNSChoices] PC2
INNER JOIN [dbo].[KnowledgeSurveyAnswers] NA2 on PC2.NA_Internal_ID = NA2.NA_Internal_ID
INNER JOIN [dbo].[KnowledgeSurveyQuestions] NQ2 on PC2.NQ_Internal_ID = NQ2.NQ_Internal_ID
INNER JOIN [dbo].[AspNetUsers] U2 on PC2.UserID = U2.Id
WHERE
U2.SU_Internal_ID=1
and NQ2.NQ_QuestionText LIKE '%Do you feel comfortable working with computers%' )
as float))
* 100 as PercentChosen
FROM [dbo].[ParticipantNSChoices] PC
INNER JOIN [dbo].[KnowledgeSurveyAnswers] NA
on PC.NA_Internal_ID = NA.NA_Internal_ID
INNER JOIN [dbo].[KnowledgeSurveyQuestions] NQ
on PC.NQ_Internal_ID = NQ.NQ_Internal_ID
INNER JOIN [dbo].[AspNetUsers] U
on PC.UserID = U.Id
WHERE
U.SU_Internal_ID=1
and NQ.NQ_QuestionText LIKE '%Do you feel comfortable working with computers%'
GROUP
BY U.SU_Internal_ID,
NQ.NQ_QuestionText,
NA.NA_AnswerText
The counts will be integers, so you need to cast as floats before dividing. You can then further format to your liking. Also, I might not have your exact denominator, because I don't know what your data looks like, but you can modify to match what you need.

in Tsql can i compare two string "MY String" to my string and show they are different

I need to do a query between two tables and find non matching fields
table 1 field locations has "my String"
table 2 field locations has "MY string"
they = by text but not by capitalization i need to return a false for this
Having the following data:
DECLARE #TableOne TABLE
(
[ID] TINYINT
,[Value] VARCHAR(12)
)
DECLARE #TableTwo TABLE
(
[ID] TINYINT
,[Value] VARCHAR(12)
)
INSERT INTO #TableOne ([ID], [Value])
VALUES (1,'my String')
INSERT INTO #TableTwo ([ID], [Value])
VALUES (1,'MY String')
You can use set Case Sentitive collation like this:
SELECT [TO].[Value]
,[TW].[Value]
FROM #TableOne [TO]
INNER JOIN #TableTwo [TW]
ON [TO].[ID] = [TW].[ID]
AND [TO].[Value] <> [TW].[Value]
COLLATE Latin1_General_CS_AS
or use HASH functions like this:
SELECT [TO].[Value]
,[TW].[Value]
FROM #TableOne [TO]
INNER JOIN #TableTwo [TW]
ON [TO].[ID] = [TW].[ID]
WHERE HASHBYTES('SHA1', [TO].[Value]) <> HASHBYTES('SHA1', [TW].[Value])
DECLARE #Table1 AS TABLE (FieldName VARCHAR(100))
DECLARE #Table2 AS TABLE (FieldName VARCHAR(100))
INSERT INTO #Table1 (FieldName) VALUES ('MY Location')
INSERT INTO #Table2 (FieldName) VALUES ('My Location')
With a default case insensitive collation order - Matches and returns results
SELECT * FROM #Table1 AS T1
INNER JOIN #Table2 AS T2
ON T1.FieldName = T2.FieldName
With a case sensitive collation order specified. Will not match
SELECT * FROM #Table1 AS T1
INNER JOIN #Table2 AS T2
ON T1.FieldName = T2.FieldName COLLATE Latin1_General_CS_AS_KS_WS
Microsoft article on collation

Resources