I have edited this legacy script a bit trying not to break a working script. It contains multiple CTEs and in the last statement calls the aggregated and calculated variables created within the CTE. The CTEs run fine from a - c, however with the addition of the final statement breaks the code: SQL Error [XX000]: ERROR: invalid value for "YYYY" in source string. Would appreciate suggestions or changes.
with a as (
select
to_date(df.launch_date, 'YYYYMMDD', FALSE) as n_date,
df.new_name, df.link, df.uid
from (
select
SPLIT_PART(descriptive_field,'_',4) as launch_date,
descriptive_field as new_name,
embedded_link as link,
autocreated_id as uid
from database.schema.table1
union
select
SPLIT_PART(ndf_descriptive_field,'_',4) as launch_date,
ndf_descriptive_field as new_name,
embedded_link as link,
autocreated_id as uid
from database.schema.table2
) as df
where n_date ~ '.*2022.*'
),
b as (
select link, uid,
case
when link ILIKE '%-type1%' THEN title_name||'-type1'
when link....
else new_name
end as new_name
from a
),
c as (
select
new_name,
case
when link ilike ....
when (link ilike .... )
end as link_grouping,
count(distinct(uid)) as dist_usr_cnt
from b
group by 1,2
)
select
c.new_name,
a.n_date,
c.link_grouping,
c.dist_usr_cnt
from c
join a on c.n_name = a.new_name
where a.n_date >= current_date-7
and a.n_date <= current_date-3
order by 2,1,3
;
Related
Using PostgreSQL I can have multiple rows of json objects.
select (select ROW_TO_JSON(_) from (select c.name, c.age) as _) as jsonresult from employee as c
This gives me this result:
{"age":65,"name":"NAME"}
{"age":21,"name":"SURNAME"}
But in SqlServer when I use the FOR JSON AUTO clause it gives me an array of json objects instead of multiple rows.
select c.name, c.age from customer c FOR JSON AUTO
[{"age":65,"name":"NAME"},{"age":21,"name":"SURNAME"}]
How to get the same result format in SqlServer ?
By constructing separate JSON in each individual row:
SELECT (SELECT [age], [name] FOR JSON PATH, WITHOUT_ARRAY_WRAPPER)
FROM customer
There is an alternative form that doesn't require you to know the table structure (but likely has worse performance because it may generate a large intermediate JSON):
SELECT [value] FROM OPENJSON(
(SELECT * FROM customer FOR JSON PATH)
)
no structure better performance
SELECT c.id, jdata.*
FROM customer c
cross apply
(SELECT * FROM customer jc where jc.id = c.id FOR JSON PATH , WITHOUT_ARRAY_WRAPPER) jdata (jdata)
Same as Barak Yellin but more lazy:
1-Create this proc
CREATE PROC PRC_SELECT_JSON(#TBL VARCHAR(100), #COLS VARCHAR(1000)='D.*') AS BEGIN
EXEC('
SELECT X.O FROM ' + #TBL + ' D
CROSS APPLY (
SELECT ' + #COLS + '
FOR JSON PATH, WITHOUT_ARRAY_WRAPPER
) X (O)
')
END
2-Can use either all columns or specific columns:
CREATE TABLE #TEST ( X INT, Y VARCHAR(10), Z DATE )
INSERT #TEST VALUES (123, 'TEST1', GETDATE())
INSERT #TEST VALUES (124, 'TEST2', GETDATE())
EXEC PRC_SELECT_JSON #TEST
EXEC PRC_SELECT_JSON #TEST, 'X, Y'
If you're using PHP add SET NOCOUNT ON; in the first row (why?).
I am coding Python and Spark SQL in Databricks and I am using spark 2.4.5.
I have two tables.
Create table IF NOT EXISTS db_xsi_ed_faits_shahgholi_ardalan.Destination
(
id Int,
Name string,
Deleted int
) USING Delta;
Create table IF NOT EXISTS db_xsi_ed_faits_shahgholi_ardalan.Source
(
id Int,
Name string,
Deleted int
) USING Delta;
I need to ran a Merge command between my source and destination. I wrote below command
%sql
MERGE INTO db_xsi_ed_faits_shahgholi_ardalan.Destination AS D
USING db_xsi_ed_faits_shahgholi_ardalan.Source AS S
ON (S.id = D.id)
-- UPDATE
WHEN MATCHED AND S.Name <> D.Name THEN
UPDATE SET
D.Name = S.Name
-- INSERT
WHEN NOT MATCHED THEN
INSERT (id, Name, Deleted)
VALUES (S.id, S.Name, S.Deleted)
-- DELETE
WHEN NOT MATCHED BY SOURCE THEN
UPDATE SET
D.Deleted = 1
When i ran this command i have below error:
It seems that we do not have NOT MATCHED BY SOURCE in spark! I need a solution to do that.
I wrote this code but still i am looking for better approach
%sql
MERGE INTO db_xsi_ed_faits_shahgholi_ardalan.Destination AS D
USING db_xsi_ed_faits_shahgholi_ardalan.Source AS S
ON (S.id = D.id)
-- UPDATE
WHEN MATCHED AND S.Name <> D.Name THEN
UPDATE SET
D.Name = S.Name
-- INSERT
WHEN NOT MATCHED THEN
INSERT (id, Name, Deleted)
VALUES (S.id, S.Name, S.Deleted)
;
%sql
-- Logical delete
UPDATE db_xsi_ed_faits_shahgholi_ardalan.Destination
SET Deleted = 1
WHERE db_xsi_ed_faits_shahgholi_ardalan.Destination.id in
(
SELECT
D.id
FROM db_xsi_ed_faits_shahgholi_ardalan.Destination AS D
LEFT JOIN db_xsi_ed_faits_shahgholi_ardalan.Source AS S ON (S.id = D.id)
WHERE S.id is null
)
I have the following table & data:
CREATE TABLE dbo.TableMapping
(
[GenericMappingKey] [nvarchar](256) NULL,
[GenericMappingValue] [nvarchar](256) NULL,
[TargetMappingKey] [nvarchar](256) NULL,
[TargetMappingValue] [nvarchar](256) NULL
)
INSERT INTO dbo.TableMapping
(
[GenericMappingKey]
,[GenericMappingValue]
,[TargetMappingKey]
,[TargetMappingValue]
)
VALUES
(
'Generic'
,'Col1Source|Col1Target;Col2Source|Col2Target;Col3Source|Col3Target;Col4Source|Col4Target;Col5Source|Col5Target;Col6Source|Col6Target'
,'Target'
,'Fruit|Apple;Car|Red;House|Bungalo;Gender|Female;Material|Brick;Solution|IT'
)
I would need to be able to automatically generate my GenericMappingValue string dynamically based on the number of column pairs in the TargetMappingValue column.
Currently, there are 6 column mapping pairs. However, if I only had two mapping column pairs in my TargetMapping such as the following...
'Fruit|Apple;Car|Red'
then I would like for the GenericMappingValue to be automatically generated (updated) such as the following since, as a consequence, I would only have 2 column pairs in my string...
'Col1Source|Col1Target;Col2Source|Col2Target'
I've started building the following query logic:
DECLARE #Mapping nvarchar(256)
SELECT #Mapping = [TargetMappingValue] from TableMapping
print #Mapping
SELECT count(*) ColumnPairCount
FROM String_split(#Mapping, ';')
The above query gives me a correct count of 6 for my column pairs.
How would I be able to continue my logic to achieve my automatically generated mapping string?
I think I understand what you are after. This should get you moving in the right direction.
Since you've tagged 2017 you can use STRING_AGG()
You'll want to split your TargetMappingValue using STRING_SPLIT() with ROW_NUMER() in a sub-query. (NOTE: We aren't guaranteed order using string_split() with ROW_NUMBER here, but will work for this situation. Example below using OPENJSON if we need to insure accurate order.)
Then you can then use that ROW_NUMBER() as the column indicator/number in a CONCAT().
Then bring it all back together using STRING_AGG()
Have a look at this working example:
DECLARE #TableMapping TABLE
(
[GenericMappingKey] [NVARCHAR](256) NULL
, [GenericMappingValue] [NVARCHAR](256) NULL
, [TargetMappingKey] [NVARCHAR](256) NULL
, [TargetMappingValue] [NVARCHAR](256) NULL
);
INSERT INTO #TableMapping (
[GenericMappingKey]
, [GenericMappingValue]
, [TargetMappingKey]
, [TargetMappingValue]
)
VALUES ( 'Generic'
, 'Col1Source|Col1Target;Col2Source|Col2Target;Col3Source|Col3Target;Col4Source|Col4Target;Col5Source|Col5Target;Col6Source|Col6Target'
, 'Target'
, 'Fruit|Apple;Car|Red;House|Bungalo;Gender|Female;Material|Brick;Solution|IT' );
SELECT [col].[GenericMappingKey]
, STRING_AGG(CONCAT('Col', [col].[ColNumber], 'Source|Col', [col].[ColNumber], 'Target'), ';') AS [GeneratedGenericMappingValue]
, [col].[TargetMappingKey]
, [col].[TargetMappingValue]
FROM (
SELECT *
, ROW_NUMBER() OVER ( ORDER BY (
SELECT 1
)
) AS [ColNumber]
FROM #TableMapping
CROSS APPLY STRING_SPLIT([TargetMappingValue], ';')
) AS [col]
GROUP BY [col].[GenericMappingKey]
, [col].[TargetMappingKey]
, [col].[TargetMappingValue];
Here's an example of what an update would look like assuming your primary key is the GenericMappingKey column:
--This what an update would look like
--Assuming your primary key is the [GenericMappingKey] column
UPDATE [upd]
SET [upd].[GenericMappingValue] = [g].[GeneratedGenericMappingValue]
FROM (
SELECT [col].[GenericMappingKey]
, STRING_AGG(CONCAT('Col', [col].[ColNumber], 'Source|Col', [col].[ColNumber], 'Target'), ';') AS [GeneratedGenericMappingValue]
, [col].[TargetMappingKey]
, [col].[TargetMappingValue]
FROM (
SELECT *
, ROW_NUMBER() OVER ( ORDER BY (
SELECT 1
)
) AS [ColNumber]
FROM #TableMapping
CROSS APPLY [STRING_SPLIT]([TargetMappingValue], ';')
) AS [col]
GROUP BY [col].[GenericMappingKey]
, [col].[TargetMappingKey]
, [col].[TargetMappingValue]
) AS [g]
INNER JOIN #TableMapping [upd]
ON [upd].[GenericMappingKey] = [g].[GenericMappingKey];
Shnugo brings up a great point in the comments in that we are not guarantee sort order with string_split() and using row number. In this particular situation it wouldn't matter as the output mappings in generic. But what if you needed to used elements from your "TargetMappingValue" column in the final "GenericMappingValue", then you would need to make sure sort order was accurate.
Here's an example showing how to use OPENJSON() and it's "key" which would guarantee that order using Shnugo example:
SELECT [col].[GenericMappingKey]
, STRING_AGG(CONCAT('Col', [col].[colNumber], 'Source|Col', [col].[colNumber], 'Target'), ';') AS [GeneratedGenericMappingValue]
, [col].[TargetMappingKey]
, [col].[TargetMappingValue]
FROM (
SELECT [tm].*
, [oj].[Key] + 1 AS [colNumber] --Use the key as our order/column number, adding 1 as it is zero based.
, [oj].[Value] -- and if needed we can bring the split value out.
FROM #TableMapping [tm]
CROSS APPLY OPENJSON('["' + REPLACE([tm].[TargetMappingValue], ';', '","') + '"]') [oj] --Basically turn the column value into JSON string.
) AS [col]
GROUP BY [col].[GenericMappingKey]
, [col].[TargetMappingKey]
, [col].[TargetMappingValue];
if the data is already in the table and you want to break it out into columns, this should work
select
v.value
,left(v.value, charindex('|',v.value) -1) col1
,reverse(left(reverse(v.value), charindex('|',reverse(v.value)) -1)) col2
from String_split(#mapping,';') v
Am getting an error that goes like this:
Insert values statement can contain only constant literal values or variable references.
these are the statements in which I am getting the errors:
INSERT INTO val.summary_numbers (metric_name, metric_val, dt_create) VALUES ('Total IP Enconters',
(SELECT
count(DISTINCT encounter_id)
FROM prod.encounter
WHERE encounter_type = 'Inpatient')
,
(SELECT min(mod_loadidentifier)
FROM ccsm.stg_demographics_baseline)
);
INSERT INTO val.summary_numbers (metric_name, metric_val, dt_create) VALUES ('Total 30d Readmits',
(SELECT
count(DISTINCT encounter_id)
FROM prod.encounter_attr
WHERE
attr_name = 'day_30_readmit' AND attr_value = 1)
,
(SELECT min(mod_loadidentifier)
FROM ccsm.stg_demographics_baseline));
Change your query like this:
insert into val.summary_numbers
select
'Total IP Enconters',
(select count(distinct encounter_id)
from prod.encounter
where encounter_type = 'Inpatient'),
(select min(mod_loadidentifier)
from ccsm.stg_demographics_baseline)
When using the ADW service, I would recommend that you consider using the CTAS operation possibly combined with a RENAME. The RENAME is a metadata operation so it is fast and the CTAS is parallel where the INSERT INTO will be row by row.
You may still have a data related issue that can be hard to determine with out the create table statement.
Thanks
I have a table T with CLOB column called XML_CLOB
Value in the column likes following:
<reportName>
<string>REPORT_A</string>
<string>REPORT_B</string>
<string>REPORT_C</string>
</reportName>
I'm trying to retrieve string value from this CLOB column and return in different rows. If I use
xmltype(xml_clob).extract('//reportName/string/text()').getstringval()
it outputs like 'REPORT_AREPORT_BREPORT_C' in the same row.
I also tried
extractValue(xmltype(xml_clob), '//reportName/string[1]')
but the problem is I don't know how much child values under tag
Is there anyway I can retrieve in different rows like:
1 REPORT_A
2 REPORT_B
3 REPORT_C
Many thanks in advance~
Oracle Setup:
CREATE TABLE table_name (xml_clob CLOB );
INSERT INTO table_name VALUES (
'<reportName>
<string>REPORT_A</string>
<string>REPORT_B</string>
<string>REPORT_C</string>
</reportName>'
);
Query 1:
SELECT x.string
FROM table_name t,
XMLTable('/reportName/string'
PASSING XMLType( t.xml_clob )
COLUMNS string VARCHAR2(50) PATH '/'
) x
Query 2:
SELECT EXTRACTVALUE( s.COLUMN_VALUE, '/string' ) AS string
FROM table_name t,
TABLE(
XMLSequence(
EXTRACT(
XMLType( t.xml_clob ),
'/reportName/string'
)
)
) s;
Output:
STRING
--------
REPORT_A
REPORT_B
REPORT_C
WITH test_table AS
(SELECT xmltype('<reportName>
<string>REPORT_A</string>
<string>REPORT_B</string>
<string>REPORT_C</string>
</reportName>' ) xml_clob
FROM dual
)
SELECT x.*
FROM test_table,
xmltable('/reportName/string'
passing test_table.xml_clob
columns report_name VARCHAR2(100) path 'text()') x