I receive an excel file with the following info:
SQL SERVER SCHEMA TABLENAME
AdventureWorksDW Dbo DimOrganization
AdventureWorksDW Dbo ...
... ... ...
I want to export all columns information from the above TABLENAME from SQL Server into excel sheet using SSIS. The excel filename should be the TABLENAME above. One TABLENAME for one excel sheet, so the package should be dynamic.
Output: DimOrganization.xlsx (TABLENAME above, etc.)
Number Name Type Required Source
1 OrganizationKey int N AdventureWorksDW.dbo.DimOrganization
2 ParentOrganizationKey int Y AdventureWorksDW.dbo.DimOrganization
3 PercentageOfOwnership nvarchar(16) Y AdventureWorksDW.dbo.DimOrganization
4 OrganizationName nvarchar(50) Y AdventureWorksDW.dbo.DimOrganization
5 CurrencyKey int Y AdventureWorksDW.dbo.DimOrganization
In order to get the column info above from SQL, my SQL Task will be:
SELECT
ROW_NUMBER() OVER(ORDER BY (SELECT NULL)) [Number],
COLUMN_NAME Name,
DATA_TYPE +
CASE
WHEN CHARACTER_MAXIMUM_LENGTH IS NULL THEN ''
ELSE '(' + CAST(CHARACTER_MAXIMUM_LENGTH AS VARCHAR) +')'
END Type,
CASE IS_NULLABLE
WHEN 'NO' THEN 'N'
ELSE 'Y'
END AS Required,
IC.TABLE_CATALOG+'.'+IC.TABLE_SCHEMA+'.'+Ic.TABLE_NAME [Source]
FROM AdventureWorksDW.INFORMATION_SCHEMA.COLUMNS IC
WHERE TABLE_NAME = 'DimOrganization'
What should I do in SSIS to create each excel file for each TABLENAME above without using script?
I believe I need to add a variables and parameters to assign the WHERE TABLE_NAME = ? This way, if I change the TABLE_NAME variable and run it, if valid, it will retrieve the info from SQL, input metadata info into excel file and save the excel as filename? Any thought?
Related
Using PostgreSQL I can have multiple rows of json objects.
select (select ROW_TO_JSON(_) from (select c.name, c.age) as _) as jsonresult from employee as c
This gives me this result:
{"age":65,"name":"NAME"}
{"age":21,"name":"SURNAME"}
But in SqlServer when I use the FOR JSON AUTO clause it gives me an array of json objects instead of multiple rows.
select c.name, c.age from customer c FOR JSON AUTO
[{"age":65,"name":"NAME"},{"age":21,"name":"SURNAME"}]
How to get the same result format in SqlServer ?
By constructing separate JSON in each individual row:
SELECT (SELECT [age], [name] FOR JSON PATH, WITHOUT_ARRAY_WRAPPER)
FROM customer
There is an alternative form that doesn't require you to know the table structure (but likely has worse performance because it may generate a large intermediate JSON):
SELECT [value] FROM OPENJSON(
(SELECT * FROM customer FOR JSON PATH)
)
no structure better performance
SELECT c.id, jdata.*
FROM customer c
cross apply
(SELECT * FROM customer jc where jc.id = c.id FOR JSON PATH , WITHOUT_ARRAY_WRAPPER) jdata (jdata)
Same as Barak Yellin but more lazy:
1-Create this proc
CREATE PROC PRC_SELECT_JSON(#TBL VARCHAR(100), #COLS VARCHAR(1000)='D.*') AS BEGIN
EXEC('
SELECT X.O FROM ' + #TBL + ' D
CROSS APPLY (
SELECT ' + #COLS + '
FOR JSON PATH, WITHOUT_ARRAY_WRAPPER
) X (O)
')
END
2-Can use either all columns or specific columns:
CREATE TABLE #TEST ( X INT, Y VARCHAR(10), Z DATE )
INSERT #TEST VALUES (123, 'TEST1', GETDATE())
INSERT #TEST VALUES (124, 'TEST2', GETDATE())
EXEC PRC_SELECT_JSON #TEST
EXEC PRC_SELECT_JSON #TEST, 'X, Y'
If you're using PHP add SET NOCOUNT ON; in the first row (why?).
I am coding Python and Spark SQL in Databricks and I am using spark 2.4.5.
I have two tables.
Create table IF NOT EXISTS db_xsi_ed_faits_shahgholi_ardalan.Destination
(
id Int,
Name string,
Deleted int
) USING Delta;
Create table IF NOT EXISTS db_xsi_ed_faits_shahgholi_ardalan.Source
(
id Int,
Name string,
Deleted int
) USING Delta;
I need to ran a Merge command between my source and destination. I wrote below command
%sql
MERGE INTO db_xsi_ed_faits_shahgholi_ardalan.Destination AS D
USING db_xsi_ed_faits_shahgholi_ardalan.Source AS S
ON (S.id = D.id)
-- UPDATE
WHEN MATCHED AND S.Name <> D.Name THEN
UPDATE SET
D.Name = S.Name
-- INSERT
WHEN NOT MATCHED THEN
INSERT (id, Name, Deleted)
VALUES (S.id, S.Name, S.Deleted)
-- DELETE
WHEN NOT MATCHED BY SOURCE THEN
UPDATE SET
D.Deleted = 1
When i ran this command i have below error:
It seems that we do not have NOT MATCHED BY SOURCE in spark! I need a solution to do that.
I wrote this code but still i am looking for better approach
%sql
MERGE INTO db_xsi_ed_faits_shahgholi_ardalan.Destination AS D
USING db_xsi_ed_faits_shahgholi_ardalan.Source AS S
ON (S.id = D.id)
-- UPDATE
WHEN MATCHED AND S.Name <> D.Name THEN
UPDATE SET
D.Name = S.Name
-- INSERT
WHEN NOT MATCHED THEN
INSERT (id, Name, Deleted)
VALUES (S.id, S.Name, S.Deleted)
;
%sql
-- Logical delete
UPDATE db_xsi_ed_faits_shahgholi_ardalan.Destination
SET Deleted = 1
WHERE db_xsi_ed_faits_shahgholi_ardalan.Destination.id in
(
SELECT
D.id
FROM db_xsi_ed_faits_shahgholi_ardalan.Destination AS D
LEFT JOIN db_xsi_ed_faits_shahgholi_ardalan.Source AS S ON (S.id = D.id)
WHERE S.id is null
)
I have a stored procedure in AZURE SQL database.In that there is a requirement to insert the records into the remote table from #temp table.
As xxxx_table is in the remote database used sp_execute_remote.
below is the scenario:
Create Procedure SP1 parameter1, Parameter2
As
select Distinct B.column1, B.Column2
into #A
from (Query1
Union
Query2) B
if (select count(1) from #A) > 0
Begin
Exec sp_execute_remote #data_source_name = N'Remotedatabase',
#stmt = N'INSERT INTO [dbo].[xxxx_table]
SELECT DISTINCT
'xxx' AS 'column1',
'xxx as 'Column2',
'xxx' AS 'Column3',
'xxx' AS 'Column4',
'xxx' AS Column4
FROM #A A INNER JOIN table1 on A.Column1 = Table1.Column2'
End
)
Getting the syntax error as below:
Incorrect syntax near 'xxx'.
Where am i going wrong? or let me know if there is another way to achieve this.
If you need to dynamically build a string in SQL single-quote the whole sentence, or use 'some text' + 'another text' to concat sentences. If you must add single quote use a double single quote ''
Example:
DECLARE #param1 int;
DECLARE #param1 VARCHAR(10);
SET #param1 = 10;
SET #param2 = 'CCDOS87'
#Stmt = 'SELECT Field1 FROM TableName WHERE Field1 = '
+ CAST(#param1 AS VARCHAR(100))
+ ' AND Field1 = '''
+ param2
+ ''''; <- This is a single '
#stmt = N'INSERT INTO [dbo].[Error_table]
SELECT DISTINCT
xxx AS column1,
xxx as Column2,
xxx AS Column3,
xxx AS Column4,
xxx AS Environment
FROM #A A INNER JOIN table1 on A.Column1 = Table1.Column2'
update
If your tables are in different databases but in the same server use:
INSERT INTO SERVER.SCHEMA.TABLE_NAME
SELECT Something
FROM SERVER.SCHEMA.TABLE_NAME
I have a table T with CLOB column called XML_CLOB
Value in the column likes following:
<reportName>
<string>REPORT_A</string>
<string>REPORT_B</string>
<string>REPORT_C</string>
</reportName>
I'm trying to retrieve string value from this CLOB column and return in different rows. If I use
xmltype(xml_clob).extract('//reportName/string/text()').getstringval()
it outputs like 'REPORT_AREPORT_BREPORT_C' in the same row.
I also tried
extractValue(xmltype(xml_clob), '//reportName/string[1]')
but the problem is I don't know how much child values under tag
Is there anyway I can retrieve in different rows like:
1 REPORT_A
2 REPORT_B
3 REPORT_C
Many thanks in advance~
Oracle Setup:
CREATE TABLE table_name (xml_clob CLOB );
INSERT INTO table_name VALUES (
'<reportName>
<string>REPORT_A</string>
<string>REPORT_B</string>
<string>REPORT_C</string>
</reportName>'
);
Query 1:
SELECT x.string
FROM table_name t,
XMLTable('/reportName/string'
PASSING XMLType( t.xml_clob )
COLUMNS string VARCHAR2(50) PATH '/'
) x
Query 2:
SELECT EXTRACTVALUE( s.COLUMN_VALUE, '/string' ) AS string
FROM table_name t,
TABLE(
XMLSequence(
EXTRACT(
XMLType( t.xml_clob ),
'/reportName/string'
)
)
) s;
Output:
STRING
--------
REPORT_A
REPORT_B
REPORT_C
WITH test_table AS
(SELECT xmltype('<reportName>
<string>REPORT_A</string>
<string>REPORT_B</string>
<string>REPORT_C</string>
</reportName>' ) xml_clob
FROM dual
)
SELECT x.*
FROM test_table,
xmltable('/reportName/string'
passing test_table.xml_clob
columns report_name VARCHAR2(100) path 'text()') x
I need a way to generically take a table and copy its data into a new table--basically the same thing that SELECT * INTO does in regular SQL Server. Is there a way to do this in SQL Azure? I only have the existing and new table names at this point.
I encountered the same problem and the author's answer is not very detailed, so I will give some more information, on how i solved it.
I needed to duplicate tables that start with a given prefix ('from_') into new tables with prefix ('to_').
Generate CREATE Statement
I use this query (found on stackoverflow) to generate all CREATE statements, for every table that starts with 'from_' prefix.
select 'create table [' + so.name + '] (' + o.list + ')' + CASE WHEN tc.Constraint_Name IS NULL THEN '' ELSE 'ALTER TABLE ' + so.Name + ' ADD CONSTRAINT ' + tc.Constraint_Name + ' PRIMARY KEY ' + ' (' + LEFT(j.List, Len(j.List)-1) + ')' END as query
OBJECTPROPERTY(object_id(TABLE_NAME), 'TableHasIdentity') as tablehasidentity
from sysobjects so
cross apply
(SELECT
' ['+column_name+'] ' +
data_type + case data_type
when 'sql_variant' then ''
when 'text' then ''
when 'ntext' then ''
when 'decimal' then '(' + cast(numeric_precision as varchar) + ', ' + cast(numeric_scale as varchar) + ')'
else coalesce('('+case when character_maximum_length = -1 then 'MAX' else cast(character_maximum_length as varchar) end +')','') end + ' ' +
case when exists (
select id from syscolumns
where object_name(id)=so.name
and name=column_name
and columnproperty(id,name,'IsIdentity') = 1
) then
'IDENTITY(' +
cast(ident_seed(so.name) as varchar) + ',' +
cast(ident_incr(so.name) as varchar) + ')'
else ''
end + ' ' +
(case when IS_NULLABLE = 'No' then 'NOT ' else '' end ) + 'NULL ' +
case when information_schema.columns.COLUMN_DEFAULT IS NOT NULL THEN 'DEFAULT '+ information_schema.columns.COLUMN_DEFAULT ELSE '' END + ', '
from information_schema.columns where table_name = so.name
order by ordinal_position
FOR XML PATH('')) o (list)
left join
information_schema.table_constraints tc
on tc.Table_name = so.Name
AND tc.Constraint_Type = 'PRIMARY KEY'
cross apply
(select '[' + Column_Name + '], '
FROM information_schema.key_column_usage kcu
WHERE kcu.Constraint_Name = tc.Constraint_Name
ORDER BY
ORDINAL_POSITION
FOR XML PATH('')) j (list)
where xtype = 'U'
AND name NOT IN ('dtproperties') AND name like 'from_%'
This query results in a set of values:
['query'] = create table [from_users_roles] ( [uid] int NOT NULL DEFAULT ((0)), [rid] int NOT NULL DEFAULT ((0)), )ALTER TABLE from_users_roles ADD CONSTRAINT from_users_roles_pkey PRIMARY KEY ([uid], [rid])
['tablehasidentity'] = 1 or 0
Now replace the prefixes in the query 'from_' with 'to_' and the CREATE Statement is finished:
create table [to_users_roles] ( [uid] int NOT NULL DEFAULT ((0)), [rid] int NOT NULL DEFAULT ((0)), )ALTER TABLE to_users_roles ADD CONSTRAINT to_users_roles_pkey PRIMARY KEY ([uid], [rid]);
Create INSERT Statement
When you want to insert data from one table to another, you have to distinguish between two cases:
TablehasIdentity == 0
INSERT INTO to_users_roles SELECT * FROM from_users_roles
TablehasIdentity == 1
This case is a bit more complex. The statement requires a column list and IDENTITY_INSERT switched on.
DECLARE #Query nvarchar(4000)
DECLARE #columnlist nvarchar(4000)
// Result of this query e.g.: "[cid], [pid], [nid], [uid], [subject]"
SET #columnlist = (SELECT SUBSTRING((SELECT ', ' + QUOTENAME(COLUMN_NAME) FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = 'from_users_roles' ORDER BY ORDINAL_POSITION FOR XML path('')), 3, 200000))
SET #query ='SET IDENTITY_INSERT to_users_roles ON; INSERT INTO to_users_roles (' + #columnlist + ') SELECT ' + #columnlist + ' FROM from_users_roles; SET IDENTITY_INSERT to_users_roles OFF'
exec sp_executesql #query;
This worked out for me pretty well.
The latest version of Azure SQL DB, now in Preview, supports the SELECT INTO syntax and no longer requires a clustered index. For a detailed description of its features, and how to use it, see http://azure.microsoft.com/en-us/documentation/articles/sql-database-preview-whats-new/
After doing more research, it looks like there is no simple way to do this. You basically have to read the table's schema information and create the new table based on that.
Select into is now supported SQL DB V12. Just upgrade your server and start using the syntax.
I found a clever trick on this blog
Instead of using "select into" use "insert select".
First you have to create the destination table. To do this, right click on the source table in SQL Management Studio, and choose "Script Table as" -> "Create To" -> "New Query Window".
Then, change the name of the table in the query, and execute the query. Below is an example where I have added today's date to the new table, calling it "Entities_2015_08_24" (the old table was called "Entities"):
CREATE TABLE [dbo].[Entities_2015_08_24](
[Url] [nvarchar](max) NULL,
[ClientID] [nvarchar](max) NULL
)
Then, do a "insert select" from the old table (Entities) into the new table (Entities_2015_08_24):
INSERT INTO [dbo].[Entities_2015_08_24]
([Url]
,[ClientID]
)
SELECT
[Url]
,[ClientID]
FROM [dbo].[Entities]
Q: Did you try it?
Q: Did you look at the SQL Azure documentation
ADDENDUM
AFAIK, you cannot use select into syntax to "clone" a table in Azure SQL. Because Azure requires a clustered index, and select into has no provision for defining one.
Details, and a potential workaround, are here:
http://blogs.msdn.com/b/windowsazure/archive/2010/05/04/select-into-with-sql-azure.aspx