I am trying to create a temp table in Synapse SQL using CTAS
IF OBJECT_ID('tempdb..#temp') IS NOT NULL DROP TABLE #temp
CREATE TABLE #temp
AS SELECT [State], LocalizedLabel FROM StateMetadata WHERE EntityName ='incident'
and getting an error message
Incorrect syntax near the keyword 'SELECT'.
Total execution time: 00:00:00.533
Error message snip
If I run SELECT statement by itself as highlighted it works
SELECT [State], LocalizedLabel FROM StateMetadata WHERE EntityName ='incident'
Select statement that works fine
and if I create temp table without using AS, it works fine as well
IF OBJECT_ID('tempdb..#temp') IS NOT NULL DROP TABLE #temp
CREATE TABLE #temp (
statecode int,
statecodename nvarchar(64)
)
SELECT * FROM #temp
Create temp table without using AS
I have tried brackets and everything I could. Not sure what I am possibly doing wrong. I am new to Synapse Analytics, Any help on this would be highly appreciated.
Thanks
Nidhi
CTAS requires a distribution option while creating. Include distribution options while creating temp tables using CTAS.
IF OBJECT_ID('tempdb..#temp') IS NOT NULL DROP TABLE #temp
CREATE TABLE #temp
WITH
(
DISTRIBUTION = ROUND_ROBIN
)
AS
SELECT StateName, StateCode FROM SP WHERE Region ='US';
select * from #temp
Related
This is my 1st time working with Azure synapse and It seems Select Insert is not working, is there any workaround for this one, where I will just use select statement and then dump it into a temporary table?
here are the error prompted
The query references an object that is not supported in distributed processing mode.
and this is my query
Select *
Into #Temp1
FROM [dbo].[TblSample]
This is the azure synapse we are currently using
ondemand-sql.azuresynapse.net
In Synapse On-Demand, the use of Temporary Tables is limited. In your case I am assuming that dbo.TblSample is an External Table which is possibly why you are facing this restriction.
Instead of using a Temp Table, can you either just JOIN the TblSample directly or use a CTE if you are SELECTing specific rows and columns?
I have a table called tbl1 in azure data bricks and I want to perform simple unpivot operation using SQL. I am new to SQL and DataBricks. I followed online tutorial to perform unpivot operation. Based on that I came up with below syntax. But. i am getting this You have an error in your SQL syntax; it seems the error is around: 'unpivot( height for details IN (ucol1, ucol2, ucol3))' error continuously.
SQL Syntax for unpivot operation.
%sql
Select date_format(X.Date,'dd-MMM')Datee
,X.width
,X.height
,X.details
,X.col1
From
(
Select
Datee,width,B.details,col1,height, from tbl1 A,
Unpivot
(
height
for details IN (
ucol1, ucol2, ucol3
)) B
GROUP BY Datee,width,B.Details,height,col1
)X
Is there anything wrong with the above SQL syntax?
Any hint would be appreciable.
Please let me know if you need any further details.
You can use stack in Spark SQL to unpivot, eg
SELECT
DATE_FORMAT( Date, 'dd-MMM') x,
STACK( 3, ucol1, ucol2, ucol3 )
FROM tbl1;
It would be helpful if you provided some simple sample data and expected results as it's not 100% clear what you need as your query does not work in any language.
Is there any Insert overwrite mode in postgres sql like below.
INSERT OVERWRITE INTO TABLE table2 select * FROM table1;
PostgreSQL has the TRUNCATE command to wipe the contents of a table but keep the table itself. You would have to use two statements:
TRUNCATE table2;
INSERT INTO table2 SELECT * FROM table1;
If you want to INSERT new records and UPDATE existing records you can use the ON CONFLICT argument of the INSERT statement:
INSERT INTO table2 (id, name)
(SELECT id, name FROM table1)
ON CONFLICT (id_pkey) DO UPDATE SET name = EXCLUDED.name;
You need to have a primary key or UNIQUE constraint to perform the conflict check. Full details can be viewed in the INSERT Statement documentation, https://www.postgresql.org/docs/current/sql-insert.html
You can also choose to DO NOTHING on conflict which has the advantage of protecting against inserting duplicate records.
I created a hive table on top of a parquet folder written via spark. In one test server it is running fine and giving out results (hive version 2.6.5.196) but in production it gives no records (hive 2.6.5.179). Could someone please point out what the exact issue could be?
If you created the table on top of an existing partition structure, you have to make it known to the table that there are partitions at this location.
MSCK REPAIR TABLE table_name; -- adds missing partitions
SELECT * FROM table_name; -- should return records now
This problem shouldn't happen if there are only files in that location, and if they are the expected format.
You can verify with:
SHOW CREATE TABLE table_name; -- to see the expected format
created hive table on top of a parquet folder written via spark.
Check for the databases that you are using is available or not using
show databases;
check the ddl of the table that you have created on your test server and the other that is there on production
show create table table_name;
Make sure that both the ddl exactly matches.
Do msck repair table table_name to load the incremental data or the data from all the partitions
select * from table_name to view records
I had a managed hive table and moved it to a different database using the following command:
alter table_name rename to new_db.table_name
The table was successfully moved and all the data is under the database now. The table is shown fine in HIVE. However when I try to read the table from Spark, it can read the schema but there is no content in there. That is, the count returns zero! What has happened? How can I fix this issue?
I loading it in spark using the following code:
val t = sqlContext.table("new_db.table_name")
Sometimes just altering the name isn't enough, I had to also alter the location.
sqlContext.sql("""
ALTER TABLE new_table
SET LOCATION "hdfs://.../new_location"
""")
And refresh the table in Spark for good measure
sqlContext.sql("""
REFRESH TABLE new_table
""")
You can double-check if the location is correct w/ describe formatted new_table