I have a couple of stored procedures that create different temporary tables.
At the end of the procedure i drop them (know that is not required, but it's good practice).
The stored procedures are executed as a part of a SSIS package. I got 4 different SQL jobs that execute the same SSIS package running in parallel.
When logging into the Azure portal and using the performance recommendation feature, I get a recommendation to fix the schema issues. It states an Sql error code 208. According to documentation that means "object not found".
Temporary tables are valid within the scope of the stored procedure and should get a unique name in the database, so I do not think where are any conflicts.
I have no idea what causes this, and the stored procedures seems to work alright. Anyone know what could be the cause here?
Simplified sample of one of the procedures:
SET NOCOUNT ON;
CREATE TABLE #tmpTransEan
(
Ean_Art_Str_id BIGINT ,
Artikler_id BIGINT
);
INSERT INTO #tmpTransEan
( Ean_Art_Str_id ,
Artikler_id
)
SELECT DISTINCT
eas.Ean_Art_Str_id ,
a.Artikler_id
FROM dbo.Artikkel_Priser ap
JOIN Ean_Art_Str eas ON eas.artikler_id = ap.Artikler_id
JOIN wsKasse_Til_Kasselogg ktk ON eas.Ean_Art_Str_id = ktk.ID_Primary
JOIN dbo.Artikler a ON a.Artikler_id = eas.artikler_id
JOIN dbo.Felles_Butikker b ON b.Butikker_id = ap.butikker_id
WHERE ktk.ID_Table = OBJECT_ID('Ean_Art_Str')
AND LEN(a.Artikkelnr) >= 8
AND ktk.Tidspunkt >= #tidspunkt
AND ( ( ap.butikker_id = #nButikker_id1
AND #Alle_artikler_til_kasse = 'N'
)
OR ( b.Databaser_id = #Databaser_id
AND #Alle_artikler_til_kasse = 'J'
)
)
AND b.Akt_kode = 'A'
AND a.Akt_kode = 'A'
AND a.Databaser_id IN ( -1, #Databaser_id )
SELECT DISTINCT
a.Artikkelnr ,
s.Storrelse ,
eas.* ,
EAN_12 = LEFT(eas.EAN_13, 12)
FROM dbo.Ean_Art_Str eas
JOIN #tmpTransEan t ON t.Artikler_id = eas.artikler_id
JOIN Artikler a ON a.Artikler_id = eas.artikler_id
JOIN dbo.Felles_Storrelser s ON s.Storrelser_id = eas.storrelser_id
DROP TABLE #tmpTransEan;
END;
Related
I have a variable SCRIPT which has two to three DML statements. I want to run them sequentially after connecting to my Oracle DB. I have tried the below but it is failing with below error
c.execute(SCRIPT)
cx_Oracle.DatabaseError: ORA-00933: SQL command not properly ended
Below is the piece of code tried.
SCRIPT="""UPDATE IND_AFRO.DRIVER
SET Emp_Id = 1000, update_user_id = 'RIBST-4059'
WHERE Emp_Id IN (SELECT Emp_Id
FROM IND_AFRO.DRIVER Ddq
WHERE NOT EXISTS
(SELECT 1
FROM IND_AFRO_AF.EMPLOYEE
WHERE Emp_Id = Ddq.Emp_Id)
AND Functional_Area_Cd = 'DC');
UPDATE IND_AFRO.APPOINTMENTS
SET Emp_Id = 1000, update_user_id = 'RIBST-4059'
WHERE Emp_Id IN (SELECT Emp_Id
FROM IND_AFRO.APPOINTMENTS Ddq
WHERE NOT EXISTS
(SELECT 1
FROM IND_AFRO_AF.EMP
WHERE Emp_Id = Ddq.Emp_Id));
UPDATE IND_AFRO.ar_application_for_aid a
SET a.EMP_ID = 1000
WHERE NOT EXISTS
(SELECT 1
FROM IND_AFRO_AF.EMP
WHERE emp_id = a.emp_id);"""
conn = cx_Oracle.connect(user=r'SYSTEM', password='ssadmin', dsn=CONNECTION)
c = conn.cursor()
c.execute(SCRIPT)
c.close()
The execute() and executemany() functions only work on one SQL or PL/SQL statement.
You can wrap the three statements in a PL/SQL BEGIN/END block like:
SQL> begin
2 insert into test values(1);
3 update test set a = 2;
4 end;
5 /
PL/SQL procedure successfully completed.
Alternatively you can split up your string into individual statements. If the statements originate from a file, you can write a wrapper to read file and execute each statement. This is a lot easier if you restrict the SQL syntax (particularly regarding line terminators). For an example, see https://github.com/oracle/python-cx_Oracle/blob/master/samples/SampleEnv.py#L116
However this means calling execute() more times, which isn't as efficient as the first solution.
Some of my tables are of type REPLICATE. I would these tables to be actually replicated (not pending) before I start querying my data. This will help me avoid data movement.
I have a script, which I found online, which runs in a loop and do a SELECT TOP 1 on all the tables which are set for replication, but sometimes the script runs for hours. It may seem as the server sometimes won't trigger replication even if you do a SELECT TOP 1 from foo.
How can you force SQL Datawarehouse to complete replication?
The script looks something like this:
begin
CREATE TABLE #tbl
WITH
( DISTRIBUTION = ROUND_ROBIN
)
AS
SELECT
ROW_NUMBER() OVER(
ORDER BY
(
SELECT
NULL
)) AS Sequence
, CONCAT('SELECT TOP(1) * FROM ', s.name, '.', t.[name]) AS sql_code
FROM sys.pdw_replicated_table_cache_state AS p
JOIN sys.tables AS t
ON t.object_id = p.object_id
JOIN sys.schemas AS s
ON t.schema_id = s.schema_id
WHERE p.[state] = 'NotReady';
DECLARE #nbr_statements INT=
(
SELECT
COUNT(*)
FROM #tbl
), #i INT= 1;
WHILE #i <= #nbr_statements
BEGIN
DECLARE #sql_code NVARCHAR(4000)= (SELECT
sql_code
FROM #tbl
WHERE Sequence = #i);
EXEC sp_executesql #sql_code;
SET #i+=1;
END;
DROP TABLE #tbl;
SET #i = 0;
WHILE
(
SELECT TOP (1)
p.[state]
FROM sys.pdw_replicated_table_cache_state AS p
JOIN sys.tables AS t
ON t.object_id = p.object_id
JOIN sys.schemas AS s
ON t.schema_id = s.schema_id
WHERE p.[state] = 'NotReady'
) = 'NotReady'
BEGIN
IF #i % 100 = 0
BEGIN
RAISERROR('Replication in progress' , 0, 0) WITH NOWAIT;
END;
SET #i = #i + 1;
END;
END
Henrik, if 'select top 1' doesn't trigger a replicated table build, then that would be a defect. Please file a support ticket.
Without looking at your system, it is impossible to know exactly what is going on. Here are a couple of things that could be in factoring into extended build time to look into:
The replicated tables are large (size, not necessarily rows) requiring long build times.
There are a lot of secondary indexes on the replicated table requiring long build times.
Replicated table builds require statirc20 (2 concurrency slots). If the concurrency slots are not available, the build will queue behind other running queries.
The replicated tables are constantly being modified with inserts, updates and deletes. Modifications require the table to be built again.
The best way is to run a command like this as part of the job which creates/updates the table:
select top 1 * from <table>
That will force its redistribution at the correct time, without the slow loop through the stored procedure.
I need to get the list of tables used in a stored procedure,However in Azure Datawarehouse sp_depends is not supported.
The other alternative I thought of having is to get the stored proc code from INFORMATION_SCHEMA.ROUTINES and then run a script to get the [schema].[tablename] from the stored procedure definition but here the issue is in storing the whole stored proc into a variable. VARCHAR(MAX)has a limit of 8000 to store and if my proc exceeds this limit then I wont be able to get the complete table list.
Try using sys.sql_expression_dependencies. The following query may help you:
SELECT ReferencingObjectType = o1.type,
ReferencingObject = SCHEMA_NAME(o1.schema_id)+'.'+o1.name,
ReferencedObject = SCHEMA_NAME(o2.schema_id)+'.'+ed.referenced_entity_name,
ReferencedObjectType = o2.type
FROM sys.sql_expression_dependencies ed
INNER JOIN sys.objects o1
ON ed.referencing_id = o1.object_id
INNER JOIN sys.objects o2
ON ed.referenced_id = o2.object_id
WHERE o1.type in ('P','TR','V', 'TF')
ORDER BY ReferencingObjectType, ReferencingObject
I am stuck with a query which takes more time in JOIN, I want to use WHERE EXISTS in place of JOIN since as performance wise EXISTS takes less time than it.
I have modified the query and it's executing as per expectation but I am not able to use sub query's columns in my main query
Here is my query
SELECT MAX(st.grade_level::integer) AS grades ,
scl.sid AS org_sourced_id
FROM schedules_53b055b75cd237fde3af904c1e726e12 sch
LEFT JOIN schools scl ON(sch.school_id=scl.school_id)
AND scl.batch_id=sch.batch_id
AND scl.client_id = sch.client_id
AND sch.run_id = scl.run_id
WHERE EXISTS
(SELECT t.term_id,t.abbreviation
FROM terms t
WHERE (sch.term = t.term_id)
AND t.batch_id=sch.batch_id
AND t.client_id = sch.client_id
AND t.run_id = sch.run_id)
AND EXISTS
(SELECT st.grade_level,
st.sid
FROM students st
WHERE (sch.student_id=st.sid)
AND st.batch_id= sch.batch_id
AND st.client_id = sch.client_id
AND st.run_id = sch.run_id)
GROUP BY scl.sid ,
sch.course_name ,
sch.course_number,
sch.school_id
And I am getting this error:
ERROR: missing FROM-clause entry for table "st"
SQL state: 42P01
Character: 29
I have only used one column here just for sample but I have to use more fields from sub query.
My main aim is that how can I achieve this with EXISTS or any alternate solution which is more optimal as performance wise
I am using pg module on Node.js since as back end I am using Node.js.
UPDATE
Query with JOIN
SELECT MAX(st.grade_level::integer) AS grades ,
scl.sid AS org_sourced_id
FROM schedules_53b055b75cd237fde3af904c1e726e12 sch
LEFT JOIN schools scl ON(sch.school_id=scl.school_id)
AND scl.batch_id=sch.batch_id
AND scl.client_id = sch.client_id
AND sch.run_id = scl.run_id
LEFT JOIN terms t ON (sch.term = t.term_id)
AND t.batch_id=sch.batch_id
AND t.client_id = sch.client_id
AND t.run_id = sch.run_id
LEFT JOIN students st ON (sch.student_id=st.sid)
AND st.batch_id= sch.batch_id
AND st.client_id = sch.client_id
AND st.run_id = sch.run_id
GROUP BY scl.sid ,
sch.course_name ,
sch.course_number,
sch.school_id
I want to perform a simple join on two tables (BusinessUnit and UserBusinessUnit), so I can get a list of all BusinessUnits allocated to a given user.
The first attempt works, but there's no override of Select which allows me to restrict the columns returned (I get all columns from both tables):
var db = new KensDB();
SqlQuery query = db.Select
.From<BusinessUnit>()
.InnerJoin<UserBusinessUnit>( BusinessUnitTable.IdColumn, UserBusinessUnitTable.BusinessUnitIdColumn )
.Where( BusinessUnitTable.RecordStatusColumn ).IsEqualTo( 1 )
.And( UserBusinessUnitTable.UserIdColumn ).IsEqualTo( userId );
The second attept allows the column name restriction, but the generated sql contains pluralised table names (?)
SqlQuery query = new Select( new string[] { BusinessUnitTable.IdColumn, BusinessUnitTable.NameColumn } )
.From<BusinessUnit>()
.InnerJoin<UserBusinessUnit>( BusinessUnitTable.IdColumn, UserBusinessUnitTable.BusinessUnitIdColumn )
.Where( BusinessUnitTable.RecordStatusColumn ).IsEqualTo( 1 )
.And( UserBusinessUnitTable.UserIdColumn ).IsEqualTo( userId );
Produces...
SELECT [BusinessUnits].[Id], [BusinessUnits].[Name]
FROM [BusinessUnits]
INNER JOIN [UserBusinessUnits]
ON [BusinessUnits].[Id] = [UserBusinessUnits].[BusinessUnitId]
WHERE [BusinessUnits].[RecordStatus] = #0
AND [UserBusinessUnits].[UserId] = #1
So, two questions:
- How do I restrict the columns returned in method 1?
- Why does method 2 pluralise the column names in the generated SQL (and can I get round this?)
I'm using 3.0.0.3...
So far my experience with 3.0.0.3 suggests that this is not possible yet with the query tool, although it is with version 2.
I think the preferred method (so far) with version 3 is to use a linq query with something like:
var busUnits = from b in BusinessUnit.All()
join u in UserBusinessUnit.All() on b.Id equals u.BusinessUnitId
select b;
I ran into the pluralized table names myself, but it was because I'd only re-run one template after making schema changes.
Once I re-ran all the templates, the plural table names went away.
Try re-running all 4 templates and see if that solves it for you.