I am using voltdb-5.0.2 and starting database as voltdb create --zookeeper portno
I have loaded a procedure from java class in voltdb.The procedure takes arguments - tableName , columnName and integer value .
This is the error on sqlcmd line - when I call the procedure as -
exec CheckDeleteProcedure arg1(NewTable) arg2 arg3.
VOLTDB ERROR: USER ABORT
Attempted to queue DML adhoc sql 'Delete from NewTable where id <=?;' from read only procedure
at checkdel.CheckDeleteProcedure.run(CheckDeleteProcedure.java:56)
Thanks for adding that detail.
The voltQueueExperimental() method is an experimental feature that is not supported.
The procedure was categorized as read-only at compile time because it does not contain any compile time constant SQLStmt objects that involve writes. At run-time, you are getting this error because a read-only procedure is not allowed to do any writes. Procedures that involve writes are handled differently (two phase commit, etc).
You could run this DELETE statement as an Ad Hoc query from the client interface.
Another option, if there are a small number of tables you want this procedure to handle would be to declare separate SQLStmt objects for each table. This would be fine as far as making the SQLStmt's compile-time constants:
final String sql1 = "DELETE FROM ";
final String sql2 = " WHERE col1 <= ?;";
final SQLStmt qry1 = new SQLStmt(sql1 + "EMPLOYEE" + sql2);
final SQLStmt qry2 = new SQLStmt(sql1 + "DEPT" + sql2);
I should also caution you about potentially deleting many records at once. I wrote a blog post that goes over that topic.
Related
I have a variable SCRIPT which has two to three DML statements. I want to run them sequentially after connecting to my Oracle DB. I have tried the below but it is failing with below error
c.execute(SCRIPT)
cx_Oracle.DatabaseError: ORA-00933: SQL command not properly ended
Below is the piece of code tried.
SCRIPT="""UPDATE IND_AFRO.DRIVER
SET Emp_Id = 1000, update_user_id = 'RIBST-4059'
WHERE Emp_Id IN (SELECT Emp_Id
FROM IND_AFRO.DRIVER Ddq
WHERE NOT EXISTS
(SELECT 1
FROM IND_AFRO_AF.EMPLOYEE
WHERE Emp_Id = Ddq.Emp_Id)
AND Functional_Area_Cd = 'DC');
UPDATE IND_AFRO.APPOINTMENTS
SET Emp_Id = 1000, update_user_id = 'RIBST-4059'
WHERE Emp_Id IN (SELECT Emp_Id
FROM IND_AFRO.APPOINTMENTS Ddq
WHERE NOT EXISTS
(SELECT 1
FROM IND_AFRO_AF.EMP
WHERE Emp_Id = Ddq.Emp_Id));
UPDATE IND_AFRO.ar_application_for_aid a
SET a.EMP_ID = 1000
WHERE NOT EXISTS
(SELECT 1
FROM IND_AFRO_AF.EMP
WHERE emp_id = a.emp_id);"""
conn = cx_Oracle.connect(user=r'SYSTEM', password='ssadmin', dsn=CONNECTION)
c = conn.cursor()
c.execute(SCRIPT)
c.close()
The execute() and executemany() functions only work on one SQL or PL/SQL statement.
You can wrap the three statements in a PL/SQL BEGIN/END block like:
SQL> begin
2 insert into test values(1);
3 update test set a = 2;
4 end;
5 /
PL/SQL procedure successfully completed.
Alternatively you can split up your string into individual statements. If the statements originate from a file, you can write a wrapper to read file and execute each statement. This is a lot easier if you restrict the SQL syntax (particularly regarding line terminators). For an example, see https://github.com/oracle/python-cx_Oracle/blob/master/samples/SampleEnv.py#L116
However this means calling execute() more times, which isn't as efficient as the first solution.
Some of my tables are of type REPLICATE. I would these tables to be actually replicated (not pending) before I start querying my data. This will help me avoid data movement.
I have a script, which I found online, which runs in a loop and do a SELECT TOP 1 on all the tables which are set for replication, but sometimes the script runs for hours. It may seem as the server sometimes won't trigger replication even if you do a SELECT TOP 1 from foo.
How can you force SQL Datawarehouse to complete replication?
The script looks something like this:
begin
CREATE TABLE #tbl
WITH
( DISTRIBUTION = ROUND_ROBIN
)
AS
SELECT
ROW_NUMBER() OVER(
ORDER BY
(
SELECT
NULL
)) AS Sequence
, CONCAT('SELECT TOP(1) * FROM ', s.name, '.', t.[name]) AS sql_code
FROM sys.pdw_replicated_table_cache_state AS p
JOIN sys.tables AS t
ON t.object_id = p.object_id
JOIN sys.schemas AS s
ON t.schema_id = s.schema_id
WHERE p.[state] = 'NotReady';
DECLARE #nbr_statements INT=
(
SELECT
COUNT(*)
FROM #tbl
), #i INT= 1;
WHILE #i <= #nbr_statements
BEGIN
DECLARE #sql_code NVARCHAR(4000)= (SELECT
sql_code
FROM #tbl
WHERE Sequence = #i);
EXEC sp_executesql #sql_code;
SET #i+=1;
END;
DROP TABLE #tbl;
SET #i = 0;
WHILE
(
SELECT TOP (1)
p.[state]
FROM sys.pdw_replicated_table_cache_state AS p
JOIN sys.tables AS t
ON t.object_id = p.object_id
JOIN sys.schemas AS s
ON t.schema_id = s.schema_id
WHERE p.[state] = 'NotReady'
) = 'NotReady'
BEGIN
IF #i % 100 = 0
BEGIN
RAISERROR('Replication in progress' , 0, 0) WITH NOWAIT;
END;
SET #i = #i + 1;
END;
END
Henrik, if 'select top 1' doesn't trigger a replicated table build, then that would be a defect. Please file a support ticket.
Without looking at your system, it is impossible to know exactly what is going on. Here are a couple of things that could be in factoring into extended build time to look into:
The replicated tables are large (size, not necessarily rows) requiring long build times.
There are a lot of secondary indexes on the replicated table requiring long build times.
Replicated table builds require statirc20 (2 concurrency slots). If the concurrency slots are not available, the build will queue behind other running queries.
The replicated tables are constantly being modified with inserts, updates and deletes. Modifications require the table to be built again.
The best way is to run a command like this as part of the job which creates/updates the table:
select top 1 * from <table>
That will force its redistribution at the correct time, without the slow loop through the stored procedure.
Unable to get the complete table data from stored procedure. For easy understanding, I have provided a simple stored procedure. Internally, we have different logic and different stored procedure that will return a table.
ALTER PROCEDURE [dbo].[SP_GetResultWeeklyUnitSold]
#fromDate VARCHAR(10),
#toDate VARCHAR(10)
AS
BEGIN
SELECT *
FROM SOOrder
WHERE OrderDate BETWEEN #fromDate AND #toDate
End
var pars = new List<PXSPParameter>();
PXSPParameter fromDate = new PXSPInParameter("#fromDate",
PXDbType.VarChar, Filters.Current.StartDate);
PXSPParameter toDate = new PXSPInParameter("#toDate", PXDbType.VarChar,
Filters.Current.EndDate);
pars.Add(fromDate);
pars.Add(toDate);
var results = PXDatabase.Execute("SP_WeeklyUnitSold", pars.ToArray());
To get the result table from stored procedure.
I didn't find any way of direct getting information from stored procedure of the whole table.
The only working way I found, was serializing output into xml at stored procedure level, and then deserializing it in Acumatica.
But for case that you've described I can't understand, why not to use ordinary PXSelect? It will be much simpler and I'm sure much more efficient.
I am working with an Oracle 11g database, release 11.2.0.3.0 - 64 bit production
I have written the following procedure which uses a cursor to collect batches of benefit_ids (which are simply of type NUMBER) from a table called benefit_info. For each benefit_id within each batch, I need to obtain the associated customers and then perform various calculations etc. So far I have the following:
CREATE OR REPLACE PROCEDURE ben_correct(in_bulk_collect_limit IN PLS_INTEGER DEFAULT 1000)
IS
TYPE ben_identity_rec IS RECORD
(
life_scd_id NUMBER,
benefit_id NUMBER
);
TYPE ben_identity_col IS TABLE OF ben_identity_rec INDEX BY PLS_INTEGER;
life_col ben_identity_col;
ben_id NUMBER;
CURSOR benefit_cur
IS
SELECT benefit_id FROM benefit_info;
TYPE benefit_ids_t IS TABLE OF NUMBER INDEX BY PLS_INTEGER;
benefit_ids benefit_ids_t;
PROCEDURE get_next_set_of_incoming(out_benefit_ids OUT NOCOPY benefit_ids_t)
IS
BEGIN
FETCH benefit_cur
BULK COLLECT INTO out_benefit_ids
LIMIT in_bulk_collect_limit;
END;
BEGIN
OPEN benefit_cur;
LOOP
get_next_set_of_incoming(benefit_ids);
/*
The code below is too slow as each benefit_id is considered
individually. Want to change FOR LOOP into LEFT JOIN of benefit_ids
*/
FOR indx IN 1 .. benefit_ids.count LOOP
ben_id := benefit_ids(indx);
SELECT c.life_scd_id, c.benefit_id
BULK COLLECT INTO life_col
FROM customer c
WHERE c.benefit_id = ben_id;
-- Now do further processing with life_col
END LOOP;
EXIT WHEN benefit_ids.count = 0;
END LOOP;
CLOSE benefit_cur;
END;
/
As indicated in the code above, the FOR indx IN 1 .. LOOP is VERY slow, particularly as there are millions of benefit_ids. However, I am aware I can replace the entire FOR LOOP with something like:
SELECT c.life_scd_id, c.benefit_id
BULK COLLECT INTO life_col
FROM customer c
LEFT JOIN table(benefit_ids) b
WHERE b.benefit_id IS NOT NULL;
However, for that to work I think I need to declare an Object type at the schema level as I think in the SELECT query you can join on pure tables or collections of objects. Therefore, from the procedure I remove
TYPE benefit_ids_t IS TABLE OF NUMBER INDEX BY PLS_INTEGER;
and instead at the schema level I have defined
CREATE OR REPLACE TYPE ben_id FORCE AS object
(
benefit_id number
);
CREATE OR REPLACE TYPE benefit_ids_t FORCE AS TABLE OF ben_id;
My revised code essentially becomes:
CREATE OR REPLACE PROCEDURE ben_correct(in_bulk_collect_limit IN PLS_INTEGER DEFAULT 1000)
IS
sql_str VARCHAR2(1000);
TYPE ben_identity_rec IS RECORD
(
life_scd_id NUMBER,
benefit_id NUMBER
);
TYPE ben_identity_col IS TABLE OF ben_identity_rec INDEX BY PLS_INTEGER;
life_col ben_identity_col;
CURSOR benefit_cur
IS
SELECT benefit_id FROM benefit_info;
--- benefit_ids_t has now been declared at schema level
benefit_ids benefit_ids_t;
PROCEDURE get_next_set_of_incoming(out_benefit_ids OUT NOCOPY benefit_ids_t)
IS
BEGIN
FETCH benefit_cur
BULK COLLECT INTO out_benefit_ids
LIMIT in_bulk_collect_limit;
END;
BEGIN
OPEN benefit_cur;
LOOP
get_next_set_of_incoming(benefit_ids);
sql_str := 'SELECT c.life_scd_id, c.benefit_id
FROM customer c
LEFT JOIN table(benefit_ids) b
WHERE b.benefit_id IS NOT NULL';
EXECUTE IMMEDIATE sql_str BULK COLLECT INTO life_col;
-- Now do further processing with life_col
EXIT WHEN benefit_ids.count = 0;
END LOOP;
CLOSE benefit_cur;
END;
/
However, this generates ORA-24344 and PLS-00386 errors, ie type mismatch found at 'OUT_BENEFIT_IDS' between FETCH cursor and INTO variables.
I sort of understand that it is complaining that benefit_ids_t is now a table of ben_ids, which are in turn objects of type number, which is in't quite the same as a table of numbers.
I've tried various attempts at resolving the issues, but I can't seem to quite get it right. Any help would be gratefully appreciated.
Also, any general comments to improve are welcome.
You don't need your table type to be of an object containing a number field, it can just be a table of numbers:
CREATE OR REPLACE TYPE benefit_ids_t FORCE AS TABLE OF number;
Or you can use a built-in type like sys.odcinumberlist, but having your own type under your control isn't a bad thing.
You don't want to use dynamic SQL though; this:
sql_str := 'SELECT c.life_scd_id, c.benefit_id
FROM customer c
LEFT JOIN table(benefit_ids) b
WHERE b.benefit_id IS NOT NULL';
EXECUTE IMMEDIATE sql_str BULK COLLECT INTO life_col;
won't work because benefit_ids isn't in scope when that dynamic statement is executed. You can just do it statically:
SELECT c.life_scd_id, c.benefit_id
BULK COLLECT INTO life_col
FROM table(benefit_ids) b
JOIN customer c
ON c.benefit_id = b.column_value;
which is closer to what you had in your original code.
Your EXIT is also in the wrong place - it will try to process rows in a loop when it doesn't find any. I wouldn't bother with the separate fetch procedure at all, it's easier to follow with the fetch directly in the loop:
BEGIN
OPEN benefit_cur;
LOOP
FETCH benefit_cur
BULK COLLECT INTO benefit_ids
LIMIT in_bulk_collect_limit;
EXIT WHEN benefit_ids.count = 0;
SELECT c.life_scd_id, c.benefit_id
BULK COLLECT INTO life_col
FROM table(benefit_ids) b
JOIN customer c
ON c.benefit_id = b.column_value;
-- Now do further processing with life_col
END LOOP;
CLOSE benefit_cur;
END;
If you did really want your object type, you could keep that, but you would need to make your cursor return instances of that object, via its default constructor:
CURSOR benefit_cur
IS
SELECT ben_id(benefit_id) FROM benefit_info;
The customer query join would then be:
SELECT c.life_scd_id, c.benefit_id
BULK COLLECT INTO life_col
FROM table(benefit_ids) b
JOIN customer c
ON c.benefit_id = b.benefit_id;
As it's an object type you can refer to it's field name, benefit_id, rather than the generic column_value from the scalar type table.
I am new to both Cassandra (2.0) and QlikView (11).
I have two keyspaces (tables) with large amount of data in Cassandra and I want to load them to QlikView.
Since I can not load the entire set, filtering is necessary.
// In QlikView's edit script
ODBC CONNECT TO [DataStax Cassandra ODBC DSN64];
LOAD idsession,
logintime,
"h_id" as hid;
SQL SELECT *
FROM Cassandra.test.sessions
WHERE logintime > '2015-06-09'
ALLOW FILTERING;
LOAD idhost,
site;
SQL SELECT *
FROM Cassandra.test.hosts
WHERE idhost in hid;
The second query does not work, error from qlikview line 3:16 no viable alternative at input 'hid'.
My question: is it possible to get the h_ids from the first query and only collect the corresponding entities from the second table?
I assume that you can't do an Exists in the DataSyntax ODBC which may help. DataStax doc
This could be done with an external program like (C#) but I really want to do this in QlikView's script file:
// Not complete code
query = select * from sessions where loginTime > '2015-06-09';
foreach (var id in query) {
query2 = "select * from hosts where idhost = " + i;
}
EDIT
This can be solved when loading XML files:
TableA:
LOAD id,
itema
FROM
[C:\test1data.xlsx]
(ooxml, embedded labels);
TableB:
LOAD idb,
itemb,
ida
FROM
[C:\test2data.xlsx]
(ooxml, embedded labels) where(Exists (id,ida));
EDIT2
Besides the great answer from #i_saw_drones another solutions is to loop through ids.
For i = 1 to NoOfRows('Sessions')
Let cur_id = Peek('hid',i - 1,'Sessions');
LOAD
idhost,
site;
SQL SELECT *
FROM Cassandra.test.hosts
WHERE idhost = $(cur_id);
NEXT i
Nevertheless was the performance not the great. It took about 30 minutes to load around 300 K lines from Cassandra. The same queries were tested in a C# program with the connector and it took 9 sec. But that was just the query. Then you should write it to XML and then load it to QlikView.
The reason that the second query fails is because the WHERE clause is expecting to find a literal string list of values to look "in". For example:
LOAD
idhost,
site;
SQL SELECT *
FROM Cassandra.test.hosts
WHERE idhost in ('ID1', 'ID2', 'ID3', 'ID4');
The hid field returned by the first query is a QlikView list and as such cannot be immediately coerced into a string. We have to do a little more scripting to obtain a list of values from the first query in literal form, and then add that to the second query as part of the WHERE clause. The easiest way to do this is to concatenate all of your hids into a string and then use the string as part of your WHERE IN clause.
ODBC CONNECT TO [DataStax Cassandra ODBC DSN64];
MyData:
LOAD
idsession,
logintime,
"h_id" as hid;
SQL SELECT *
FROM Cassandra.test.sessions
WHERE logintime > '2015-06-09'
ALLOW FILTERING;
hid_entries:
LOAD
chr(39) & hids & chr(39) as hids;
LOAD
concat(hid, chr(39) & ',' & chr(39)) as hids;
LOAD DISTINCT
hid
RESIDENT MyData;
LET hid_values = '(' & peek('hids',0,'hid_entries') & ')';
DROP TABLE hid_entries;
LOAD
idhost,
site;
SQL SELECT *
FROM Cassandra.test.hosts
WHERE idhost in $(hid_values);