I have to generate set of xml files by querying oracle db table based on some condition.
the xml file structure little complex and their is a limit on file size if limit croees new file need to be created. what is the best approch to this. using select XMLElemnt in oracle plsql or doin it in C# after querying data from db using cursor?
performamce also to be considered and this process need to be done faster.
In oracle 11.2 you can transform the results from a ref cursor into a XML(CLOB) using dbms_xmlgen.
EXAMPLE:
PROCEDURE GetXMLByRefCursor
(
irfCursor in REF CURSOR,
oclXMLDocument out nocopy clob,
isbRootTag in varchar2 default null,
isbResultTag in varchar2 default null,
inuNullHandling in number default dbms_xmlgen.DROP_NULLS
)
IS
nuContext dbms_xmlgen.ctxHandle;
BEGIN
nuContext:= dbms_xmlgen.newContext(irfCursor);
dbms_xmlgen.setPrettyPrinting(nuContext, false);
dbms_xmlgen.setNullHandling(nuContext, inuNullHandling);
if (isbRootTag is not null) then
dbms_xmlgen.setRowSetTag(nuContext, isbRootTag);
end if;
if (isbResultTag is not null) then
dbms_xmlgen.setRowTag(nuContext, isbResultTag);
end if;
oclXMLDocument := dbms_xmlgen.getXML(nuContext);
dbms_xmlgen.closeContext(nuContext);
END GetXMLFromRefCursor;
In PLSQL you cannot easily manage resulting XML size.
Performance-wise, it can be better either in PLSQL or in C#, that depends on the speed of the .NET-side machine and the speed of the OracleDB-side machine.
Also, if you create XML in PLQSL and then return in to the caller C# tier, you'll have to do it via CLOBs. Since this would involve temporary CLOBs, some copying in the Oracle's temp tablespace might be involved, thus decreasing performance.
All in all, I suggest doing it rather in C#.
Related
Using Prisma client 3 I'm trying to create a stored procedure.
The motivation behind it is:
I need to query a table that will be created on run time.
To do this, I need to use dynamic queries,
and I read that stored procedures will be the better practice in this case (pass the table name as a parameter).
I would like for each member of my team to have the updated version of the stored procedure (like all the tables in Prisma)
So, what I've decided to do is to create the stored procedure with prisma.$executeRaw when the app starts and call it when I need.
The code:
let prisma = new PrismaClient();
let res = await prisma.$executeRawUnsafe(`
CREATE PROCEDURE \`module-events\`.GetAllProducts()
BEGIN
select 555;
END
`);
The result:
Invalid `prisma.$executeRaw()` invocation:
Raw query failed. Code: `1295`. Message: `This command is not supported in the prepared statement protocol yet`
As you can see the $executeRawUnsafe() returns the same results. Is there any way to create a stored procedure with Prisma? Is there a way to run a "free style" query that is not limited by Prisma?
I understood from this answer that it is possible to create the stored procedure:
You could also use $executeRaw to generate the stored procedure or use the tool/CLI of your choice.
I am trying to execute this query but as userdefined(Create type) types are not supportable in azure data warehouse. and i want to use it in stored procedure.
CREATE TYPE DataTypeforCustomerTable AS TABLE(
PersonID int,
Name varchar(255),
LastModifytime datetime
);
GO
CREATE PROCEDURE usp_upsert_customer_table #customer_table DataTypeforCustomerTable READONLY
AS
BEGIN
MERGE customer_table AS target
USING #customer_table AS source
ON (target.PersonID = source.PersonID)
WHEN MATCHED THEN
UPDATE SET Name = source.Name,LastModifytime = source.LastModifytime
WHEN NOT MATCHED THEN
INSERT (PersonID, Name, LastModifytime)
VALUES (source.PersonID, source.Name, source.LastModifytime);
END
GO
CREATE TYPE DataTypeforProjectTable AS TABLE(
Project varchar(255),
Creationtime datetime
);
GO
CREATE PROCEDURE usp_upsert_project_table #project_table DataTypeforProjectTable READONLY
AS
BEGIN
MERGE project_table AS target
USING #project_table AS source
ON (target.Project = source.Project)
WHEN MATCHED THEN
UPDATE SET Creationtime = source.Creationtime
WHEN NOT MATCHED THEN
INSERT (Project, Creationtime)
VALUES (source.Project, source.Creationtime);
END
Is there any alternative way to do this.
You've got a few challenges there, because most of what you're trying to convert is not the way to do things on ASDW.
First, as you point out, CREATE TYPE is not supported, and there is no equivalent alternative.
Next, the code appears to be doing single inserts to a table. That's really bad on ASDW, performance will be dreadful.
Next, there's no MERGE statement (yet) for ASDW. That's because UPDATE is not the best way to handle changing data.
And last, stored procedures work a little differently on ASDW, they're not compiled, but interpreted each time the procedure is called. Stored procedures are great for big chunks of table-level logic, but not recommended for high volume calls with single-row operations.
I'd need to know more about the use case to make specific recommendations, but in general you need to think in tables rather than rows. In particular, focus on the CREATE TABLE AS (CTAS) way of handling your ELT.
Here's a good link, it shows how the equivalent of a Merge/Upsert can be handled using a CTAS:
https://learn.microsoft.com/en-us/azure/sql-data-warehouse/sql-data-warehouse-develop-ctas#replace-merge-statements
As you'll see, it processes two tables at a time, rather than one row. This means you'll need to review the logic that called your stored procedure example.
If you get your head around doing everything in CTAS, and separately around Distribution, you're well on your way to having a high performance data warehouse.
Temp tables in Azure SQL Data Warehouse have a slightly different behaviour to box product SQL Server or Azure SQL Database - they exist at the session level. So all you have to do is convert your CREATE TYPE statements to temp tables and split the MERGE out into separate INSERT / UPDATE / DELETE statements as required.
Example:
CREATE TABLE #DataTypeforCustomerTable (
PersonID INT,
Name VARCHAR(255),
LastModifytime DATETIME
)
WITH
(
DISTRIBUTION = HASH( PersonID ),
HEAP
)
GO
CREATE PROCEDURE usp_upsert_customer_table
AS
BEGIN
-- Add records which do not already exist
INSERT INTO customer_table ( PersonID, Name, LastModifytime )
SELECT PersonID, Name, LastModifytime
FROM #DataTypeforCustomerTable AS source
WHERE NOT EXISTS
(
SELECT *
FROM customer_table target
WHERE source.PersonID = target.PersonID
)
...
Simply load the temp table and execute the stored proc. See here for more details on temp table scope.
If you are altering a large portion of the table then you should consider the CTAS approach to create a new table, then rename it as suggested by Ron.
Im create new trigger audit using store procedure cause want flexible column in the trigger audit
im using Oracle 12 C ..
CREATE OR REPLACE PROCEDURE DBADMIN.TEST3 (OUTPUT OUT SYS_REFCURSOR,
TABLE_NAME IN VARCHAR2)
IS
N NUMBER;
BEGIN
N := 0;
EXECUTE IMMEDIATE '
CREATE OR REPLACE TRIGGER DBADMIN.TA_EMPLOYEES3
AFTER INSERT OR DELETE OR UPDATE
ON DBADMIN.EMPLOYEES
FOR EACH ROW
DECLARE
SID VARCHAR2 (30);
BEGIN
SELECT SYS_CONTEXT ('' USERENV '', '' IP_ADDRESS '') INTO IP FROM DUAL;
SELECT SEQ#
INTO SID1
FROM v$session
WHERE audsid = (SELECT USERENV ('' SESSIONID '') FROM DUAL);
IF INSERTING
THEN
INSERT INTO DBADMIN.DBLOG_MONITORING_DETAIL2 (SID,
COLUMNS,
OLDVALUE,
NEWVALUE)
VALUES (SID1,
i.COLUMN_NAME,
'for row in (SELECT column_name from user_tab_columns where table_name=''EMPLOYEES'' loop
execute immediate '':old.row.column_name '';
end loop;
/
32 26 PLS-00103: Encountered the symbol "FOR"
i think im bad logic in my script .. can give me better logic or repair my script its better ?? .
In Oracle, you should really rarely use dynamic SQL to create database objects. People sometimes abuse that functionality which is - in my opinion - what you're trying to do. The fact that you can do it doesn't mean that you should do it, e.g. you can poke your eye with a pencil, but you shouldn't do that.
Rule of thumb with dynamic SQL:
forget about EXECUTE IMMEDIATE
Create a local variable (a large VARCHAR2 or CLOB, depending on what you're doing)
compose the statement (CREATE TRIGGER in your case) and
store it into that variable
display it on the screen using DBMS_OUTPUT.PUT_LINE
copy/paste it and run it as a standalone statement
if it succeeds, you've done a good job so you can now use it in EXECUTE IMMEDIATE
if it fails, you'll have to try a little bit harder, debug it, fix errors and repeat the cycle
As of the error you got: this:
for row in (SELECT column_name from user_tab_columns where table_name=''EMPLOYEES'' loop
execute immediate '':old.row.column_name '';
end loop;
is invalid as the INSERT INTO target; you can't mix SQL and PL/SQL that way. It won't work at all, regardless dynamic SQL you use. Besides, it is obvious that it is wrong (missing closing bracket, what exactly are you executing immediately? old_row.column_name? How would you execute a column name?
In my opinion (once again), you shouldn't do it that way. If you want to create a trigger, do it - but not dynamically.
I wrote a transformation in xquery which unquotes an XML-String and inserts a element with its content. This works fine.
I need to create a collection dependant on the root element of this element as well. I can't do this on new documents as xdmp:document-add-collections() is not working. How do I add the collection to new Documents in transformations?
Here my ServerSide xQuery Code:
xquery version "1.0-ml";
module namespace transform = "http://marklogic.com/rest-api/transform/smtextdocuments";
import module namespace mem = "http://xqdev.com/in-mem-update" at '/MarkLogic/appservices/utils/in-mem-update.xqy';
declare function transform(
$context as map:map,
$params as map:map,
$content as document-node()
) as document-node()
{
let $uri := base-uri($content)
let $doccont := $content/smtextdocuments/documentcontent
let $newcont := xdmp:unquote($doccont)
let $contname := node-name($newcont/*)
let $result := if ( exists($content/smtextdocuments/content))
then mem:node-replace($content/smtextdocuments/content, <content>11{$newcont}</content>)
else mem:node-insert-after($doccont, <content>{$newcont}</content>)
let $log := xdmp:log($content)
return (
$result,
xdmp:document-add-collections($uri, fn:string($contname)),
xdmp:document-remove-collections($uri, "raw")
)
};
The script ist running with the java api (4.0.4) create methode via parameter ServerTransform transform. As per documentation the transformation script is running before the document is stored in the Database.
Its a new document; I need to transform the content and then create the collection.
I can see the document after the create, the content is available. Just the collection is missing. I can try xdmp:document-insert method but is it correct writing the document while create is running?.
The transform mechanism of the Java API / REST API takes responsibility for the document write. At present, there's no way for the transform to supply collections to the writer. That would be a reasonable request for enhancement.
The transform shouldn't attempt to write the document, because the writer would also attempt to write the same document.
One alternative would be to transform the document in Java before writing it and specify the collection as part of the write request.
Another alternative would be to rewrite the transform as a resource service extension, implement the write within the resource service extension, and modify the Java client to send the document to the resource service extension.
Depending on the model, a final alternative might be to use a range index on an element within the document to collect documents into sets instead of using a collection on the document.
Hoping that helps,
What do you mean by "new documents"? Is the document already inserted into the MarkLogic database at the time you are adjusting the collections of it? If not, you may want to modify your return to ($result, xdmp:document-insert($uri, $result, xdmp:default-permissions(), fn:string($contname)) ) for that case.
Otherwise, can you edit your question to share the error or problem more specifically you are facing?
It is a pity that REST transforms do not allow this, like MLCP transforms do. Until changed you have the options drawn by ehennum, or you can consider delaying adding of collections to a pre- or post-commit trigger. It takes some overhead, but it sometimes makes perfect sense to do something like that in a trigger, since it makes sure it is always enforced, and a good place to do content validation, audit logging, and things like that as well.
HTH!
I've been using SqlList() to receive result sets from SPs and it is handy.
var people = db.SqlList<Person>("EXEC GetRockstarsAged #age", new { "age", 42 });
but how can I use this OrmLite to get multiple result sets from a single SP?
Using the above approach only seems to retrieve the first result set.
Unfortunately, ServiceStack.OrmLite does not support multiple result sets unless combined with Dapper. ServiceStack MARS (Multiple Active Result Sets) using ORMLite and Output Parameters
Alternatively, you can use the .Net SqlCommand.
Return multiple recordsets from stored proc in C#
ServiceStack.OrmLite V4 notes: https://github.com/ServiceStack/ServiceStack.OrmLite
ServiceStack.OrmLite V3 notes: https://github.com/ServiceStack/ServiceStack.OrmLite/tree/v3