psycopg2.errors.InvalidFunctionDefinition: create function must specify volatility attribute (IMMUTABLE|STABLE|VOLATILE) - python-3.x

I'm trying to create a User defiend function in AWS Redshift using psycopg2, heres' the code:
CREATE OR REPLACE FUNCTION generate_create_table_statement(p_table_name varchar)
RETURNS text AS
$BODY$
DECLARE
v_table_ddl text;
column_record record;
BEGIN
FOR column_record IN
SELECT
b.nspname as schema_name,
b.relname as table_name,
a.attname as column_name,
pg_catalog.format_type(a.atttypid, a.atttypmod) as column_type,
CASE WHEN
(SELECT substring(pg_catalog.pg_get_expr(d.adbin, d.adrelid) for 128)
FROM pg_catalog.pg_attrdef d
WHERE d.adrelid = a.attrelid AND d.adnum = a.attnum AND a.atthasdef) IS NOT NULL THEN
'DEFAULT '|| (SELECT substring(pg_catalog.pg_get_expr(d.adbin, d.adrelid) for 128)
FROM pg_catalog.pg_attrdef d
WHERE d.adrelid = a.attrelid AND d.adnum = a.attnum AND a.atthasdef)
ELSE
''
END as column_default_value,
CASE WHEN a.attnotnull = true THEN
'NOT NULL'
ELSE
'NULL'
END as column_not_null,
a.attnum as attnum,
e.max_attnum as max_attnum
FROM
pg_catalog.pg_attribute a
INNER JOIN
(SELECT c.oid,
n.nspname,
c.relname
FROM pg_catalog.pg_class c
LEFT JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
WHERE c.relname ~ ('^('||p_table_name||')$')
AND pg_catalog.pg_table_is_visible(c.oid)
ORDER BY 2, 3) b
ON a.attrelid = b.oid
INNER JOIN
(SELECT
a.attrelid,
max(a.attnum) as max_attnum
FROM pg_catalog.pg_attribute a
WHERE a.attnum > 0
AND NOT a.attisdropped
GROUP BY a.attrelid) e
ON a.attrelid=e.attrelid
WHERE a.attnum > 0
AND NOT a.attisdropped
ORDER BY a.attnum
LOOP
IF column_record.attnum = 1 THEN
v_table_ddl:='CREATE TABLE '||column_record.schema_name||'.'||column_record.table_name||' (';
ELSE
v_table_ddl:=v_table_ddl||',';
END IF;
IF column_record.attnum <= column_record.max_attnum THEN
v_table_ddl:=v_table_ddl||chr(10)||
' '||column_record.column_name||' '||column_record.column_type||' '||column_record.column_default_value||' '||column_record.column_not_null;
END IF;
END LOOP;
v_table_ddl:=v_table_ddl||');';
RETURN v_table_ddl;
END;
$BODY$
LANGUAGE plpgsql;
I'm getting this error:
psycopg2.errors.InvalidFunctionDefinition: create function must specify volatility attribute
(IMMUTABLE|STABLE|VOLATILE)
Where exactly do I put this attribute in my query? I've looked around but found no examples that did this.

Please check this link:
https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_FUNCTION.html
You forgot to specify the volatility attribute. It need to be specified after the 'return' statement like this:
CREATE OR REPLACE FUNCTION generate_create_table_statement(p_table_name varchar)
RETURNS text
stable
AS
$BODY$
Furthermore it seems, that plpgsql is not a supported language for function creation at redshift.
Supported languages:
$$ LANGUAGE { plpythonu | sql }

Related

Postgresql query not accept array of string in where section

At the next query:
CREATE OR REPLACE FUNCTION public.test_function(statuses character varying)
RETURNS TABLE("Id" uuid,"FirstName" character varying)
LANGUAGE plpgsql
AS $function$
declare
temptable text;
*status* text;
begin
if(statuses is not null and statuses != '') then
select string_agg(item, ',')
into temptable
from
(select '''' || Trim(regexp_split_to_table) || '''' as item
from regexp_split_to_table(statuses, E',')
where regexp_split_to_table != '') A;
* status* := temptable;
end if;
return query select
mp.mid "Id",
mp.first_name "FirstName"
from data_table as mp
where mp.status in (*status*);
END;
$function$;
status aren't recognized as value on the query execution.
What is necessary for that postgresql detect the string value for example ['New','Evaluation']

Python how to pass variables to SQLite complex SQL update Query

I have this SQL query that I confirmed works in SQLite. It updates two columns in the Table. I have 144 columns that need to be updated using the same query. How can I, using Python, pass along variables so I can use the same query to update all of them?
Here is my query to update one column:
UPDATE GBPAUD_TA AS t1
SET _1m_L3_Time = COALESCE(
(
SELECT
MIN(
CASE t1.Action
WHEN 'Buy' THEN CASE WHEN (t2._1M_55 >= t2.Low AND t2._1M_55 < t2.Open) THEN t2.Date_Time END
WHEN 'Sell' THEN CASE WHEN (t2._1M_55 <= t2.High AND t2._1M_55 < t2.Open) THEN t2.Date_Time END
END
)
FROM GBPAUD_DATA t2
WHERE t2.Date_Time >= t1.Open_Date AND t2.Date_Time <= t1.New_Closing_Time
),
t1._1m_L3_Time
);
UPDATE GBPAUD_TA
SET _1m_L3_Price = (SELECT _1M_55
FROM GBPAUD_DATA
WHERE Date_Time = GBPAUD_TA._1m_L3_Time)
where EXISTS (SELECT _1M_55
FROM GBPAUD_DATA
WHERE Date_Time = GBPAUD_TA._1m_L3_Time)
Here is my query showing the variables that I would need to automatically insert:
UPDATE GBPAUD_TA AS t1
SET Variable1 = COALESCE(
(
SELECT
MIN(
CASE t1.Action
WHEN 'Buy' THEN CASE WHEN (t2.Variable2 >= t2.Low AND t2.Variable2< t2.Open) THEN t2.Date_Time END
WHEN 'Sell' THEN CASE WHEN (t2.Variable2 <= t2.High AND t2.Variable2< t2.Open) THEN t2.Date_Time END
END
)
FROM GBPAUD_DATA t2
WHERE t2.Date_Time >= t1.Open_Date AND t2.Date_Time <= t1.New_Closing_Time
),
t1.Variable1
);
UPDATE GBPAUD_TA
SET Variable3 = (SELECT Variable2
FROM GBPAUD_DATA
WHERE Date_Time = GBPAUD_TA.Variable1)
where EXISTS (SELECT Variable2
FROM GBPAUD_DATA
WHERE Date_Time = GBPAUD_TA.Variable1)
I have a total of 3 Variables.
Based upon googling and reading, I found a possible way by using host variables: I use the "?" in place of the variable, combine the variables into a tuple, and then use "executemany()"?
I tried this, but it did not work. It gave me an error:
"cursor.executemany(sql_update_query, SLTuple)
OperationalError: near "?": syntax error"
So what should I do? Any guidance is much appreciated!
Found the answer after I figured out the proper terminology: string formatting and interloping. Found the answer here.

I wrote a function to break a string into parts delimitted by $ Now I need to access it with select from but getting error

I have created following function to split a given string delimited by $.
I want to call the function as part of SQL query and get the result as rows.
CREATE OR REPLACE FUNCTION string_tokenize2
( p_string IN CLOB
-- p_delim in varchar2
)
RETURN SYS_REFCURSOR
AS
cur1 SYS_REFCURSOR;
BEGIN
OPEN cur1 FOR
SELECT regexp_substr(p_string
,'[^$]+'
,1
,LEVEL) AS str
FROM sys.dual
CONNECT BY LEVEL <= regexp_count(p_string
,'\$') + 1;
RETURN cur1;
END string_tokenize2;
/
However when I tried it using in a SQL resulted in following error.
Kindly assist me how to proceed here ,
select string_tokenize2('')from dual;
put your IN variable in the ( )
Do you have any constraint to stick to the function as it is ? OR
Why not create an object type and change the function to return a table type and then use it in SQL ?
--create the type
CREATE OR REPLACE TYPE string_tokenize2_obj IS TABLE OF VARCHAR2(4000);
--Function changes
CREATE OR REPLACE FUNCTION string_tokenize2(p_string IN CLOB)
RETURN string_tokenize2_obj AS
l_tab string_tokenize2_obj;
BEGIN
SELECT to_char(str)
BULK COLLECT
INTO l_tab
FROM (SELECT regexp_substr(p_string
,'[^$]+'
,1
,LEVEL) str
FROM dual
CONNECT BY LEVEL <= regexp_count(p_string
,'\$') + 1);
RETURN l_tab;
END string_tokenize2;
/
SQL> SELECT column_value str FROM TABLE(string_tokenize2('abc$def$geh$ijkl'));
STR
--------------------------------------------------------------------------------
abc
def
geh
ijkl
SQL>
You can either
1) use xmltable and xmltype to read data from sys_refcursor functions results if you need that data in select statement:
ie you can use the same your function without any changes, but read it using xmltable+xmltype:
select *
from xmltable(
'/ROWSET/ROW'
passing xmltype(string_tokenize2('1$2$3$4'))
columns
str varchar2(100) path 'STR'
);
Results:
STR
---------------------------------------
1
2
3
4
2) or use plsql implicit results if you are just need to get data from any client: https://oracle-base.com/articles/12c/implicit-statement-results-12cr1
Example:
CREATE OR REPLACE PROCEDURE string_tokenize2
( p_string IN CLOB
-- p_delim in varchar2
)
AS
cur1 SYS_REFCURSOR;
BEGIN
OPEN cur1 FOR
SELECT regexp_substr(p_string
,'[^$]+'
,1
,LEVEL) AS str
FROM sys.dual
CONNECT BY LEVEL <= regexp_count(p_string
,'\$') + 1;
DBMS_SQL.RETURN_RESULT(cur1);
END string_tokenize2;
/
begin
string_tokenize2('1,2,3,4');
end;
Result:
SQL> exec string_tokenize2('1,2,3,4');
ResultSet #1
STR
---------------------------------------
1,2,3,4
3) or just return sys_refcursor to client app as a bind variable for fetching:
begin
:res := string_tokenize2('1,2,3,4');
end;
Example in sql*plus:
SQL> var res refcursor
SQL> exec :res := string_tokenize2('1$2$3$4');
SQL> print res
STR
--------------------
1
2
3
4
4) Or varchar2 collections/varrays as suggested by other users

Does Azure Databricks SparkSQL support recursive queries

I am moving data from SQL Server to Azure data lake gen2 and converting SQL queries with recursive queries.
Here is an sample SQL query with recursion using CTE (Common Table Expression)
WITH RECURSIVE AS BOM
(SELECT p.MItemId AS RootPartNumber,
p.MItemId AS PartNumber,
NULL AS ParentPartNumber,
0 AS BomLevel,
1.0 AS Quantity
FROM PartItem p
UNION ALL
SELECT BOM.RootPartNumber,
CAST(BSM.ChildItem AS string) AS PartNumber,
CAST(DB.PartNumber AS string) AS ParentPartNumber,
BOM.BomLevel + 1 as BomLevel,
BSM.Quantity AS Quantity
FROM PartItemBomList BSM
INNER JOIN BOM ON BOM.PartNumber = BSM.ParentItem
INNER JOIN PartItem p ON p.MItemId = BSM.ChildItem
WHERE BSM.IsDeleted = 0
)
SELECT * FROM BOM
I have tried changing the query embedding the recursion within a FROM clause as shown below with no success.
SELECT * FROM
(SELECT p.MItemId AS RootPartNumber,
p.MItemId AS PartNumber,
NULL AS ParentPartNumber,
0 AS BomLevel,
1.0 AS Quantity
FROM PartItem p
WHERE p.PartType = 'Cloud-OrderableAssembly'
UNION ALL
SELECT BOM.RootPartNumber,
CAST(BSM.ChildItem AS string) AS PartNumber,
CAST(DB.PartNumber AS string) AS ParentPartNumber,
BOM.BomLevel + 1 as BomLevel,
BSM.Quantity AS Quantity
FROM PartItemBomList BSM
INNER JOIN BOM ON BOM.PartNumber = BSM.ParentItem
INNER JOIN PartItem p ON p.MItemId = BSM.ChildItem
WHERE BSM.IsDeleted = 0
) as BOM
Here is the error I get from Azure Databricks session.
Error in SQL statement: AnalysisException: Table or view not found: BOM; line 16 pos 22
The problem is here
INNER JOIN BOM ON BOM.PartNumber = BSM.ParentItem
This is the inner query and as I understand the BOM is defined outside , and so this part of the query runs BOM does not exist .
If I were you i could have tried to fix the below query by running directly on the SQL .
The way BOM is referenced below is not correct
SELECT p.MItemId AS RootPartNumber,
p.MItemId AS PartNumber,
NULL AS ParentPartNumber,
0 AS BomLevel,
1.0 AS Quantity
FROM PartItem p
WHERE p.PartType = 'Cloud-OrderableAssembly'
UNION ALL
SELECT BOM.RootPartNumber,
CAST(BSM.ChildItem AS string) AS PartNumber,
CAST(DB.PartNumber AS string) AS ParentPartNumber,
BOM.BomLevel + 1 as BomLevel,
BSM.Quantity AS Quantity
FROM PartItemBomList BSM
INNER JOIN BOM ON BOM.PartNumber = BSM.ParentItem
INNER JOIN PartItem p ON p.MItemId = BSM.ChildItem
WHERE BSM.IsDeleted = 0

Could not add table 'SELECT('

Error given when adding query that runs in SQL developer but not in MS Query. Seems to not like my nested query.
Code I am using:
SELECT ORDER_DATE
,SALES_ORDER_NO
,CUSTOMER_PO_NUMBER
,DELIVER_TO
,STATUS
,ITEM_NUMBER
,DESCRIPTION
,ORD_QTY
,SUM(QUANTITY) AS ON_HAND
,PACKAGE_ID
,PACKAGE_STATUS
,MAX(TRAN_DATE) AS LAST_TRANSACTION
,MIN(DAYS) AS DAYS
FROM (
SELECT TRUNC(SH.ORDER_DATE) AS ORDER_DATE
,SH.SALES_ORDER_NO
,SH.CUSTOMER_PO_NUMBER
,SH.SHIP_CODE AS DELIVER_TO
,SH.STATUS
,SB.ITEM_NUMBER
,IM.DESCRIPTION
,SB.ORD_QTY
,BID.QUANTITY
,SPM.PACKAGE_ID
,CASE
WHEN SPM.SHIPPED = 'Y'
THEN 'SHIPPED'
WHEN SPM.STATUS = 'C'
THEN 'PACKED'
WHEN BID.QUANTITY IS NOT NULL
THEN 'AVAILABLE'
WHEN BID.QUANTITY IS NULL
THEN 'UNAVAILABLE'
END AS PACKAGE_STATUS
,CASE
WHEN SPM.SHIPPED = 'Y'
THEN TRUNC(SPM.BILLING_DATE)
WHEN SPM.STATUS = 'C'
THEN TRUNC(SPM.END_TIME)
WHEN BID.QUANTITY IS NOT NULL
THEN TRUNC(BID.ACTIVATION_TIME)
END AS TRAN_DATE
,CASE
WHEN SPM.SHIPPED = 'Y'
THEN ROUND(SYSDATE - SPM.BILLING_DATE, 0)
WHEN SPM.STATUS = 'C'
THEN ROUND(SYSDATE - SPM.END_TIME, 0)
WHEN BID.QUANTITY IS NOT NULL
THEN ROUND(SYSDATE - BID.ACTIVATION_TIME, 0)
END AS DAYS
FROM SO_HEADER SH
LEFT JOIN SO_BODY SB ON SB.SO_HEADER_TAG = SH.SO_HEADER_TAG
LEFT JOIN SO_PACKAGE_MASTER SPM ON SPM.PACKAGE_ID = SB.PACKAGE_ID
LEFT JOIN ITEM_MASTER IM ON IM.ITEM_NUMBER = SB.ITEM_NUMBER
LEFT JOIN V_BIN_ITEM_DETAIL BID ON BID.ITEM_NUMBER = SB.ITEM_NUMBER
WHERE SH.ORDER_TYPE = 'MSR'
AND (
SB.REASON_CODE IS NULL
OR SB.REASON_CODE NOT LIKE 'CANCEL%'
)
AND (
IM.DESCRIPTION NOT LIKE '%CKV%'
OR IM.DESCRIPTION IS NULL
AND IM.ITEM_NUMBER IS NOT NULL
)
)
WHERE PACKAGE_STATUS <> 'SHIPPED'
GROUP BY ORDER_DATE
,SALES_ORDER_NO
,CUSTOMER_PO_NUMBER
,DELIVER_TO
,STATUS
,ITEM_NUMBER
,DESCRIPTION
,ORD_QTY
,PACKAGE_ID
,PACKAGE_STATUS
ORDER BY (
CASE PACKAGE_STATUS
WHEN 'AVAILABLE'
THEN 1
WHEN 'UNAVAILABLE'
THEN 2
WHEN 'PACKED'
THEN 3
END
)
,LAST_TRANSACTION;
Is there any option I can select that will allow me to run this query?

Resources