When using Redshift, I would like to get the names of all the procedure that were created in a schema, along with their definition.
I know you can use the SHOW PROCEDURE command to get the definition but that requires to have the procedure name.
In SVV_TABLE there is only information regarding tables and view but not procedure.
So if anyone knows how to get that ?
Redshift doesn't have a system view for that yet but you can use tbe PG_PROC table and join it with pg_namespace to filter on schema name.
select proname, proargnames, prosrc
from PG_PROC
join pg_namespace on pg_namespace.oid = pg_proc.pronamespace
where nspname = 'your_schema_name' and procsecdef = true;
The procsecdef = true is to get only stored procedure definition otherwise you also get python UDF.
Related
Using Prisma client 3 I'm trying to create a stored procedure.
The motivation behind it is:
I need to query a table that will be created on run time.
To do this, I need to use dynamic queries,
and I read that stored procedures will be the better practice in this case (pass the table name as a parameter).
I would like for each member of my team to have the updated version of the stored procedure (like all the tables in Prisma)
So, what I've decided to do is to create the stored procedure with prisma.$executeRaw when the app starts and call it when I need.
The code:
let prisma = new PrismaClient();
let res = await prisma.$executeRawUnsafe(`
CREATE PROCEDURE \`module-events\`.GetAllProducts()
BEGIN
select 555;
END
`);
The result:
Invalid `prisma.$executeRaw()` invocation:
Raw query failed. Code: `1295`. Message: `This command is not supported in the prepared statement protocol yet`
As you can see the $executeRawUnsafe() returns the same results. Is there any way to create a stored procedure with Prisma? Is there a way to run a "free style" query that is not limited by Prisma?
I understood from this answer that it is possible to create the stored procedure:
You could also use $executeRaw to generate the stored procedure or use the tool/CLI of your choice.
I have created stored procedure in snowflake which is executed fine in snowflake UI and also from server by using snowsql. Now I want to execute procedure from python program, I tried to execute from python, here are the steps that I have followed:
establish the connection to snowflake ( successfully able to connect.)
cs = ctx.cursor()
Used appropriate role,warehouse,database and schema.
tried to execute procedure like this:
cs.execute("call test_proc('value1', 'value2')")
x = cs.fetchall()
print(x)
But getting an erorr:
snowflake.connector.errors.ProgrammingError: 002140 (42601): SQL
compilation error: Unknown function test_proc
Can you please help me to resolve this problem.
Thanks,
When connecting to Snowflake using Python connector you could define DATABASE/SCHEMA
conn = snowflake.connector.connect(
user=USER,
password=PASSWORD,
account=ACCOUNT,
warehouse=WAREHOUSE,
database=DATABASE,
schema=SCHEMA
);
Once you have it set up, you could call your stored procedure without using fully-qualified name:
cs.execute("call test_proc('value1', 'value2')");
Alternative way is:
Using the Database, Schema, and Warehouse
Specify the database and schema in which you want to create tables. Also specify the warehouse that will provide resources for executing DML statements and queries.
For example, to use the database testdb, schema testschema and warehouse tiny_warehouse (created earlier):
conn.cursor().execute("USE WAREHOUSE tiny_warehouse_mg")
conn.cursor().execute("USE DATABASE testdb_mg")
conn.cursor().execute("USE SCHEMA testdb_mg.testschema_mg")
Actually, I have to have command like this
cs.execute("call yourdbname.schemaname.test_proc('value1', 'value2')")
and It is working as expected.
Thanks
I am trying to execute this query but as userdefined(Create type) types are not supportable in azure data warehouse. and i want to use it in stored procedure.
CREATE TYPE DataTypeforCustomerTable AS TABLE(
PersonID int,
Name varchar(255),
LastModifytime datetime
);
GO
CREATE PROCEDURE usp_upsert_customer_table #customer_table DataTypeforCustomerTable READONLY
AS
BEGIN
MERGE customer_table AS target
USING #customer_table AS source
ON (target.PersonID = source.PersonID)
WHEN MATCHED THEN
UPDATE SET Name = source.Name,LastModifytime = source.LastModifytime
WHEN NOT MATCHED THEN
INSERT (PersonID, Name, LastModifytime)
VALUES (source.PersonID, source.Name, source.LastModifytime);
END
GO
CREATE TYPE DataTypeforProjectTable AS TABLE(
Project varchar(255),
Creationtime datetime
);
GO
CREATE PROCEDURE usp_upsert_project_table #project_table DataTypeforProjectTable READONLY
AS
BEGIN
MERGE project_table AS target
USING #project_table AS source
ON (target.Project = source.Project)
WHEN MATCHED THEN
UPDATE SET Creationtime = source.Creationtime
WHEN NOT MATCHED THEN
INSERT (Project, Creationtime)
VALUES (source.Project, source.Creationtime);
END
Is there any alternative way to do this.
You've got a few challenges there, because most of what you're trying to convert is not the way to do things on ASDW.
First, as you point out, CREATE TYPE is not supported, and there is no equivalent alternative.
Next, the code appears to be doing single inserts to a table. That's really bad on ASDW, performance will be dreadful.
Next, there's no MERGE statement (yet) for ASDW. That's because UPDATE is not the best way to handle changing data.
And last, stored procedures work a little differently on ASDW, they're not compiled, but interpreted each time the procedure is called. Stored procedures are great for big chunks of table-level logic, but not recommended for high volume calls with single-row operations.
I'd need to know more about the use case to make specific recommendations, but in general you need to think in tables rather than rows. In particular, focus on the CREATE TABLE AS (CTAS) way of handling your ELT.
Here's a good link, it shows how the equivalent of a Merge/Upsert can be handled using a CTAS:
https://learn.microsoft.com/en-us/azure/sql-data-warehouse/sql-data-warehouse-develop-ctas#replace-merge-statements
As you'll see, it processes two tables at a time, rather than one row. This means you'll need to review the logic that called your stored procedure example.
If you get your head around doing everything in CTAS, and separately around Distribution, you're well on your way to having a high performance data warehouse.
Temp tables in Azure SQL Data Warehouse have a slightly different behaviour to box product SQL Server or Azure SQL Database - they exist at the session level. So all you have to do is convert your CREATE TYPE statements to temp tables and split the MERGE out into separate INSERT / UPDATE / DELETE statements as required.
Example:
CREATE TABLE #DataTypeforCustomerTable (
PersonID INT,
Name VARCHAR(255),
LastModifytime DATETIME
)
WITH
(
DISTRIBUTION = HASH( PersonID ),
HEAP
)
GO
CREATE PROCEDURE usp_upsert_customer_table
AS
BEGIN
-- Add records which do not already exist
INSERT INTO customer_table ( PersonID, Name, LastModifytime )
SELECT PersonID, Name, LastModifytime
FROM #DataTypeforCustomerTable AS source
WHERE NOT EXISTS
(
SELECT *
FROM customer_table target
WHERE source.PersonID = target.PersonID
)
...
Simply load the temp table and execute the stored proc. See here for more details on temp table scope.
If you are altering a large portion of the table then you should consider the CTAS approach to create a new table, then rename it as suggested by Ron.
I have a DataFrame from a JDBC table hitting MySql and I need to filter it using a UUID. The data is stored in MySql using binary(16) and when querying out in spark is converted to Array[Byte] as expected.
I'm new to spark and have been trying various ways to pass a variable of type UUID into the DataFrame's filter method.
Ive tried statements like
val id: UUID = // other logic that looks this up
df.filter(s"id = $id")
df.filter("id = " convertToByteArray(id))
df.filter("id = " convertToHexString(id))
All of these error with different messages.
I just need to somehow pass in Binary types but can't seem to put my finger on how to do so properly.
Any help is greatly appreciated.
After reviewing even more sources online, I found a way to accomplish this without using the filter method.
When I'm reading from my sparkSession, I just use an adhoc table instead of table name, as follows:
sparkSession.read.jdbc(connectionString, s"(SELECT id, {other col omitted) FROM MyTable WHERE id = 0x$id) AS MyTable", props)
This pre-filters the results for me and then I just work with the data frame as I need.
If anyone knows of a solution using filter, I'd still love to know it as that would be useful in some cases.
I'm using the Postgres extension 'earthdistance' for lat/long distance calculation.
I'm also using Sequelize to access the database and I want to define a getter
method for calculation and sorting by distance from a set of coordinates.
The following query works fine:
SELECT name,
earth_distance(ll_to_earth( 51.5241182, -0.0758046 ),
ll_to_earth(latitude, longitude)) as distance_from_current_location
FROM "Branches"
ORDER BY distance_from_current_location ASC;
And I can use it using sequelize.query(), but I want to keep all the model queries part of the model.
How can I specify WHERE conditions from inside a getter method in the model definition?
Thanks!
Your best bet is probably to wrap the query in a stored procedure and pass in the arguments you want to use in the where clause. As stored procedures are compiled, this will perform better than a Dynamic SQL where you generate the WHERE clause on the fly.
Add whatever parameters and types to your stored proc as you need, and the result will look something like this:
CREATE FUNCTION GetEarthDistance (v_Foo bigint) RETURNS type AS $$
DECLARE
v_Name varchar(256);
BEGIN
SELECT name INTO v_Name,
earth_distance(ll_to_earth( 51.5241182, -0.0758046 ),
ll_to_earth(latitude, longitude)) as distance_from_current_location
FROM Branches
WHERE somecol > v_foo
ORDER BY distance_from_current_location ASC;
RETURN v_Name;
END;
$$ LANGUAGE 'plpgsql';