I have created stored procedure in snowflake which is executed fine in snowflake UI and also from server by using snowsql. Now I want to execute procedure from python program, I tried to execute from python, here are the steps that I have followed:
establish the connection to snowflake ( successfully able to connect.)
cs = ctx.cursor()
Used appropriate role,warehouse,database and schema.
tried to execute procedure like this:
cs.execute("call test_proc('value1', 'value2')")
x = cs.fetchall()
print(x)
But getting an erorr:
snowflake.connector.errors.ProgrammingError: 002140 (42601): SQL
compilation error: Unknown function test_proc
Can you please help me to resolve this problem.
Thanks,
When connecting to Snowflake using Python connector you could define DATABASE/SCHEMA
conn = snowflake.connector.connect(
user=USER,
password=PASSWORD,
account=ACCOUNT,
warehouse=WAREHOUSE,
database=DATABASE,
schema=SCHEMA
);
Once you have it set up, you could call your stored procedure without using fully-qualified name:
cs.execute("call test_proc('value1', 'value2')");
Alternative way is:
Using the Database, Schema, and Warehouse
Specify the database and schema in which you want to create tables. Also specify the warehouse that will provide resources for executing DML statements and queries.
For example, to use the database testdb, schema testschema and warehouse tiny_warehouse (created earlier):
conn.cursor().execute("USE WAREHOUSE tiny_warehouse_mg")
conn.cursor().execute("USE DATABASE testdb_mg")
conn.cursor().execute("USE SCHEMA testdb_mg.testschema_mg")
Actually, I have to have command like this
cs.execute("call yourdbname.schemaname.test_proc('value1', 'value2')")
and It is working as expected.
Thanks
Related
I was trying to dump my PostgreSQL database created via SQLalchemy using python script. Though I have successfully created a database and all the data are getting inserted via web parsing in the ORM I have mapped with. But when I am trying to take a dump for all my insert queries using this
tab = Table(table.__tablename__, MetaData())
x = tab.insert().compile(
dialect=postgresql.dialect(),
compile_kwargs={"literal_binds": True},
)
logging.info(f"{x}")
I am adding values using ORM like this:
for value in vertex_type_values:
data = table(
Type=value["type"],
Name=value["name"],
SizeX=value["size_x"],
SizeY=value["size_y"],
SizeZ=value["size_z"],
)
session.add(data)
session.commit()
here table is the model which i have designed and imported from my local library and vertex_type_values which I have extracted and yield in my script
I am getting the output as
INSERT INTO <tablename> DEFAULT VALUES
So my question is how to get rid of Default Values and get actual values so that I can directly use insert command if my DB crash anytime? I need to know raw SQL for insert command
When using Redshift, I would like to get the names of all the procedure that were created in a schema, along with their definition.
I know you can use the SHOW PROCEDURE command to get the definition but that requires to have the procedure name.
In SVV_TABLE there is only information regarding tables and view but not procedure.
So if anyone knows how to get that ?
Redshift doesn't have a system view for that yet but you can use tbe PG_PROC table and join it with pg_namespace to filter on schema name.
select proname, proargnames, prosrc
from PG_PROC
join pg_namespace on pg_namespace.oid = pg_proc.pronamespace
where nspname = 'your_schema_name' and procsecdef = true;
The procsecdef = true is to get only stored procedure definition otherwise you also get python UDF.
i am trying to work with multiple database and schemas using simple vertical partitioning in sqlalchemy and python .
Have create two database engines and configured successfully to sessionmaker()
Session = sessionmaker()
Session.configure(binds={BaseA:engine1, BaseB:engine2})
Able to get the required sql query generated successfully
driverssql = session.query(drivers)
but when i execute the above query to fetch the requslts i get the follwing error :
resultset=session.execute(driversql)
sqlalchemy.exc.UnboundExecutionError: Could not locate a bind configured on SQL expression or this Session (how can i associate the correct engine with execute statement)
I'm seeing here two variants:
You can create 2 sessionmakers here and use them separately according to an engine.
You can choose necessary engine when executing a query:
engine1 = create_engine(first_db)
engine2 = create_engine(second_db)
session.execute(drivers, bind=engine1)
I'd like to run a query using Spark SQL in Airflow, it looks like the SparkSQLOperator is perfect for this (https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/operators/spark_sql_operator.py)
However, I can't figure out how the connection must be configured.
In DB Visualizer, I can connect to the Hive database using:
driver : jdbc
database url : jdbc:hive2://myserver.com:10000/default
database userid : me
database password : mypassword
Applying these settings to the spark_sql_default connection gives me:
enter[2017-12-12 11:35:33,774] {models.py:1462} ERROR - Cannot execute
on hive2://myserver.com:10000/default. Error code is: 1. Output: ,
Stderr:
Any ideas?
Spark isn't a database, so you configure your Spark connection just like you would if you were submitting regular Spark jobs, rather than the JDBC-like parameters you're used to.
If you look at the signature of the operator:
def __init__(self,
sql,
conf=None,
conn_id='spark_sql_default',
executor_cores=None,
executor_memory=None,
keytab=None,
master='yarn',
name='default-name',
num_executors=None,
yarn_queue='default',
*args,
**kwargs)
You need to create a connection (see models.Connection) that specifies the Spark master, which would be one of 'yarn', 'local[x]', or 'spark://hostname:port'. That should really be it, since they default the rest for you and it is likely handled by the underlying Spark config.
Your SQL or HQL code/script can be passed into the first parameter.
I created a new table in the Bluemix SQL Database service by uploading a csv (baseball.csv) and took the default table name of "baseball".
I created a simple app in Node.js which is just trying to select data from the table with select * from baseball, but I keep getting the following error:
[IBM][CLI Driver][DB2/NT] SQL0204N "USERxxxx.BASEBALL" in an undefined name
Why can't it find my database table?
This issue seems independent of bluemix, rather it is usage error.
This error is possibly caused by following:
The object identified by name is not defined in the database.
User response
Ensure that the object name (including any required qualifiers) is correctly specified in the SQL statement and it exists.
try running "list tables" from command prompt to check if your table spelling is correct or not.
http://www-01.ibm.com/support/knowledgecenter/SSEPGG_9.7.0/com.ibm.db2.luw.messages.sql.doc/doc/msql00204n.html?cp=SSEPGG_9.7.0%2F2-6-27-0-130
I created the table from SQL Database web UI in bluemix and took the default name of baseball. It looks like this creates a case-sensitive table name.
Unfortunately for me, the sql_db libary (and all db2 clients I believe) auto-capitalizes the SQL query into "SELECT * FROM BASEBALL"
The solution was to either
A. Explicitly name my table BASEBALL in the web UI; or
B. Modify my sql query by quoting the table name:
select * from "baseball"
More info at http://www.ibm.com/developerworks/data/library/techarticle/0203adamache/0203adamache.html#N10121