I'm trying to convert a SQL server query into Snowflake but it uses an unsupported query type:
FROM Policy p
LEFT JOIN CurrencyHistory ch
ON ch.CurrencyID = 'USD'
AND ch.CurrencyTo = p.CurrencyID
AND ch.PeriodFrom = (SELECT MAX(ch2.PeriodFrom)
FROM CurrencyHistory ch2
WHERE ch2.CurrencyID = ch.CurrencyID
AND ch2.CurrencyTo = ch.CurrencyTo)
I tried adapting the solution from this but couldn't get it to work as the join within the subquery is exclusively on the CurrencyHistory table unlike the other issue.
Any help is much appreciated.
Related
I have setup an AWS Lambda function with python to ingest requests from a CSV and then query an AWS Serverless Aurora PostgreSQL database based on this request. The function works when the requests are less then 1K but I get errors due to a hard limit in the data API. I am trying to figure out a way to break up the query into smaller queries once this limit is hit but not sure how to do this in python. Can someone suggest a way to break up a request into smaller chunks so I do not hit the data API limit?
Snippet of Code Used:
#Set Database connection params
engine = wr.data_api.rds.connect( resource_arn = resource_arn, database=database_name, secret_arn=secret_arn)
#read in s3 csv and select ids
read_df = wr.s3.read_csv(path=s3_path_in)
requested_ids = read_df["ids"]
in_ID = requested_ids.tolist()
in_query= str(tuple(in_ID))
#query postgres
query_id = """
select c.*
from table1 b
INNER JOIN table2 c on b.id = c.id
where b.id in %s
""" % in_query
out_ids = wr.data_api.rds.read_sql_query(query_id, engine)
one way that i can think of is to use the LIMIT <row_count> clause of the postgres SQL and dynamically pass the row_count to your query .
select c.*
from table1 b
INNER JOIN table2 c on b.id = c.id
where b.id in <>
order by <primary_key>
limit 999
PostgreSQL LIMIT
I have a couple of stored procedures that create different temporary tables.
At the end of the procedure i drop them (know that is not required, but it's good practice).
The stored procedures are executed as a part of a SSIS package. I got 4 different SQL jobs that execute the same SSIS package running in parallel.
When logging into the Azure portal and using the performance recommendation feature, I get a recommendation to fix the schema issues. It states an Sql error code 208. According to documentation that means "object not found".
Temporary tables are valid within the scope of the stored procedure and should get a unique name in the database, so I do not think where are any conflicts.
I have no idea what causes this, and the stored procedures seems to work alright. Anyone know what could be the cause here?
Simplified sample of one of the procedures:
SET NOCOUNT ON;
CREATE TABLE #tmpTransEan
(
Ean_Art_Str_id BIGINT ,
Artikler_id BIGINT
);
INSERT INTO #tmpTransEan
( Ean_Art_Str_id ,
Artikler_id
)
SELECT DISTINCT
eas.Ean_Art_Str_id ,
a.Artikler_id
FROM dbo.Artikkel_Priser ap
JOIN Ean_Art_Str eas ON eas.artikler_id = ap.Artikler_id
JOIN wsKasse_Til_Kasselogg ktk ON eas.Ean_Art_Str_id = ktk.ID_Primary
JOIN dbo.Artikler a ON a.Artikler_id = eas.artikler_id
JOIN dbo.Felles_Butikker b ON b.Butikker_id = ap.butikker_id
WHERE ktk.ID_Table = OBJECT_ID('Ean_Art_Str')
AND LEN(a.Artikkelnr) >= 8
AND ktk.Tidspunkt >= #tidspunkt
AND ( ( ap.butikker_id = #nButikker_id1
AND #Alle_artikler_til_kasse = 'N'
)
OR ( b.Databaser_id = #Databaser_id
AND #Alle_artikler_til_kasse = 'J'
)
)
AND b.Akt_kode = 'A'
AND a.Akt_kode = 'A'
AND a.Databaser_id IN ( -1, #Databaser_id )
SELECT DISTINCT
a.Artikkelnr ,
s.Storrelse ,
eas.* ,
EAN_12 = LEFT(eas.EAN_13, 12)
FROM dbo.Ean_Art_Str eas
JOIN #tmpTransEan t ON t.Artikler_id = eas.artikler_id
JOIN Artikler a ON a.Artikler_id = eas.artikler_id
JOIN dbo.Felles_Storrelser s ON s.Storrelser_id = eas.storrelser_id
DROP TABLE #tmpTransEan;
END;
I have a parent/child table setup - Items/ItemDetails. This part works:
var q = db.From<Item>(); //various where clauses based on request
items = db.Select<Item>(q);
q = q.Select(a => a.ITEM_NO);
itemDetails = db.Select<ItemDetail>(x => Sql.In(x.ITEM_NO, q));
Trying to add paging to improve the performance of this request for large data sets, I'm having trouble getting the .Limit(skip, rows) function to work in the SQL.In statement of the child table.
var q = db.From<Item>().Limit(skip, rows);
items = db.Select<Item>(q);
q = q.Select(a => a.ITEM_NO);
itemDetails = db.Select<ItemDetail>(x => Sql.In(x.ITEM_NO, q));
It works when limiting the results in the first select, but when used in the child data pull I get "Only one expression can be specified in the select list when the subquery is not introduced with EXISTS."
The SQL that comes out changes the where subquery to:
WHERE "ITEM_NO" IN (SELECT * FROM (SELECT "ITEM_NO", ROW_NUMBER() OVER
(ORDER BY "ITEM"."ITEM_NO") As RowNum
FROM "ITEM") AS RowConstrainedResult WHERE RowNum > 5 AND RowNum <= 15)
I understand the SQL error is because I am selecting more than one column in the IN clause. Is there a better way to write this to avoid the error?
Thanks
If you're using SQL Server 2012 or later you should use SqlServer2012Dialect.Provider, e.g:
container.Register<IDbConnectionFactory>(c =>
new OrmLiteConnectionFactory(connString, SqlServer2012Dialect.Provider));
Which lets OrmLite use the paging support added in SQL Server 2012 instead of resorting to use the windowing function hack required to implement paging for earlier versions of SQL Server.
I'm trying to figure out how to replicate the below query in SQLAlchemy
SELECT c.company_id AS company_id,
(SELECT policy_id FROM associative_table at WHERE at.company_id = c.company_id) AS policy_id_ref,
(SELECT `default` FROM policy p WHERE p.policy_id = policy_id_ref) AS `default`,
FROM company c;
Note that this is a stripped down, basic example of what I'm really dealing with. The actual schema supports data and relationship versioning that requires the subqueries to include additional conditions, sorting, and limiting, making it impractical (if not impossible) for them to be joins.
The crux of the problem is in how the second subquery relies on policy_id_ref -- the value obtained from the first subquery. In SQLAlchemy, this is effectively what I have now:
ct = aliased(classes.company)
at = aliased(classes.associative_table)
pt = aliased(classes.policy)
policy_id_ref = session.query(at.policy_id).\
filter(at.company_id == ct.company_id).\
label('policy_id_ref')
policy_default = session.query(pt.default).\
filter(pt.id == 'policy_id_ref').\
label('default')
query = session.query(ct.company_id,policy_id_ref,policy_default)
The pull from the "company" table works fine as does the first subquery that retrieves the "policy_id_ref" column. The problem is the second subquery that has to reference that "policy_id_ref" column. I don't know how to write its filter in such a way that it literally renders "policy_id_ref" in the resulting query, to match the label of the first subquery.
Suggestions?
Thanks in advance
You can write your query as
select(
Companies.company_id,
AssociativeTable.policy_id.label('policy_id_ref'),
Policy.default.label('policy_default'),
).select_from(
Companies,
).join(
AssociativeTable,
AssociativeTable.company_id == Companies.company_id,
).join(
Policy,
AssociativeTable.policy_id == Policy.id
)
but in case you need reference to label from subquery => use literal_column
from sqlalchemy import func, select, literal_column
session.query(
func.array_agg(
literal_column('batch_info'),
JSONB
).label('history')
).select_from(
select(
func.jsonb_build_object(
'batch_id', AccountingQueueBatch.id,
'batch_label', AccountingQueueBatch.label,
).label('batch_info')
).select_from(
AccountingQueueBatch,
)
)
I am using version 2.1 of SubSonic. I am trying to build to build a relatively simple query where I get a list of Roles for a User using UserId as a parameter. (UserId is a string...)
SubSonic.SqlQuery sel = new SubSonic.Select().From(Data.Tables.Role).InnerJoin(Data.Tables.UserRole, Data.UserRole.Columns.RoleId, Data.Tables.Role, Data.Role.Columns.Id).InnerJoin(Data.Tables.User, Data.User.Columns.Id, Data.Tables.UserRole, Data.UserRole.Columns.UserId).Where("[dbo].[User].[Id]").IsEqualTo(userId);
this generates the query
SELECT [dbo].[Role].[Id], [dbo].[Role].[PID], [dbo].[Role].[Name]
FROM [dbo].[Role]
INNER JOIN [dbo].[UserRole] ON [dbo].[Role].[Id] = [dbo].[UserRole].[RoleId]
INNER JOIN [dbo].[User] ON [dbo].[UserRole].[UserId] = [dbo].[User].[Id]
WHERE [dbo].[User].[Id] = #[dbo].[User].[Id]0
which fails. If I replace the Where with .Where(Data.User.Columns.Id) this generates the query
SELECT [dbo].[Role].[Id], [dbo].[Role].[PID], [dbo].[Role].[Name]
FROM [dbo].[Role]
INNER JOIN [dbo].[UserRole] ON [dbo].[Role].[Id] = [dbo].[UserRole].[RoleId]
INNER JOIN [dbo].[User] ON [dbo].[UserRole].[UserId] = [dbo].[User].[Id]
WHERE [dbo].[Role].[Id] = #Id0
which uses the Role table in the Where clause instead of the User table.
Is this a bug, or am I doing something incorrectly? What would be the correct way to do this? Thanks.
This is fixed in version 2.2 - I would suggest upgrading.