The end of this article suggests that a materialized view can contain flatten ... https://www.snowflake.com/blog/snowflake-materialized-views-a-fast-zero-maintenance-accurate-solution/
But if I try it I get an error ... "More than one table referenced in the view definition"
Anybody got any examples of how to create a MV with flatten?
CREATE OR REPLACE MATERIALIZED VIEW MV_SOMETHING
AS
SELECT
F.ID,
A.VALUE:Type::string AS ATTRIBUTE_TYPE,
A.VALUE:Value::string AS ATTRIBUTE_VALUE
FROM
FRAME F
, lateral flatten(input => F.ENVIRONMENT) A;
The following works fine for me. You need FLATTEN() for ARRAYs, but not for OBJECTs.
Maybe you are mixing the two?
CREATE OR REPLACE TABLE T AS
SELECT 1 ID, ARRAY_CONSTRUCT(1, 2, 3) A, OBJECT_CONSTRUCT('X', 2, 'Y', 3) O;
CREATE OR REPLACE MATERIALIZED VIEW T_MV AS
SELECT
ID,
O:X::NUMBER O_X,
O:Y::NUMBER O_Y,
VALUE A
FROM T, LATERAL FLATTEN(INPUT => A);
Related
I am new to Azure Data Factory, and I have searched everywhere for a solution that may be implemented for my necessity, but I haven't found any.
My Problem:
I have a table in Azure Database with a column containing a mathematical operation, about 50 columns containing the variables for the operation and one last column where I need to update the result of the mathematical operation, like this:
Example of the table
What I want to do is to fill up the column "result" with the result of the mathematical operation, contained in the column "Operation", using the other columns values in the expression. This is just an example table, my actual table has about 50 columns of values, so it is not a solution for me to use a "replace" operation.
There are probably a few ways to do this but I would not use Data Factory, unless you need to orchestrate this activity as part of a wider pipeline. As you have some compute handy via Azure SQL Database, I would make best use of that unless you have a specific reason not to do so. T-SQL has dynamic SQL and the EXEC command to help. Use a cursor to run through the distinct list of formulas and execute it dynamically. A simplified example:
DROP TABLE IF EXISTS dbo.formulas;
CREATE TABLE dbo.formulas (
Id INT PRIMARY KEY,
formula VARCHAR(100) NOT NULL,
a INT NOT NULL,
b INT NOT NULL,
c INT NOT NULL,
d INT NOT NULL,
e INT NOT NULL,
--...
result INT
);
-- Set up test data
INSERT INTO dbo.formulas ( Id, formula, a, b, c, d, e )
VALUES
( 1, '(a+b)/d', 1, 20, 2, 3, 1 ),
( 2, '(c+b)*(a+e)', 0, 1, 2, 3, 4 ),
( 3, 'a*(d+e+c)', 7, 10, 6, 2, 1 )
SET NOCOUNT ON
-- Create local fast_forward ( forward-only, read-only ) cursor
-- Get the distinct formulas for the table
DECLARE formulaCursor CURSOR FAST_FORWARD LOCAL FOR
SELECT DISTINCT formula
FROM dbo.formulas
-- Cursor variables
DECLARE #sql NVARCHAR(MAX)
DECLARE #formula NVARCHAR(100)
OPEN formulaCursor
FETCH NEXT FROM formulaCursor INTO #formula
WHILE ##fetch_status = 0
BEGIN
SET #sql = 'UPDATE dbo.formulas
SET result = ' + #formula + '
--OUTPUT inserted.id -- optionally output updated ids
WHERE formula = ''' + #formula + ''';'
PRINT #sql
-- Update each result field for the current formula
EXEC(#sql)
FETCH NEXT FROM formulaCursor INTO #formula
END
CLOSE formulaCursor
DEALLOCATE formulaCursor
GO
SET NOCOUNT OFF
GO
-- Check the results
SELECT *
FROM dbo.formulas;
Cursors have a bad reputation for performance but i) here I'm using the distinct list of formulas and ii) sometimes it's the only way. I can't think of a nice set-based way of doing this - happy to be corrected. CLR is not available to you. If performance is a major issue for you you may need to think about alternatives; there's an interesting discussion on a similar problem here.
My results:
If your database was an Azure Synapse Analytics dedicated SQL pool then you could look at Azure Synapse Notebooks to achieve the same outcome.
I've using a sql Table to generate filters on each dimensions for a value in a SSAS Cube.
The MDX Query is based on the column Query below, the calculated member is:
AGGREGATE
(
IIF(Query= "" or ISEMPTY(Query),
[Code].[_KeyQuery].[ALL],
StrToTuple('('+ Query+')')
),[Measures].[Value]
)
I have to work with pivot Table in Excel. It works perfectly, the value is correctly filter on each dimension member. If i use a query like this, it's ok.
[Level].[LevelCode].&[A],[Status].[StatusCode].&[ST]
But now i need adding the possibility to filter on multiple dimensions members. For exemple, using a query :
[Level].[LevelCode].&[A],[Level].[LevelCode].&[X],[Status].[StatusCode].&[ST]
It doesn't works, i've try changing the query like this:
{[Level].[LevelCode].&[A],[Level].[LevelCode].&[X]},[Status].[StatusCode].&[ST]
but the StrToTuple() function causes error. I don't know how to filter in multiple values for a same dimension hierarchy.
If it will always be a tuple then no need to use AGGREGATE just a tuple should return the value:
IIF(
Query= "" OR ISEMPTY(Query),
(
[Code].[_KeyQuery].[ALL]
,[Measures].[Value]
)
,StrToTuple('('+ Query +',[Measures].[Value])')
)
Or this version:
StrToTuple(
'('
+ IIF(
Query= "" OR ISEMPTY(Query)
,[Code].[_KeyQuery].[ALL]
,Query
)
+',[Measures].[Value])'
)
possible approach for decision between tuple and set
Add a column to your control table "TupleOrSet" with values of either "T" or "S". Then you could amend your code to something like this:
IIF(
Query= "" OR ISEMPTY(Query),
(
[Code].[_KeyQuery].[ALL]
,[Measures].[Value]
)
,IIF(
TupleOrSet = "T"
,StrToTuple('('+ Query +',[Measures].[Value])')
,AGGREGATE( StrToSet('{'+ Query +'}'), [Measures].[Value])
)
)
note
A tuple is a definite point in the cube space so cannot therefore be made up of two members from the same hierarchy - this would create coordinates that are non-determinant
I searched a lot about sorting elements by sum of votes (in another model), like I do in SQL here :
SELECT item.* FROM item
LEFT JOIN (
SELECT
vote.item,
SUM(vote.value) AS vote.rating
FROM vote
GROUP BY vote.item
) AS res ON item.id = vote.item
ORDER BY res.rating DESC
Is there a way to do it via waterline methods ?
I think you can't do the left join with simple waterline methods, but you can use the .query method to execute your raw SQL syntax.
Sails MySQL adapter makes sum('field') conflict with sort('field'). It will generate SQL query like:
SELECT SUM(table.field) AS field FROM table ORDER BY table.field;
But I want:
SELECT SUM(table.field) AS field FROM table ORDER BY field;
It same as:
SELECT SUM(table.field) AS f FROM table ORDER BY f;
My solution is using lodash.sortBy() to process results. https://lodash.com/docs/4.16.4#sortBy
I have created a query with a subquery in Access, and cannot link it in Excel 2003: when I use the menu Data -> Import External Data -> Import Data... and select the mdb file, the query is not present in the list. If I use the menu Data -> Import External Data -> New Database Query..., I can see my query in the list, but at the end of the import wizard I get this error:
Too few parameters. Expected 2.
My guess is that the query syntax is causing the problem, in fact the query contains a subquery. So, I'll try to describe the query goal and the resulting syntax.
Table Positions
ID (Autonumber, Primary Key)
position (double)
currency_id (long) (references Currency.ID)
portfolio (long)
Table Currency
ID (Autonumber, Primary Key)
code (text)
Query Goal
Join the 2 tables
Filter by portfolio = 1
Filter by currency.code in ("A", "B")
Group by currency and calculate the sum of the positions for each currency group an call the result: sumOfPositions
Calculate abs(sumOfPositions) on each currency group
Calculate the sum of the previous results as a single result
Query
The query without the final sum can be created using the Design View. The resulting SQL is:
SELECT Currency.code, Sum(Positions.position) AS SumOfposition
FROM [Currency] INNER JOIN Positions ON Currency.ID = Positions.currency_id
WHERE (((Positions.portfolio)=1))
GROUP BY Currency.code
HAVING (((Currency.code) In ("A","B")));
in order to calculate the final SUM I did the following (in the SQL View):
SELECT Sum(Abs([temp].[SumOfposition])) AS sumAbs
FROM [SELECT Currency.code, Sum(Positions.position) AS SumOfposition
FROM [Currency] INNER JOIN Positions ON Currency.ID = Positions.currency_id
WHERE (((Positions.portfolio)=1))
GROUP BY Currency.code
HAVING (((Currency.code) In ("A","B")))]. AS temp;
So, the question is: is there a better way for structuring the query in order to make the export work?
I can't see too much wrong with it, but I would take out some of the junk Access puts in and scale down the query to this, hopefully this should run ok:
SELECT Sum(Abs(A.SumOfPosition)) As SumAbs
FROM (SELECT C.code, Sum(P.position) AS SumOfposition
FROM Currency As C INNER JOIN Positions As P ON C.ID = P.currency_id
WHERE P.portfolio=1
GROUP BY C.code
HAVING C.code In ("A","B")) As A
It might be worth trying to declare your parameters in the MS Access query definition and define their datatypes. This is especially important when you are trying to use the query outside of MS Access itself, since it can't auto-detect the parameter types. This approach is sometimes hit or miss, but worth a shot.
PARAMETERS [[Positions].[portfolio]] Long, [[Currency].[code]] Text ( 255 );
SELECT Sum(Abs([temp].[SumOfposition])) AS sumAbs
FROM [SELECT Currency.code, Sum(Positions.position) AS SumOfposition
FROM [Currency] INNER JOIN Positions ON Currency.ID = Positions.currency_id
WHERE (((Positions.portfolio)=1))
GROUP BY Currency.code
HAVING (((Currency.code) In ("A","B")))]. AS temp;
I have solved my problems thanks to the fact that the outer query is doing a trivial sum. When choosing New Database Query... in Excel, at the end of the process, after pressing Finish, an Import Data form pops up, asking
Where do you want to put the data?
you can click on Create a PivotTable report... . If you define the PivotTable properly, Excel will display only the outer sum.
I want to perform a simple join on two tables (BusinessUnit and UserBusinessUnit), so I can get a list of all BusinessUnits allocated to a given user.
The first attempt works, but there's no override of Select which allows me to restrict the columns returned (I get all columns from both tables):
var db = new KensDB();
SqlQuery query = db.Select
.From<BusinessUnit>()
.InnerJoin<UserBusinessUnit>( BusinessUnitTable.IdColumn, UserBusinessUnitTable.BusinessUnitIdColumn )
.Where( BusinessUnitTable.RecordStatusColumn ).IsEqualTo( 1 )
.And( UserBusinessUnitTable.UserIdColumn ).IsEqualTo( userId );
The second attept allows the column name restriction, but the generated sql contains pluralised table names (?)
SqlQuery query = new Select( new string[] { BusinessUnitTable.IdColumn, BusinessUnitTable.NameColumn } )
.From<BusinessUnit>()
.InnerJoin<UserBusinessUnit>( BusinessUnitTable.IdColumn, UserBusinessUnitTable.BusinessUnitIdColumn )
.Where( BusinessUnitTable.RecordStatusColumn ).IsEqualTo( 1 )
.And( UserBusinessUnitTable.UserIdColumn ).IsEqualTo( userId );
Produces...
SELECT [BusinessUnits].[Id], [BusinessUnits].[Name]
FROM [BusinessUnits]
INNER JOIN [UserBusinessUnits]
ON [BusinessUnits].[Id] = [UserBusinessUnits].[BusinessUnitId]
WHERE [BusinessUnits].[RecordStatus] = #0
AND [UserBusinessUnits].[UserId] = #1
So, two questions:
- How do I restrict the columns returned in method 1?
- Why does method 2 pluralise the column names in the generated SQL (and can I get round this?)
I'm using 3.0.0.3...
So far my experience with 3.0.0.3 suggests that this is not possible yet with the query tool, although it is with version 2.
I think the preferred method (so far) with version 3 is to use a linq query with something like:
var busUnits = from b in BusinessUnit.All()
join u in UserBusinessUnit.All() on b.Id equals u.BusinessUnitId
select b;
I ran into the pluralized table names myself, but it was because I'd only re-run one template after making schema changes.
Once I re-ran all the templates, the plural table names went away.
Try re-running all 4 templates and see if that solves it for you.