Sequelize - order and exclude - node.js

I have a table of items with foreign keys for two tables, and I try to do FindAll with order, limit, exclude and include, something like this:
Items.findAll({
order: [['creationDate', 'DESC']],
limit: 10,
include: [
{
model: Artists,
required: false,
attributes: ['id', 'name',]
},
{
model: Albums,
required: false,
},
],
attributes: { exclude: ['creationDate'] }
})
But I get an error:
DatabaseError [SequelizeDatabaseError]: (conn = 837, no: 1054, SQLState: 42S22) Unknown column 'Items.creationDate' in 'order clause'
If I delete this line:
attributes: { exclude: ['creationDate'] }
The error disappears.
What should I do to enable the query with the desired attributes?
Thanks in advance

findAll with include tries to combine data from multiple tables. In SQL, Subqueries and JOIN both achieves this use case. However, there is a slight difference in how data is combined and that what you are seeing here. By default, Sequelize generates Subqueries for findAll + include and subQuery params is to tell Sequelize use Subquery (true) or JOIN (false).
For more about Subqueries vs JOIN: https://www.essentialsql.com/what-is-the-difference-between-a-join-and-subquery/#:~:text=Joins%20versus%20Subqueries,tables%20into%20a%20single%20result.&text=Subqueries%20can%20be%20used%20to,for%20use%20in%20a%20query
Back to Sequelize,
subQuery: true (default)
This will generate SQL like following
SELECT ...
FROM (SELECT ...(attributes except creationDate) FROM `Items` ...)
LEFT OUTER JOIN `Artists` ...
...
ORDER BY `Items`.`creationDate` DESC
LIMIT 10;
This SQL doesn't work, since nested Subquery is executed first and the Subquery is excluding creationDate and when main SQL with ORDER is executed, creationDate does not exist.
Then if you turn off Subquery.
subQuery: false
This will generate SQL like following
SELECT ... (Artist's, Album's attributes and Item's attributes except creationDate)
FROM `Items`
LEFT OUTER JOIN `Artists` ...
...
ORDER BY `Items`.`creationDate` DESC
LIMIT 10;
This will work because this won't filter attributes when ORDER is executed and exclude happens at top level.

Related

Prevent Sequelize Join from returning primary key of include table

I have a sequelize query which renders the correct SQL statement with the exception that it will return the primary key of the joined table. As this query is using Group By the additional field is unacceptable but I am not certain how to remove it.
userDB.tblUsers.belongsTo(userDB.tblSMSSent, { foreignKey: 'collector', targetKey: 'user' });
userDB.tblSMSSent.hasMany(userDB.tblUsers);
let users = await userDB.tblUsers.findAll({
attributes: ['collector', 'tblUsers.id'],
include: {
model: userDB.tblSMSSent,
as: 'tblSMSSent',
attributes: [[sequelize.fn('COUNT', 'tblSMSSent.id'), 'numMessages']]
},
group: ['tblUsers.collector', 'tblUsers.id'],
logging: console.log
})
The SQL rendered is such:
SELECT
[tblUsers].[collector],
[tblUsers].[id],
[tblSMSSent].[id] AS [tblSMSSent.id],
COUNT(N'tblSMSSent.id') AS [tblSMSSent.numMessages]
FROM [dbo].[tblUsers] AS [tblUsers] LEFT OUTER JOIN [dbo].[tblSMSSent] AS [tblSMSSent] ON [tblUsers].[collector] = [tblSMSSent].[user]
GROUP BY [tblUsers].[collector], [tblUsers].[id];
I need this query without tblSMSSent.id included. I have tried using exclude: tblSMSSent.id as outlined in this article in the include attributes but that has not succeeded either. How am I able to correctly exclude this column from my query?
Added the raw:true tag in the JSON sent to sequelize. This forced only the inclusion of the required fields without primary keys.

Using sequelize distinct option correctly (on postgres)

I'm trying to use the distinct clause to filter out some rows from a query. I have two models called Parcel and DeliveryProblem. Each parcel can have n delivery problems and I must get a list of Parcels through the DeliveryProblem side of the relation.
Obviously this causes the result to have many duplicated Parcels which I'd like to filter out. This is what I have tried using:
const problems = await DeliveryProblem.findAll({
limit: 20,
offset: (page - 1) * 20,
attributes: ['parcel_id', 'id'],
include: [
{
model: Parcel,
as: 'parcel',
distinct: ['parcel_id']
attributes: ['product', 'start_date'],
include: [
{
model: Deliveryman,
as: 'deliveryman',
attributes: ['name'],
},
],
},
],
});
It seems sequelize simply ignores the distinct property and keeps returning me the full set. Is there any other way to achieve the distinct set of rows?
You can use the distinct option with count and aggregate functions only.

Sequelize : Get Subquery/Raw Query as model for include

I have gone through the Sequelize doc, but can't find anything helpful
What I want to do is to add raw query or custom model in include, is it possible ?
model.Post.findAll({
include: [{
model: model.User,
attributes: ['fullname', 'profilepic'],
},
{
model: model.PostComments,
},
{
model: "Raw Query"
}
]
}
What I want to achieve is like :
Select post_id, count(*) as total_likes from post_likes group by post_id
I can't achieve this by using simply include, so what I want to do is create a table/model from above query and then use it inside include.
If I use group by withing include it gives group by from top level, and I want to apply group by just for the post_like table.
Please let me know, if it found confusing or not clear.
I was looking to do the same, use a raw query dynamically formed inside the include but there's no possible way to use include without a model https://sequelize.org/master/class/lib/model.js~Model.html#static-method-findAll .
For the purposes of my MySQL I turned my inner join (include) into a where in. I was doing the inner join to avoid the exception This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery . I get this exception because I have a subquery with LIMIT applied.
If it helps anyone:
let rows = await j_model.findAll({
attributes: [...j_columns, `${association}.meta_key_id`],
include: [
{
model: um_model,
as: association,
attributes: ['um_id'],
on: {'j_id' : {$col: 'j.j_id'} }
],
where: {'j_id': {$in: sequelize.literal(`(SELECT * FROM (${massive_inner_raw_query}) as tmp)`)}},
logging: console.log
});
The actual magic is in this line:
where: {'j_id': {$in: sequelize.literal(`(SELECT * FROM (${massive_inner_raw_query}) as tmp)`)}}
The SELECT * FROM removes that exception and lets me do a where in instead of the wanted INNER JOIN. Maybe you can apply a similar deal to your problem.
You can use
Model.findAll({
attributes: [[models.sequelize.literal('CASE WHEN "field1" = true THEN 55
ELSE 23 END'), 'field3']]
}
OR
Model.findAll({
attributes: { include: [[models.sequelize.literal('CASE WHEN "field1" = true THEN 55 ELSE 23 END'), 'field3']]}
}

Adding limit to sequelize findAll destroys the query?

I'm trying to return a group of Models, paginated using limit and offset, including the grouped count of that model's favorites. A fairly trivial thing to attempt.
Here's my basic query setup with sequelize:
var perPage = 12;
var page = 1;
return Model.findAll({
group: [ 'model.id', 'favorites.id' ],
attributes: [
'*',
[ Sequelize.fn('count', Sequelize.col('favorites.id')), 'favorites_count' ]
],
include: [
{ attributes: [], model: Favorite },
],
offset: perPage * page
This generates the (fairly) expected query:
SELECT "model"."id",
"model".*,
Count("favorites"."id") AS "favorites_count",
"favorites"."id" AS "favorites.id"
FROM "model" AS "model"
LEFT OUTER JOIN "favorite" AS "favorites"
ON "model"."id" = "favorites"."model_id"
GROUP BY "model"."id",
"favorites"."id" offset 12;
Ignoring the fact that it quotes the tables, and that it selects favorites.id (forcing me to add it to the group by clause), and that it has randomly aliased things to their exact name or to a nonsensical name like "favorites.id" (all undesired), it seems to have worked. But now let's complete the pagination and add the limit to the query:
...
offset: perPage * page
limit: perPage
It now generates this query:
SELECT "model".*,
"favorites"."id" AS "favorites.id"
FROM (SELECT "model"."id",
"model".*,
Count("favorites"."id") AS "favorites_count"
FROM "model" AS "model"
GROUP BY "model"."id",
"favorites"."id"
LIMIT 12 offset 12) AS "model"
LEFT OUTER JOIN "favorite" AS "favorites"
ON "model"."id" = "favorites"."model";
In completely baffling behavior, it has generated an inner query and applied the limit only to that, then aliased that as "model".
As a sanity check I looked in the docs for findAll, but the docs do not seem to think that command exists.
I suspect I am doing something wrong, but I can't figure out what it is. This behavior is quite bizzarre, and I'm hoping my sleep deprivation is the cause of my confusion.
I'm using version 2.0.6
This turned out to be a bug in Sequelize. See https://github.com/sequelize/sequelize/issues/3623 and https://github.com/sequelize/sequelize/pull/3671

Having trouble adding a where clause to a join table using Sequelizer for NodeJS

I am executing the following where. The key logic that is causing me problems is in line #7 where I'm trying to specify a where condition on a join table.
models.Portfolio.findAll({
include: [{
model: models.PortfolioPermissions,
}],
where: models.sequelize.or(
{'userId': userId},
{'PortfolioPermissions.userId': userId}
),
order: [['startDate', 'DESC']]
})
You can see the resulting query below has a major problem in line #9. Sequelize is prepending my where clause with portfolios table which is messing everything up.
SELECT `portfolios`.*,
`sharedUsers`.`id` AS `sharedUsers.id`,
`sharedUsers`.`permissions` AS `sharedUsers.permissions`,
`sharedUsers`.`userId` AS `sharedUsers.userId`,
`sharedUsers`.`portfolioId` AS `sharedUsers.portfolioId`
FROM `portfolios`
LEFT OUTER JOIN `portfolioPermissions` AS `sharedUsers` ON `portfolios`.`id` = `sharedUsers`.`portfolioId`
WHERE (`portfolios`.`userId`=1
OR `portfolios`.`PortfolioPermissions.userId`=1)
ORDER BY `startDate` DESC;
If someone could please point me in the right direction here, I'd really appreciate it. Thank you so much. I'm using Postgres FYI, probably not relevant though.
Condition on a joined table should be put in the relevant include entry, not on the global where condition.
In your case something like the following should work:
models.Portfolio.findAll({
include: [{
model: models.PortfolioPermissions,
where: {'userId': userId}
}],
where: models.sequelize.or(
{'userId': userId}
),
order: [['startDate', 'DESC']]
})

Resources