I'm trying to return a group of Models, paginated using limit and offset, including the grouped count of that model's favorites. A fairly trivial thing to attempt.
Here's my basic query setup with sequelize:
var perPage = 12;
var page = 1;
return Model.findAll({
group: [ 'model.id', 'favorites.id' ],
attributes: [
'*',
[ Sequelize.fn('count', Sequelize.col('favorites.id')), 'favorites_count' ]
],
include: [
{ attributes: [], model: Favorite },
],
offset: perPage * page
This generates the (fairly) expected query:
SELECT "model"."id",
"model".*,
Count("favorites"."id") AS "favorites_count",
"favorites"."id" AS "favorites.id"
FROM "model" AS "model"
LEFT OUTER JOIN "favorite" AS "favorites"
ON "model"."id" = "favorites"."model_id"
GROUP BY "model"."id",
"favorites"."id" offset 12;
Ignoring the fact that it quotes the tables, and that it selects favorites.id (forcing me to add it to the group by clause), and that it has randomly aliased things to their exact name or to a nonsensical name like "favorites.id" (all undesired), it seems to have worked. But now let's complete the pagination and add the limit to the query:
...
offset: perPage * page
limit: perPage
It now generates this query:
SELECT "model".*,
"favorites"."id" AS "favorites.id"
FROM (SELECT "model"."id",
"model".*,
Count("favorites"."id") AS "favorites_count"
FROM "model" AS "model"
GROUP BY "model"."id",
"favorites"."id"
LIMIT 12 offset 12) AS "model"
LEFT OUTER JOIN "favorite" AS "favorites"
ON "model"."id" = "favorites"."model";
In completely baffling behavior, it has generated an inner query and applied the limit only to that, then aliased that as "model".
As a sanity check I looked in the docs for findAll, but the docs do not seem to think that command exists.
I suspect I am doing something wrong, but I can't figure out what it is. This behavior is quite bizzarre, and I'm hoping my sleep deprivation is the cause of my confusion.
I'm using version 2.0.6
This turned out to be a bug in Sequelize. See https://github.com/sequelize/sequelize/issues/3623 and https://github.com/sequelize/sequelize/pull/3671
Related
I have a table of items with foreign keys for two tables, and I try to do FindAll with order, limit, exclude and include, something like this:
Items.findAll({
order: [['creationDate', 'DESC']],
limit: 10,
include: [
{
model: Artists,
required: false,
attributes: ['id', 'name',]
},
{
model: Albums,
required: false,
},
],
attributes: { exclude: ['creationDate'] }
})
But I get an error:
DatabaseError [SequelizeDatabaseError]: (conn = 837, no: 1054, SQLState: 42S22) Unknown column 'Items.creationDate' in 'order clause'
If I delete this line:
attributes: { exclude: ['creationDate'] }
The error disappears.
What should I do to enable the query with the desired attributes?
Thanks in advance
findAll with include tries to combine data from multiple tables. In SQL, Subqueries and JOIN both achieves this use case. However, there is a slight difference in how data is combined and that what you are seeing here. By default, Sequelize generates Subqueries for findAll + include and subQuery params is to tell Sequelize use Subquery (true) or JOIN (false).
For more about Subqueries vs JOIN: https://www.essentialsql.com/what-is-the-difference-between-a-join-and-subquery/#:~:text=Joins%20versus%20Subqueries,tables%20into%20a%20single%20result.&text=Subqueries%20can%20be%20used%20to,for%20use%20in%20a%20query
Back to Sequelize,
subQuery: true (default)
This will generate SQL like following
SELECT ...
FROM (SELECT ...(attributes except creationDate) FROM `Items` ...)
LEFT OUTER JOIN `Artists` ...
...
ORDER BY `Items`.`creationDate` DESC
LIMIT 10;
This SQL doesn't work, since nested Subquery is executed first and the Subquery is excluding creationDate and when main SQL with ORDER is executed, creationDate does not exist.
Then if you turn off Subquery.
subQuery: false
This will generate SQL like following
SELECT ... (Artist's, Album's attributes and Item's attributes except creationDate)
FROM `Items`
LEFT OUTER JOIN `Artists` ...
...
ORDER BY `Items`.`creationDate` DESC
LIMIT 10;
This will work because this won't filter attributes when ORDER is executed and exclude happens at top level.
I'm trying to use the distinct clause to filter out some rows from a query. I have two models called Parcel and DeliveryProblem. Each parcel can have n delivery problems and I must get a list of Parcels through the DeliveryProblem side of the relation.
Obviously this causes the result to have many duplicated Parcels which I'd like to filter out. This is what I have tried using:
const problems = await DeliveryProblem.findAll({
limit: 20,
offset: (page - 1) * 20,
attributes: ['parcel_id', 'id'],
include: [
{
model: Parcel,
as: 'parcel',
distinct: ['parcel_id']
attributes: ['product', 'start_date'],
include: [
{
model: Deliveryman,
as: 'deliveryman',
attributes: ['name'],
},
],
},
],
});
It seems sequelize simply ignores the distinct property and keeps returning me the full set. Is there any other way to achieve the distinct set of rows?
You can use the distinct option with count and aggregate functions only.
Designing a PostgreSQL database that will be queried by a Node API using Sequelize. Currently, I have a table called recipes that has columns called ingredients and instructions. Those columns are stored for a given as an array of strings like {Tomatoes, Onions}.
That method of storage worked fine for simply fetching and rendering a recipe on the client side. But it wasn't working well for fuzzy search querying because, using Sequelize all I could do was ingredients: { [Op.contains] : [query] }. So if a user typed tomatoes there was no way to write a "fuzzy" search query that would return a recipe with an ingredient Tomatoes.
And then I read this in the PostgreSQL documentation:
Arrays are not sets; searching for specific array elements can be a sign of database misdesign. Consider using a separate table with a row for each item that would be an array element. This will be easier to search, and is likely to scale better for a large number of elements.
Now I'm considering storing ingredients and instructions as separate tables, but I have a couple of questions.
1) As a recipe can have multiple ingredients related to it, should I just use a foreign key for each ingredient and the Sequelize hasMany relationship? That seems correct to me, except that I'm now potentially duplicating common ingredients each time a new recipe is created that uses that ingredient.
2) What would be the best way to write a fuzzy search query so that a user could search the main columns of the recipes table (e.g. title, description) and additionally apply their query to the instructions and ingredients tables?
Essentially I'd like to end up with a fuzzy search query applied to the three tables that looks something like this...
const recipes = await req.context.models.Recipe.findAll({
where: {
[Op.or]: [
{ title: { [Op.iLike]: '%' + query + '%' } },
{ description: { [Op.iLike]: '%' + query + '%' } },
{ ingredients: { ingredient: { [Op.iLike]: '%' + query + '%' } } },
{ instructions: { instruction: { [Op.iLike]: '%' + query + '%' } } }
]
}
});
Thanks!
I have done this, i happen to use graphql in my node layer with sequelize, and i have filter objects that do this type of thing. You'll just need some include statements in your Recipie.findAll.. after your initial where clause where you evaluate whether you are searching title or description or both type thing. i sent my search params in with prefix's i could strip off that told me what sequelize op's i would want to use on them and just ran my args through a utility method to create my where clause, but i know there are many ways to skin that cat. i just did not want to clutter up my resolvers with tonnes of hardcoded ops and conditional clauses was all.... your include might look something like this
include: [{
model: models.Ingredient,
as: 'Ingredients',
through: { some join table specifying keys where necessary since this
is many to many }
where: {some conditional code around your search param},
}, {
model: models.Instruction,
as: 'Instructions',
where: {some conditional code around your search param},
}],
There is good documentation around multiple includes, or nested includes in the sequelize docs, but from what i see above you have a fairly good understanding of what you need to do. To uncomplicate things a bit, i'd start with just searching on your fields from recipie (title, description) before you add the includes and get that working, then it will be a little clearer how you want to form your where clauses.
alternativley.. you can skip the includes and write associations in your models and call them with getters and pass the where clauses to those... i do that as well and again well documented stuff now.. Sequelize has really upped their game
Recipie.associate = function (models) {
models.Recipie.hasMany(models.Ingredient, { as: 'Ingredients', through: "recipie_ingredient" foreignKey: 'recipie_id'});
};
now you have a getter for Ingredients, and if you declare belongsToMany targetting back at Recipie in the Ingredient model then you'll have a getter there as well, and you can pass your search string to that via where clause and get all recipies that have a given ingredient or ingredient list type thing.... Clear as mud?
I have a model Booking, which is having hasMany relation with hotels, and hotel is having one to one relation with supppliers.
What i need is, get all booking where supplier_id = 33333.
I am trying this
BOOKINGS.findAll({
where: {
'hotels.supplier.supplier_id' : '32',
},
include: [
{
model: HOTELS,
include: [
{
model: SUPPLIERS,
],
}
],
limit : 30,
offset: 0
})
It throws error like hotels.supplier... column not found.. I tried all things because on docs of sequelze it only gives solution to add check which adds where inside the include which i can't use as it adds sub queries.
I don't want to add where check alongwith supplier model inside the include array, because it adds sub queries, so If i am having 1000 bookings then for all bookings it will add sub query which crashes my apis.
I need a solutions like this query in Sequelize.
Select col1,col2,col3 from BOOKINGS let join HOTELS on BOOKINGS.booking_id = HOTELS.booking_id, inner join SUPPLIERS on BOOKINGS.supplier_id = SUPPLIERS.supplier_id
Adding a where in the include object will not add a sub query. It will just add a where clause to the JOIN which is being applied to the supplier model. It will not crash your API in anyway. You can test it out on your local machine plenty of times to make sure.
BOOKINGS.findAll({
include: [
{
model: HOTELS,
include: [
{
model: SUPPLIERS,
where: { supplier_id: 32 }
}
]
}
],
limit: 30,
offset: 0
})
If you still want to use the query on the top level you can use sequelize.where+ sequelize.literal but you will need to use the table aliases that sequelize assigns. e.g this alias for supplier table will not work hotels.supplier.supplier_id. Sequelize assings table aliases like in the example I have shown below:
BOOKINGS.findAll({
where: sequelize.where(sequelize.literal("`hotels->suppliers`.supplier_id = 32")),
include: [
{
model: HOTELS,
include: [SUPPLIERS]
}
],
limit: 30,
offset: 0
})
I have gone through the Sequelize doc, but can't find anything helpful
What I want to do is to add raw query or custom model in include, is it possible ?
model.Post.findAll({
include: [{
model: model.User,
attributes: ['fullname', 'profilepic'],
},
{
model: model.PostComments,
},
{
model: "Raw Query"
}
]
}
What I want to achieve is like :
Select post_id, count(*) as total_likes from post_likes group by post_id
I can't achieve this by using simply include, so what I want to do is create a table/model from above query and then use it inside include.
If I use group by withing include it gives group by from top level, and I want to apply group by just for the post_like table.
Please let me know, if it found confusing or not clear.
I was looking to do the same, use a raw query dynamically formed inside the include but there's no possible way to use include without a model https://sequelize.org/master/class/lib/model.js~Model.html#static-method-findAll .
For the purposes of my MySQL I turned my inner join (include) into a where in. I was doing the inner join to avoid the exception This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery . I get this exception because I have a subquery with LIMIT applied.
If it helps anyone:
let rows = await j_model.findAll({
attributes: [...j_columns, `${association}.meta_key_id`],
include: [
{
model: um_model,
as: association,
attributes: ['um_id'],
on: {'j_id' : {$col: 'j.j_id'} }
],
where: {'j_id': {$in: sequelize.literal(`(SELECT * FROM (${massive_inner_raw_query}) as tmp)`)}},
logging: console.log
});
The actual magic is in this line:
where: {'j_id': {$in: sequelize.literal(`(SELECT * FROM (${massive_inner_raw_query}) as tmp)`)}}
The SELECT * FROM removes that exception and lets me do a where in instead of the wanted INNER JOIN. Maybe you can apply a similar deal to your problem.
You can use
Model.findAll({
attributes: [[models.sequelize.literal('CASE WHEN "field1" = true THEN 55
ELSE 23 END'), 'field3']]
}
OR
Model.findAll({
attributes: { include: [[models.sequelize.literal('CASE WHEN "field1" = true THEN 55 ELSE 23 END'), 'field3']]}
}