I'm trying to use the distinct clause to filter out some rows from a query. I have two models called Parcel and DeliveryProblem. Each parcel can have n delivery problems and I must get a list of Parcels through the DeliveryProblem side of the relation.
Obviously this causes the result to have many duplicated Parcels which I'd like to filter out. This is what I have tried using:
const problems = await DeliveryProblem.findAll({
limit: 20,
offset: (page - 1) * 20,
attributes: ['parcel_id', 'id'],
include: [
{
model: Parcel,
as: 'parcel',
distinct: ['parcel_id']
attributes: ['product', 'start_date'],
include: [
{
model: Deliveryman,
as: 'deliveryman',
attributes: ['name'],
},
],
},
],
});
It seems sequelize simply ignores the distinct property and keeps returning me the full set. Is there any other way to achieve the distinct set of rows?
You can use the distinct option with count and aggregate functions only.
Related
I have this model:
const UserSchema = new Schema({
following: [
{
type: Schema.Types.ObjectId,
ref: "users",
},
],
});
module.exports = User = mongoose.model("users", UserSchema);
I would like to use populate on the following field, BUT return only the first 5 elements of that array.
Is this possible to do this directly with populate?
I know I can do this array truncation lately after getting all the array elements. But, what I am doing is in a complex query, and I wouldn't like to complicate it any further.
Thank you.
https://mongoosejs.com/docs/populate.html#limit-vs-perDocumentLimit
Populate does support a limit option, however, it currently does not limit on a per-document basis for backwards compatibility. For example, suppose you have 5 following:
Limit
if you want to get a total of 5 following regardless of the number of users
const users = await User.find().populate({
path: 'following',
options: { limit: 5 }
});
That's because, in order to avoid executing a separate query for each document, Mongoose instead queries for fans using numDocuments * limit as the limit. If you need the correct limit, you should use the perDocumentLimit option (new in Mongoose 5.9.0). Just keep in mind that populate() will execute a separate query for each story, which may cause populate() to be slower.
perDocumentLimit
if you want to get a total of 5 following per user
A special option that tells Mongoose to execute a separate query
for each User to make sure we get 5 following for each user.
const users = await User.find().populate({
path: 'following',
options: { perDocumentLimit: 5 }
});
I have a table of items with foreign keys for two tables, and I try to do FindAll with order, limit, exclude and include, something like this:
Items.findAll({
order: [['creationDate', 'DESC']],
limit: 10,
include: [
{
model: Artists,
required: false,
attributes: ['id', 'name',]
},
{
model: Albums,
required: false,
},
],
attributes: { exclude: ['creationDate'] }
})
But I get an error:
DatabaseError [SequelizeDatabaseError]: (conn = 837, no: 1054, SQLState: 42S22) Unknown column 'Items.creationDate' in 'order clause'
If I delete this line:
attributes: { exclude: ['creationDate'] }
The error disappears.
What should I do to enable the query with the desired attributes?
Thanks in advance
findAll with include tries to combine data from multiple tables. In SQL, Subqueries and JOIN both achieves this use case. However, there is a slight difference in how data is combined and that what you are seeing here. By default, Sequelize generates Subqueries for findAll + include and subQuery params is to tell Sequelize use Subquery (true) or JOIN (false).
For more about Subqueries vs JOIN: https://www.essentialsql.com/what-is-the-difference-between-a-join-and-subquery/#:~:text=Joins%20versus%20Subqueries,tables%20into%20a%20single%20result.&text=Subqueries%20can%20be%20used%20to,for%20use%20in%20a%20query
Back to Sequelize,
subQuery: true (default)
This will generate SQL like following
SELECT ...
FROM (SELECT ...(attributes except creationDate) FROM `Items` ...)
LEFT OUTER JOIN `Artists` ...
...
ORDER BY `Items`.`creationDate` DESC
LIMIT 10;
This SQL doesn't work, since nested Subquery is executed first and the Subquery is excluding creationDate and when main SQL with ORDER is executed, creationDate does not exist.
Then if you turn off Subquery.
subQuery: false
This will generate SQL like following
SELECT ... (Artist's, Album's attributes and Item's attributes except creationDate)
FROM `Items`
LEFT OUTER JOIN `Artists` ...
...
ORDER BY `Items`.`creationDate` DESC
LIMIT 10;
This will work because this won't filter attributes when ORDER is executed and exclude happens at top level.
I have a model Booking, which is having hasMany relation with hotels, and hotel is having one to one relation with supppliers.
What i need is, get all booking where supplier_id = 33333.
I am trying this
BOOKINGS.findAll({
where: {
'hotels.supplier.supplier_id' : '32',
},
include: [
{
model: HOTELS,
include: [
{
model: SUPPLIERS,
],
}
],
limit : 30,
offset: 0
})
It throws error like hotels.supplier... column not found.. I tried all things because on docs of sequelze it only gives solution to add check which adds where inside the include which i can't use as it adds sub queries.
I don't want to add where check alongwith supplier model inside the include array, because it adds sub queries, so If i am having 1000 bookings then for all bookings it will add sub query which crashes my apis.
I need a solutions like this query in Sequelize.
Select col1,col2,col3 from BOOKINGS let join HOTELS on BOOKINGS.booking_id = HOTELS.booking_id, inner join SUPPLIERS on BOOKINGS.supplier_id = SUPPLIERS.supplier_id
Adding a where in the include object will not add a sub query. It will just add a where clause to the JOIN which is being applied to the supplier model. It will not crash your API in anyway. You can test it out on your local machine plenty of times to make sure.
BOOKINGS.findAll({
include: [
{
model: HOTELS,
include: [
{
model: SUPPLIERS,
where: { supplier_id: 32 }
}
]
}
],
limit: 30,
offset: 0
})
If you still want to use the query on the top level you can use sequelize.where+ sequelize.literal but you will need to use the table aliases that sequelize assigns. e.g this alias for supplier table will not work hotels.supplier.supplier_id. Sequelize assings table aliases like in the example I have shown below:
BOOKINGS.findAll({
where: sequelize.where(sequelize.literal("`hotels->suppliers`.supplier_id = 32")),
include: [
{
model: HOTELS,
include: [SUPPLIERS]
}
],
limit: 30,
offset: 0
})
I have gone through the Sequelize doc, but can't find anything helpful
What I want to do is to add raw query or custom model in include, is it possible ?
model.Post.findAll({
include: [{
model: model.User,
attributes: ['fullname', 'profilepic'],
},
{
model: model.PostComments,
},
{
model: "Raw Query"
}
]
}
What I want to achieve is like :
Select post_id, count(*) as total_likes from post_likes group by post_id
I can't achieve this by using simply include, so what I want to do is create a table/model from above query and then use it inside include.
If I use group by withing include it gives group by from top level, and I want to apply group by just for the post_like table.
Please let me know, if it found confusing or not clear.
I was looking to do the same, use a raw query dynamically formed inside the include but there's no possible way to use include without a model https://sequelize.org/master/class/lib/model.js~Model.html#static-method-findAll .
For the purposes of my MySQL I turned my inner join (include) into a where in. I was doing the inner join to avoid the exception This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery . I get this exception because I have a subquery with LIMIT applied.
If it helps anyone:
let rows = await j_model.findAll({
attributes: [...j_columns, `${association}.meta_key_id`],
include: [
{
model: um_model,
as: association,
attributes: ['um_id'],
on: {'j_id' : {$col: 'j.j_id'} }
],
where: {'j_id': {$in: sequelize.literal(`(SELECT * FROM (${massive_inner_raw_query}) as tmp)`)}},
logging: console.log
});
The actual magic is in this line:
where: {'j_id': {$in: sequelize.literal(`(SELECT * FROM (${massive_inner_raw_query}) as tmp)`)}}
The SELECT * FROM removes that exception and lets me do a where in instead of the wanted INNER JOIN. Maybe you can apply a similar deal to your problem.
You can use
Model.findAll({
attributes: [[models.sequelize.literal('CASE WHEN "field1" = true THEN 55
ELSE 23 END'), 'field3']]
}
OR
Model.findAll({
attributes: { include: [[models.sequelize.literal('CASE WHEN "field1" = true THEN 55 ELSE 23 END'), 'field3']]}
}
I'm trying to return a group of Models, paginated using limit and offset, including the grouped count of that model's favorites. A fairly trivial thing to attempt.
Here's my basic query setup with sequelize:
var perPage = 12;
var page = 1;
return Model.findAll({
group: [ 'model.id', 'favorites.id' ],
attributes: [
'*',
[ Sequelize.fn('count', Sequelize.col('favorites.id')), 'favorites_count' ]
],
include: [
{ attributes: [], model: Favorite },
],
offset: perPage * page
This generates the (fairly) expected query:
SELECT "model"."id",
"model".*,
Count("favorites"."id") AS "favorites_count",
"favorites"."id" AS "favorites.id"
FROM "model" AS "model"
LEFT OUTER JOIN "favorite" AS "favorites"
ON "model"."id" = "favorites"."model_id"
GROUP BY "model"."id",
"favorites"."id" offset 12;
Ignoring the fact that it quotes the tables, and that it selects favorites.id (forcing me to add it to the group by clause), and that it has randomly aliased things to their exact name or to a nonsensical name like "favorites.id" (all undesired), it seems to have worked. But now let's complete the pagination and add the limit to the query:
...
offset: perPage * page
limit: perPage
It now generates this query:
SELECT "model".*,
"favorites"."id" AS "favorites.id"
FROM (SELECT "model"."id",
"model".*,
Count("favorites"."id") AS "favorites_count"
FROM "model" AS "model"
GROUP BY "model"."id",
"favorites"."id"
LIMIT 12 offset 12) AS "model"
LEFT OUTER JOIN "favorite" AS "favorites"
ON "model"."id" = "favorites"."model";
In completely baffling behavior, it has generated an inner query and applied the limit only to that, then aliased that as "model".
As a sanity check I looked in the docs for findAll, but the docs do not seem to think that command exists.
I suspect I am doing something wrong, but I can't figure out what it is. This behavior is quite bizzarre, and I'm hoping my sleep deprivation is the cause of my confusion.
I'm using version 2.0.6
This turned out to be a bug in Sequelize. See https://github.com/sequelize/sequelize/issues/3623 and https://github.com/sequelize/sequelize/pull/3671