Here's my case.
I'm building a system in which you have users (consider them admins) that create employees and a performance report is created for every employee. The users can only view employees that were created by users from the same company. However, another company can create a report for an employee from a different company using a search field. Once a report is created for that employee, then they can view that employee within the employees list, if the report is not there, they wouldn't have seen it within that list.
Note: The model names are changed but have same characteristics
I'm trying to have an endpoint which returns all employees based on either of these two scenarios:
Return all employees created by the user which their company belong to the same company as the user initiating the request.
Return all employees that (if they don't belong to the same company as the user who created the employee) have a report associated to them and the report was created by a user from the same company as the user initiating the request.
Here's a brief ERD that explains the relationship between the models:
Here's the code which uses nested joins:
async function getAll(company) {
return await Employee.findAll({
where: {
[Op.or]: [
{
'$User.company$': { [Op.eq]: company }
},
{
'$Report.User.company$': { [Op.eq]: company }
}
]
},
include: [
{
attributes: [],
model: User.scope('withoutPassword'),
},
{
attributes: [],
model: Report,
include: {
attributes: [],
model: User.scope('withoutPassword'),
}
}
]
})
.then(employees => {
return employees
});
}
I'm trying to use the distinct clause to filter out some rows from a query. I have two models called Parcel and DeliveryProblem. Each parcel can have n delivery problems and I must get a list of Parcels through the DeliveryProblem side of the relation.
Obviously this causes the result to have many duplicated Parcels which I'd like to filter out. This is what I have tried using:
const problems = await DeliveryProblem.findAll({
limit: 20,
offset: (page - 1) * 20,
attributes: ['parcel_id', 'id'],
include: [
{
model: Parcel,
as: 'parcel',
distinct: ['parcel_id']
attributes: ['product', 'start_date'],
include: [
{
model: Deliveryman,
as: 'deliveryman',
attributes: ['name'],
},
],
},
],
});
It seems sequelize simply ignores the distinct property and keeps returning me the full set. Is there any other way to achieve the distinct set of rows?
You can use the distinct option with count and aggregate functions only.
In my database, I have the following relevant tables: users, listings, and reviews
users have many listings
listings have many reviews
reviews has a column value, which is a float between 0 and 5.
I'd like to be able to come up with a rating for a user, which is defined as:
The average value of all reviews for listings owned by the user.
In Sequelize, I have this:
User.findById(userId, {
include: [
{
model: Listing,
order: [
[Review, 'updatedAt', 'desc']
],
include: [
{
model: Review,
include: [User]
}
],
}
]
})
This successfully gets me:
All user fields
All listings owned by the user
All reviews for those listings
I do have enough information at this point to calculate the user average rating, but that's assuming that I never put any limit on the number of listings or reviews in this query. In order for the application to scale, I'll need limits, and so I have to be able to use aggregate functions to calculate the average rating on a listing and user.
I'm quite new to Sequelize and SQL in general. How might I accomplish this?
I have gone through the Sequelize doc, but can't find anything helpful
What I want to do is to add raw query or custom model in include, is it possible ?
model.Post.findAll({
include: [{
model: model.User,
attributes: ['fullname', 'profilepic'],
},
{
model: model.PostComments,
},
{
model: "Raw Query"
}
]
}
What I want to achieve is like :
Select post_id, count(*) as total_likes from post_likes group by post_id
I can't achieve this by using simply include, so what I want to do is create a table/model from above query and then use it inside include.
If I use group by withing include it gives group by from top level, and I want to apply group by just for the post_like table.
Please let me know, if it found confusing or not clear.
I was looking to do the same, use a raw query dynamically formed inside the include but there's no possible way to use include without a model https://sequelize.org/master/class/lib/model.js~Model.html#static-method-findAll .
For the purposes of my MySQL I turned my inner join (include) into a where in. I was doing the inner join to avoid the exception This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery . I get this exception because I have a subquery with LIMIT applied.
If it helps anyone:
let rows = await j_model.findAll({
attributes: [...j_columns, `${association}.meta_key_id`],
include: [
{
model: um_model,
as: association,
attributes: ['um_id'],
on: {'j_id' : {$col: 'j.j_id'} }
],
where: {'j_id': {$in: sequelize.literal(`(SELECT * FROM (${massive_inner_raw_query}) as tmp)`)}},
logging: console.log
});
The actual magic is in this line:
where: {'j_id': {$in: sequelize.literal(`(SELECT * FROM (${massive_inner_raw_query}) as tmp)`)}}
The SELECT * FROM removes that exception and lets me do a where in instead of the wanted INNER JOIN. Maybe you can apply a similar deal to your problem.
You can use
Model.findAll({
attributes: [[models.sequelize.literal('CASE WHEN "field1" = true THEN 55
ELSE 23 END'), 'field3']]
}
OR
Model.findAll({
attributes: { include: [[models.sequelize.literal('CASE WHEN "field1" = true THEN 55 ELSE 23 END'), 'field3']]}
}
I'm managing a MongoDB database for a building products store. The most immediate collection is products, right?
There are quite several products, however they all belong to one among a set of 5-8 categories and then to one subcatefory among a small set of subcategories.
For example:
-Electrical
*Wires
p1
p2
..
*Tools
p5
pn
..
*Sockets
p11
p23
..
-Plumber
*Pipes
..
*Tools
..
PVC
..
I will use Angular at web site client side to show whole products catalog, I think about AJAX for querying the right subset of products I want.
Then, I wonder whether I should manage one only collection like:
{
MainCategory1: {
SubCategory1: {
{},{},{},{},{},{},{}
}
SubCategory2: {
{},{},{},{},{},{},{}
}
SubCategoryn: {
{},{},{},{},{},{},{}
}
},
MainCategory2: {
SubCategory1: {
{},{},{},{},{},{},{}
}
SubCategory2: {
{},{},{},{},{},{},{}
}
SubCategoryn: {
{},{},{},{},{},{},{}
}
},
MainCategoryn: {
SubCategory1: {
{},{},{},{},{},{},{}
}
SubCategory2: {
{},{},{},{},{},{},{}
}
SubCategoryn: {
{},{},{},{},{},{},{}
}
}
}
Or a single collection per each category. The number of documents might not be higher than 500. However I care about a balance for:
quick DB answer,
easy server side DB querying, and
client-side Angular code for rendering results to html.
I'm using mongodb node.js module, not Mongoose now.
What CRUD operations will I do?
Inserts of products, I'd also like to have a way to obtain autogenerated ids (maybe sequential) per each new register. However, as it might seem natural I wouldn't offer the _id to the user.
Querying the whole documents set of a subcategory. Maybe just obtaining a few attributes at first.
Querying whole or a specific subset of attributes of a document (product) in particular.
Modifying a product's attributes values.
I agree client side should get the easiest result to render. However, to nest categories into products is still a bad idea. The trade off is once you want to change, for example, the name of a category, it will be a disaster. And if you think about the possible usecases, for example:
list all categories
find all subcategories of a certain category
find all products in a certain category
You'll find it hard to do these stuff with your data structure.
I had same situation in my current project. So here's what I do for your reference.
First, categories should be in a separate collection. DON'T nest categories into each other, as it will complicate the procedure to find all subcategories. The traditional way for finding all subcategories is to maintain an idPath property. For example, your categories are divided into 3 levels:
{
_id: 100,
name: "level1 category"
parentId: 0, // means it's the top category
idPath: "0-100"
}
{
_id: 101,
name: "level2 category"
parentId: 100,
idPath: "0-100-101"
}
{
_id: 102,
name: "level3 category"
parentId: 101,
idPath: "0-100-101-102"
}
Note with idPath, parentId is not necessary anymore. It's for you to understand the structure easier.
Once you need to find all subcategories of category 100, simply do the query:
db.collection("category").find({_id: /^0-100-/}, function(err, doc) {
// whatever you want to do
})
With category stored in a separate collection, in your product you'll need to reference them by _id, just like when we use RDBMS. For example:
{
... // other fields of product
categories: [100, 101, 102, ...]
}
Now if you want to find all products in a certain category:
db.collection("category").find({_id: new RegExp("/^" + idPath + "-/"}, function(err, categories) {
var cateIds = _.pluck(categories, "_id"); // I'm using underscore to pluck category ids
db.collection("product").find({categories: { $in: cateIds }}, function(err, products) {
// products are here
}
})
Fortunately, category collection is usually very small, with only hundreds of records inside (or thousands). And it doesn't varies a lot. So you can always store a live copy of categories inside memory, and it can be constructed as nested objects like:
[{
id: 100,
name: "level 1 category",
... // other fields
subcategories: [{
id: 101,
... // other fields
subcategories: [...]
}, {
id: 103,
... // other fields
subcategories: [...]
},
...]
}, {
// another top1 category
}, ...]
You may want to refresh this copy every several hours, so:
setTimeout(3600000, function() {
// refresh your memory copy of categories.
});
That's all I get in mind right now. Hope it helps.
EDIT:
to provide int ID for each user, $inc and findAndModify is very useful. you may have a idSeed collection:
{
_id: ...,
seedValue: 1,
forCollection: "user"
}
When you want to get an unique ID:
db.collection("idSeed").findAndModify({forCollection: "user"}, {}, {$inc: {seedValue: 1}}, {}, function(err, doc) {
var newId = doc.seedValue;
});
The findAndModify is an atomic operator provided by mongodb. It will guarantee thread safety. and the find and modify actually happens in a "transaction".
2nd question is in my answer already.
query subsets of properties is described with mongodb Manual. NodeJS API is almost the same. Read the document of projection parameter.
update subsets is also supported by $set of mongodb operator.