Implementing mongodb pagination along with match query? - node.js

I have user document as this
users = [
{
_id:'',
name:'jay',
email:'jay#gmail.com',
role: 'actor',
status: true // isActive
},
{
_id:'',
name:'ram',
email:'ram123#gmail.com',
role: 'electrician',
status: false // isActive
},
...... so on
]
I want to apply pagination and also some filters to retrieve data
filter = {
role: 'actor',
order: -1 //descending sort,
sortOn: 'name' // apply sort on name field
search: 'ja', // match the string starting with 'ja',
status: true,
size:25,
page: 1 // means documents from 1-25, page2 means 26-50
}
How can this be achieved?
I am using mongoose as well.

Using your filter object you can do something like this:
Use these steps to ensure a good pagination:
Sort by any value (to ensure not get random positions)
Skip by the number of pages
Limit by the number of elements into page
So, the query will be something like (not tested but you can see the idea):
const elementsPerPage = filter.size
const nSkip = elementsPerPage * filter.page
const sort = {[filter.sortOn]:filter.order}
YourModel.find({/*yourquery*/})
.limit(elementsPerPage)
.skip(nSkip)
.sort(sort)
Also, you can use filter values into your query, something like:
YourModel.find({
role: filter.role,
status:filter.status,
name:{ $regex: filter.search}
})
This query is like this example.
Also, is not defined what calues do you want to use, the condition etc, so, with this, you can use if/else to add or not values into query.
For example:
var query = {}
if(filter.search){
query.name = {$regex: filter.search}
}
So all together can be:
const elementsPerPage = filter.size
const nSkip = elementsPerPage * filter.page
const sort = {[filter.sortOn]:filter.order}
var query = {}
if(filter.search){
query.name = {$regex: filter.search}
}
if(filter.role){
query.role = filter.role
}
if(filter.status){
query.status = filter.status
}
YourModel.find(query)
.limit(elementsPerPage)
.skip(nSkip)
.sort(sort)
Note that this has not been tested, but as I've said before you can see the idea with this example.

Related

Pulling data with 'Where' and 'Include' statements at the same time

I have managed to get my include statements working with my foreign keys however when I try to add a 'where' statement to the findAll statement I am getting the below error. I have checked my foreign keys and models numerous times and I can't see any problems with them.
Do I have the syntax correct in the below API? The data in the keys is important so I don't want to have to spend more time on a workaround. I have seen other examples that have the where statement inside one of the include keys but the fields I want pulled are on the main table so I don't think this is the same situation.
Unhandled rejection Error: Include unexpected. Element has to be either a Model, an Association or an object.
API
const sequelize = require("pg");
const { Sequelize, Op, Model, DataTypes } = require("sequelize");
exports.schoolDiveLogApproval = (req, res) => {
try {
const schoolID = req.params.schoolID;
diveLog.findAll({
include: [{
model: diveType, as: 'diveTypeID_FK2',
}, {
model: diveSchool, as: 'diveSchoolID_FK',
}, {
model: current, as: 'currentID_FK',
}, {
model: visibility, as: 'visibilityID_FK',
}, {
model: user, as: 'userID_FK1',
}, {
model: diveSpot, as: 'diveSpotID_FK2'
}],
where: {
[Op.and]: [
{diveSchoolID: schoolID},
{diveVerifiedBySchool: false}
]
},
})
...........
Update
Error message
{
"message": "missing FROM-clause entry for table \"diveLog\""
}
Controller API with raw SQL
exports.schoolDiveLogApproval = async (req, res) => {
try {
const schoolID = req.params.schoolID;
const { QueryTypes } = require('sequelize');
await diveLog.sequelize.query(
'SELECT * '+
'FROM "diveLogs" '+
'LEFT OUTER JOIN "diveTypes" AS "diveTypeID_FK2" ON "diveLogs"."diveTypeID" = "diveTypeID_FK2"."diveTypeID" ' +
'LEFT OUTER JOIN "diveSchools" AS "diveSchoolID_FK" ON "diveLog"."diveSchoolID" = "diveSchoolID_FK"."diveSchoolID" ' +
'LEFT OUTER JOIN "currents" AS "currentID_FK" ON "diveLog"."currentID" = "currentID_FK"."currentID" ' +
'LEFT OUTER JOIN "visibilities" AS "visibilityID_FK" ON "diveLog"."visibilityID" = "visibilityID_FK"."visibilityID" ' +
'LEFT OUTER JOIN "userLogins" AS "userID_FK" ON "diveLog"."userID" = "userID_FK1"."userID" ' +
'LEFT OUTER JOIN "diveSpots" AS "diveSpotID_FK2" ON "diveLog"."diveSpotID" = "diveSpotFK2"."diveSpotID" ' +
'WHERE "diveLogs"."diveSchoolID" = ? ' +
'AND "diveLogs"."diveVerifiedBySchool" = ?',
{
replacements: [schoolID, false],
type: QueryTypes.SELECT
}
)
All the table and column names look correct as per the pgAdmin database so I can't see where the problem could be other than the syntax.
I have went off the way the SQL is executed in the IDE terminal so it should execute in theory. Is this more likely be to do with the way I pass the id?
Well, I've really tried some minutes around this sequelize, but I'd really suggest you to learn and use raw SQL, its way easier and universal than that.
After half an hour of trying to set up and understand this sqlize and see so many weird issues like these:
selecting non-existing columns like id, adding an "s" out of nowhere to the name of the table "user_infoS(?)".. so I gave up and came up with a query that might help you, if I understood your problem correctly.
If you send proper schema I can help you further, but yeah, sorry I can't spend more time in this sequelize, this thing is atrocious.
const { QueryTypes } = require('sequelize');
await sequelize.query(
'SELECT *'+
'FROM diveLog l'+
'JOIN diveType t ON l.diveTypeId = t.id'+
'JOIN diveSchool s ON l.diveSchoolId = s.id'+
'JOIN current c ON l.currentId = c.id'+
'JOIN visibility v ON l.visibilityId = v.id'+
'JOIN user u ON l.userId = u.id'+
'JOIN diveSpot sp ON l.diveSpotId = sp.id'+
'WHERE diveSchoolID = ?'+
'AND diveVerifiedBySchool = ?',
{
replacements: [schoolID, false],
type: QueryTypes.SELECT
}
);
With regards to the first error in your API, ensure you are requiring your models like below:
const { diveLog, diveType, diveSchool, diveSchool, current, visibility, user, diveSpot } = require('../models');
The second error message:
{
"message": "relation "divelogs" does not exist"
}
Indicates that the table "divelogs" cannot be found in your database, double check if the table exists, if it does not exist, run your migrations to create the table. Please note that, if enabled Sequelize will automatically make your table names plural. If you define a model called User, Sequelize will create the table as Users
When I went back to using the original Sequelize with the include / where clauses I removed the [Op and] from the API, then re-declared each of the models at the top of the file and it worked.
exports.schoolDiveLogApproval = async (req, res) => {
try {
const schoolID = req.params.schoolID;
diveLog.findAll({
include: [{
model: diveType, as: 'diveTypeID_FK2',
}, {
model: diveSchool, as: 'diveSchoolID_FK',
}, {
model: current, as: 'currentID_FK',
}, {
model: visibility, as: 'visibilityID_FK',
}, {
model: user, as: 'userID_FK1',
}, {
model: diveSpot, as: 'diveSpotID_FK2'
}],
where: [{
diveSchoolID: req.params.schoolID,
diveVerifiedBySchool: false
}],
})
.then((diveLog) => {
const schoolDiveLogList = [];
for (i = 0; i < diveLog.length; i++) {
Second error: you are committing a typo on aliases using sometimes "devLogS" and other times "devLoG" (in the singular).
Try more before asking here, we won't do your work for you.

how to query mongoDb with array of boolean fields?

My mongoose model schema looks like -
{
email: { type: String},
Date: {type: Date},
isOnboarded: {type: Boolean},
isVip: {type: Boolean},
isAdult: {type: Boolean}
}
In my frontend I have 3 checkboxes for "isVip", "isOnboarded" and "isAdult" options. If they are checked I'm adding them to an array, which I'll pass to the server. Let's say if "isVip" and "isAdult" are checked, I will pass [isVip, isAdult] in post api to server. Now how can I write a query to get all the documents with the fields in array as true i.e in above example how can I retrieve all docs with {isVip: true, isAdult:true}
I'm having trouble because the array values keep changing, it can be only one field or 3 fields. I couldn't find a way to give condition inside mongoose query.
User.find(
{ [req.query.array]: true},
{ projection: { _id: 0 } }
)
User is my mongoose model.
I want something like this (documents with the value 'true' for the fields given in the array) and 'req.query.array' is the array with field names I passed from frontend.
You have to create your object in JS and pass then to mongo in this way:
var query = {}
if(isVip) query["isVip"] = true;
if(isOnboarded) query["isOnboarded"] = true;
if(isAdult) query["isAdult"] = true;
And then you can use the mongoose method you want, for example:
var found = await model.find(query)
And this will return the document that matches the elements.
Also, note that this is to create and object to be read by the query, so you can use this into an aggregation pipeline or whatever you vant
Check the output:
var query = {}
query["isVip"] = true;
query["isOnboarded"] = true;
query["isAdult"] = true;
console.log(query)
Is the same object that is used to do the query here
{
"isVip": true,
"isOnboarded": true,
"isAdult": true
}
Also, to know if your post contains "isVip", "isOnboarded" or "isAdult" is a javascript question, not a mongo one, but you can use something like this (I assume you pass a string array):
var apiArray = ["isVip","isAdult"]
var isVip = apiArray.includes("isVip")
var isAdult = apiArray.includes("isAdult")
var isOnboarded = apiArray.includes("isOnboarded")
console.log("isVip: "+isVip)
console.log("isAdult: "+isAdult)
console.log("isOnboarded: "+isOnboarded)

Mongoose, NodeJS & Express: Sorting by column given by api call

I'm currently writing a small API for a cooking app. I have a Recipe model and would like to implement sorting by columns based on the req Parameter given.
I'd like to sort by whatever is passed in the api call. the select parameter works perfectly fine, I can select the columns to be displayed but when I try to sort anything (let's say by rating) the return does sort but I'm not sure what it does sort by.
The code i'm using:
query = Recipe.find(JSON.parse(queryStr));
if(req.query.select){
const fields = req.query.select.split(',').join(' ');
query = query.select(fields);
}
if(req.query.sort){
const sortBy = req.query.sort.split(',').join(' ');
query = query.sort({ sortBy: 1 });
} else {
query = query.sort({ _id: -1 });
}
The result, when no sorting is set: https://pastebin.com/rPLv8n5s
vs. the result when I pass &sort=rating: https://pastebin.com/7eYwAvQf
also, when sorting my name the result is also mixed up.
You are not using the value of sortBy but the string "sortBy". You will need to create an object that has the rating as an object key.
You need the sorting object to look like this.
{
rating: 1
}
You can use something like this so it will be dynamic.
if(req.query.sort){
const sortByKey = req.query.sort.split(',').join(' ');
const sortByObj = {};
sortByObj[sortByKey] = 1; // <-- using sortBy as the key
query = query.sort(sortByObj);
} else {
query = query.sort({ _id: -1 });
}

Is there a way I can use Group By and Count with Type Orm Repository

I am new here recently joined and New in Type ORM My code that I am trying
FIRST QUERY: I would like to use this approach but not sure how I can group by with count on the column and then order by desc on that column
const result = await this.jobViewsRepository.find({
relations: ["jobs"],
loadEagerRelations: true,
order: { id: "DESC" },
skip: offset,
take: limit,
}
);
I am trying if I can use this in my above query
SECOND QUERY: IT'S WORKING FOR ME PERFECTLY THE RESULT I AM LOOKING
const res = await this.jobViewsRepository.createQueryBuilder('jobViews')
.addSelect("COUNT(jobViews.user_id) AS jobViews_total_count" )
.leftJoinAndSelect(Jobs, "jobs", "jobs.id = jobViews.job_id")
.where("jobs.user_id != :id", { id: user_id })
.groupBy("jobViews.job_id")**
.orderBy('jobViews_total_count', 'DESC')**
.limit(limit)
.offset(offset)
.getRawMany();
Please if any can help me out in this will be really appreciated
Thanks
At least in the current version there is no way to do this feature (neither in the documentation nor in the web)
I believe you can use .query to write your own query
Now is the only one way is to use queryBuilder And .groupBy("user.id") with .addGroupBy("user.id")
https://orkhan.gitbook.io/typeorm/docs/select-query-builder#adding-group-by-expression
Or write raw query:
import { getManager } from 'typeorm';
const entityManager = getManager();
const someQuery = await entityManager.query(`
SELECT
fw."X",
fw."Y",
ew.*
FROM "table1" as fw
JOIN "table2" as ew
ON fw."X" = $1 AND ew.id = fw."Y";
`, [param1]);
you use findAndCount to count the result size
result = await this.jobViewsRepository.findAndCount({ ... })
the result = [data,count]

Filter, sort, limit, skip subdocument mongoose

const A = mongoose.Schema({
...
bs: [B.schema],
...
});
So basically i have two schemas, and one is subdocument of another.
From my datatable i get params. like filter, page limit, page, sort...
What i need to do is, to create query that will with _id from A schema get all his B schemas and always sort, limit, skip, filter with params. that i sent
I tried something like this
b = await A.find({'_id' : idA},
{ 'bs' :
{ $slice: [ offset * limit, limit ]
}
});
And it's working but i can't still figure out how to filter and sort.
So if somebody have some idea welcome to share.
P.S Sorry for bad english
Regards,
Salesh
What you're trying to do is not find A documents that fulfill your array criteria, but to modify the results to accommodate to your needs. You can do this with two approaches, depending on where you want the processing to be done:
1. Use MongoDB Aggregation. The processing is done in the DB.
The aggregation pipeline is a series of steps you determine that documents go through being queried and transformed.
A rough untested (and probably syntactically wrong) example would be:
A.aggregate([
{ $match: { _id: "id" }},
{ $project: {
bs: {
$filter: { input: "$bs" , as: "filteredBs" , cond: { /* conditions object */} }},
}
},
{ $slice: ["$filteredBs", offset * limit, limit ] }
/* ... */
]);
2. Get the document by Id and process the array on your server.
Here you're just limited by javascript and its array capabilites.
const found = A.findById('id');
const bs = A.bs.filter( /* filter function */ ).slice() // ... whatever you want.
A.bs = bs;
return A;

Resources