I'm trying to get pagination working properly, but for some reason, mongoose is returning zero items.
Here's the relevant code in nodejs:
if (req.query.name) {
match.name = req.query.name;
}
// initial pageSize is 2
// initial page is 1
pageSize = parseInt(req.query.pagesize);
page = parseInt(req.query.page);
const foods = await Food.find({ match })
.skip(pageSize * (page - 1))
.limit(pageSize);
res.send({
foods,
});
Now, I've used this same technique using .populate with a different route and have gotten the result I'm looking for like so:
await req.user
.populate({
path: 'pets',
options: {
limit: req.query.pagesize,
skip: req.query.pagesize * (req.query.page - 1),
},
})
.execPopulate();
I can't figure out what's wrong. There's 20 items in the database. If I remove the .limit and .skip and just have the .find, I retrieve all the items.
It seems like you are pasting in the wrong value to .find()
Shouldn't you place .find({name: req.query.name}) or .find(match) in your case??
Now it says ({match: {name: req.query.name}}) wich doesn't work I believe
Related
I am applying Mongoose Paginate on a page that was previously just displaying all records. My original code in my controller looks like this:
let allEvents = await Event.find(queryObject).where('date').gte(todayFormatted).sort({'date': 1, 'startTime': 1}).populate("vendors");
Applying the pagination, the new code looks like this:
const events = await Event.paginate(queryObject, {
page: req.query.page || 1,
limit: 30,
populate: 'vendors likes',
sort: { 'date': 1, 'startTime': 1 }
});
However, I still need to apply the condition 'where date is greater than or equal to todayFormatted'. But it seems that Paginate does not natively support where conditions; can anyone please advise how I could add a similar condition to filter the records (i.e. not display events with a date in the past)?
Thanks
you can always write it this way, you will be able to paginate with page (default of 1) and per (default of 30)
const { per, page } = req.query
const limit = parseInt(per || 30)
const skip = parseInt(per || 30) * ((parseInt(page) || 1) - 1)
queryObject.date = { $gte: todayFormatted }
let events = await Event
.find(queryObject)
.limit(limit)
.skip(skip)
.sort({ 'date': 1, 'startTime': 1 })
.populate("vendors");
if you are already filtering by date in the query, you can add $and
https://docs.mongodb.com/manual/reference/operator/query/and/
I have user document as this
users = [
{
_id:'',
name:'jay',
email:'jay#gmail.com',
role: 'actor',
status: true // isActive
},
{
_id:'',
name:'ram',
email:'ram123#gmail.com',
role: 'electrician',
status: false // isActive
},
...... so on
]
I want to apply pagination and also some filters to retrieve data
filter = {
role: 'actor',
order: -1 //descending sort,
sortOn: 'name' // apply sort on name field
search: 'ja', // match the string starting with 'ja',
status: true,
size:25,
page: 1 // means documents from 1-25, page2 means 26-50
}
How can this be achieved?
I am using mongoose as well.
Using your filter object you can do something like this:
Use these steps to ensure a good pagination:
Sort by any value (to ensure not get random positions)
Skip by the number of pages
Limit by the number of elements into page
So, the query will be something like (not tested but you can see the idea):
const elementsPerPage = filter.size
const nSkip = elementsPerPage * filter.page
const sort = {[filter.sortOn]:filter.order}
YourModel.find({/*yourquery*/})
.limit(elementsPerPage)
.skip(nSkip)
.sort(sort)
Also, you can use filter values into your query, something like:
YourModel.find({
role: filter.role,
status:filter.status,
name:{ $regex: filter.search}
})
This query is like this example.
Also, is not defined what calues do you want to use, the condition etc, so, with this, you can use if/else to add or not values into query.
For example:
var query = {}
if(filter.search){
query.name = {$regex: filter.search}
}
So all together can be:
const elementsPerPage = filter.size
const nSkip = elementsPerPage * filter.page
const sort = {[filter.sortOn]:filter.order}
var query = {}
if(filter.search){
query.name = {$regex: filter.search}
}
if(filter.role){
query.role = filter.role
}
if(filter.status){
query.status = filter.status
}
YourModel.find(query)
.limit(elementsPerPage)
.skip(nSkip)
.sort(sort)
Note that this has not been tested, but as I've said before you can see the idea with this example.
I have a store table that has a column lat_long which stores each store latitude/longitude as geometry point in sequelize. I want to be searching for stores based on proximity of the latitude/longitude from the request body from the front end. Have tried to exploit many related topics(answer) concerning this in Stackoverflow but what have learned is that the query can be written like this.
const location = sequelize.literal(`ST_GeomFromText('POINT(${lng} ${lat})', 4326)`);
User.findAll({
attributes: [[sequelize.fn('ST_Distance_Sphere', sequelize.literal('geolocation'), location),'distance']],
order: 'distance',
limit: 10,
logging: console.log
})
.then(function(instance){
console.log(instance);
})
Have tried it and it works fine but the challenge am having is that; in this example and almost all the sample I saw sofar they made use of the attribute which return only distance but I want to return all the fields in the table. Then I tried removing the attribute then the response I got was not searched based on the geometry(distance) and I also tried to use where but am getting an error.
Shown below is my latest query which is returning error please how can I go about this? thank you.
export const getNearestStoreWithCategory = async (lat, long) => {
try {
const location = sequelize.literal(`ST_GeomFromText('POINT(${lat} ${long})')`);
const distance = sequelize.fn('ST_Distance_Sphere', sequelize.literal('lat_long'), location)
const storeArr = await Store.findAll({
order: distance,
where: sequelize.where(distance),
limit: 20
})
return storeArr
} catch (error) {
return error
}
}
From the doc at https://sequelize.org/v4/manual/tutorial/querying.html, I'm guessing what you are missing in the sample code you had is including the distance to the fields to be returned as the response from the db rather that only computing the distance and returning;
const location = sequelize.literal(`ST_GeomFromText('POINT(${lng} ${lat})', 4326)`);
User.findAll({
attributes: {
include: [[sequelize.fn('ST_Distance_Sphere', sequelize.literal('geolocation'), location),'distance']]
},
order: 'distance',
limit: 10,
logging: console.log
})
.then(function(instance){
console.log(instance);
})
This should give a query like;
SELECT id, OTHER_FIELDS_OF_THIS_TABLE, ST_Distance_Sphere(geolocation) AS distance ...
Now, this is untested code, but you could give it a try and comment if it helps or not.
I am doing a really large Mongoose call with nested population, and then doing some processing on the data.
However, because of the nested populate call I am hitting the 16MB limit for a BSON document in Mongoose.
What's a solid way around this?
let allDocuments = await exampleModel.find({
condition: true
}).populate({path: 'tvShows', populate: {path: 'views'}});
Maybe I can break into into multiple calls? But I'm not sure a logical way to do that. Thanks.
You can implement your own paging using skip and limit query parameters.
const query = yourModel.find({ /* your conditions here */ });
const batchSize = 100;
function mergeAllReducer( accumulator, currentValue ) {
return accumulator.concat( currentValue );
}
query.count().then( total => {
let skip = 0;
const allQueries = [];
while ( skip < total ) {
allQueries.push( query.find()
.skip( skip )
.limit( batchSize )
.populate({ path: 'tvShows', populate: { path: 'views' }})
);
skip += batchSize;
}
return Promise.all( allQueries );
})
.then( arrayOfArrays => arrayOfArrays.reduce( mergeAllReducer, [] ))
.then( result => {
// do something with your populated result
});
Please note that you will still have to deal with all this memory being used and your javascript array size might still be larger than you VM can handle, so you could possibly try handling in batches as well, instead of working with the whole array.
I want to make a pagination feature in my Collection. How can find a documents with 'start' and 'limit' positions and get total document number in a single query?
You can't get both results in one query; the best you can do is to get them both using one Mongoose Query object:
var query = MyModel.find({});
query.count(function(err, count) {...});
query.skip(5).limit(10).exec('find', function(err, items) {...});
Use a flow control framework like async to cleanly execute them in parallel if needed.
You can use the plugin Mongoose Paginate:
$ npm install mongoose-paginate
After in your routes or controller, just add :
Model.paginate({}, { page: 3, limit: 10 }, function(err, result) {
// result.docs
// result.total
// result.limit - 10
// result.page - 3
// result.pages
});
If you plan to have a lot of pages, you should not use skip/limit, but rather calculate ranges.
See Scott's answer for a similar question: MongoDB - paging
UPDATE :
Using skip and limit is not good for pagination. Here is the discussion over it.
#Wes Freeman, Gave a good answer. I read the linked pose, you should use range query. n = 0; i = n+10; db.students.find({ "id" : { $gt: n, $lt: (n+i)} } );
OLD ANSWER (don't use this)
use something like
n = db.students.find().skip(n).limit(10);
//pass n dynamically, so for page 1 it should be 0 , page 2 it should be 10 etc
more documentation at http://www.mongodb.org/display/DOCS/Advanced+Queries
user.find({_id:{$nin:friends_id},_id:{$ne:userId}},function(err,user_details){
if (err)
res.send(err);
response ={
statusCode:200,
status:true,
values:user_details
}
res.json(response);
}).skip(10).limit(1);
I am using this function,
You can check if prev and next data is available or not.
async (req, res) => {
let { page = 1, size = 10 } = req.query
page = parseInt(page)
size = parseInt(size)
const query = {}
const totalData = await Model.find().estimatedDocumentCount()
const data = await Model.find(query).skip((page - 1) * size).limit(size).exec()
const pageNumber = Math.ceil(totalData / size)
const results = {
currentPage: page,
prevPage: page <= 1 ? null : page - 1,
nextPage: page >= pageNumber ? null : page + 1,
data
}
res.json(results) }
To know more estimatedDocumentCount