I have this problem , i don't know how to make a pagination in Mongoose text search module, i am totally beginner, so please help, this is my code:
searchModul.create({ title: title, description: description }, function (err) {
if (err) return handleError(err);
searchModul.textSearch(title, function (err, output) {
if (err) return handleError(err);
res.render('search', {
title : 'title',
results : output.results
});
});
});
and also i want to know how to display that pagination in the search.ejs view. and thanks in advance.
To implement pagination, use $limit to define a limit for each query, and $skip to navigate pages.
From the docs:
$limit takes a positive integer that specifies the maximum number of documents to pass along.
$skip takes a positive integer that specifies the maximum number of documents to skip.
There are previous questions like this one, check out the answers here and a more detailed tutorial here.
Hope that helps!
I think you want something like this:
searchModul.find({ $text: { $search: title }}).skip(50).limit(50).exec(function (err, output) {
if (err) return handleError(err);
res.render('search', {
title : 'title',
results : output.results
});
});
This will return the second 50 items that match. .skip(50) the number of items skip and .limit(50) is the number of items to return.
const field = req.query.field;
const search = {};
search[field] = req.query.searchValue;
search["role"] = req.params._role;
search["status"] = parseInt(req.query.status);
user = await Users.paginate(search, {
limit: parseInt(req.query.limit),
page: parseInt(req.query.page),
});
OR
var ObjectId = mongoose.Types.ObjectId;
let _id = new ObjectId(req.query.q);;
user = await Users.paginate(
{
role: req.params._role,
_id: _id,
},
{
limit: parseInt(req.query.limit),
page: parseInt(req.query.page),
});
Related
Apologies if incorrect wording. I have a question regarding MongoDb query. Say I want to get a list of books with specified isbn if passed. If not passed, I would still want the list based on other parameters. Below is my code snippet
async findBooks(rawNameValues) {
try {
const nameValues = this._validate('findBooks', rawNameValues);
//#TODO
console.log('find books', nameValues);
const authorsTitle = new RegExp(`.*${nameValues.authorsTitleSearch}.*`);
console.log('Authors Title', authorsTitle);
const cursor = await this.db.collection(COLLECTIONS.BOOK_CATALOG).find(
{
isbn: '' || nameValues.isbn.toString(),
title: {$regex: authorsTitle}
}
).toArray();
return cursor || [];
} catch (err) {
console.log('Caught Error', err);
}}
However it get caught if no isbn is passed. Any help would be appreciated.
try this before filter option for existance of isbn:
let custom_filter = nameValues.isbn ? {isbn:
nameValues.isbn.toString(), title: {$regex: authorsTitle}} :
{title: {$regex: authorsTitle}};
let query = await
this.db.collection(COLLECTIONS.BOOK_CATALOG).find(custom_filter)
I have a json data which contains many objects. I want to limit the data for pagination and I need the total items count. Please help.
Model.find().skip((pageNumber-1)*limit).limit(limit).exec()
I want the count and skipped data in response.
You can use async library for running 2 queries at once. In your case you can run one query to get the number of documents and another for pagination.
Example with 'User' model:
var async = require('async');
var User = require('./models/user');
var countQuery = function(callback){
User.count({}, function(err, count){
if(err){ callback(err, null) }
else{
callback(null, count);
}
}
};
var retrieveQuery = function(callback){
User.find({}).skip((page-1)*PAGE_LIMIT)
.limit(PAGE_LIMIT)
.exec(function(err, doc){
if(err){ callback(err, null) }
else{
callback(null, doc);
}
}
};
async.parallel([countQuery, retrieveQuery], function(err, results){
//err contains the array of error of all the functions
//results contains an array of all the results
//results[0] will contain value of doc.length from countQuery function
//results[1] will contain doc of retrieveQuery function
//You can send the results as
res.json({users: results[1], pageLimit: PAGE_LIMIT, page: page, totalCount: results[0]});
});
async allows you to run a number of queries in parallel depending on the hardware you are using. This would be faster than using 2 independent queries to get count and get the required documents.
Hope this helps.
I have solved it with $facet and aggregate the following way in mongoose v3+:
const [{ paginatedResult, [{ totalCount }] }] = await Model.aggregate([{
$facet: {
paginatedResult: [
{ $match: query },
{ $skip: skip },
{ $limit: limit }
],
totalCount: [
{ $match: query },
{ $count: 'totalCount' }
]
}
}])
where the totalCount refers the total number of records matching the search query while the paginatedResult is only the paginated slice of them.
The problem with these solutions is that for every request you are doing two queries. This becomes problematic when you have a complex data structure and large data set as performance becomes an issue. Consider instead creating a special function that listens for the /resource?count=true or /resource/count GET methods and returns only the count.
You need to perform 2 queries to achieve that. One to get results and another to get total items amount with .count().
For example code you can watch at on of "paginator" for mongoose mongoose-paginate.
To performe only one query, you may use the find() method associated with promises and array slices. A small example would be:
getPaginated(query, skip, limit){
return this.model.find(query)
.lean()
.then((value)=>{
if (value.length === 0) return {userMessage: 'Document not found'};
const count = value.length;
//skip===0 must be handled
const start = parseInt(limit)*parseInt(skip - 1);
const end = start + parseInt(reqQuery.pagesize);
//slicing the array
value = value.slice(start,end);
//could return it another way...
value.push( { 'querySize': count });
return value;
})
.catch((reason)=>{
//...handling code
});
}
I have a function that is needed to get results.
When I give 1 as _id filter everything is OK.
collectionPersonnel
.find({ '_id' : 1 })
.toArray(function (err, personnel) {
console.log(personnel);
});
If I give filter another way for instance user[0]['personnel_id'] -that is store 1- then I get only [] result;
collectionPersonnel
.find({ '_id' : user[0]['personnel_id'] })
.toArray(function (err, personnel) {
console.log(personnel);
});
And then I've tried another way. But it doesn't work because I used a string(user[0]['personnel_id']) instead of an ObjectID.
var ObjectID = require('mongodb').ObjectID;
var personnelPK_Hex = (user[0]['personnel_id']).toHexString();
var personnelPK = ObjectID.createFromHexString(personnelPK_Hex);
What should I do?
Edit
All of my codes are below;
module.exports = {
show: function(req, res) {
User.native(function(err, collectionUser) {
if(err) {
console.log("There is no exist a User by current_id");
};
collectionUser
.find({'_id' : req.param('id')})
.toArray(function (err, user) {
Personnel.native(function(err, collectionPersonnel) {
if(err) {
// handle error getting mongo collection
console.log("There is no exist a Personel by current _id");
};
if(!collectionPersonnel) {
console.log("There is no exist a Personel by current _id");
};
// var ObjectID = require('mongodb').ObjectID;
// var personnelPK_Hex = (user[0]['personnel_id']).toHexString();
// var personnelPK = ObjectID.createFromHexString(personnelPK_Hex);
collectionPersonnel
.find({ '_id' : user[0].personnel_id })
.toArray(function (err, personnel) {
console.log(personnel);
});
});
});
});
}
};
And console's output is;
[]
Solved
Just like apsillers's said. I had given a numeric _id to collection, incorrectly.
I've fixed _id value and everything is OK.
Thank you all...
user[0]['personnel_id'] might be a string. For Mongo, "1" is different from 1, which is why your literal number 1 worked, but your variable (which holds a string) does not.
Instead, try using a unary plus to convert the string to a number: +user[0]['personnel_id'].
try to use like user[0].personal_id instead of user[0]['personnel_id'] please provide your schema design that would be better to figure out what exactly you are missing.
i tried like this
collectionPersonnel
.find({ '_id' : user[0].personnel_id })
.toArray(function (err, personnel) {
console.log(personnel);
});
I've got myself a question regarding associations in Sails.js version 0.10-rc5. I've been building an app in which multiple models are associated to one another, and I've arrived at a point where I need to get to nest associations somehow.
There's three parts:
First there's something like a blog post, that's being written by a user. In the blog post I want to show the associated user's information like their username. Now, everything works fine here. Until the next step: I'm trying to show comments which are associated with the post.
The comments are a separate Model, called Comment. Each of which also has an author (user) associated with it. I can easily show a list of the Comments, although when I want to display the User's information associated with the comment, I can't figure out how to populate the Comment with the user's information.
In my controller i'm trying to do something like this:
Post
.findOne(req.param('id'))
.populate('user')
.populate('comments') // I want to populate this comment with .populate('user') or something
.exec(function(err, post) {
// Handle errors & render view etc.
});
In my Post's 'show' action i'm trying to retrieve the information like this (simplified):
<ul>
<%- _.each(post.comments, function(comment) { %>
<li>
<%= comment.user.name %>
<%= comment.description %>
</li>
<% }); %>
</ul>
The comment.user.name will be undefined though. If I try to just access the 'user' property, like comment.user, it'll show it's ID. Which tells me it's not automatically populating the user's information to the comment when I associate the comment with another model.
Anyone any ideals to solve this properly :)?
Thanks in advance!
P.S.
For clarification, this is how i've basically set up the associations in different models:
// User.js
posts: {
collection: 'post'
},
hours: {
collection: 'hour'
},
comments: {
collection: 'comment'
}
// Post.js
user: {
model: 'user'
},
comments: {
collection: 'comment',
via: 'post'
}
// Comment.js
user: {
model: 'user'
},
post: {
model: 'post'
}
Or you can use the built-in Blue Bird Promise feature to make it. (Working on Sails#v0.10.5)
See the codes below:
var _ = require('lodash');
...
Post
.findOne(req.param('id'))
.populate('user')
.populate('comments')
.then(function(post) {
var commentUsers = User.find({
id: _.pluck(post.comments, 'user')
//_.pluck: Retrieves the value of a 'user' property from all elements in the post.comments collection.
})
.then(function(commentUsers) {
return commentUsers;
});
return [post, commentUsers];
})
.spread(function(post, commentUsers) {
commentUsers = _.indexBy(commentUsers, 'id');
//_.indexBy: Creates an object composed of keys generated from the results of running each element of the collection through the given callback. The corresponding value of each key is the last element responsible for generating the key
post.comments = _.map(post.comments, function(comment) {
comment.user = commentUsers[comment.user];
return comment;
});
res.json(post);
})
.catch(function(err) {
return res.serverError(err);
});
Some explanation:
I'm using the Lo-Dash to deal with the arrays. For more details, please refer to the Official Doc
Notice the return values inside the first "then" function, those objects "[post, commentUsers]" inside the array are also "promise" objects. Which means that they didn't contain the value data when they first been executed, until they got the value. So that "spread" function will wait the acture value come and continue doing the rest stuffs.
At the moment, there's no built in way to populate nested associations. Your best bet is to use async to do a mapping:
async.auto({
// First get the post
post: function(cb) {
Post
.findOne(req.param('id'))
.populate('user')
.populate('comments')
.exec(cb);
},
// Then all of the comment users, using an "in" query by
// setting "id" criteria to an array of user IDs
commentUsers: ['post', function(cb, results) {
User.find({id: _.pluck(results.post.comments, 'user')}).exec(cb);
}],
// Map the comment users to their comments
map: ['commentUsers', function(cb, results) {
// Index comment users by ID
var commentUsers = _.indexBy(results.commentUsers, 'id');
// Get a plain object version of post & comments
var post = results.post.toObject();
// Map users onto comments
post.comments = post.comments.map(function(comment) {
comment.user = commentUsers[comment.user];
return comment;
});
return cb(null, post);
}]
},
// After all the async magic is finished, return the mapped result
// (or an error if any occurred during the async block)
function finish(err, results) {
if (err) {return res.serverError(err);}
return res.json(results.map);
}
);
It's not as pretty as nested population (which is in the works, but probably not for v0.10), but on the bright side it's actually fairly efficient.
I created an NPM module for this called nested-pop. You can find it at the link below.
https://www.npmjs.com/package/nested-pop
Use it in the following way.
var nestedPop = require('nested-pop');
User.find()
.populate('dogs')
.then(function(users) {
return nestedPop(users, {
dogs: [
'breed'
]
}).then(function(users) {
return users
}).catch(function(err) {
throw err;
});
}).catch(function(err) {
throw err;
);
Worth saying there's a pull request to add nested population: https://github.com/balderdashy/waterline/pull/1052
Pull request isn't merged at the moment but you can use it installing one directly with
npm i Atlantis-Software/waterline#deepPopulate
With it you can do something like .populate('user.comments ...)'.
sails v0.11 doesn't support _.pluck and _.indexBy use sails.util.pluck and sails.util.indexBy instead.
async.auto({
// First get the post
post: function(cb) {
Post
.findOne(req.param('id'))
.populate('user')
.populate('comments')
.exec(cb);
},
// Then all of the comment users, using an "in" query by
// setting "id" criteria to an array of user IDs
commentUsers: ['post', function(cb, results) {
User.find({id:sails.util.pluck(results.post.comments, 'user')}).exec(cb);
}],
// Map the comment users to their comments
map: ['commentUsers', function(cb, results) {
// Index comment users by ID
var commentUsers = sails.util.indexBy(results.commentUsers, 'id');
// Get a plain object version of post & comments
var post = results.post.toObject();
// Map users onto comments
post.comments = post.comments.map(function(comment) {
comment.user = commentUsers[comment.user];
return comment;
});
return cb(null, post);
}]
},
// After all the async magic is finished, return the mapped result
// (or an error if any occurred during the async block)
function finish(err, results) {
if (err) {return res.serverError(err);}
return res.json(results.map);
}
);
You could use async library which is very clean and simple to understand. For each comment related to a post you can populate many fields as you want with dedicated tasks, execute them in parallel and retrieve the results when all tasks are done. Finally, you only have to return the final result.
Post
.findOne(req.param('id'))
.populate('user')
.populate('comments') // I want to populate this comment with .populate('user') or something
.exec(function (err, post) {
// populate each post in parallel
async.each(post.comments, function (comment, callback) {
// you can populate many elements or only one...
var populateTasks = {
user: function (cb) {
User.findOne({ id: comment.user })
.exec(function (err, result) {
cb(err, result);
});
}
}
async.parallel(populateTasks, function (err, resultSet) {
if (err) { return next(err); }
post.comments = resultSet.user;
// finish
callback();
});
}, function (err) {// final callback
if (err) { return next(err); }
return res.json(post);
});
});
As of sailsjs 1.0 the "deep populate" pull request is still open, but the following async function solution looks elegant enough IMO:
const post = await Post
.findOne({ id: req.param('id') })
.populate('user')
.populate('comments');
if (post && post.comments.length > 0) {
const ids = post.comments.map(comment => comment.id);
post.comments = await Comment
.find({ id: commentId })
.populate('user');
}
Granted this is an old question, but a much simpler solution would be to loop over the comments,replacing each comment's 'user' property (which is an id) with the user's full detail using async await.
async function getPost(postId){
let post = await Post.findOne(postId).populate('user').populate('comments');
for(let comment of post.comments){
comment.user = await User.findOne({id:comment.user});
}
return post;
}
Hope this helps!
In case anyone is looking to do the same but for multiple posts, here's one
way of doing it:
find all user IDs in posts
query all users in 1 go from DB
update posts with those users
Given that same user can write multiple comments, we're making sure we're reusing those objects. Also we're only making 1 additional query (whereas if we'd do it for each post separately, that would be multiple queries).
await Post.find()
.populate('comments')
.then(async (posts) => {
// Collect all comment user IDs
const userIDs = posts.reduce((acc, curr) => {
for (const comment of post.comments) {
acc.add(comment.user);
}
return acc;
}, new Set());
// Get users
const users = await User.find({ id: Array.from(userIDs) });
const usersMap = users.reduce((acc, curr) => {
acc[curr.id] = curr;
return acc;
}, {});
// Assign users to comments
for (const post of posts) {
for (const comment of post.comments) {
if (comment.user) {
const userID = comment.user;
comment.user = usersMap[userID];
}
}
}
return posts;
});
My first attempt at building something with Angular + express + mongodb, so I'm probably going about this completely the wrong way. Express is being used to serve up json. Angular then takes care of all the views etc.
I'm using Mongoose to interact with Mongo.
I have the following database schema:
var categorySchema = new mongoose.Schema({
title: String, // this is the Category title
retailers : [
{
title: String, // this is the retailer title
data: { // this is the retailers Data
strapLine: String,
img: String , // this is the retailer's image
intro: String,
website: String,
address: String,
tel: String,
email: String
}
}
]
});
var Category = mongoose.model('Category', categorySchema);
and in Express I have a couple of routes to get the data:
app.get('/data/categories', function(req, res) {
// Find all Categories.
Category.find(function(err, data) {
if (err) return console.error(err);
res.json(data)
});
});
// return a list of retailers belonging to the category
app.get('/data/retailer_list/:category', function(req, res) {
//pass in the category param (the unique ID), and use that to do our retailer lookup
Category.findOne({ _id: req.params.category }, function(err, data) {
if (err) return console.error(err);
res.json(data)
});
});
The above works - I'm just having big problems trying to get at a single retailer. I'm passing the category, and retailer id through... I've tried all sorts of things - from doing a find on the category, then a findOne on the contents within... but I just cant get it to work. I'm probably going about this all wrong...
I found this thread here: findOne Subdocument in Mongoose and implemented the solution - however, it returns all my retailers - and not just the one I want.
// Returns a single retailer
app.get('/data/retailer_detail/:category/:id', function(req, res) {
//pass in the category param (the unique ID), and use that to do our retailer lookup
Category.findOne({_id: req.params.category , 'retailers.$': 1}, function(err, data) {
console.log(data);
if (err) return console.error(err);
res.json(data)
});
});
Thanks,
Rob
Now that I see your full filter/query, you should be able to use the array positional operator in this case as part of the projection rather than doing client side filtering:
app.get('/data/retailer_detail/:category/:id', function(req, res) {
//pass in the category param (the unique ID), and use that to do our retailer lookup
Category.findOne({
/* query */
_id: req.params.category ,
'retailers._id' : req.params.id
},
{ /* projection */
"retailers.$" : 1
},
function(err, data) {
var retailer = _.where(data.retailers , { id : req.params.id });
if (err) return console.error(err);
res.json(retailer)
});
});
For the { "retailers.$" : 1 } to work properly, the query must include a field from an element in the array. The $ operator returns the first match only.
The guys next door use Mongo + Express and gave me some pointers: they explained to me how mongo worked, and advised I should use underscore.js to assist with my filter.
They said I needed to pull the entire category out - and then run the filter. I don't strictly need , 'retailers._id' : req.params.id} but they said to leave it in as it guaranteed that the category would only be returned if an item within it contained that information. I still don't really know why or how... So can't really mark this as solved.. it it solved, but I don't really get why as yet - so will do more reading :)
app.get('/data/retailer_detail/:category/:id', function(req, res) {
//pass in the category param (the unique ID), and use that to do our retailer lookup
Category.findOne({_id: req.params.category , 'retailers._id' : req.params.id}, function(err, data) {
var retailer = _.where(data.retailers , { id : req.params.id });
if (err) return console.error(err);
res.json(retailer)
});
});