This question already has an answer here:
mongoDB array pagination
(1 answer)
Closed 5 years ago.
Currently, I have this confusion which is to paginate an array of items in a Schema.
So technically, every user has a cart and I want to paginate the cart's items. For example let say the cart has 18 items stored. I want to paginate the cart to only 5 items.
Here's the schema
const UserSchema = new Schema({
email: { type: String, unique: true, lowercase: true},
cart: [{
type: Schema.Types.ObjectId, ref: 'Product'
}],
});
Here's the route
My current Approach
router.get('/cart', (req, res, next) => {
User.findOne({ _id: req.user._id })
.populate({
path: 'cart',
options: { limit: 5 }
})
.exec((err, user) => {
res.render('order/cart', { user: user });
});
});
My current approach will definitely limit the items shown from 18 items to only 5 items, but this creates a new problem which now I can't calculate the total price of the cart because it only limit to 5 items per request.
You can do the pagination on the client side, this way you'll get all the products from the server, but display only five on the client.
It will allow you calculate the average or other calculation on all the data. also it will allow you filtering and sorting all the data.
If you have a large amount of data, it will take long time for the first time for the client to load the data from the server, so for large amount of data it may be not the right way to go, in this case you probably want to load the data in chuncks, and maybe in the background, and
perform the calculation using the server maybe also in the background -for optimization.
You can find pros and cons of server/client pagentaion https://dzone.com/articles/pagination-server-side-or-clie
Related
Hello I am new to nodejs and mongodb.
I have 3 models:
"user" with fields "name phone"
"Shop" with fields "name, address"
"Member" with fields "shop user status". (shop and user hold the "id" of respective collections).
Now when I create "shops" api to fetch all shop, then I need to add extra field "isShopJoined" which is not part of the model. This extra field will true if user who see that shop is joined it otherwise it will be false.
The problem happens when I share my model with frontend developers like Android/iOS and others, They will not aware of that extra field until they see the API response.
So is it ok if I add extra field in shops listing which is not part of the model? Or do I need to add that extra field in model?
Important note
All the code below has NOT been tested (yet, I'll do it when I can setup a minimal environment) and should be adapted to your project. Keep in mind that I'm no expert when it comes to aggregation with MongoDB, let alone with Mongoose, the code is only here to grasp the general idea and algorithm.
If I understood correctly, you don't have to do anything since the info is stored in the Member collection. But it forces the front-end to do an extra-request (or many extra-requests) to have both the list of Shops and to check (one by one) if the current logged user is a Member of the shop.
Keep in mind that the front-end in general is driven by the data (and so, the API/back-end), not the contrary. The front-end will have to adapt to what you give it.
If you're happy with what you have, you can just keep it that way and it will work, but that might not be very effective.
Assuming this:
import mongoose from "mongoose";
const MemberSchema = new mongoose.Schema({
shopId: {
type: ObjectId,
ref: 'ShopSchema',
required: true
},
userId: {
type: ObjectId,
ref: 'UserSchema',
required: true
},
status: {
type: String,
required: true
}
});
const ShopSchema = new mongoose.Schema({
name: {
type: String,
required: true
},
address: {
//your address model
}
});
const UserSchema = new mongoose.Schema({
name: {
type: String,
required: true
},
phone: {
type: String,
required: true,
},
// Add something like this
shopsJoined: {
type: Array,
default: [],
required: true
}
});
You could tackle this problem via 2 ways:
MongoDB Aggregates
When retrieving (back-end side) the list of shops, if you know the user that made the request, instead of simply returning the list of Shops, you could return an aggregate of Shops and Members resulting in an hybrid document containing both the info of Shops and Models. That way, the front-end have all the info it needs with one back-end request.
Important note
The following code might not work as-is and you'll have to adapt it, I currently have nothing to test it against. Keep in mind I'm not very familiar with aggregates, let alone with Mongoose, but you'll get the general idea by looking the code and comments.
const aggregateShops = async (req, res, next) => {
try {
// $lookup will merge the "Model" and "Shop" documents into one
// $match will return only the results matching the condition
const aggreg = await Model.aggregate({$lookup: {
from: 'members', //the name of the mongodb collection
localField: '_id', //the "Shop" field to match with foreign collection
foreignField: 'shopId', //the "Member" field to match with local collection
as: 'memberInfo' //the field name in which to store the "Member" fields;
}, {
$match: {memberInfo: {userId: myUserId}}
}});
// the result should be an array of object looking like this:
/*{
_id: SHOP_OBJECT_ID,
name: SHOP_NAME,
address: SHOP_ADDRESS,
memberInfo: {
shopId: SHOP_OBJECT_ID,
userId: USER_OBJECT_ID,
status: STATUS_JOINED_OR_NOT
}
}*/
// send back the aggregated result to front-end
} catch (e) {
return next(e);
}
}
Drop the Members collection and store the info elsewhere
Instinctively, I would've gone this way. The idea is to either store an array field shopsJoined in the User model, or a membersJoined array field in the Shops model. That way, the info is retrieved no matter what, since you still have to retrieve the Shops and you already have your User.
// Your PATCH route should look like this
const patchUser = async (req, res, next) => {
try {
// How you chose to proceed here is up to you
// I tend to facilitate front-end work, so get them to send you (via req.body) the shopId to join OR "un-join"
// They should already know what shops are joined or not as they have the User
// For example, req.body.shopId = "+ID" if it's a join, or req.body.shopId = "-ID" if it's an un-join
if (req.body.shopId.startsWith("+")) {
await User.findOneAndUpdate(
{ _id: my_user_id },
{ $push: { shopsJoined: req.body.shopId } }
);
} else if (req.body.shopId.startsWith("-")) {
await User.findOneAndUpdate(
{ _id: my_user_id },
{ $pull: { shopsJoined: req.body.shopId } }
);
} else {
// not formatted correctly, return error
}
// return OK here depending on the framework you use
} catch (e) {
return next(e);
}
};
Of course, the above code is for the User model, but you can do the same thing for the Shop model.
Useful links:
MongoDB aggregation pipelines
Mongoose aggregates
MongoDB $push operator
MongoDB $pull operator
Yes you have to add the field to the model because adding it to the response will be only be a temporary display of the key but what if you need that in the future or in some list filters, so its good to add it to the model.
If you are thinking that front-end will have to be informed so just go it, and also you can set some default values to the "isShopJoined" key let it be flase for the time.
My collection has 1000-ish records. I have a flutter app, which fetches data from a Heroku API, where the backend is based in NodeJs. While fetching, it sorts the entire collection based on the number of a certain "Vacant" field in descending order. It takes nearly 15 seconds to fetch that data. I really don't think 1000 documents is a lot of data, so what can be the optimal method of approaching this problem?
UPDATE 1: This is the code where I'm fetching the data, where I'm sorting based on the 'Vacant' field.
UPDATE 2: My region on Heroku was set in the US, that's why there was a huge delay in the response time. Shifted to AWS with a server close to me, the response is in milliseconds now.
const dataSchema = new mongoose.Schema({
District: String,
Name: String,
Vacant: Number,
Address: String,
PhoneNumber: String
})
app.get('/:state', (req, res) => {
const State = mongoose.model(req.params.state, dataSchema)
State.find().sort([['Vacant', -1]]).exec((err, foundData) => {
if (err) {
console.log(err)
} else {
res.send(foundData)
}
})
})
I'm stuck with some complex query joins in mongoose (using nodejs)
I have the model User
{ active_payment: {type: mongoose.Schema.Types.ObjectId, ref: 'Payment'}}
And the model payment:
{status : Number, creation_date: Date}
I want to find all the users with the a paument of a status = 2.
I've tried:
User.find({'active_payment.status': 2,}).populate('active_payment')
But its not working. Is there any way to do it without having to sort all the users by a for loop?
I would also like to sort users by the creation_date of the payment.
You can not reach status in User.find() directly, Please try this one
User
.find({})
.populate({
path: 'active_payment',
match: { status: 2},
})
.exec(function(err, users) {
users = users.filter(function(user) {
return user.active_payment; // filter the user by active_payment field
});
});
With match to filter the status is 2 populate documents, then filter user after populate though active_payment.
Is it possible to query on associated fields properties in sails. For example, let's consider these two model:
Product:
attributes: {
name:{
type:'string'
},
code:{
type:'string',
required:true,
unique:true
},
inventory:{
collection:'inventory',
via:'product'
}
}
Inventory
attributes: {
product:{
model:'product',
required:true
},
quantity:{
type:'float'
}
}
Now is there any way to get Inventory records that have Products of certain codes. One way is to get products of those specific codes and then get Inventory records having those products. But is it possible to do a single find query to get desired result? I have tried following, but this does not filter products on code.
Inventory
.find()
.populate('product',{where:{code:{'contains':'something'}}})
.exec(function (err, inventories) {
//do something
})
In fact with your request you take all Inventory and populate only product with code:{'contains':'something'} other inventory will have empty products.
You can make your association two way and do this :
Product
.find().where({code:{'contains':'something'}})
.populate('inventory')
.exec(function (err, products) {
//do something
//Build an array of inventory with async on all products
})
As I know there is no other way to do this actually :/
No, you can`t. Best option you have - it is NOT use sails ever!
This question already has answers here:
Retrieve only the queried element in an object array in MongoDB collection
(18 answers)
Closed 8 years ago.
Given a schema like this:
UserSchema = new Schema({
name: String,
donations: [
{
amount: Number,
charity: {
type: Schema.Types.ObjectId,
ref: 'charity'
}
}
]
});
I have the (string) ids of a user and of a nonprofit and I want to get the amount the user donated. Im using this query with lodash:
User.findOne({
_id: userId
}, function(err, user) {
var amount = _.find(user.donations, {charity: charityId});
console.log("the amount is: ", amount);
});
Here the amount returns undefined even though it shouldn't also im guessing i shouldn't have to use lodash. What's the right way to get to the amount donated given a specific userId and charityId?
This is pretty similar to the possible duplicate I linked to, as you can do this without using lodash as:
User.findOne({
_id: userId,
'donations.charity': charityId
}, {
'donations.$': 1
}, function(err, user) {
console.log("the amount is: ", user.donations[0].amount);
});