Mongoose aggregate, Match, Count, Group - node.js

I am trying to send a list of total paid and unpaid client with count along with data from my node API.
In mongoose method, I am stuck at thinking how to go further.
can anyone suggest the best way to achieve this?
router.get("/", ensureAuthenticated, (req, res) => {
Loan.aggregate([
{
$match: {
ePaidunpaid: "Unpaid"
}
}
]).then(function(data) {
console.log(data);
res.render("dashboard", { admin: req.user.eUserType, user: req.user,data:data });
});
});
Loan Model:
const Loan = new Schema({
sName: { type: String },
sPurpose: [String],
sBankName: String,
sBranchName: [String],
nTotalFees: { type: Number },
ePaidunpaid: { type: String ,default:'Unpaid'},
sCashOrCheque: { type: String },
});
Outcome:
Details of a user with a count of paid and unpaid clients
[
Paid:{
// Paid users
},
Unpaid:{
// Unpaid Users
},
]

Well in that case, try this -
Loan.aggregate([
{
$group: {
_id: "$ePaidunpaid",
data: { $push: "$$ROOT" },
count: { $sum: 1 }
}
}
]);
Output would be something like this -
{
"_id": "Paid",
"data": [
// All the documents having ePaidunpaid = Paid
{ _id: "asdasd123 1eqdsada", sName: "Some name", // Rest of the fields },
{ _id: "asdasd123 1eqdsada", sName: "Some name", // Rest of the fields }
],
count: 2
},
{
"_id": "Unpaid",
"data": [
// All the documents of having ePaidunpaid = Unpaid
{ _id: "asdasd123 1eqdsada", sName: "Some name", // Rest of the fields },
{ _id: "asdasd123 1eqdsada", sName: "Some name", // Rest of the fields }
],
count: 2
},
Explanation
First stage of the pipeline $group groups all the documents according to ePaidunpaidfield which only have two values Paid or Unpaid thus rendering only two documents respectively.
Next step is to accumulate original data (documents) being grouped together. This is achieved using $push accumulator on data field, pushing $$ROOT which effectively references the document currently being processed by pipeline stage.
Since you needed count of all paid and unpaid users hence a $sum accumulator to count all the items in each group.

Related

How to sum all amount paid by users

I am finding it defficult to add up all amount paid by customers that ordered items
Order Schema
const orderschema = new Mongoose.Schema({
created: { type: Date, default: Date.now() },
amount: { type: Number, default: 0 }
User: [{ type: Mongoose.Schema.ObjectId, ref: 'Users'}]
...
})
Route
Get('/total-amount', total-amount)
Controller
Exports.total-amount = () => {
Order.find()...
}
I don't know what to add here to get the total amount made by all customers.
Using NodeJS and MongoDB.
Thank you for you help
You can use $sum in an aggregation stage like this:
First $group all (without _id is to group all values)
Then create field total which is the sum of al amount.
And an optional stage, $project to output only total field.
db.order.aggregate({
"$group": {
"_id": null,
"total": {
"$sum": "$amount"
}
}
},
{
"$project": {
"_id": 0
}
})
Example here
To add into a controller using nodeJS and Mongoose you can use something like this piece of code:
Exports.total - amount = (req, res) => {
Order.aggregate({
"$group": {
"_id": null,
"total": {
"$sum": "$amount"
}
}
}, {
"$project": {
"_id": 0
}
}).then(response => {
res.status(200).send(response)
}).catch(e => res.status(400).send())
}
Note hoy the operation is done using your mongoose model (in this case Order). You are calling aggregate method in the same way you call find method for example: Instead of doing
yourModel.find()
Is
yourModel.aggregate()
And the response will be:
[
{
"total": 6
}
]
So even you can update your controller to add a if/else block like this:
if(response[0].total)
res.status(200).send(response[0].total)
else
res.status(404).send()

Mongoose: Infinite scroll with filtering

I have these two models:
User.js
const UserSchema = new Schema({
profile: {
type: Schema.Types.ObjectId,
ref: "profiles",
},
following: [
{
type: Schema.Types.ObjectId,
ref: "users",
},
],
});
module.exports = User = mongoose.model("users", UserSchema);
Profile.js
const ProfileSchema = new Schema({
videoURL: {
type: String,
},
});
module.exports = Profile = mongoose.model("profiles", ProfileSchema);
Here's an example of a User document:
{
"following": [
{
"profile":{
"videoURL":"video_url_1"
}
},
{
"profile":{
"videoURL":"video_url_2"
}
},
{
"profile":{}
},
{
"profile":{
"videoURL":"video_url_3"
}
},
{
"profile":{
"videoURL":"video_url_4"
}
},
{
"profile":{
"videoURL":"video_url_5"
}
},
{
"profile":{}
},
{
"profile":{
"videoURL":"video_url_6"
}
}
]
}
I am trying to implement an infinite scroll of the videos of the users followed by the connected user.
This means, I will have to filter user.following.profile.videoURL
WHERE videoURL exists
Suppose, I will be loading two videos, by two videos:
Response 1: ["video_url_1","video_url_2"]
Response 2: ["video_url_3","video_url_4"]
Response 3: ["video_url_5","video_url_6"]
Usually, infinite scroll is easy because all I have to load the documents 2 by 2 by order of storage without filtering on any field.
Example: Displaying the followed users two by two in an infinite scroll
User.findById(user_id).populate({
path: "following",
options: {
skip: 2 * page,
limit: 2,
},
});
But, now I have to perform filtering on each followed_user.profile.video, and return two by two. And I don't see how I can perform BOTH the filtering and the infinite scroll at the same time.
NOTE: According to the documentation:
In general, there is no way to make populate() filter stories based on properties of the story's author. For example, the below query won't return any results, even though author is populated.
const story = await Story.
findOne({ 'author.name': 'Ian Fleming' }).
populate('author').
exec();
story; // null
So I suppose, there is no way for me to use populate to filter based user.followers, based on each user.follower.profile.videoURL
I am not sure it is possible with populate method, but you can try aggregation pipeline,
$match user_id condition
$lookup with aggregation pipeline in users collection for following
$match following id condition
$lookup with profile for following.profile
$match videoURL should exists
$project to show profile field and get first element using $arrayElemAt
$slice to do pagination in following
let page = 0;
let limit = 2;
let skip = limit * page;
User.aggregate([
{ $match: { _id: mongoose.Types.ObjectId(user_id) } },
{
$lookup: {
from: "users",
let: { following: "$following" },
pipeline: [
{ $match: { $expr: { $in: ["$_id", "$$following"] } } },
{
$lookup: {
from: "profiles",
localField: "profile",
foreignField: "_id",
as: "profile"
}
},
{ $match: { "profile.videoURL": { $exists: true } } },
{
$project: {
profile: { $arrayElemAt: ["$profile", 0] }
}
}
],
as: "following"
}
},
{
$addFields: {
following: {
$slice: ["$following", skip, limit]
}
}
}
])
Playground
Suggestion:
You can improve your schema design,
removing profile schema and add profile object in users collection, so you can achieve easily your requirement using populate method,
put match condition in following populate for videoURL exists
const UserSchema = new Schema({
profile: {
type: {
videoURL: {
type: String
}
}
},
following: [
{
type: Schema.Types.ObjectId,
ref: "users"
}
]
});
module.exports = User = mongoose.model("users", UserSchema);
User.findById(user_id).populate({
path: "following",
match: {
"profile.videoURL": { $ne: null }
},
options: {
skip: 2 * page,
limit: 2,
}
});
So what you want is table with infinite scroll and:
You can opt given ways to approach your problem :
Load data (first page) into grid.
Set filter on a col.
Load data again, this time using the filter.

Mongoose full text search not filtering correctly

So basically i have model with a bunch of string fields like so:
const Schema: Schema = new Schema(
{
title: {
type: String,
trim: true
},
description: {
type: String,
trim: true
},
...
}
);
Schema.index({ '$**': 'text' });
export default mongoose.model('Watch', Schema);
where I index all of them.
Now when I search being that this schema is used as a ref for another model I do a search like this where user is an instance of the other model
const { search, limit = 5 } = req.query;
const query = search && { match: { $text: { $search: new RegExp(search, 'i') } } };
const { schemaRes } = await user
.populate({
path: 'schema',
...query,
options: {
limit
}
})
.execPopulate();
and the searching itself seems to work ok, the problem is when search fields starts to be more specific it seems to me the it does not regard it well.
Example
db
{ title: 'Rolex', name: 'Submariner', description: 'Nice' }
{ title: 'Rolex', name: 'Air-King', description: 'Nice' }
When the search param is Rolex I get both items which is ok but when the search param becomes Rolex Air-King i keep on getting both items which to me is not ok because I would rather get only one.
Is there something I could do to achieve this?
Returning both items is correct, since both items match your search params, but with different similarity score.
You can output the similarity score to help sorting the result.
user.aggregate([
{ $match: { $text: { $search: "Rolex Air-King" } } },
{ $set: { score: { $meta: "textScore" } } }
])
// new RegExp("Rolex Air-King", 'i') is not necessary and even invalid,
// as $search accepts string and is already case-insensitive by default
The query will return
[{
"_id": "...",
"title": "Rolex",
"name": "Air-King",
"description": "Nice",
"score": 2.6
},
{
"_id": "....",
"title": "Rolex",
"name": "Submariner",
"description": "Nice",
"score": 1.1
}]
Since the second result item matches your search query (even partially), MongoDB returns it.
You could use the score to help sort the items. But determining the right threshold to filter the result is complex, as the score depends on the word count as well.
On a side note: You can assign different weights to the fields if they are not equally important
https://docs.mongodb.com/manual/tutorial/control-results-of-text-search/

Using aggregation query to get list of users with total transaction count and transaction detail as embedded document

I am trying to get a list of users with total transaction count and each user should have latest transaction detail as embedded object using MongoDB's aggregate pipelines to fetch results in GET API.
I have the following database schema:
User: _id, name, phone, address
Product: _id, name, unit_price, description
Transaction: _id, date, product_id(ref to Product), user_id(ref to User), quantity, total_price
Expected Response JSON
[
{
name:"",
phone:"",
address:"",
total_transaction:
latest_transaction_detail: {
product_id:
quantity:
total_price:
}
},
{
name:"",
phone:"",
address:"",
total_transaction:
latest_transaction_detail: {
product_id:
quantity:
total_price:
}
}
]
How do I generate an aggregate query to return the above?
You can achieve this by running an aggregation query.
A lookup stage will join your User collection with your Transaction collection (no need to join Product in your expected result). Its pipeline is splitted with $facet, for both get the count result and the latest transaction for that user
A project stage will reshape your data and extract array elements to documents.
Here's such a query :
db.User.aggregate(
[
{
$lookup:
{
from: "Transaction",
let: { userId: "$_id" },
pipeline: [
{$facet:
{count:[{$match:{$expr:{$eq:["$$userId","$user_id"]}}}, {$count:"total_transaction"}],
latest:[
{$match:{$expr:{$eq:["$$userId","$user_id"]}}},
{$sort:{date:-1}},
{$limit:1}]
} }],
as: "transactions"
}
},
{
$project: {
last_name:1,
phone:1,
address:1,
total_transaction : {
$let:{
vars:{
count:{
$arrayElemAt:["$transactions.count",0]
}
},
in:{
$arrayElemAt:["$$count.total_transaction",0]
}
}
},
latest_transaction : {
$let:{
vars:{
latest:{
$arrayElemAt:["$transactions.latest",0]
}
},
in:{
$arrayElemAt:["$$latest",0]
}
}
},
}
},
]
);

Mongoose sort the aggregated result

I'm having a lot of difficulty in solving this mongodb (mongoose) problem.
There is this schema 'Recommend' (username, roomId, ll and date) and its collection contains recommendation of user.
I need to get a list of most recommended rooms (by roomId). Below is the schema and my tried solution with mongoose query.
var recommendSchema = mongoose.Schema({
username: String,
roomId: String,
ll: { type: { type: String }, coordinates: [ ] },
date: Date
})
recommendSchema.index({ ll: '2dsphere' });
var Recommend = mongoose.model('Recommend', recommendSchema);
Recommend.aggregate(
{
$group:
{
_id: '$roomId',
recommendCount: { $sum: 1 }
}
},
function (err, res) {
if (err) return handleError(err);
var resultSet = res.sort({'recommendCount': 'desc'});
}
);
The results returned from the aggregation pipeline are just plain objects. So you do the sorting as a pipeline stage, not as a separate operation:
Recommend.aggregate(
[
// Grouping pipeline
{ "$group": {
"_id": '$roomId',
"recommendCount": { "$sum": 1 }
}},
// Sorting pipeline
{ "$sort": { "recommendCount": -1 } },
// Optionally limit results
{ "$limit": 5 }
],
function(err,result) {
// Result is an array of documents
}
);
So there are various pipeline operators that can be used to $group or $sort or $limit and other things as well. These can be presented in any order, and as many times as required. Just understanding that one "pipeline" stage flows results into the next to act on.

Resources