How to sum all amount paid by users - node.js

I am finding it defficult to add up all amount paid by customers that ordered items
Order Schema
const orderschema = new Mongoose.Schema({
created: { type: Date, default: Date.now() },
amount: { type: Number, default: 0 }
User: [{ type: Mongoose.Schema.ObjectId, ref: 'Users'}]
...
})
Route
Get('/total-amount', total-amount)
Controller
Exports.total-amount = () => {
Order.find()...
}
I don't know what to add here to get the total amount made by all customers.
Using NodeJS and MongoDB.
Thank you for you help

You can use $sum in an aggregation stage like this:
First $group all (without _id is to group all values)
Then create field total which is the sum of al amount.
And an optional stage, $project to output only total field.
db.order.aggregate({
"$group": {
"_id": null,
"total": {
"$sum": "$amount"
}
}
},
{
"$project": {
"_id": 0
}
})
Example here
To add into a controller using nodeJS and Mongoose you can use something like this piece of code:
Exports.total - amount = (req, res) => {
Order.aggregate({
"$group": {
"_id": null,
"total": {
"$sum": "$amount"
}
}
}, {
"$project": {
"_id": 0
}
}).then(response => {
res.status(200).send(response)
}).catch(e => res.status(400).send())
}
Note hoy the operation is done using your mongoose model (in this case Order). You are calling aggregate method in the same way you call find method for example: Instead of doing
yourModel.find()
Is
yourModel.aggregate()
And the response will be:
[
{
"total": 6
}
]
So even you can update your controller to add a if/else block like this:
if(response[0].total)
res.status(200).send(response[0].total)
else
res.status(404).send()

Related

Mongodb (mongoose) aggregate is returning the JSON given instead of the sum asked for

I am an extreme newbie to MongDB - using mongoose npm module in my node.js app
given the following data:
_id:622e41bf09c90e08689eb0aa
title:"amazon"
start:"2022-03-13 12:10:55"
amount:"14.11"
_id:622e41bf09c90e08689eb0ab
title:"amazon"
start:"2022-03-13 12:10:55"
amount:"14.11"
const db = mongoose.model('upgrade', schema);
const sum = db.aggregate(
[{
$project : {
_id: '$_id',
total : {
$sum: "amount"
}
}
}]);
console.log(__line__, sum._pipeline[0].$project);
// 143 { _id: '$_id', total: { '$sum': '$amount' } }
what I would like to see in the results is the total of the 2 amounts (28.22)
Can someone please help me get the right syntax to get the desired result?
I really like the idea of working with JSON as opposed to RDBMS
Thank you!
UPDATE
const sum = db.aggregate([{ $group : { _id: null, total : { $sum: { $toDouble : "$amount" }}}}]);
still logging: 143 { _id: null, total: { '$sum': { '$toDouble': '$amount' } } }
UPDATE 2
I have just added things to the schema definition:
const schema = new mongoose.Schema({
name: String,
title: String,
amount: mongoose.Decimal128,
start: { type: Date, default: Date.now },
});
but I am still just getting an empty array in the log
UPDATE 3
console.log(line, mongoose.connection.readyState); shows connecting (2) - so there must be something wrong with my connection string
db.aggregate is asynchronous. You have to wait for the response. This can be done with the await keyword if you are inside an async function or with the .then function.
Using await:
const sum = await db.aggregate([{ $group : { _id: null, total : { $sum: { $toDouble : "$amount" }}}}]);
console.log(sum);
Using .then
db.aggregate([{ $group : { _id: null, total : { $sum: { $toDouble : "$amount" }}}}])
.then((sum) => console.log(sum));

Mongoose SUM get stacked

I'm trying to make trivial SUM on mongoDB to count number of prices for single client.
My collection:
{"_id":"5d973c71dd93adfbda4c7272","name":"Faktura2019006","clientId":"5d9c87a6b9676069c8b5e15b","expiration":"2019-10-02T01:11:18.965Z","price":999999,"userId":"123"},
{"_id":"5d9e07e0b9676069c8b5e15d","name":"Faktura2019007","clientId":"5d9c87a6b9676069c8b5e15b","expiration":"2019-10-02T01:11:18.965Z","price":888,"userId":"123"}
What I tried:
// invoice.model.js
const mongoose = require("mongoose");
const InvoiceSchema = mongoose.Schema({
_id: String,
name: String,
client: String,
userId: String,
expiration: Date,
price: Number
});
module.exports = mongoose.model("Invoice", InvoiceSchema, "invoice");
and
// invoice.controller.js
const Invoice = require("../models/invoice.model.js");
exports.income = (req, res) => {
console.log("Counting Income");
Invoice.aggregate([
{
$match: {
userId: "123"
}
},
{
$group: {
total: { $sum: ["$price"] }
}
}
]);
};
What happen:
When I now open a browser and code above is being called, I get console log 'Counting Income' in terminal however in browser it's just loading forever and nothing happen.
Most likely I just miss some stupid minor thing but I'm trying to find it out for quite a long time without any success so any advise is welcome.
The reason that the controller never finishes is because you are not ending the response process (meaning, you need to use the res object and send something back to the caller).
In order to get the aggregate value, you also need to execute the pipeline (see this example).
Also, as someone pointed out in the comments, you need to add _id: null in your group to specify that you are not going to group by any specific field (see the second example here).
Finally, in the $sum operator, for what you're trying to do, you just need to remove the array brackets since you only want to sum on a single field (see a few examples down here).
Here is the modified code:
// invoice.controller.js
const Invoice = require("../models/invoice.model.js");
exports.income = (req, res) => {
console.log("Counting Income");
Invoice.aggregate([
{
$match: {
userId: "123"
}
},
{
$group: {
_id: null,
total: { $sum: "$price" }
}
}
]).then((response) => {
res.json(response);
});
};
Edit for your comment about when an empty array is returned.
If you want to always return the same type of object, I would control that in the controller. I'm not sure if there is a fancy way to do this with the aggregate pipeline in mongo, but this is what I would do.
Invoice.aggregate([
{
$match: {
userId: "123"
}
},
{
$group: {
_id: null,
total: { $sum: "$price" }
}
},
{
$project: {
_id: 0,
total: "$total"
}
}
]).then((response) => {
if (response.length === 0) {
res.json({ total: 0 });
} else {
// always return the first (and only) value
res.json(response[0]);
}
});
Here, if you find a userId of 123, then you would get this as the return:
{
"total": 1000887
}
But if you change the userId to, say, 1123 which doesn't exist in your db, the result will be:
{
"total": 0
}
This way, your client can always consume the same type of object.
Also, the reason I put the $project pipeline stage in there was to suppress the _id field (see here for more info).

Mongoose aggregate, Match, Count, Group

I am trying to send a list of total paid and unpaid client with count along with data from my node API.
In mongoose method, I am stuck at thinking how to go further.
can anyone suggest the best way to achieve this?
router.get("/", ensureAuthenticated, (req, res) => {
Loan.aggregate([
{
$match: {
ePaidunpaid: "Unpaid"
}
}
]).then(function(data) {
console.log(data);
res.render("dashboard", { admin: req.user.eUserType, user: req.user,data:data });
});
});
Loan Model:
const Loan = new Schema({
sName: { type: String },
sPurpose: [String],
sBankName: String,
sBranchName: [String],
nTotalFees: { type: Number },
ePaidunpaid: { type: String ,default:'Unpaid'},
sCashOrCheque: { type: String },
});
Outcome:
Details of a user with a count of paid and unpaid clients
[
Paid:{
// Paid users
},
Unpaid:{
// Unpaid Users
},
]
Well in that case, try this -
Loan.aggregate([
{
$group: {
_id: "$ePaidunpaid",
data: { $push: "$$ROOT" },
count: { $sum: 1 }
}
}
]);
Output would be something like this -
{
"_id": "Paid",
"data": [
// All the documents having ePaidunpaid = Paid
{ _id: "asdasd123 1eqdsada", sName: "Some name", // Rest of the fields },
{ _id: "asdasd123 1eqdsada", sName: "Some name", // Rest of the fields }
],
count: 2
},
{
"_id": "Unpaid",
"data": [
// All the documents of having ePaidunpaid = Unpaid
{ _id: "asdasd123 1eqdsada", sName: "Some name", // Rest of the fields },
{ _id: "asdasd123 1eqdsada", sName: "Some name", // Rest of the fields }
],
count: 2
},
Explanation
First stage of the pipeline $group groups all the documents according to ePaidunpaidfield which only have two values Paid or Unpaid thus rendering only two documents respectively.
Next step is to accumulate original data (documents) being grouped together. This is achieved using $push accumulator on data field, pushing $$ROOT which effectively references the document currently being processed by pipeline stage.
Since you needed count of all paid and unpaid users hence a $sum accumulator to count all the items in each group.

Aggregation Timing Out

I am using aggregates to query my schema for counts over date ranges, my problem is i am not getting any response from the server (Times out everytime), other mongoose queries are working fine (find, save, etc.) and when i call aggregates it depends on the pipeline (when i only use match i get a response when i add unwind i don't get any).
Connection Code:
var promise = mongoose.connect('mongodb://<username>:<password>#<db>.mlab.com:<port>/<db-name>', {
useMongoClient: true,
replset: {
ha: true, // Make sure the high availability checks are on
haInterval: 5000 // Run every 5 seconds
}
});
promise.then(function(db){
console.log('DB Connected');
}).catch(function(e){
console.log('DB Not Connected');
console.errors(e.message);
process.exit(1);
});
Schema:
var ProspectSchema = new Schema({
contact_name: {
type: String,
required: true
},
company_name: {
type: String,
required: true
},
contact_info: {
type: Array,
required: true
},
description:{
type: String,
required: true
},
product:{
type: Schema.Types.ObjectId, ref: 'Product'
},
progression:{
type: String
},
creator:{
type: String
},
sales: {
type: Schema.Types.ObjectId,
ref: 'User'
},
technical_sales: {
type: Schema.Types.ObjectId,
ref: 'User'
},
actions: [{
type: {type: String},
description: {type: String},
date: {type: Date}
}],
sales_connect_id: {
type: String
},
date_created: {
type: Date,
default: Date.now
}
});
Aggregation code:
exports.getActionsIn = function(start_date, end_date) {
var start = new Date(start_date);
var end = new Date(end_date);
return Prospect.aggregate([
{
$match: {
// "actions": {
// $elemMatch: {
// "type": {
// "$exists": true
// }
// }
// }
"actions.date": {
$gte: start,
$lte: end
}
}
}
,{
$project: {
_id: 0,
actions: 1
}
}
,{
$unwind: "actions"
}
,{
$group: {
_id: "actions.date",
count: {
$sum: 1
}
}
}
// ,{
// $project: {
// _id: 0,
// date: {
// $dateToString: {
// format: "%d/%m/%Y",
// date: "actions.date"
// }
// }
// // ,
// // count : "$count"
// }
// }
]).exec();
}
Calling the Aggregation:
router.get('/test',function(req, res, next){
var start_date = req.query.start_date;
var end_date = req.query.end_date;
ProspectCont.getActionsIn(start_date,end_date).then(function(value, err){
if(err)console.log(err);
res.json(value);
});
})
My Main Problem is that i get no response at all, i can work with an error message the issue is i am not getting any so i don't know what is wrong.
Mongoose Version: 4.11.8
P.s. I tried multiple variations of the aggregation pipeline, so this isn't my first try, i have an aggregation working on the main prospects schema but not the actions sub-document
You have several problems here, mostly by missing concepts. Lazy readers can skip to the bottom for the full pipeline example, but the main body here is in the explanation of why things are done as they are.
You are trying to select on a date range. The very first thing to check on any long running operation is that you have a valid index. You might have one, or you might not. But you should issue: ( from the shell )
db.prospects.createIndex({ "actions.date": 1 })
Just to be sure. You probably really should add this to the schema definition so you know this should be deployed. So add to your defined schema:
ProspectSchema.index({ "actions.date": 1 })
When querying with a "range" on elements of an array, you need to understand that those are "multiple conditions" which you are expecting to match elements "between". Whilst you generally can get away with querying a "single property" of an array using "Dot Notation", you are missing that the application of [$gte][1] and $lte is like specifying the property several times with $and explicitly.
Whenever you have such "multiple conditions" you always mean to use $elemMatch. Without it, you are simply testing every value in the array to see if it is greater than or less than ( being some may be greater and some may be lesser ). The $elemMatch operator makes sure that "both" are applied to the same "element", and not just all array values as "Dot notation" exposes them:
{ "$match": {
"actions": {
"$elemMatch": { "date": { "$gte": start, "$lte: end } }
}
}}
That will now only match documents where the "array elements" fall between the specified date. Without it, you are selecting and processing a lot more data which is irrelevant to the selection.
Array Filtering: Marked in Bold because it's prominence cannot be ignored. Any initial $match works just like any "query" in that it's "job" is to "select documents" valid to the expression. This however does not have any effect on the contents of the array in the documents returned.
Whenever you have such a condition for document selection, you nearly always intend to "filter" such content from the array itself. This is a separate process, and really should be performed before any other operations that work with the content. Especially [$unwind][4].
So you really should add a $filter in either an $addFields or $project as is appropriate to your intend "immediately" following any document selection:
{ "$project": {
"_id": 0,
"actions": {
"$filter": {
"input": "$actions",
"as": "a",
"in": {
"$and": [
{ "$gte": [ "$$a.date", start ] },
{ "$lte": [ "$$a.date", end ] }
]
}
}
}
}}
Now the array content, which you already know "must" have contained at least one valid item due to the initial query conditions, is "reduced" down to only those entries that actually match the date range that you want. This removes a lot of overhead from later processing.
Note the different "logical variants" of $gte and $lte in use within the $filter condition. These evaluate to return a boolean for expressions that require them.
Grouping It's probably just as an attempt at getting a result, but the code you have does not really do anything with the dates in question. Since typical date values should be provided with millisecond precision, you general want to reduce them.
Commented code suggests usage of $dateToString within a $project. It is strongly recommended that you do not do that. If you intend such a reduction, then supply that expression directly to the grouping key within $group instead:
{ "$group": {
"_id": {
"$dateToString": {
"format": "%Y-%m-%d",
"date": "$actions.date"
}
},
"count": { "$sum": 1 }
}}
I personally don't like returning a "string" when a natural Date object serializes properly for me already. So I like to use the "math" approach to "round" dates instead:
{ "$group": {
"_id": {
"$add": [
{ "$subtract": [
{ "$subtract": [ "$actions.date", new Date(0) ] },
{ "$mod": [
{ "$subtract": [ "$actions.date", new Date(0) ] },
1000 * 60 * 60 * 24
]}
],
new Date(0)
]
},
"count": { "$sum": 1 }
}}
That returns a valid Date object "rounded" to the current day. Mileage may vary on preferred approaches, but it's the one I like. And it takes the least bytes to transfer.
The usage of Date(0) represents the "epoch date". So when you $subtract one BSON Date from another you end up with the milliseconds difference between the two as an integer. When $add an integer value to a BSON Date, you get a new BSON Date representing the sum of the milliseconds value between the two. This is the basis of converting to numeric, rounding to the nearest start of day, and then converting numeric back to a Date value.
By making that statement directly within the $group rather than $project, you are basically saving what actually gets interpreted as "go through all the data and return this calculated value, then go and do...". Much the same as working through a pile of objects, marking them with a pen first and then actually counting them as a separate step.
As a single pipeline stage it saves considerable resources as you do the accumulation at the same time as calculating the value to accumulate on. When you think it though much like the provided analogy, it just makes a lot of sense.
As a full pipeline example you would put the above together as:
Prospect.aggregate([
{ "$match": {
"actions": {
"$elemMatch": { "date": { "$gte": start, "$lte: end } }
}
}},
{ "$project": {
"_id": 0,
"actions": {
"$filter": {
"input": "$actions",
"as": "a",
"in": {
"$and": [
{ "$gte": [ "$$a.date", start ] },
{ "$lte": [ "$$a.date", end ] }
]
}
}
}
}},
{ "$unwind": "$actions" },
{ "$group": {
"_id": {
"$dateToString": {
"format": "%Y-%m-%d",
"date": "$actions.date"
}
},
"count": { "$sum": 1 }
}}
])
And honestly if after making sure an index is in place, and following that pipeline you still have timeout problems, then reduce the date selection down until you get a reasonable response time.
If it's still taking too long ( or the date reduction is not reasonable ) then your hardware simply is not up to the task. If you really have a lot of data then you have to be reasonable with expectations. So scale up or scale out, but those things are outside the scope of any question here.
As it stands those improvements should make a significant difference over any attempt shown so far. Mostly due to a few fundamental concepts that are being missed.

Mongoose sort the aggregated result

I'm having a lot of difficulty in solving this mongodb (mongoose) problem.
There is this schema 'Recommend' (username, roomId, ll and date) and its collection contains recommendation of user.
I need to get a list of most recommended rooms (by roomId). Below is the schema and my tried solution with mongoose query.
var recommendSchema = mongoose.Schema({
username: String,
roomId: String,
ll: { type: { type: String }, coordinates: [ ] },
date: Date
})
recommendSchema.index({ ll: '2dsphere' });
var Recommend = mongoose.model('Recommend', recommendSchema);
Recommend.aggregate(
{
$group:
{
_id: '$roomId',
recommendCount: { $sum: 1 }
}
},
function (err, res) {
if (err) return handleError(err);
var resultSet = res.sort({'recommendCount': 'desc'});
}
);
The results returned from the aggregation pipeline are just plain objects. So you do the sorting as a pipeline stage, not as a separate operation:
Recommend.aggregate(
[
// Grouping pipeline
{ "$group": {
"_id": '$roomId',
"recommendCount": { "$sum": 1 }
}},
// Sorting pipeline
{ "$sort": { "recommendCount": -1 } },
// Optionally limit results
{ "$limit": 5 }
],
function(err,result) {
// Result is an array of documents
}
);
So there are various pipeline operators that can be used to $group or $sort or $limit and other things as well. These can be presented in any order, and as many times as required. Just understanding that one "pipeline" stage flows results into the next to act on.

Resources