I have two collections. The first collection is distributors and the second collection is reports.
reports collection contains about 300K of data. I need to get the data for the specified time interval. But when using a lockup, the request slows down a lot. It takes an average of 7-8 seconds.
My query is like:
getFilterByDistributorTest = async ( req: Request, res: Response ): Promise<void> => {
DistributorModel.aggregate( [
{
$lookup: {
from: 'drugreports',
let: { "dataID": "$_id" },
pipeline: [
{
$match: { $expr: { $ne: ["$is_updated", null] } }
},
{ $match: { $expr: { $eq: ["$mode_40_distributor", "$$dataID"] } } },
{ $match: { mode_40_date: { $gte: new Date( "01-01-2020" ), $lte: new Date( "02-02-2020" ) } } },
{
$group:
{
_id: "$mode_40_distributor",
totalQuantity: { $sum: "$quantity" },
totalPricesUSD: { $sum: "$sum_price.usd" },
totalPricesUZS: { $sum: "$sum_price.uzs" },
totalPricesRUB: { $sum: "$sum_price.rub" },
totalPricesEUR: { $sum: "$sum_price.eur" },
}
},
{
$project: {
_id: 1,
usd: "$totalPricesUSD",
uzs: "$totalPricesUZS",
rub: "$totalPricesRUB",
eur: "$totalPricesEUR",
qty: "$totalQuantity",
}
}
],
as: "filterByTurnover_"
}
},
{ $unwind: { path: "$filterByTurnover_", preserveNullAndEmptyArrays: true } },
{ $limit: 25 },
] )
.exec( function ( err, result ) {
if ( err ) {
return res.status( 404 ).json( {
status: "Error",
code: 10004,
} );
}
return res.json( {
status: "Success",
data: result
} );
} );
};
The above query takes place in 7-8 seconds. It's too slow. What are the options to speed up the query?
If I do not use $lockup, the request starts in 0.006 seconds. If I use $lockup it lasts 7-8 seconds.
Thank you!
Related
I want to write a MongoDB query in NodeJS where it return the matching documents as well as the count of documents too. For ex consider the below code -
const result = await Student.aggregate(
[
{
$match: {
...filter
}
},
{
$project: {
_id: 1,
payment: 1,
type: 1,
BirthDate: 1
}
},
{
$sort: { StudentData: -1 }
},
{
$count: 'count'
},
{
$skip: skip
},
{
$limit: limit
}
]
);
Here I want to save two things in the result variable - the number of documents and individually all the documents.
let [{ totalItems, result }] = await Student.aggregate(
[
{
$match: {
...filter
}
},
{
$project: {
_id: 1,
payment: 1,
type: 1,
BirthDate: 1
}
},
{
$facet: {
result: [
{
$sort: { BirthDate: -1 },
},
{
$skip: skip
},
{
$limit: limit
}
],
totalItems: [{ $count: 'count' }]
}
},
{
$addFields: {
totalItems: {
$arrayElemAt: ["$totalItems.count", 0]
},
}
}
]
);
i want to find profit of each account using mongodb and node.js i wrote this code but the main problem is i $match stage of aggregation pipeline i want to filter document according to accountNum and also rate but in my query filtering according to account number not work it return an empty array when i comment the account number and only filter by rate and grouping according to account number it work but i want to only find the profit of one person not all and filter data in $match state according to rate and accountNum both.
my code that return null value(filtering according to rate and accountNum in $match. i have some data with that account number):
const profit = await roznamcha.aggregate([
{
$match: {
rate: { $ne: 0 },
accountNum:{$eq:'$req.query.accountNum'}
}
},
{
$addFields: {
resultMultiply: {
$divide: [
{ $multiply: ["$credit", { $subtract: [100, "$rate"] }] }, 100
]
},
resultMultiplyde: {
$divide: [
{ $multiply: ["$debit", { $subtract: [100, "$rate"] }] }, 100
]
},
}
},
{
$group: {
_id: 0,
sumcredit: {
$sum: '$resultMultiply'
},
sumdebit: {
$sum: '$resultMultiplyde'
}
}
},
{
$addFields: {
finalProfit: { $subtract: ["$sumcredit", "$sumdebit"] }
}
}
])
res.status(201).json({
status: 'success',
data: {
profit
}
})
})
the code that work and return a value (filter according to rate only in $match stage)
const profit = await roznamcha.aggregate([
{
$match: {
rate: { $ne: 0 },
}
},
{
$addFields: {
resultMultiply: {
$divide: [
{ $multiply: ["$credit", { $subtract: [100, "$rate"] }] }, 100
]
},
resultMultiplyde: {
$divide: [
{ $multiply: ["$debit", { $subtract: [100, "$rate"] }] }, 100
]
},
}
},
{
$group: {
_id: '$accountNum',
sumcredit: {
$sum: '$resultMultiply'
},
sumdebit: {
$sum: '$resultMultiplyde'
}
}
},
{
$addFields: {
finalProfit: { $subtract: ["$sumcredit", "$sumdebit"] }
}
}
])
res.status(201).json({
status: 'success',
data: {
profit
}
})
})
I am trying to find the latest "order" in "orders" array in the whole collection (Not only in the one object).
Data:
[
{
_id: 1,
orders: [
{
title: 'Burger',
date: {
$date: '2021-07-18T13:12:08.717Z',
},
},
],
},
{
_id: 2,
orders: [
{
title: 'Salad',
date: {
$date: '2021-07-18T13:35:01.586Z',
},
},
],
},
];
Code:
var restaurant = await Restaurant.findOne({
'orders.date': 1,
});
Rather simple:
db.collection.aggregate([
{ $project: { latest_order: { $max: "$orders.date" } } }
])
If you like to get the full order use this:
db.collection.aggregate([
{
$project: {
latest_order: {
$first: {
$filter: {
input: "$orders",
cond: { $eq: [ "$$this.date", { $max: "$orders.date" } ] }
}
}
}
}
},
{ $sort: { "latest_order.date": 1 } },
{ $limit: 1 }
])
Mongo Playground
You have to use aggregation for that
db.collection.aggregate([
{ $unwind: "$orders" },
{ $sort: { "orders.date": -1 } },
{ $limit: 1 },
{
"$group": {
"_id": "$_id",
"orders": { "$first": "$orders" }
}
}
])
Working Mongo playground
I'm trying to make a query in my javascript code, when I try to execute the query it in robo3t it works, but when I try it in my angular code, it doesn't can you please help me?
Here is the code in robo3t.
db.getCollection('interviews').aggregate({
$match: {
status: {
$ne: 'Callback'
},
dateInserted: {
$gte: ISODate("2019-02-07 00:00:00"),
$lte: ISODate("2019-02-08 00:00:00")
},
'insertedBy.adminId': '5c353f840fe0fd000440df01'
}
},
{
$group: {
_id: {
insertedBy: '$insertedBy.email'
},
timeExported: {$first: '$dateInserted'},
total: {
$sum: 1
}
},
},
{
$limit: 100
}
)
and the result shows:
result image
Now here is my code in angular
query = [{
$match: {
status: {
$ne: 'Callback'
},
dateInserted: {
$gte: new Date("2019-02-07 00:00:00").toISOString(),
$lte: new Date("2019-02-08 00:00:00").toISOString()
},
'insertedBy.adminId': localStorage.getItem('_lgu_')
}
},
{
$group: {
_id: {
insertedBy: '$insertedBy.email'
},
timeExported: {$last: '$dateInserted'},
total: {
$sum: 1
}
},
},
{
$limit: 100
},
{
$sort: {
total: 1
}
}
]
Now when I try the query in angular, it doesn't give any result and when I remove the date condition:
dateInserted: {
$gte: new Date("2019-02-07 00:00:00").toISOString(),
$lte: new Date("2019-02-08 00:00:00").toISOString()
},
It will give a result but not what I am expecting.
I am new to nodejs, I am trying to get the all duplicate documents in a collection in mongoDB for that I have tried the following query in mongo shell
db.collection.aggregate([
{
$group: {
_id: {
ProductName: "$ProductName"
},
uniqueIds: {
$addToSet: "$_id"
},
count: {
$sum: 1
}
}
},
{
$match: {
count: {
$gte: 2
}
}
},
{
$sort: {
count: -1
}
}
])
In mongo shell result:
{
"_id" : {
"ProductName" : "Sony Mobile"
},
"uniqueIds" : [
ObjectId("5728ce42a069270e00e59910"),
ObjectId("5728cde6a069270e00e5990e")
],
"count" : 2
},
{
"_id" : {
"ProductName" : "Nokia Mobile"
},
"uniqueIds" : [
ObjectId("5728ce42a069270e00e59920"),
ObjectId("5728cde6a069270e00e5990f")
],
"count" : 2
}
In mongo shell it gaves the result what i want correctly, but i tried the same query in nodejs server side function like below
Company.aggregate([
{
$group: {
_id: {
Proname: "$Proname"
},
uniqueIds: {
$addToSet: "$_id"
},
count: {
$sum: 1
}
}
},
{
$match: {
count: {
$gte: 2
}
}
},
{
$sort: {
count: -1
}
}
]).then(function (dupProds) {
console.log("ALL DUPLICATE PRDCTS : " + JSON.stringify(dupProds));
})
};
It shows me an error that Compnay.aggregate(...).then is not a function, I tried in different ways but no use, now how can get the result same as like I got in mongoshell.
Use exec instead of then for mongoose and in callback function parameters error then result.
Company.aggregate([
{
$group: {
_id: {
Proname: "$Proname"
},
uniqueIds: {
$addToSet: "$_id"
},
count: {
$sum: 1
}
}
},
{
$match: {
count: {
$gte: 2
}
}
},
{
$sort: {
count: -1
}
}
]).exec(function (err,dupProds) {
if(err) {
// return err;
}
console.log("ALL DUPLICATE PRDCTS : ", dupProds);
// return dupProds
})
};
Actually then worked on promise so if you want to use then you need to promising. so can use like
aggregate([{..}]).exec().then(function(result){..})
where aggregate([{..}]).exec() return promise
You need call exec before then:
Company.aggregate(params).exec().then(function (dupProds) {
console.log("ALL DUPLICATE PRDCTS : " + JSON.stringify(dupProds));
})
};
[ http://mongoosejs.com/docs/api.html#aggregate_Aggregate-exec ]