Mongo/Node: Filtering By Single Properties? - node.js

I am dealing with a query with a criteria object that is being passed as the first argument to this query:
module.exports = (criteria, sortProperty, offset = 0, limit = 20) => {
// write a query that will follow sort, offset, limit options only
// do not worry about criteria yet
console.log(criteria);
const query = Artist.find({ age: { $gte: 19, $lte: 44 } })
.sort({ [sortProperty]: 1 })
.skip(offset)
.limit(limit);
return Promise.all([query, Artist.count]).then(results => {
return {
all: results[0],
count: results[1],
offset: offset,
limit: limit
};
});
};
By default, the criteria object has a single name property that is an empty string.
The age property points to an object that has both min and max values assigned to it. I also have a yearsActive property inside of the criteria object and that also has a min and max value.
So three different properties: age, name and yearsActive.
This has been an extremely challenging one for me and if you look above that's as far as I got.
When my criteria property is console logged it only has a name { name: "" }. It has no yearsActive or age by default when it first starts. So that is where the point of the sliders come in. When I start moving these sliders around on the frontend, then it gets the age and yearsActive appended to the criteria object.
So I need to figure out how to update the query to consider for example the different ages and I have been considering using an if conditional inside a helper function.

Regarding to the comment that I left you.
You have three states at least one when you retrieve the data to the UI. In this case, I would recommend you use aggregation in order to retrieve the data as a model as your business.
For example, the problem as you have is that sometimes you don't know about the max or min value for age or yearsActive, but also you should have an identifier that could be an ObjectId which will be used to update the model identified by that property.
Artist.aggregate([
{
$match: { age: { $gte: 19, $lte: 44 } }
},
{
$sort: { yourProperty: 1 }
},
{
$skip: 10
},
{
$limit: 10
},
{
$project: {
// You set your properties to retrieve with the 1 as flag
propertieX: 1,
"another.property": 1,
"age.max": {
$cond: {
if: { $eq: [ "", "$age.max" ] },
then: 0, // Or the value that you want to set it
else: "$age.max"
}
}
}
}]);
The other state is when you do the query according to the parameters that you're submitting from the form.
If you assurance to retrieve a model with the logic as you want. For example you should return this model in every request using $project and applying the default values when doesn't exist the manipulation in the front-end side as in the searching should be easy to manage.
{
ObjectId: YOUR_OBJECT_ID,
age: {
min: YOUR_MIN_VALUE,
max: YOUR_MAX_VALUE
},
yearsActive: {
min: YOUR_MIN_VALUE,
max: YOUR_MAX_VALUE
}
}
Finally, when you would send the data to save it you should sent the entire model that you returned but the must important thing is identify only that element by the ObjectId to do the update.
NOTE: This is an approach that I will do according with the information that I understand from your question, If I'm bad with me interpretation let me know, and if you want to share more information or open a repository to understand in code, should more easy to me understand the problem.

So what I decided to do since the code would look messy to throw all inside the Artist.find({}) was to create a separate helper function:
const buildQuery = (criteria) => {
console.log(criteria);
};
This helper function is being called with the criteria object and I have to form up the object in such a way that it will represent the query the way in which I want to search the Artist collection.
What made this difficult to wrap my head around was the not very well formed object for searching over a collection with its random properties such as age which has a min and a max which Mongo does not know how to deal with by default. MongoDB does not know what min and max mean exactly.
So inside the helper function I made a separate object to return from this function thats going to represent the actual query that I want to send off to Mongo.
const buildQuery = (criteria) => {
console.log(criteria);
const query = {};
};
I am not modifying the object in anyway, I am just reading some of the desired search results or what the user wants to see from this UI object and so I made this object called query and I added the idea of age.
const buildQuery = (criteria) => {
console.log(criteria);
const query = {};
query.age = {};
};
I decided to do an if conditional inside of the helper function for the specific age range that I want to find.
const buildQuery = (criteria) => {
console.log(criteria);
const query = {};
if (criteria.age) {
query.age = {};
}
};
So this is where the Mongo query operators come into play. The two operators I want to be concerned with is the greater than or equal to ($gte) and the less than or equal to ($lte) operators.
This is how I actually implemented in practice:
const buildQuery = (criteria) => {
console.log(criteria);
const query = {};
if (criteria.age) {
query.age = {
$gte: criteria.age.min,
$lte: criteria.age.max
};
}
};
The query object here will eventually be returned from the buildQuery function:
const buildQuery = (criteria) => {
console.log(criteria);
const query = {};
if (criteria.age) {
query.age = {
$gte: criteria.age.min,
$lte: criteria.age.max
};
}
return query;
};
That query object will be passed off to the find operation:
module.exports = (criteria, sortProperty, offset = 0, limit = 20) => {
// write a query that will follow sort, offset, limit options only
// do not worry about criteria yet
const query = Artist.find(buildQuery(criteria))
.sort({ [sortProperty]: 1 })
.skip(offset)
.limit(limit);
return Promise.all([query, Artist.count]).then(results => {
return {
all: results[0],
count: results[1],
offset: offset,
limit: limit
};
});
};
const buildQuery = (criteria) => {
console.log(criteria);
const query = {};
if (criteria.age) {
query.age = {
$gte: criteria.age.min,
$lte: criteria.age.max
};
}
return query;
};
So what I am doing here is to get the equivalent of Artist.find({ age: { $gte: minAge, $lte: maxAge }).
So for yearsActive I decided to implement something that is nearly identical:
const buildQuery = criteria => {
console.log(criteria);
const query = {};
if (criteria.age) {
query.age = {
$gte: criteria.age.min,
$lte: criteria.age.max
};
}
if (criteria.yearsActive) {
}
return query;
};
So if the user changes the slider, I am going to expect my criteria object to have a yearsActive property defined on it like so:
const buildQuery = criteria => {
console.log(criteria);
const query = {};
if (criteria.age) {
query.age = {
$gte: criteria.age.min,
$lte: criteria.age.max
};
}
if (criteria.yearsActive) {
query.yearsActive = {
$gte: criteria.yearsActive.min,
$lte: criteria.yearsActive.max
}
}
return query;
};

Related

Is this possible to update multiple data with sequlize?

I have created add product API like this. This is working fine. I'm posting successfully varient data by sharing product id as a foreign key, but I'm confused about how can I update product data. Can I update data by using this code?
try {
const { name, description, photo, tag, productId ,lableType,reccomendedProduct, varientDetails, isTex,GSTrate,GSTtyp, HSNcode, categoryId, subCategoryId, videoUpload,} = req.body;
const data= db.product.findOne({ where: { id: productId },
include: [{ model: db.tagModel, attributes: ["id","name","productId"]}, { model: db.reccomendProduct, attributes: ["id", "productName","productId"]},
{ model: db.varientModel, attributes: ["id", "sort","sku","productId","waightunitno","unit","mrp","discount","price","stock","minstock","outofstock"]}]
}).then(product => {
if (product) {
db.product.update({
categoryId: categoryId ? categoryId : product.categoryId,
subCategoryId: subCategoryId ? subCategoryId : product.subCategoryId,
name:name,
description:description,
lableType:lableType,
isTex:isTex,
// photo:req.file ? req.file.location:'',
photo:photo,
GSTrate:GSTrate,
GSTtyp:GSTtyp,
HSNcode:HSNcode,
categoryId:categoryId,
subCategoryId:subCategoryId,
videoUpload:videoUpload }, { where: { id: product.id }
})
}
if(varientDetails ) {
db.varientModel.findAll ({ where: { productId:productId }})
.then(varient => {
console.log(varient+data)
for (let i=0; i < varientDetails.length; i++) {
db.varientModel.update({
productId:productId,
sort:varientDetails[i].sort,
sku: varientDetails[i].sku,
waightunitno:varientDetails[i].waightunitno,
unit:varientDetails[i].unit,
mrp:varientDetails[i].mrp,
discount: varientDetails[i].discount,
price: varientDetails[i].price,
stock: varientDetails[i].stack,
minstock: varientDetails[i].minstock,
outofstock: varientDetails[i].outofstock
}, { where: { productId:productId[i] }
})
}
})
}
Yes, there are ways to do it.
I don't find them as expressive and as clear as multiple one.
1. Creating Query on own
You can create function like this
function updateUsers(updateFirstValue, updateSecondValue, productIds) {
let query = "";
for (let index = 0; index < productIds.length; index++) {
query += `update tableName set firstUpdateValue="${updateFirstValue[index]}",secondUpdateValue="${updateSecondValue[index]}" where productId="${productIds[index]}";`;
}
return query;
}
//This is vulnerable to SQL Injection so change according to your needs
//It's just idea of doing it
let firstUpdates = [800, 900, 10];
let secondUpdates = [1.23, 2.23, 8.97];
let productIds = [1, 9, 3];
let generatedQuery = updateUsers(firstUpdates, secondUpdates, productIds);
console.log(generatedQuery);
// to run this with sequelize we can execute plain query with this
//sequelize.query(generatedQuery);
2. Using bulkCreate and updateOnDuplicate
let updatingValue = [
{productId:1, sort:100,sku:800},
{productId:2, sort:800,sku:8.27},
{productId:3, sort:400,sku:7.77}
];
model.bulkCreate(updatingValue,{
fields:["productid","sort","sku"],
updateOnDuplicate: ["sort","sku"]
}
// these are the column name that gets updated if primaryKey(productId) gets matched you have to update these accordingly
)
It had problem before but is updated now this PR
Other methods but quite complicated are also here.

Building a product-filter in NodeJS/MongoDB: only works when selecting all filters

I am building a product-filter in nodeJS. It works when i hard-code the following find()
const products = await Product.find({
price: {
$gte: price[0],
$lte: price[1],
}, category: categories, shipping: shipping})
The problem is of course that it only gives result if entered all the search options in the front-end.
However i also want the filter to work if i choose to NOT enter all the search options.
Thus i came up with the following code, to store all filters in an array, but it doesn't give any results.
The output of the array is =
{ price: { '$gte': 0, '$lte': 40820 } },
{
category: [
'60657e1d328b25043581e46f',
'601ef1ea8e06fc04cdaa5080',
'601eeaeb8e06fc04cdaa507f'
]
},
{ shipping: 'Yes' }
] <<<_--- content of searchArry????
This is my code, but it returns no results =
exports.searchFilters = async (req, res) => {
try {
const categories = req.body.categoryIds;
const price = req.body.price;
const shipping = req.body.shipping;
console.log(price, "<<<------- price data received in backend???");
console.log(categories, "<<--- categories data received in backend???");
console.log(shipping, "<<---- shipping data received in backend");
const searchArray = [];
if (req.body.price) {
const price = {
$gte: req.body.price[0],
$lte: req.body.price[1],
}
searchArray.push({ "price" : price });
}
if (req.body.categoryIds) {
searchArray.push({ "category" : categories });
}
if (req.body.shipping) {
searchArray.push({ "shipping" : shipping });
}
console.log(searchArray, "<<<_--- content of searchArry????");
const products = await Product.find({searchArray})
.populate("category", "_id name")
.populate("subs", "_id name")
.populate("postedBy", "_id name")
.exec();
res.json(products);
// console.log(products, "<<--producte?? ") console.log(products.length, "<<--- lengte") } catch(err) { console.log(err); }}
} catch(err) {
console.log(err);
}
}
Your hardcoded call looks like this:
Product.find({price, category, shipping}) // the arg is an object
And the second call (with searchArray) looks like:
Product.find({searchArray: [{price}, {category}, {shipping}]}) // the arg is an object containing an array of objects
Try merging the elements of searchArray into a single object:
Product.find(Object.assign({}, ...searchArray))
Note: Be aware that {variableName} is a shorthand for {variableName: variableName} or {"variableName": variableName}.

How do I find out the total number of items in a collection? mongoose

I have 100 items in the collection, each item has a "money" field, I need to get the total amount of money from all, preferably without crutches. i don't know how to did it.
const allMoney = async () => {
let count = 0;
await User.find({}).sort({ money: -1 }).forEach(plr => { count+=plr.money });
return count;
}
"user is my model"
Try using the Mongoose aggregation framework with the $group and
$sum operators, just like this:
user.aggregate([
{
$group: {
_id: null,
count: { $sum: "$money" }
}
}
])
The _id: null expression is necessary because a group specification must include it obligatorily. Otherwise Mongoose will throw an error.
You can use aggregate with a group which will easily fetch the total amount for you.
Refer: https://docs.mongodb.com/manual/reference/operator/aggregation/group/
for more clarification.
const allMoney = async () => {
var aggregate = Scheme.aggregate()
aggregate.group({
_id: null,
totalMoney: {$sum: '$money'}
)}
aggregate.exec(function(err, result) {
if (err) return err;
return result;
});
}

How to use MongoDB $ne on nested object property

I have a node API which connects to a mongoDB through mongoose. I am creating an advanced results middleware that enabled selecting, filtering, sorting, pagination etc. based on a Brad Traversy course Node.js API Masterclass With Express & MongoDB. This is all good.
I am adapting the code from the course to be able to use the $ne (not equal) operator and I want to be able to get a model that is not equal to a nested property (user id) of the model. I am using this for an explore feature to see a list of things, but I don't want to show the user their own things. I am having trouble figuring out how to access the id property.
********************* UPDATE *********************
It seems all the documentation I've read recommends writing const injected like this:
const injected = {
'user._id': { "$ne": req.user.id }
};
but for some reason it is not working. I can query top level properties that are just a plain string value like this:
const injected = {
access: { "$ne": "public" }
};
but not a property on an object. Does anyone know why? Is it because the property I want to query is an id? I've also tried:
const injected = {
'user._id': { "$ne": mongoose.Types.ObjectId(req.user.id) }
};
which also does not work...
So the model looks like this:
{
name: 'Awesome post',
access: 'public',
user: {
_id: '2425635463456241345', // property I want to access
}
}
then the actual advanced results middleware looks like this and it's the 'injected' object where I am trying to access id. In the course brad uses this syntax to use lte (/?averageCost[lte]=10000) but I do not get any results with my ne. Can anyone help me here?
const advancedResults = (model, populate) => async (req, res, next) => {
let query;
const injected = {
access: 'public',
'user._id[ne]': req.user.id, // I don't think user._id[ne] is correct
};
}
// Copy req.query
const reqQuery = { ...req.query, ...injected };
console.log('injected: ', injected);
// Fields to exclude
const removeFields = ['select', 'sort', 'page', 'limit'];
// Loop over removeFields and delete them from reqQuery
removeFields.forEach(param => delete reqQuery[param]);
// Create query string
let queryStr = JSON.stringify(reqQuery);
// Create operators ($gt, $gte, etc)
queryStr = queryStr.replace(/\b(gt|gte|lt|lte|in|ne)\b/g, match => `$${match}`);
// Finding resource and remove version
query = model.find(JSON.parse(queryStr)).select('-__v');
// Select Fields
if (req.query.select) {
const fields = req.query.select.split(',').join(' ');
query = query.select(fields);
}
// Sort
if (req.query.sort) {
const sortBy = req.query.sort.split(',').join(' ');
query = query.sort(sortBy);
} else {
query = query.sort('-createdAt');
}
// Pagination
const page = parseInt(req.query.page, 10) || 1;
const limit = parseInt(req.query.limit, 10) || 25;
const startIndex = (page - 1) * limit;
const endIndex = page * limit;
const total = await model.countDocuments(JSON.parse(queryStr));
query = query.skip(startIndex).limit(limit);
if (populate) {
query = query.populate(populate);
}
// Executing query
const results = await query;
// Pagination result
const pagination = {};
if (endIndex < total) {
pagination.next = {
page: page + 1,
limit,
};
}
if (startIndex > 0) {
pagination.prev = {
page: page - 1,
limit,
};
}
res.advancedResults = {
success: true,
count: results.length,
pagination,
data: results,
};
next();
};
module.exports = advancedResults;
Answering your question about how to use $ne:
The use of $ne is as follows:
"field":{
"$ne": yourValue
}
Into your query should be like:
"user._id": {
"$ne": req.user.id
}
Example here
$ne operator will return all document where the field value don't match with the given value.
As you have done, to acces the nested field is necessary use the dot notation.
Also, to ensure it works, if your schema defines _id as ObjectId maybe is necessary parse req.user.id to ObjectId.
But if in your schema is a string then should works.
So try (not tested at all):
const injected = {
'user._id': { "$ne": req.user.id }
};

MongoDB - find one and add a new property

Background: Im developing an app that shows analytics for inventory management.
It gets an office EXCEL file uploaded, and as the file uploads the app convert it to an array of JSONs. Then, it comapers each json object with the objects in the DB, change its quantity according to the XLS file, and add a timestamp to the stamps array which contain the changes in qunatity.
For example:
{"_id":"5c3f531baf4fe3182cf4f1f2",
"sku":123456,
"product_name":"Example",
"product_cost":10,
"product_price":60,
"product_quantity":100,
"Warehouse":4,
"stamps":[]
}
after the XLS upload, lets say we sold 10 units, it should look like that:
{"_id":"5c3f531baf4fe3182cf4f1f2",
"sku":123456,
"product_name":"Example",
"product_cost":10,
"product_price":60,
"product_quantity":90,
"Warehouse":4,
"stamps":[{"1548147562": -10}]
}
Right now i cant find the right commands for mongoDB to do it, Im developing in Node.js and Angular, Would love to read some ideas.
for (let i = 0; i < products.length; i++) {
ProductsDatabase.findOneAndUpdate(
{"_id": products[i]['id']},
//CHANGE QUANTITY AND ADD A STAMP
...
}
You would need two operations here. The first will be to get an array of documents from the db that match the ones in the JSON array. From the list you compare the 'product_quantity' keys and if there is a change, create a new array of objects with the product id and change in quantity.
The second operation will be an update which uses this new array with the change in quantity for each matching product.
Armed with this new array of updated product properties, it would be ideal to use a bulk update for this as looping through the list and sending
each update request to the server can be computationally costly.
Consider using the bulkWrite method which is on the model. This accepts an array of write operations and executes each of them of which a typical update operation
for your use case would have the following structure
{ updateOne :
{
"filter" : <document>,
"update" : <document>,
"upsert" : <boolean>,
"collation": <document>,
"arrayFilters": [ <filterdocument1>, ... ]
}
}
So your operations would follow this pattern:
(async () => {
let bulkOperations = []
const ids = products.map(({ id }) => id)
const matchedProducts = await ProductDatabase.find({
'_id': { '$in': ids }
}).lean().exec()
for(let product in products) {
const [matchedProduct, ...rest] = matchedProducts.filter(p => p._id === product.id)
const { _id, product_quantity } = matchedProduct
const changeInQuantity = product.product_quantity - product_quantity
if (changeInQuantity !== 0) {
const stamps = { [(new Date()).getTime()] : changeInQuantity }
bulkOperations.push({
'updateOne': {
'filter': { _id },
'update': {
'$inc': { 'product_quantity': changeInQuantity },
'$push': { stamps }
}
}
})
}
}
const bulkResult = await ProductDatabase.bulkWrite(bulkOperations)
console.log(bulkResult)
})()
You can use mongoose's findOneAndUpdate to update the existing value of a document.
"use strict";
const ids = products.map(x => x._id);
let operations = products.map(xlProductData => {
return ProductsDatabase.find({
_id: {
$in: ids
}
}).then(products => {
return products.map(productData => {
return ProductsDatabase.findOneAndUpdate({
_id: xlProductData.id // or product._id
}, {
sku: xlProductData.sku,
product_name: xlProductData.product_name,
product_cost: xlProductData.product_cost,
product_price: xlProductData.product_price,
Warehouse: xlProductData.Warehouse,
product_quantity: productData.product_quantity - xlProductData.product_quantity,
$push: {
stamps: {
[new Date().getTime()]: -1 * xlProductData.product_quantity
}
},
updated_at: new Date()
}, {
upsert: false,
returnNewDocument: true
});
});
});
});
Promise.all(operations).then(() => {
console.log('All good');
}).catch(err => {
console.log('err ', err);
});

Resources