MongoDB find documents using an array of objects as search argument - node.js

Is there a way to search a mongodb database using an array of objects as search arguments?
Lets say I have the following search preferences:
preferences = [{
product: "PRODUCT_A",
minqty: 5,
maxqty: 50
},
{
product: "PRODUCT_B",
minqty: 100,
maxqty: 500
}
]
In my database I have Jobs with the following structure:
{
jobName: "job name",
items: [{
product: "PRODUCT_A"
qty: 25
},
{
product: "PRODUCT_F"
qty: 300
}
]
}
I would like to query the database using preferences and returning any jobs that match at least one of the criteria's.
ATTEMPT 1:
I managed to use all my preferences as filters, but $match is cumulative, the way it's written it works like && in javascript. My goal is to have "match THIS || match THAT".
let pipeline = [];
preferences.map((item) => {
let search = {
product: item.product,
quantity: { $gte: item.minqty, $lte: item.maxqty },
};
return pipeline.push({
$match: {
items: {
$elemMatch: search,
},
},
});
});
const jobs = await Job.aggregate(pipeline);
ATTEMPS 2 (SUCCESS):
let search = [];
preferences.map((item, index) => {
let arguments = {
product: item.product,
quantity: { $gte: item.minqty, $lte: item.maxqty },
};
search.push(arguments);
});
let pipeline = [
{
$match: {
items: {
$elemMatch: {
$or: search,
},
},
},
},
];
const jobs = await Job.aggregate(pipeline);

Use aggregation
Denormalize items using $Unwind
Once denormalized, you can use simple match with $or
Use $lte and $gte
And update the question with your attempts of these or post a new one.

Related

It is possible to ignore fields if it doesn't exists in document?

I have a model:
const schema = new Schema({
// ....
conditions: {},
// ....
});
Conditions - nested document and I can save anything into it with any key. And let's say we have such conditions:
{
"conditions": {
"age": 10,
"name": "John"
}
}
This is located on the base. Now, I want to find this document, but since I don't know what fields are there, I am facing problems...
const conditions = {
'conditions.age': 10,
'conditions.name': 'John',
'conditions.surname': 'White' // surname doesn't exists
}
const result = await Model.find(conditions);
console.log(result) // [];
And the question is, is it possible to exclude from the filter the fields that are missing in the document? So that find() simply skipped them, did not take them into account...
Use Logical Query Operators $and and $or as below-
const conditions = {
$and: [
{ 'conditions.age': 10, },
{ 'conditions.name': 'John', },
{ $or: [{ 'conditions.surname': { $exists: false } }, { 'conditions.surname': 'White' }] }
]
}
const result = await Model.find(conditions);

Mongoose: Infinite scroll with filtering

I have these two models:
User.js
const UserSchema = new Schema({
profile: {
type: Schema.Types.ObjectId,
ref: "profiles",
},
following: [
{
type: Schema.Types.ObjectId,
ref: "users",
},
],
});
module.exports = User = mongoose.model("users", UserSchema);
Profile.js
const ProfileSchema = new Schema({
videoURL: {
type: String,
},
});
module.exports = Profile = mongoose.model("profiles", ProfileSchema);
Here's an example of a User document:
{
"following": [
{
"profile":{
"videoURL":"video_url_1"
}
},
{
"profile":{
"videoURL":"video_url_2"
}
},
{
"profile":{}
},
{
"profile":{
"videoURL":"video_url_3"
}
},
{
"profile":{
"videoURL":"video_url_4"
}
},
{
"profile":{
"videoURL":"video_url_5"
}
},
{
"profile":{}
},
{
"profile":{
"videoURL":"video_url_6"
}
}
]
}
I am trying to implement an infinite scroll of the videos of the users followed by the connected user.
This means, I will have to filter user.following.profile.videoURL
WHERE videoURL exists
Suppose, I will be loading two videos, by two videos:
Response 1: ["video_url_1","video_url_2"]
Response 2: ["video_url_3","video_url_4"]
Response 3: ["video_url_5","video_url_6"]
Usually, infinite scroll is easy because all I have to load the documents 2 by 2 by order of storage without filtering on any field.
Example: Displaying the followed users two by two in an infinite scroll
User.findById(user_id).populate({
path: "following",
options: {
skip: 2 * page,
limit: 2,
},
});
But, now I have to perform filtering on each followed_user.profile.video, and return two by two. And I don't see how I can perform BOTH the filtering and the infinite scroll at the same time.
NOTE: According to the documentation:
In general, there is no way to make populate() filter stories based on properties of the story's author. For example, the below query won't return any results, even though author is populated.
const story = await Story.
findOne({ 'author.name': 'Ian Fleming' }).
populate('author').
exec();
story; // null
So I suppose, there is no way for me to use populate to filter based user.followers, based on each user.follower.profile.videoURL
I am not sure it is possible with populate method, but you can try aggregation pipeline,
$match user_id condition
$lookup with aggregation pipeline in users collection for following
$match following id condition
$lookup with profile for following.profile
$match videoURL should exists
$project to show profile field and get first element using $arrayElemAt
$slice to do pagination in following
let page = 0;
let limit = 2;
let skip = limit * page;
User.aggregate([
{ $match: { _id: mongoose.Types.ObjectId(user_id) } },
{
$lookup: {
from: "users",
let: { following: "$following" },
pipeline: [
{ $match: { $expr: { $in: ["$_id", "$$following"] } } },
{
$lookup: {
from: "profiles",
localField: "profile",
foreignField: "_id",
as: "profile"
}
},
{ $match: { "profile.videoURL": { $exists: true } } },
{
$project: {
profile: { $arrayElemAt: ["$profile", 0] }
}
}
],
as: "following"
}
},
{
$addFields: {
following: {
$slice: ["$following", skip, limit]
}
}
}
])
Playground
Suggestion:
You can improve your schema design,
removing profile schema and add profile object in users collection, so you can achieve easily your requirement using populate method,
put match condition in following populate for videoURL exists
const UserSchema = new Schema({
profile: {
type: {
videoURL: {
type: String
}
}
},
following: [
{
type: Schema.Types.ObjectId,
ref: "users"
}
]
});
module.exports = User = mongoose.model("users", UserSchema);
User.findById(user_id).populate({
path: "following",
match: {
"profile.videoURL": { $ne: null }
},
options: {
skip: 2 * page,
limit: 2,
}
});
So what you want is table with infinite scroll and:
You can opt given ways to approach your problem :
Load data (first page) into grid.
Set filter on a col.
Load data again, this time using the filter.

MongoDB aggregation $match with $or

Is there a way to search a mongodb database using an array of objects as search arguments?
Lets say I have the following search preferences:
preferences = [{
product: "PRODUCT_A",
minqty: 5,
maxqty: 50
},
{
product: "PRODUCT_B",
minqty: 100,
maxqty: 500
}
]
In my database I have Jobs with the following structure:
{
jobName: "job name",
items: [{
product: "PRODUCT_A"
qty: 25
},
{
product: "PRODUCT_F"
qty: 300
}
]
}
I would like to query the database using preferences and returning any jobs that match at least one of the criteria's.
ATTEMPT 1:
I managed to use all my preferences as filters, but $match is cumulative, the way it's written it works like && in javascript. My goal is to have "match THIS || match THAT".
let pipeline = [];
preferences.map((item) => {
let search = {
product: item.product,
quantity: { $gte: item.minqty, $lte: item.maxqty },
};
return pipeline.push({
$match: {
items: {
$elemMatch: search,
},
},
});
});
const jobs = await Job.aggregate(pipeline);
ATTEMPS 2 (SUCCESS):
let search = [];
preferences.map((item, index) => {
let arguments = {
product: item.product,
quantity: { $gte: item.minqty, $lte: item.maxqty },
};
search.push(arguments);
});
let pipeline = [
{
$match: {
items: {
$elemMatch: {
$or: search,
},
},
},
},
];
const jobs = await Job.aggregate(pipeline);
I think you can create your search object by reducing the preferences array and use the $or operator. When you map the preferences array it is returning an array that will perform and operation. you need an object like -
{
$or: [{product1, quantity1}, {product2, quantity2}]
}
I guess you got my point.

How to pass an optional argument in Mongoose/MongoDb

I have the following query:
Documents.find({
$and: [
{
user_id: {$nin:
myUserId
}
},
{ date: { $gte: dateMax, $lt: dateMin } },
{documentTags: {$all: tags}}
],
})
What I'm trying to do is make the documentTags portion of the query optional. I have tried building the query as follows:
let tags = " ";
if (req.body.tags) {
tags = {videoTags: {$all: req.body.tags}};
}
let query = {
$and: [
{
user_id: {$nin:
myUserId
}
},
{ date: { $gte: dateMax, $lt: dateMin } },
tags
],
}
and then Document.find(query). The problem is no matter how I modify tags (whether undefined, as whitespace, or otherwise) I get various errors like $or/$and/$nor entries need to be full objects and TypeError: Cannot read property 'hasOwnProperty' of undefined.
Is there a way to build an optional requirement into the query?
I tried the option below and the query is just returning everything that matches the other fields. For some reason it isn't filtering by tags. I did a console.log(queryArr) and console.log(query) get the following respectively:
[
{ user_id: { '$nin': [Array] } },
{
date: {
'$gte': 1985-01-01T00:00:00.000Z,
'$lt': 2020-01-01T00:00:00.000Z
}
},
push: { documentTags: { '$all': [Array] } }
]
console.log(query)
{
'$and': [
{ user_id: [Object] },
{ date: [Object] },
push: { documentTags: [Object] }
]
}
You are almost there. Instead you could construct the object outside the query and just put the constructed query in $and when done..
let queryArr = [
{
user_id: {$nin: myUserId}
},
{ date: { $gte: dateMax, $lt: dateMin } }
];
if (req.body.tags) {
queryArr.push({videoTags: {$all: req.body.tags}});
}
let query = {
$and: queryArr
}
Now you can control the query by just pushing object into the query Array itself.
I figured out why it wasn't working. Basically, when you do myVar.push it creates a key-value pair such as [1,2,3,push:value]. This would work if you needed to append a k-v pair in that format, but you'll have difficulty using it in a query like mine. The right way for me turned out to be to use concact which appends the array with just the value that you set, rather than a k-v pair.
if (req.body.tags){
queryArgs = queryArgs.concat({documentTags: {$all: tags}});
}
let query = {
$and: queryArgs
}

Check whether or not a booking is possible

I'm currently implementing a booking system. The seller can specify how many items of the given type are available. The rule is simple: There can never be more bookings then available items.
Now I would like to find out with how many existing bookings a new booking is in conflict in order to check if the limit is reached.
The following diagram should give you a little insight on what I'm trying to do.
https://ibb.co/4pJk8XV
In this example the maximum amount of concurrent bookings is 2. As you can see there are already 3 bookings. One of which has no end date specified.
Only viewing the existing bookings there are never more than 2 bookings at the time.
Now I would like to check whether a new booking is possible. I know the start date for every booking. For bookings with no specified end date, the end date will be null.
I'm trying to achieve this using Mongoose.
There is no existing code regarding this problem.
Looking at each example separately: The first one with no end date should fail, since between the 07th and 09th there would be 3 bookings at a time. The second one should be fine as there is only one existing booking on the 06th.
We will start by finding all intersections with the newDocument i'm only going to make one assumption which is that the "newBooking" endDate is typed date (if its null then we make it a very far futuristic date)
let newBookingStartDate = newBooking.startDate;
let newBookingEndDate = newBooking.endDate ? newBooking.endDate : new Date().setYear(3000);
Now for the query:
let results = await db.collection.aggregate([
{
$addFields: {
tmpEndDate: {
$cond: [
{$ne: ["$endDate", null]},
"$endDate",
newBookingEndDate
]
}
}
},
{
$match: {
$or: [
{
$and: [
{
startDate: {$lt: newBookingStartDate},
},
{
tmpEndDate: {$gt: newBookingStartDate}
},
]
},
{
$and: [
{
startDate: {$gte: newBookingStartDate},
},
{
startDate: {$lt: newBookingEndDate}
},
]
},
],
}
},
])
We match documents by diving into two cases:
document.startDate is lower than newDocument.startDate - in this case all we need to check if the document.endDate is greater then the newDocument.startDate, if it is then we have an intersection
document.startDate is greater or equal to newDocument.startDate - in this case we just need to check the document.startDate is less than the newDocument.endDate and again we'll get an intersection
Now we need to iterate over the documents we found and calculate intersections between them by running the same query:
for (let i = 0; i < results.length; i++) {
let doc = results[i];
let otherIds = results.map(val => val._id);
let docStartDate = doc.startDate;
let docEndDate = doc.endDate ? doc.endDate : new Date().setYear(3000);
let count = await db.collection.aggregate([
{
$match: {
_id: {$in: otherIds}
}
},
{
$addFields: {
tmpEndDate: {
$cond: [
{$ne: ["$endDate", null]},
"$endDate",
docEndDate
]
}
}
},
{
$match: {
$or: [
{
$and: [
{
startDate: {$lt: docStartDate},
},
{
tmpEndDate: {$gt: docStartDate}
},
]
},
{
$and: [
{
startDate: {$gte: docStartDate},
},
{
startDate: {$lt: docEndDate}
},
]
},
],
}
},
{
$count: "count"
}
])
if (count[0].count >= 3){
return false;
}
}
If any of the count results is 3 or greater (3 because i didn't remove the curr document ID from the array and it will always intersect with itself) return false as inserting a new document will set you over the threshold.

Resources