Resolve custom user condition from a collection - mongodb - node.js

The idea was that exercises should only be returned where applicable. All condition properties are stored in the user collection.
As an an example
exercises collection
title: string,
conditions?: [
{ property: string, // field from the user collection
operator: string, // (gt, eq, gte, lte, ...)
value: string
}
]
Each exercise can have one or more conditions.
It could look like this:
conditions: [
{ property: 'age',
operator: 'gte'
value: '18'
}
]
Current solution
With my current solution in the backend, I go through the result of mongodb and only return the one where the condition applies or is empty.
The problem
Unfortunately, that's not the best-performing solution.
.
Would that be possible with the aggregation framework?
Unfortunately, that overwhelms my aggregation knowledge.
Open for any solution ideas? :)
best regards

Related

Search string value inside an array of objects inside an object of the jsonb column- TypeORM and Nest.js

the problem I am facing is as follows:
Search value: 'cooking'
JSON object::
data: {
skills: {
items: [ { name: 'cooking' }, ... ]
}
}
Expected result: Should find all the "skill items" that contain 'cooking' inside their name, using TypeORM and Nest.js.
The current code does not support search on the backend, and I should implement this. I want to use TypeORM features, rather than handling it with JavaScript.
Current code: (returns data based on the userId)
const allItems = this.dataRepository.find({ where: [{ user: { id: userId } }] })
I investigated the PostgreSQL documentation regarding the PostgreSQL functions and even though I understand how to create a raw SQL query, I am struggling to convert this to the TypeORM equivalent.
Note: I researched many StackOverflow issues before creating this question, but do inform me If I missed the right one. I will be glad to investigate.
Can you help me figure out the way to query this with TypeORM?
UPDATE
Let's consider the simple raw query:
SELECT *
FROM table1 t
WHERE t.data->'skills' #> '{"items":[{ "name": "cooking"}]}';
This query will provide the result for any item within the items array that will match exact name - in this case, "cooking".
That's totally fine, and it can be executed as a raw request but it is certainly not easy to maintain in the future, nor to use pattern matching and wildcards (I couldn't find a solution to do that, If you know how to do it please share!). But, this solution is good enough when you have to work on the exact matches. I'll keep this question updated with the new findings.
use Like in Where clause:
servicePoint = await this.servicePointAddressRepository.find({
where: [{ ...isActive, name: Like("%"+key+"%"), serviceExecutive:{id: userId} },
{ ...isActive, servicePointId: Like("%"+key+"%")},
{ ...isActive, branchCode: Like("%"+key+"%")},
],
skip: (page - 1) * limit,
take: limit,
order: { updatedAt: "DESC" },
relations:["serviceExecutive","address"]
});
This may help you! I'm matching with key here.

Mongoose partial field search without RegEx

Let's say I have this schema:
var mongoose = require("mongoose")
var userSchema = new mongoose.Schema({
name: {type: String},
// other fields
}, { collation: { locale: "en_US", strength: 1 } });
I use collation so that the search is case-insensitive
Then let's say I have a document with name "Dave"
{
name: "Dave",
// other fields
}
then, I search for it but without writing the whole word
var userList = {
.find({name: "da"})
.exec();
}
How can I make this work without using a regex expression? Which are quite slow. I have tried doing an index and then searching with the $text method but I don't know how to make it so that it searches only a specific field within the document.
I believe using REGEX is your best solution. What you are trying to do is literally what regex is designed for. Yeah it's slow, but any other option you try to implement will probably be slower.
Creating a text index, and using the $text is only designed to match full words so you cannot use this method.
If you are truly desperate, and really don't want to use regex you can try something else... Try creating and storing an object, with each possible substring in the document. Object lookup is O(1) time, which means it will be faster, but the tradeoff is you are storing an absurd amount of data in the database. If this is ok with you, then give 'er a try.
Let's use Dave for example. The object you store could look something like this:
{
"d": 1,
"da": 1,
"dav": 1,
"dave"" 1
}
We can store this object in a field called substrings. Then when we do the database lookup, it's as simple as:
User.find({ 'substrings.da': { $exists: true }})
But please consider using regex... It's so much simpler, so much cleaner and it's designed for exactly what you want.

How do you search a Model's attributes of type 'array'

I have a Model, Pet, with an attribute favoriteFoods.
// Pet.js
module.exports = {
attributes: {
name: {
type: 'string'
},
favoriteFoods: {
type: 'array'
}
}
};
favoriteFoods is stored as an array, because it's a free-text field for the user.
Now, is there a way to find based on values of that array?
I'm actually interested in querying from the URL, the following example DOES work, but it's not ideal.
/pet?where={"favoriteFoods":{"contains":"Best Food"}}
Is there a better way? A string comparison doesn't seem like the best solution.
For example, if one example entry has favoriteFoods: ["Best Food", "Best Treats"] and another has favoriteFoods: ["Second Best Food", "Best Treats"], then the query above will return 2 results. Our goal is a result w/ an exact match to one of the entries in the array, so this does not give us the intended result.
The simplest solution, although maybe not the most performant is to simply encapsulate in quotes. The following would find "Best Food" and not "Second Best Food"
/pet?where={"favoriteFoods":{"contains":"\"Best Food\""}}
Sub-document queries are being implemented in the next version of waterline. Your other other option would be to go native or query depending on your adapter.
In this instance your adapter would make the biggest difference.

Mongoose - get length of array in model

I have this Mongoose schema:
var postSchema = mongoose.Schema({
postId: {
type: Number,
unique: true
},
upvotes: [
{
type: Number,
unique: true
}
]
});
what the best query to use to get the length of the upvotes array? I don't believe I need to use aggregation because I only want to query for one model, just need the length of the upvotes array for a given model.
Really struggling to find this info online - everything I search for mentions the aggregation methodology which I don't believe I need.
Also, as a side note, the unique schema property of the upvotes array doesn't work, perhaps I am doing that wrong.
find results can only include content from the docs themselves1, while aggregate can project new values that are derived from the doc's content (like an array's length). That's why you need to use aggregate for this, even though you're getting just a single doc.
Post.aggregate([{$match: {postId: 5}}, {$project: {upvotes: {$size: '$upvotes'}}}])
1Single exception is the $meta projection operator to project a $text query result's score.
I'm not normally a fan of caching values, but it might be an option (and after finding this stackoverflow answer is what I'm going to do for my use case) to calculate the length of the field when the record is updated in the pre('validate') hook. For example:
var schema = new mongoose.Schema({
name: String,
upvoteCount: Number,
upvotes: [{}]
});
schema.pre('validate', function (next) {
this.upvoteCount = this.upvotes.length
next();
});
Just note that you need to do your updates the mongoose way by loading the object using find and then saving changes using object.save() - don't use findOneAndUpdate
postSchema.virtual('upvoteCount').get(function () {
return this.upvotes.length
});
let doc = await Post.findById('foobar123')
doc.upvoteCount // length of upvotes
My suggestion would be to pull the entire upvotes fields data and use .length property of returned array in node.js code
//logic only, not a functional code
post.find( filterexpression, {upvote: 1}, function(err, res){
console.log(res.upvotes.length);
});
EDIT:
Other way of doing would be stored Javascript. You can query the
upvote and count the same in mongodb side stored Javascript using
.length

Mongoose Query: compare two values on same document

How can I query a Mongo collection using Mongoose to find all the documents that have a specific relation between two of their own properties?
For example, how can I query a characters collections to find all those characters that have their currentHitPoints value less than their maximumHitPoints value? Or all those projects that have their currentPledgedMoney less than their pledgeGoal?
I tried to something like this:
mongoose.model('Character')
.find({
player: _currentPlayer
})
.where('status.currentHitpoints').lt('status.maximumHitpoints')
.exec(callback)
but I am getting errors since the lt argument must be a Number. The same goes if I use $.status.maximumHitpoints (I was hoping Mongoose would be able to resolve it like it does when doing collection operations).
Is this something that can be done within a Query? I would expect so, but can't find out how. Otherwise I can filter the whole collection with underscore but I suspect that is going to have a negative impact on performance.
PS: I also tried using similar approaches with the find call, no dice.
MongoDB 3.6 and above supports aggregation expressions within the query language:
db.monthlyBudget.find( { $expr: { $gt: [ "$spent" , "$budget" ] } } )
https://docs.mongodb.com/manual/reference/operator/query/expr/
Thanks to Aniket's suggestion in the question's comments, I found that the same can be done with Mongoose using the following syntax:
mongoose.model('Character')
.find({
player: _currentPlayer
})
.$where('this.status.currentHitpoints < this.status.maximumHitpoints')
.exec(callback)
Notice the $where method is used instead of the where method.
EDIT: To expand on Derick's comment below, a more performance sensitive solution would be to have a boolean property inside your Mongoose schema containing the result of the comparison, and update it everytime the document is saved. This can be easily achieved through the use of Mongoose Schema Plugin, so you would have something like:
var CharacterSchema = new mongoose.Schema({
// ...
status: {
hitpoints: Number,
maxHitpoints: Number,
isInFullHealth: {type: Boolean, default: false}
}
})
.plugin(function(schema, options) {
schema.pre('save', function(next) {
this.status.isInFullHealth = (this.status.hitPoints >= this.status.maxHitpoints);
next();
})
})
mongoose.model('Character')
.find({
player: _currentPlayer, $expr: { $lt: ['$currentHitpoints', '$maximumHitpoints'] }
})
This above query means find the record which has currentHitpoints less than maximumHitpoints
Starting in MongoDB 5.0, the $eq, $lt, $lte, $gt, and $gte comparison operators placed in an $expr operator can use an index on the from collection referenced in a $lookup stage.
Example
The following operation uses $expr to find documents where the spent amount exceeds the budget:
db.monthlyBudget.find( { $expr: { $gt: [ "$spent" , "$budget" ] } } )

Resources