check substring in mongo - node.js

I have filter with mongo aggregation:
query['$or'] =
{
firstName: {
$search: filter.keyword,
},
},
{
lastName: {
$regex: filter.keyword, $options: 'xi',
},
},
And later I attach the query to the aggregation.
Have user: firstName: 'john'. lastName: 'snow'.
If filter.keyword = 'jo', 'john', 'john ', all work, return 1 result, but if filter.keyword = 'john s', return 0.
I understand that I need to check not with regex but on the occurrence of the field value in the resulting string, then all subsequent characters will be ignored and the result will be returned to me. I can’t split, and check for an entry in the array, it will give an error for complex names.
This is probably a very stupid question, but I can’t verify the occurrence of the name John in the string 'John S'.
Later query will be attached to aggregation in $match.
Thanks for your answers!!!

You can continue to use regex and break word :
filter.keyword.split(" ").join("|")
This will search for firstname matching "john" or "s" :
$regex: "john|s"
Otherwise you can directly use $text : https://docs.mongodb.com/manual/reference/operator/query/text/
But it will apply a $or on each word, as the solution i propose
And finally you can use $indexOfBytes (substring search):
db.a.insert({name: "john", firstname: "snow"})
db.a.aggregate([{$project: {byteLocation:{$indexOfBytes:["john s", "$name"]}}}])
// { "_id" : ObjectId("5cadb07ddf1f1fe1647e252f"), byteLocation: 0}
db.a.aggregate([{$project: {byteLocation:{$indexOfBytes:["f john s", "$name"]}}}])
// { "_id" : ObjectId("5cadb07ddf1f1fe1647e252f"), byteLocation: 2}

Related

Find a document by INC value in MongoDB

From my website client I am sending an API request to the backend Node.js server and finding a specific document by ID like so:
var ObjectID = require('mongodb').ObjectID;
db.collection('users').find({"_id": new ObjectID(req.body._id)})
Where req.body._id equals the entire ID string, such as '5d0381ad681a2a3aa1dc5872'
However I would prefer to send only the INC (ever incrementing value) portion of the string as the argument: 'c5872'
How should I find a specific document based on just the INC value? (I'm assuming the INC is unique)
Any help is greatly appreciated.
You can temporarily transform your _id using in a string to be able to use a regular expression :
db.users.aggregate([
{
$addFields: { id: { $toString: '$_id' } }
},
{
$match: { id: /c5872$/ }
}
])
A cleaner solution is to create the field directly with a substring to avoid using a regular expression :
db.users.aggregate([
{
$addFields: { id: { $substr: [{ $toString: '$_id' }, 19, 24] } }
},
{
$match: { id: 'c5872' }
}
])
https://docs.mongodb.com/manual/reference/operator/aggregation/addFields/
https://docs.mongodb.com/manual/reference/operator/aggregation/substr/
You could create an additional ID field and a unique index to ensure that indexed fields do not store duplicate values, but note that for a five hex string, we have "1,048,576" possible combinations (16 ^ 5 = 1048576).
Note: Using a regular expression could lead to a collection scan.
A regular expression is a "prefix expression" if it starts with a
caret (^) or a left anchor (\A), followed by a string of simple
symbols. For example, the regex /^abc.*/ will be optimized by matching
only against the values from the index that start with abc.
Regular expression and index use

NodeJS MongoDB Where Array contains any element of Array

I have a collection of content objects. Each document of this collection contains an array with tags:
{
_id: ....,
title: 'Title 1',
details: { ..., tags: ['politic', 'usa'] }
},
{
_id: ....,
title: 'Title 2',
details: { ..., tags: ['art', 'modern'] }
}
The user should be able to filter for tags. For individuals and several.
Is there any way to query for that?
Example:
User search for content with one of the following tags:
['politic', 'french'] => Title1
['music', 'autumn'] => no result
['usa', 'art'] => Title1 & Title2
['modern'] => Title2
What I tried:
const aggregate = [{ $match: { "details.tags": 'music' } }];
mongodb.collection("content").aggregate(aggregate).toArray();
This works fine for searching by one tag.
If I change 'music' to an array like ['music', 'usa'] I don't get any result.
#EDIT1
I added an Index to the collection:
db.content.createIndex( { "details.tags": 1 });
Unfortunately, the aggregation query still returns an empty result. That's why I tried also a find:
db.content.find({"details.tags": ['music', 'usa']})
But also without success.
In order to find multiple values in an array, you should use the $in-operator:
db.collection.find({
"details.tags": {
$in: [
"usa",
"art"
]
}
})
See this example on mongoplayground: https://mongoplayground.net/p/vzeHNdLhq0j

match all if the user doesn't specify the field value MongoDB

I am building an API where I have several fields that are optional in my get request. So I want MongoDB to match all values for those optional fields if the user does not specify it. I have come up with this solution:
db.collection(expenses_collection).find(username: username, category: {$regex:"/" + category + "/"}, payment_type: {$regex:"/" + payment_type + "/"}})
Where if category and payment_type are not specified by the user I set them to ".*":
const {category=".*", payment_type=".*"} = req.query;
However, mongodb is still not matching any data. Any help is appreciated. Thanks a lot.
The issue is with your regex string. To match any string value, you have to use this pattern (this matches any string): (.*?)
Consider input documents:
{ _id: 1, name: "John", category: "cat 1", payment_type: "cash" },
{ _id: 2, name: "Jane", category: "cat 2", payment_type: "credit card" }
Usage to match any category field value:
let categoryStr = /(.*?)/
db.exp.find( { category: categoryStr } )
The query returns all documents.
So, in your application for the category value not specified the code can be like this:
if (category is empty or null) { // category not specified by user
categoryStr = /(.*?)/
}
Similarly, for the payment_type field also.
Then query would be:
db.exp.find( {
username: usernameStr,
category: categoryStr,
payment_type: paymentStr
} )
NOTE: The code tests fine with MongoDB NodeJS driver APIs.
Isn't this what exists is made for?
{category: { $exists: true }, payment_type: { $exists: true }}

How to find text from populated model in mongoose?

We have two models:
User
const userSchema = new mongoose.Schema({
name: {
type: String
},
email: {
type: String
},
vehicleId: {
type: mongoose.Schema.Types.ObjectId,
ref: 'vehicle'
}
})
Vehicle
const vehicleSchema = new mongoose.Schema({
name: {
type: String
},
color: {
type: String
}
})
Now, we have to find user input from user's ( name, email ) & also from vehicle's ( name, color )
- If user search "pink" then we have to find that if any user has with name, email or them vehicle include "pink" keyword or not.
- If string match from reference model then we have to return whole record.
Can you please help us, how to achieve this?
You can first run a query on a Vehicle model to fetch all vehicles that have name pink
let vehicleIds=await Model.Vehicle.aggregate([
{$match:{$or:[
{name:{$regex:"pink",$options:"i"}},
{color:{$regex:"pink",$options:"i"}
]}},
{$group:{_id:null,ids:{$push:"$_id"}}}
])
here if vehicleIds is not empty, then vehicles[0].ids will have vehcileIds that have pink in name or color
let vehcileIds=[]
if(vehicles.length) vehcileIds=vehicles[0].ids
after above query runs run another query in the user model
let users= await Models.User.find({$or:[
{name:{$regex:"pink",$options:"i"}},
{email:{$regex:"pink",$options:"i"}
{vehicleId:{$in:}}
]})
I think, the best thing you can do is, Text Search.
ANSWER 1:
1.1. Step one
You have to create text index on your collection to perform text search.
create text index with
db.user.createIndex({"name":"text", "email": "text"})
db.vehicle.createIndex({"name": "text", "color": "text"})
we have to create text index on both collection as we want to perform text search on both collections.
*
Note:
as we are creating a text index on "name", "email" from User & "name", "color" from Vehicle, you can only perform text search on this four fields in respective collections.
you can assign weights to fields so that you can sort your result according to text score. (you must do this while creating index).
*
1.2. Step two
(i am assuming you are using Javascript and mongoose here.)
db.collection("users").aggregate({$match:{$text: {$search: "pink"}}},
(err, userData: any)=>{
if(err){
return err;
}
if(userdata.length){
let vehicleIds = userData.map((user) => mongoose.Types.ObjectId(user.vehicleId));
db.collection("vehicles").aggregate({$match:{$text:{$search:"pink", _id: {$in: vehicleIds}}}}, (err, vehicleData)=>{
//this will be your vehicle details
})
})
}else{
console.log("no Data found");
}
*
The problem with this approach is you have to fire two queries because of the restrictions in Text Search and extra overhead of text index on collections.
Good Thing is you get a lot of ways to perform a search, you can sort results according to relevance with text score, it is fast than the regular expression and you get better results, you get stemming, you get stop words (please ref links that are given).
*
ANSWER 2:
you can use regular expression
db.collection("users").aggregate({$match:{$or:[{name:{$regex: /pink/}}, {email:{$regex: /pink/}}]}},
{$lookup:{from:"vehicles", localField:"vehicleId", foreignFild:"_id", as:"vehicleDetail"}},
{$unwind:{path:"$vehicleDetail"}},
{$match:{$or:[{name:{$regex:/pink/}},{color:{$regex:/pink/}}]}}
(err, data)=>{
})
ANSWER 3:
If you dont want to prefer above options, you can fire normal query too
db.collection("users").aggregate({$match:{$or:[{name:{$regex: /pink/}}, {email:{$regex: /pink/}}]}},
{$lookup:{from:"vehicles", localField:"vehicleId", foreignFild:"_id", as:"vehicleDetail"}},
{$unwind:{path:"$vehicleDetail"}},
{$match:{$or:[{name:{$regex:/pink/}},{color:{$regex:/pink/}}]}}
(err, data)=>{
})
Hope this helps. :-)
Correct me if I am wrong anywhere. :-)
Below steps you should follow:
Find user by name and email.
Populate vehicle
Again match with the color.
db.getCollection('User').aggregate([
{ $match: { email: "h#gmail.com", name: "Hardik"}},
{ $lookup: {from: 'Vehicle', localField: 'vehicleId', foreignField: '_id', as: 'vehicle'} },
{ $match: { "vehicle.color": {$regex: '.*'+ "Pink" +'.*', $options: 'si'}}},
])
Data:
User
{
"_id" : ObjectId("5d51bd5ef5fc3d6486b40ffb"),
"email" : "h#gmail.com",
"name" : "Hardik",
"vehicleId" : ObjectId("5d539786f5fc3d6486b4252b")
}
Vehicle
{
"_id" : ObjectId("5d539786f5fc3d6486b4252b"),
"name" : "Bike",
"color" : "Pink"
}
Output:
{
"_id" : ObjectId("5d51bd5ef5fc3d6486b40ffb"),
"email" : "h#gmail.com",
"name" : "Hardik",
"vehicleId" : ObjectId("5d539786f5fc3d6486b4252b"),
"vehicle" : [
{
"_id" : ObjectId("5d539786f5fc3d6486b4252b"),
"name" : "Bike",
"color" : "Pink"
}
]
}
Hope this helps!

mongoDB, finding a field that is NOT of a specific type

I have some legacy data that stored a string instead of an Object Id, I would like a way to FIND these records...
so as an example -
user : {
_id: ObjectId('234wer234wer234wer')
books_read: [
{
title: "Best book ever",
_id: "123qwe234wer345ert456rty"
},
{
title: "Worst book ever",
_id: "223qwe234wer345ert456rty"
},
{
title: "A Tail of Two Cities",
_id: ObjectId("323qwe234wer345ert456rty")
}
]
}
In this scenario, you can see 2 of the 3 books have _id of strings NOT ObjectIds... I want to update these to ObjectIds - but first I have to find them.
any suggestions greatly appreciated.

Resources