sorting alpha with mongoose - node.js

I'm trying to sort via mongoose 3.6.20 and I am receiving some unexpected results.
I have a list of companies with a name. At first I thought that maybe it was sorting in a case sensitive way. Which based on articles, I expect was true.
I'm now using a virtual property to down case the sort field. However, I'm still getting unexpected results.
CompanySchema.virtual('name_lower').get(function(){
return this.name.toLowerCase();
});
and when I sort
Company.find().sort({ name_lower: 1 });
I'm getting it in the following order:
company name
google
company name (yes a duplicate for testing)
I'm also outputting the value of my virtual property and it looks right. There is no whitespace or funky characters that would result in the 2nd 'company name' from appearing after google.
Using nodejs, express, mongoose.
What am I missing or doing incorrectly?
Update:
Based on the information provided in the answers, I refactored my schema to include some normalized fields and hooked into the pre save event of my document, where I update those normalized fields and sort using them in all future queries.
CompanySchema.pre('save', function(next){
this.normalized_name = this.name;
});
Next, is in the schema I use:
var CompanySchema = mongoose.Schema({
...
normalized_name: { type: String, set: normalize },
...
});
Where normalize is a function that for now, returns a lowercase version of the value passed into it. However, this allows me to expand on it later really fast, and I can quickly do the same to other fields that I might need to sort against.

As of MongoDB v3.4, case insensitive sorting can be done using collation method:
Company.find()
.collation({locale: "en" }) //or whatever collation you want
.sort({name:'asc'})
.exec(function(err, results) {
// use your case insensitive sorted results
});

Unfortunately MongoDB and Mongoose does not currently support complex sorting, so there are 2 options:
As you said, create a new field with the names sanitized to be all lowercase
Run a big for loop over all the data and update each company name to it's lower case form:
db.CompanyCollection.find().forEach(
function(e) {
e.CompanyName = e.CompanyName.toLowerCase();
db.CompanyCollection.save(e);
}
)
or
db.CompanyCollection.update({_id: e._id}, {$set: {CompanyName: e.CompanyName.toLowerCase()
Please see Update MongoDB collection using $toLower and Mongoose: Sort alphabetically as well for more info.

I want to put out that in this hook:
CompanySchema.pre('save', function(next){
this.normalized_name = this.name;
});
You'll have to call next(); at the end, if you want the normalized_name to be saved in the database, so the pre save hook would look like:
CompanySchema.pre('save', function(next){
this.normalized_name = this.name;
next();
});

This answer seems to be more helpful to me. I had to consider diacritics along with the case so I had used strength:3.
Mongoose: Sort alphabetically

Related

Mongoose & MongoDB: Retrieve results narrowed by multiple parameters

I need to get data from MongoDB that is first narrowed by one initial category, say '{clothing : pants}' and then a subsequent search for pants of a specific size, using an array like sizes = ['s','lg','6', '12'].
I need to return all of the results where 'pants' matches those 'sizes'.
I've started a search with:
Product.find({$and:[{categories:req.body.category, size:{$in:req.body.sizes}}]},
function(err, products) {
if (err) { console.log(err); }
return res.send(products)
});
I really don't know where to go from there. I've been all over the Mongoose docs.
Some direction would be very helpful.
The mongoose queries can receive object like Mongodb would. So you can pass the search parameters separated by ,
Product.find({categories:req.body.category, size:{$in:['s','lg','6', '12']}})
For more information on $in, check here
For more information on $and operator, check here (note we can ommit the $and operator in some cases and that is what I did)

Update by Id not finding document

I am attempting to perform a relatively simple update using Mongoose:
var location = locationService.getLocation(typedLat, typedLong, travelRadius);
DocumentModel.update({_id : passedInId }, { location : location }, function(err, resp){
next(err, resp);
});
In this case passedInId is the string:
"561ee6bbe4b0f25b4aead5c8"
And in the object (test data I created) the id is:
"_id": ObjectId("561ee6bbe4b0f25b4aead5c8")
When I run the above however, matched documents is 0. I assume this is because passedInId is being treated like a string, however when I type it to an ObjectId:
var queryId = ObjectId(passedInId)
The result is the same, the document doesn't match. What am I missing? This seems like it should be obvious....
Mongoose will correctly interpret a string as an ObjectId. One of the following must be the case:
That record is not in the collection. Run a query in the mongo shell to check.
Mongoose I'd looking in collection other than the one containing your test data. Remember, by default, mongo will lowercase the name under which you register your model and will add an a "s" to it.
Lastly, and your answer speaks to this, maybe your model it's just not being updated with any new information.
This behavior was because I had not yet updated the model in mongoose to include the location element. There is no error, it just doesn't match or do anything.

node.js mongodb projection ignored when there is a criterion

I am writing a node.js application using express, mongodb, and monk.
When I do a find with criteria only or with projections only, I get the expected result, but when I do a find with both, the full documents are returned, i.e., the projection is not performed. My code looks like this:
var collection = db.get('myDB');
collection.find({field1: "value"},{field2: 1, _id: 0},function(e,docs) {
...do stuff with docs...
});
It returns not just field2 but all fields of all the docs matching the criterion on field1. I can get field2 from this, but I don't like the inefficiency of it.
Is there a way to use both criteria and projections?
Monk uses a space-delimited string for field projection where you prefix a field name with - to exclude it.
So it should be:
var collection = db.get('myDB');
collection.find({field1: "value"}, 'field2 -_id', function(e,docs) {
...do stuff with docs...
});

Mongoose: How to populate 2 level deep population without populating fields of first level? in mongodb

Here is my Mongoose Schema:
var SchemaA = new Schema({
field1: String,
.......
fieldB : { type: Schema.Types.ObjectId, ref: 'SchemaB' }
});
var SchemaB = new Schema({
field1: String,
.......
fieldC : { type: Schema.Types.ObjectId, ref: 'SchemaC' }
});
var SchemaC = new Schema({
field1: String,
.......
.......
.......
});
While i access schemaA using find query, i want to have fields/property
of SchemaA along with SchemaB and SchemaC in the same way as we apply join operation in SQL database.
This is my approach:
SchemaA.find({})
.populate('fieldB')
.exec(function (err, result){
SchemaB.populate(result.fieldC,{path:'fieldB'},function(err, result){
.............................
});
});
The above code is working perfectly, but the problem is:
I want to have information/properties/fields of SchemaC through SchemaA, and i don't want to populate fields/properties of SchemaB.
The reason for not wanting to get the properties of SchemaB is, extra population will slows the query unnecessary.
Long story short:
I want to populate SchemaC through SchemaA without populating SchemaB.
Can you please suggest any way/approach?
As an avid mongodb fan, I suggest you use a relational database for highly relational data - that's what it's built for. You are losing all the benefits of mongodb when you have to perform 3+ queries to get a single object.
Buuuuuut, I know that comment will fall on deaf ears. Your best bet is to be as conscious as you can about performance. Your first step is to limit the fields to the minimum required. This is just good practice even with basic queries and any database engine - only get the fields you need (eg. SELECT * FROM === bad... just stop doing it!). You can also try doing lean queries to help save a lot of post-processing work mongoose does with the data. I didn't test this, but it should work...
SchemaA.find({}, 'field1 fieldB', { lean: true })
.populate({
name: 'fieldB',
select: 'fieldC',
options: { lean: true }
}).exec(function (err, result) {
// not sure how you are populating "result" in your example, as it should be an array,
// but you said your code works... so I'll let you figure out what goes here.
});
Also, a very "mongo" way of doing what you want is to save a reference in SchemaC back to SchemaA. When I say "mongo" way of doing it, you have to break away from your years of thinking about relational data queries. Do whatever it takes to perform fewer queries on the database, even if it requires two-way references and/or data duplication.
For example, if I had a Book schema and Author schema, I would likely save the authors first and last name in the Books collection, along with an _id reference to the full profile in the Authors collection. That way I can load my Books in a single query, still display the author's name, and then generate a hyperlink to the author's profile: /author/{_id}. This is known as "data denormalization", and it has been known to give people heartburn. I try and use it on data that doesn't change very often - like people's names. In the occasion that a name does change, it's trivial to write a function to update all the names in multiple places.
SchemaA.find({})
.populate({
path: "fieldB",
populate:{path:"fieldC"}
}).exec(function (err, result) {
//this is how you can get all key value pair of SchemaA, SchemaB and SchemaC
//example: result.fieldB.fieldC._id(key of SchemaC)
});
why not add a ref to SchemaC on SchemaA? there will be no way to bridge to SchemaC from SchemaA if there is no SchemaB the way you currently have it unless you populate SchemaB with no other data than a ref to SchemaC
As explained in the docs under Field Selection, you can restrict what fields are returned.
.populate('fieldB') becomes populate('fieldB', 'fieldC -_id'). The -_id is required to omit the _id field just like when using select().
I think this is not possible.Because,when a document in A referring a document in B and that document is referring another document in C, how can document in A know which document to refer from C without any help from B.

Query MongoDB for documents with non-empty intersection of arrays

So - I have a Heroku + Node + Express + MongoDB app in development, and I've been playing around all day trying to figure out how to get the mongo query I want. In a mongo collection, every document has a property 'fruits', which is an array of strings like
fruits : ['apple', 'orange', 'lemon']
The size and contents of fruits is arbitrary. I want to query the database for all documents whose fruits array has at least one element in common with another array which I provide when I make the query. How can I achieve this?
What I've tried so far centers around the $where query and some server-side JS. If I do something like
mongo.Db.connect(mongoUri, function(err, db) {
db.collection('Users', function(er, collection) {
collection.find( {$where: 'false' } ).toArray(function(err, matches){
console.log(matches)
...
});
});
});
I get an empty array - good! And if I change the false to true, I get the whole DB - also good! Things get less good with something like the following:
collection.find( {$where: function(){return false} } ).toArray(function(err, matches){...
I get the whole DB again! I thought this would pretty obviously return the same empty array as the very first example I gave above, as per the syntax presented at http://docs.mongodb.org/manual/reference/operator/query/where/
I was trying the above out just to see if I could get anything to work with $where, but further problems arise when I try to add server-side JS to the mix. I had a look at
http://docs.mongodb.org/manual/tutorial/store-javascript-function-on-server/
and added the following to the system.js collection in my DB:
{
"_id": "isMatch",
"value": "function(){return false}"
}
and modified the above example with the line
collection.find( {$where: 'isMatch()' } ).toArray(function(err, matches){...
thinking that isMatch would now be in mongo's bag of tricks when it eval's the string (I assume it must be eval'ing the string), and the string syntax was the best bet from the playings around described above. Sadly this results in a console log of null - another flavour of wrong, I expected again an empty array.
Can anyone help me achieve the query I described above? I suspect I can work out the nitty gritty myself if you kind folks can help me understand 1. How to use the function syntax instead of the string syntax for $where, and 2. How to correctly declare a JS function in system.js for use with $where. Thanks in advance!
Your $where queries work fine when I try them in the shell, but you don't need to use $where for this query. Instead, you can use an $in that targets your fruits field to find docs that contain at least one matching fruit:
var fruits = ['apple', 'orange', 'lemon'];
db.test.find({fruits: {$in: fruits}});
Don't quite seem to know what was the case in 2013, but now I think you have to use $elemMatch along with $in that targets your fruits field to find docs that contain at least one matching fruit:
var fruits = ['apple', 'orange', 'lemon'];
db.test.find({fruits: {$elemMatch: {$in: fruits}}});

Resources