I want to set 2 indexes for now, perhaps a 3rd but wanted to know how I can test if they are actually working? Do I need to use with mongo shell or is there a way to check using Node.js during development? I also saw an example of the indexes being created in mongoDb Compass. I am using mongoDb Atlas so wondered if I must just set the index in Compass or do I still need to do it in my mongoose schema?
Also, the mongoose docs say you should set autoIndex to false. Is the below then correct?
const mongoose = require("mongoose");
const Schema = mongoose.Schema;
const userSchema = new Schema({
firstName: {
type: String,
},
lastName: {
type: String,
},
});
userSchema.set("autoIndex", false);
userSchema.index({ firstName: 1, lastName: 1 });
module.exports = mongoose.model("User", userSchema);
There are a bunch of different questions here, let's see if we can tackle them in order.
I want to set 2 indexes for now, perhaps a 3rd
This isn't a question from your side, but rather from mine. What are the indexes that you are considering and what queries will you be running?
The reason I ask is because I only see a single index definition provided in the question ({ firstName: 1, lastName: 1 }) and no query. Normally indexes are designed specifically to support the queries, so the first step towards ensuring a successful indexing strategy is to make sure they align appropriately with the anticipated workload.
how I can test if they are actually working? Do I need to use with mongo shell or is there a way to check using Node.js during development?
There are a few ways to approach this, which include:
Using the explain() method to confirm that the winningPlan is using the index as expected. This is often done via the MongoDB Shell or via Compass.
Using the $indexStats aggregation stage to confirm that usage counters of the index are incrementing as expected when the application runs.
Taking a look at some of the tabs in the Atlas UI such as Performance Advisor or the Profiler which may help alert you to unoptimized operations and missing indexes.
I am using mongoDb Atlas so wondered if I must just set the index in Compass or do I still need to do it in my mongoose schema?
You can use Compass (or the Atlas UI, or the MongoDB Shell) to create your indexes. I would recommend against doing this in the application directly.
Also, the mongoose docs say you should set autoIndex to false. Is the below then correct?
As noted above, I would go further and remove index creation from the application code altogether. There can be some unintended side effects of making the application directly responsible for index management, which is one of the reasons that Mongoose no longer recommends using the autoIndex functionality.
Related
What I am really trying to do is to make indexes for filtering and string matching of documents based on their property values.
I know that mongodb has built in operators such as $text that are very helpful with this sort of functionality.
I'm not sure how to access these operators while using mongoose, or if there are any methods i need to use to access them.
I want to use mongoose to still define schema and models but need the functionality of native mongodb.
Is this possible?
Below are my views, Please add if I miss anything or if something needs to be modified or well-explained :
1. You will still be able to use mongoDB's native functionalities on using Mongoose models.
2. Mongoose is a kind of wrapper on top of native mongoDB-driver.
3. It would be very useful if you want to have schema based collections/data.
4. Additionally it would provide few more features than native mongoDB's driver. You might see few syntax differences between those two.
5. Few examples like `.findByIdAndUpdate()` & `.populate()` are mongoose specific, which has equivalent functionalities available in mongoDB driver/mongoDB as well.
6. In general it's quiet common to define mongoose models and use those over mongoDB's functionality in coding(As in node.js - You would write all of your same DB queries on Mongoose models, queries that you execute in DB).
Point 2 :
Mongoose is an object document modeling (ODM) layer that sits on top of Node's MongoDB driver. If your coming from SQL, it's similar to an ORM for a relational database.
Point 3 :
In code if you're using mongoose models to implement your write queries, unless you define a field in model - it wouldn't be added to DB though you pass it in request. Additionally you can do multiple things like making a field unique/required etc.. it's kind of making your mongoDB data look like schema based. If your collections data is more like random data(newsfeed kind of thing where fields are not same for each document & you can't predict data) then you might not care of using mongoose.
Point 6 :
Let's say you use mongo shell or a client like mongo compass/robo3T and execute a query that's like this :
db.getCollection('yourCollection').find(
{
$text: {
$search: 'employeeName',
$diacriticSensitive: false
},
country: 'usa'
},
{
employee_id: 1,
name: 1
}
).sort({ score: { $meta: 'textScore' } });
you would do same on mongoose model(As yourCollectionModel is already defined) :
yourCollectionModel.find(
{
$text: {
$search: 'employeeName',
$diacriticSensitive: false
},
country: 'usa'
},
{
employee_id: 1,
name: 1
}
).sort({ score: { $meta: 'textScore' } });
You would see key functionality difference more on writes rather than reads while using mongoose, though all the above is not about performance - If you ask me, I can say you might see much performance gains using mongoose.
Ref : Mongoose Vs MongoDB Driver
Can a FOXX application automatically create an index? I have a collection (model) where I need a field to be used as a unique index for performance. I could create the hash after the fact, but I just wanted to be sure it wasn't available using the model definition. If so, where can I find documentation?
A secondary question is how to create an index in FOXX? I know how to do it in arangojs but I can't seem to find it in the FOXX documentation. Scratch this question. I figured this out: db.collection.createIndex(). But boy was that hidden deep in the "misc" section of the documentation.
The index API is not part of the Foxx API but the general ArangoDB API (Foxx is merely the framework ArangoDB provides for building and managing microservices) and can be found in the ArangoDB documentation: https://docs.arangodb.com/IndexHandling/WorkingWithIndexes.html
'use strict';
var myCollection = applicationContext.collection('my-data');
myCollection.ensureIndex({type: 'hash', fields: ['a', 'b'], unique: true});
In ArangoDB 2.x Foxx provides wrappers around collections and documents (i.e. datasets stored in those collections) called repositories and models respectively. Each repository represents a collection and each model represents a document. ArangoDB 3.0 will provide a new, simplified API that gets rid of this additional complexity by encouraging you to use the underlying collection APIs ArangoDB already provides.
In order to use the index-specific methods on Foxx repositories (like the geo queries for collections with geo indexes) you need to define the repository with the additional indexes property like so:
'use strict';
var Foxx = require('org/arangodb/foxx').Repository;
var MyModel = Foxx.Model.extend({/* ... */});
var MyRepo = Foxx.Repository.extend({
indexes: [
// same syntax as collection.ensureIndex:
{type: 'hash', fields: ['a', 'b'], unique: true}
]
});
var repo = new MyRepo(applicationContext.collection('my-data'), {
model: MyModel
});
When the repository is instantiated (i.e. new MyRepo(/* ... */) is invoked), Foxx will ensure the indexes are created as necessary.
See the documentation at https://docs.arangodb.com/Foxx/Develop/Repository.html#defining-indexes.
Alternatively if you don't want to use Foxx repositories you can simply define the indexes in your setup script after creating the collection, using the regular index API above. Either way you don't need to worry about running the code multiple times: ensureIndex will do nothing if the index already exists.
I just want to know if when I set a mongoose connection and I define some models, (previously adding their appropriate requires on app.js, or wathever), the model, if not exist, will be created automatically the first time when I run node app.js?
Is this kind of logic correct?
If not, do I have to create before my mongoDB collections, models and so on?
I was thinking to an automatic creation of the mongo db collection when I first run the app.js
Thanks!
Michele Prandina
Schemas (and models) are a client-side (node.js) manifestation of your data model. A few things, like the indexes you've defined, are created upon first use (like saving a document for example). Nearly everything else is delay created, including collections.
If you want consistent behavior regarding your models (and their associated schemas), you'll need to make sure they're loaded prior to any access of the associated database. It doesn't really matter where you put them, as long as they are created/executed prior to usage. You might for example:
app.js
models\Cheese.js
\Cracker.js
Then, in app.js:
var Cheese = require('Cheese.js');
var Cracker = require('Cracker.js');
Assuming, of course, you've exported the models:
model.exports = mongoose.model('Cheese',
new mongoose.Schema({
name: String,
color: String
})
);
Can someone please stop me from going insane!
I have a MongoDB database which has a simply database that was created and populated via Mongoose, this works great I can perform finds woth no problems at all.
I went into the Mongo console and created a new database with just use newDB and the performed a simple insert, I inserted several records and they appeared fine within Mongo. I can find on them and so all the Mongo operations but when I try to perform a find on this database Mongoose returns a null???
I have noticed that the database I created in Mongo console does not create the '__v' field which I believe is for Mongoose internal indexing uses, I have created this field in my custom tables but still no joy I just cannot create data from outside of Mongoose and use it within my app??????
I have spent hours looking into this and reading maybe I just missed something but honestly I cannot find a thing on this and many people must hit this every week????
**Sorry here is the code I am running against the database:
exports.adduser = function(req, res){
var mongoose = require("mongoose");
mongoose.connect("localhost/nm", function(err){
if(err)throw(err);
console.log("Connected to MongoDB successfully...")
var schema = mongoose.Schema({
Firstname: String,
Lastname: String,
MiddleInitial: String,
Password: String,
Username: String
});
var auser = mongoose.model("Users", schema);
auser.find({}, function(err, alist){
console.log(">>>>"+alist);
});
});
**
Thanks again!!!!! for your input it is very much appreciated....
try inspecting mongo instance with
show dbs
use <dbName>
in mongo shell to make sure you are using the right database and then
show collections
or alternatively
db.getCollectionNames()
To see if your collections are there or not.
__v is a document version property incremented only for array operations(mongoose 3).
Your connection string might be wrong.
Anyone experiencing this problem and I know there are a good few read the comment by robertklep above it solved my problems very quickly!
DOnt know how the accept a comment sorry!
I have a mongoose User schema built like this:
var UserSchema = new Schema({
username: { type: String, required: true, index: { unique: true } },
password: { type: String, required: true },
salt: { type: String, required: true}
});
I want to be able to send this user object to the client side of my application but I don't want to sned the password or salt fields.
So I added he following code to my user model module
U
serSchema.methods.forClientSide = function() {
console.log('in UserSchema.methods.forClientSide');
console.log(this);
//var userForClientSide=_.omit(this,'passsword','salt');
var userForClientSide={_id:this._id, username:this.username };
console.log(userForClientSide);
return userForClientSide;
}
I have required the underscore module (its installed locally via a dependency in my package.js).
not the commented out line - I was expecting it to omit the password and salt fields of the user object but it did not do anything :( the logged object had the full set of properties.
when replaced with the currently used like var userForClientSide={_id:this._id, username:this.username }; it gets the results I want but:
1) I want to know why does the _.omit not work.
2) I don't like my current workaround very much because it actually selects some properties instead of omitting the ones I don't like so if I will add any new propertes to the scema I will have to add them here as well.
This is my first attempt at writing something using node.js/express/mongodb/mongoose etc. so It is very possible hat I am missing some other better solution to this issue (possibly some feature of mongoose ) feel free to educate me of the right way to do things like this.
so basically I want to know both what is the right way to do this and why did my way not work.
thanks
1) I want to know why does the _.omit not work.
Mongoose uses defineProperty and some heavy metaprogramming. If you want to use underscore, first call user.toJSON() to get a plain old javascript object that will work better with underscore without all the metaprogramming fanciness, functions, etc.
A better solution is to use mongo/mongoose's fields object and pass the string "-password -salt" and therefore just omit getting these back from mongo at all.
Another approach is to use the mongoose Transform (search for "tranform" on that page). Your use case is the EXACT use case the documentation uses as an example.
You can also make your mongoose queries "lean" by calling .lean() on your query, in which case you will get back plain javascript objects instead of mongoose model instances.
However, after trying each of these things, I'm personally coming to the opinion that there should be a separate collection for Account that has the login details and a User collection, which will make leaking the hashes extremely unlikely even by accident, but any of the above will work.