Can a FOXX application automatically create an index? I have a collection (model) where I need a field to be used as a unique index for performance. I could create the hash after the fact, but I just wanted to be sure it wasn't available using the model definition. If so, where can I find documentation?
A secondary question is how to create an index in FOXX? I know how to do it in arangojs but I can't seem to find it in the FOXX documentation. Scratch this question. I figured this out: db.collection.createIndex(). But boy was that hidden deep in the "misc" section of the documentation.
The index API is not part of the Foxx API but the general ArangoDB API (Foxx is merely the framework ArangoDB provides for building and managing microservices) and can be found in the ArangoDB documentation: https://docs.arangodb.com/IndexHandling/WorkingWithIndexes.html
'use strict';
var myCollection = applicationContext.collection('my-data');
myCollection.ensureIndex({type: 'hash', fields: ['a', 'b'], unique: true});
In ArangoDB 2.x Foxx provides wrappers around collections and documents (i.e. datasets stored in those collections) called repositories and models respectively. Each repository represents a collection and each model represents a document. ArangoDB 3.0 will provide a new, simplified API that gets rid of this additional complexity by encouraging you to use the underlying collection APIs ArangoDB already provides.
In order to use the index-specific methods on Foxx repositories (like the geo queries for collections with geo indexes) you need to define the repository with the additional indexes property like so:
'use strict';
var Foxx = require('org/arangodb/foxx').Repository;
var MyModel = Foxx.Model.extend({/* ... */});
var MyRepo = Foxx.Repository.extend({
indexes: [
// same syntax as collection.ensureIndex:
{type: 'hash', fields: ['a', 'b'], unique: true}
]
});
var repo = new MyRepo(applicationContext.collection('my-data'), {
model: MyModel
});
When the repository is instantiated (i.e. new MyRepo(/* ... */) is invoked), Foxx will ensure the indexes are created as necessary.
See the documentation at https://docs.arangodb.com/Foxx/Develop/Repository.html#defining-indexes.
Alternatively if you don't want to use Foxx repositories you can simply define the indexes in your setup script after creating the collection, using the regular index API above. Either way you don't need to worry about running the code multiple times: ensureIndex will do nothing if the index already exists.
Related
I want to set 2 indexes for now, perhaps a 3rd but wanted to know how I can test if they are actually working? Do I need to use with mongo shell or is there a way to check using Node.js during development? I also saw an example of the indexes being created in mongoDb Compass. I am using mongoDb Atlas so wondered if I must just set the index in Compass or do I still need to do it in my mongoose schema?
Also, the mongoose docs say you should set autoIndex to false. Is the below then correct?
const mongoose = require("mongoose");
const Schema = mongoose.Schema;
const userSchema = new Schema({
firstName: {
type: String,
},
lastName: {
type: String,
},
});
userSchema.set("autoIndex", false);
userSchema.index({ firstName: 1, lastName: 1 });
module.exports = mongoose.model("User", userSchema);
There are a bunch of different questions here, let's see if we can tackle them in order.
I want to set 2 indexes for now, perhaps a 3rd
This isn't a question from your side, but rather from mine. What are the indexes that you are considering and what queries will you be running?
The reason I ask is because I only see a single index definition provided in the question ({ firstName: 1, lastName: 1 }) and no query. Normally indexes are designed specifically to support the queries, so the first step towards ensuring a successful indexing strategy is to make sure they align appropriately with the anticipated workload.
how I can test if they are actually working? Do I need to use with mongo shell or is there a way to check using Node.js during development?
There are a few ways to approach this, which include:
Using the explain() method to confirm that the winningPlan is using the index as expected. This is often done via the MongoDB Shell or via Compass.
Using the $indexStats aggregation stage to confirm that usage counters of the index are incrementing as expected when the application runs.
Taking a look at some of the tabs in the Atlas UI such as Performance Advisor or the Profiler which may help alert you to unoptimized operations and missing indexes.
I am using mongoDb Atlas so wondered if I must just set the index in Compass or do I still need to do it in my mongoose schema?
You can use Compass (or the Atlas UI, or the MongoDB Shell) to create your indexes. I would recommend against doing this in the application directly.
Also, the mongoose docs say you should set autoIndex to false. Is the below then correct?
As noted above, I would go further and remove index creation from the application code altogether. There can be some unintended side effects of making the application directly responsible for index management, which is one of the reasons that Mongoose no longer recommends using the autoIndex functionality.
What I am really trying to do is to make indexes for filtering and string matching of documents based on their property values.
I know that mongodb has built in operators such as $text that are very helpful with this sort of functionality.
I'm not sure how to access these operators while using mongoose, or if there are any methods i need to use to access them.
I want to use mongoose to still define schema and models but need the functionality of native mongodb.
Is this possible?
Below are my views, Please add if I miss anything or if something needs to be modified or well-explained :
1. You will still be able to use mongoDB's native functionalities on using Mongoose models.
2. Mongoose is a kind of wrapper on top of native mongoDB-driver.
3. It would be very useful if you want to have schema based collections/data.
4. Additionally it would provide few more features than native mongoDB's driver. You might see few syntax differences between those two.
5. Few examples like `.findByIdAndUpdate()` & `.populate()` are mongoose specific, which has equivalent functionalities available in mongoDB driver/mongoDB as well.
6. In general it's quiet common to define mongoose models and use those over mongoDB's functionality in coding(As in node.js - You would write all of your same DB queries on Mongoose models, queries that you execute in DB).
Point 2 :
Mongoose is an object document modeling (ODM) layer that sits on top of Node's MongoDB driver. If your coming from SQL, it's similar to an ORM for a relational database.
Point 3 :
In code if you're using mongoose models to implement your write queries, unless you define a field in model - it wouldn't be added to DB though you pass it in request. Additionally you can do multiple things like making a field unique/required etc.. it's kind of making your mongoDB data look like schema based. If your collections data is more like random data(newsfeed kind of thing where fields are not same for each document & you can't predict data) then you might not care of using mongoose.
Point 6 :
Let's say you use mongo shell or a client like mongo compass/robo3T and execute a query that's like this :
db.getCollection('yourCollection').find(
{
$text: {
$search: 'employeeName',
$diacriticSensitive: false
},
country: 'usa'
},
{
employee_id: 1,
name: 1
}
).sort({ score: { $meta: 'textScore' } });
you would do same on mongoose model(As yourCollectionModel is already defined) :
yourCollectionModel.find(
{
$text: {
$search: 'employeeName',
$diacriticSensitive: false
},
country: 'usa'
},
{
employee_id: 1,
name: 1
}
).sort({ score: { $meta: 'textScore' } });
You would see key functionality difference more on writes rather than reads while using mongoose, though all the above is not about performance - If you ask me, I can say you might see much performance gains using mongoose.
Ref : Mongoose Vs MongoDB Driver
I am working with SailsJs+MongoDB API. I have to create New colletion in mongoDB .Name of colletion will be in request Parameter.
example:
Suppose I want to create 'Users' collection in 'mongoDbDatabase' database
by following request.
{
"collectionName" : "Users",
"dbName" :"mongoDbDatabase"
}
Now is there any way to create dynamic collection in mongoDB using req.param('collectionName) variable ?
To use all the tools that Sails provides, you have to add code (before you start your app) for each Model / Collection you are planning to interact with. As a result, creating a collection dynamically will mean you can't take advantage of the whole data - framework sails provides.
Are you sure you need a dynamic collection? Why not a defined collection differentiated by attributes?
If you really need to do this from Sails, it looks like you can get access to the underlying raw mongo database:
var db = AnyModel.getDatastore().manager; // the database will be as defined in config/models.js or config/connections.js
var collectionName = 'Widgets';
db.createCollection(collectionName);
// note, even if this works, something like 'Widgets.find' will not.
I'm using MongoDB and Mongoose in a REST API. Some deployments require a replica set, thus separate read/write databases, so as a result I have separate read/write connections in the API. However, more simple deployments don't need a replica-set, and in those cases I point my read/write connections to the same MongoDB instance and database.
My general approach is to create all models for both connections at API start up. Even when read/write conns are connecting to same database, I am able to create the same models on both connections without error.
let ReadUser = dbRead.model('User', userSchema);
let WriteUser = dbWrite.model('User', userSchema);
// no error even when dbRead and dbWrite point to same DB
Trouble comes when until I start using Mongoose Discriminators.
let ReadSpecialUser = ReadUser.discriminator('SpecialUser', specialUserSchema);
let WriteSpecialUser = WriteUser.discriminator('SpecialUser', specialUserSchema);
// Results in this Error when read and write point to same DB:
// Error: Discriminator with name "SpecialUser" already exists
I'm look for an elegant way to deal with this. Is there a way to query the db for discriminators that are already in use?
According to the Mongoose API docs the way to do this is to use Model.discriminators. So in the case above it would be
ReadUser.discriminators
or
WriteUser.discriminators
However this doesn't return anything for me. What does work is using
Object.keys(Model.discriminators)
As expected this gets you an array of strings of the discriminator names you've set previously.
If you want to use the existing discriminator model and know its name what you can do is use Model.discriminators.discriminatorName. In your example it would be:
let ReadSpecialUserDocument = new ReadUser.discriminators.SpecialUser({
key: value,
key: value,
});
ReadSpecialUserDocument.save()
This can be useful when you need to reuse the discriminator at different times, and its name is tied to your data in some way.
I am creating an expressjs api and using mongodb. I have a decent understanding of indexes and I understand that they are expensive to create when there is data in the database.
In MS Sql Server you would create indexes when creating your database tables. My question is do I handle this creation of indexes in a post call in my express app or do I achieve this using scripts when deploymening my application?
For example I need Geospatial indexing.
Would index creation be handled in the express app like this?
//express post call
let col = db.collection( 'collection' );
col.createIndex( // someIndex );
col.insertOne( //Some document );
I am looking for the best method to creating the 'initial' state of my mongodb and specifically creating indexes I will need for certain collections before these collections contain any documents.
So, It may happen, You have a lot of data in your database while deployment and you do not want your Indexing terrible. Here's what MongoDB can Help. You can do indexing in Background which will not prevent all read and write operations to the database while the index builds.A simple Command:
db.collection.createIndex( { a: 1 }, { background: true } )
Check the Manual For details.
https://docs.mongodb.org/manual/tutorial/build-indexes-in-the-background/