I just want to know if when I set a mongoose connection and I define some models, (previously adding their appropriate requires on app.js, or wathever), the model, if not exist, will be created automatically the first time when I run node app.js?
Is this kind of logic correct?
If not, do I have to create before my mongoDB collections, models and so on?
I was thinking to an automatic creation of the mongo db collection when I first run the app.js
Thanks!
Michele Prandina
Schemas (and models) are a client-side (node.js) manifestation of your data model. A few things, like the indexes you've defined, are created upon first use (like saving a document for example). Nearly everything else is delay created, including collections.
If you want consistent behavior regarding your models (and their associated schemas), you'll need to make sure they're loaded prior to any access of the associated database. It doesn't really matter where you put them, as long as they are created/executed prior to usage. You might for example:
app.js
models\Cheese.js
\Cracker.js
Then, in app.js:
var Cheese = require('Cheese.js');
var Cracker = require('Cracker.js');
Assuming, of course, you've exported the models:
model.exports = mongoose.model('Cheese',
new mongoose.Schema({
name: String,
color: String
})
);
Related
I want to set 2 indexes for now, perhaps a 3rd but wanted to know how I can test if they are actually working? Do I need to use with mongo shell or is there a way to check using Node.js during development? I also saw an example of the indexes being created in mongoDb Compass. I am using mongoDb Atlas so wondered if I must just set the index in Compass or do I still need to do it in my mongoose schema?
Also, the mongoose docs say you should set autoIndex to false. Is the below then correct?
const mongoose = require("mongoose");
const Schema = mongoose.Schema;
const userSchema = new Schema({
firstName: {
type: String,
},
lastName: {
type: String,
},
});
userSchema.set("autoIndex", false);
userSchema.index({ firstName: 1, lastName: 1 });
module.exports = mongoose.model("User", userSchema);
There are a bunch of different questions here, let's see if we can tackle them in order.
I want to set 2 indexes for now, perhaps a 3rd
This isn't a question from your side, but rather from mine. What are the indexes that you are considering and what queries will you be running?
The reason I ask is because I only see a single index definition provided in the question ({ firstName: 1, lastName: 1 }) and no query. Normally indexes are designed specifically to support the queries, so the first step towards ensuring a successful indexing strategy is to make sure they align appropriately with the anticipated workload.
how I can test if they are actually working? Do I need to use with mongo shell or is there a way to check using Node.js during development?
There are a few ways to approach this, which include:
Using the explain() method to confirm that the winningPlan is using the index as expected. This is often done via the MongoDB Shell or via Compass.
Using the $indexStats aggregation stage to confirm that usage counters of the index are incrementing as expected when the application runs.
Taking a look at some of the tabs in the Atlas UI such as Performance Advisor or the Profiler which may help alert you to unoptimized operations and missing indexes.
I am using mongoDb Atlas so wondered if I must just set the index in Compass or do I still need to do it in my mongoose schema?
You can use Compass (or the Atlas UI, or the MongoDB Shell) to create your indexes. I would recommend against doing this in the application directly.
Also, the mongoose docs say you should set autoIndex to false. Is the below then correct?
As noted above, I would go further and remove index creation from the application code altogether. There can be some unintended side effects of making the application directly responsible for index management, which is one of the reasons that Mongoose no longer recommends using the autoIndex functionality.
I recently started using Node.js + Express.js (generated with pug) + pg-promise for handling db.
My first target is to obtain data from Postgres (already set up) and display it pretty using render and pug. Let's say it is user list from Users table.
On this restful tutorial I have learned how to get data and return it as JSON - it worked.
Based on Mozilla's tutorial I seperated my code:
routes/users.js: where for '/' I call user_controller.user_list method (using router.get)
controllers/userController.js I have exported user_list where I would like to ask model for data and call render if I have results
queries.js which is kinda my model? But I'm not sure. It has API: connection to db with promises and one function for every query I am going to use in Controllers. I believe I should have like one Model file per table (or any logical entity) but where to store pgp connections?
This file is based on first tutorial I mentioned
// queries.js (connectionString is set properly to my postgres)
var pgp = require('pg-promise')(options);
var db = pgp(connectionString);
function getUsers(req, res, next) {
db.any('SELECT (user_id, username) FROM public.users ORDER BY user_id ASC LIMIT 1000')
.then(function (data) {
res.json({ data: data });
})
.catch(function (err) {
return next(err);
});
}
module.exports = {
getUsers: getUsers
};
Here starts my problem as most tutorials uses mongoose which is very model-db-schema-friendly and what I have is simple 'SELECT ...' string I pass to pg-promise's any() function.
Therefore I have no model class like User.
In userControllers.js I don't know how to call getUsers() to handle its data. Returning JS object from getUsers() would be nice.
Also: where should I call render? In controller or only in
db.any(...).then(function (data) { <--here--> })
Before, I also tried to embed whole Postgres handling into Controller but from db.any() I got this array for handling:
[{ row: '(1,John)' },{ row: '(2,Amy)' },{ row: '(50,Peter)' } ]
Didn't know how go from there as I probably lost my API functionality as well ;-)
I am browsing through multiple tutorials how to handle MVC but usually they handle MongoDB and
satisfy readers with res.send() not render().
I am not sure that I understand what your question is exactly about, but since I do not have enough reputation to comment, I'll do my best to help you with your interrogations. :)
First, regarding the queries.js file, it is IMO not exactly a model, but rather a DAO (Data Access Object) file. DAO comes between you Model (which is actually you database) and your Controller layers. There usually is a DAO file per object (User, Pet, whatever you want) in your data model.
When the data model is rather complex, it can be useful to use an Object Relational Mapping (ORM) such as Mongoose to map your database and execute complexe processes on your objects. In such a case, you might need a specific file per object so as to describe your model and store your queries. But since you don't need an ORM, you DAO can directly interact with your database. That is why you do not have a User.js file.
Regarding the way the db object should be used, I think you should refer directly to pg-promise documentation on the matter.
IMPORTANT: For any given connection, you should only create a single
Database object in a separate module, to be shared in your application
(see the code example below). If instead you keep creating the
Database object dynamically, your application will suffer from loss in
performance, and will be getting a warning in a development
environment (when NODE_ENV = development)
As a matter of fact, a db object in pg-promise sort of represents the database itself and is actually designed for the simultaneous use of several databases, which does not seem to be your case for the moment.
Finally, when it comes to the render function, I believe it should be in the controller, as your DAO is not supposed to know how the data it has gathered is going to be used.
Modularity is always a time-saving choice on the long-term.
Furthermore, note that you might later need a Business Layer between your DAO and your controller, in order to preprocess and postprocess data you are going to persist or to display. In such a case, if you need for instance to ask for data from your database, you will need to render data after it is processed by the Business layer. If the render is made in the DAO layer, it will not be possible.
In the link I provided earlier to pg-promise's db object connection, you will also find documentation on the any() method. You might already have looked it up.
It specifically states that it returns
A promise object that represents the query result:
When no rows are returned, it resolves with an empty array.
When 1 or more rows are returned, it resolves with the array of rows.
so your returned data is a JS Array. If you want to make it a JS object, just use
JSON.stringify(yourArray) to process your data before rendering it in your controller.
But I wonder if Pug is not able to use your data directly.
Also, if you cannot get any data out of your DAO, maybe you should check that your data object is not empty, as such a case is tolerated by the any() method. If you expect your query to always return something, you might want to consider using the many() or the one() methods.
I hope this helps you.
I am working with SailsJs+MongoDB API. I have to create New colletion in mongoDB .Name of colletion will be in request Parameter.
example:
Suppose I want to create 'Users' collection in 'mongoDbDatabase' database
by following request.
{
"collectionName" : "Users",
"dbName" :"mongoDbDatabase"
}
Now is there any way to create dynamic collection in mongoDB using req.param('collectionName) variable ?
To use all the tools that Sails provides, you have to add code (before you start your app) for each Model / Collection you are planning to interact with. As a result, creating a collection dynamically will mean you can't take advantage of the whole data - framework sails provides.
Are you sure you need a dynamic collection? Why not a defined collection differentiated by attributes?
If you really need to do this from Sails, it looks like you can get access to the underlying raw mongo database:
var db = AnyModel.getDatastore().manager; // the database will be as defined in config/models.js or config/connections.js
var collectionName = 'Widgets';
db.createCollection(collectionName);
// note, even if this works, something like 'Widgets.find' will not.
I'm using MongoDB and Mongoose in a REST API. Some deployments require a replica set, thus separate read/write databases, so as a result I have separate read/write connections in the API. However, more simple deployments don't need a replica-set, and in those cases I point my read/write connections to the same MongoDB instance and database.
My general approach is to create all models for both connections at API start up. Even when read/write conns are connecting to same database, I am able to create the same models on both connections without error.
let ReadUser = dbRead.model('User', userSchema);
let WriteUser = dbWrite.model('User', userSchema);
// no error even when dbRead and dbWrite point to same DB
Trouble comes when until I start using Mongoose Discriminators.
let ReadSpecialUser = ReadUser.discriminator('SpecialUser', specialUserSchema);
let WriteSpecialUser = WriteUser.discriminator('SpecialUser', specialUserSchema);
// Results in this Error when read and write point to same DB:
// Error: Discriminator with name "SpecialUser" already exists
I'm look for an elegant way to deal with this. Is there a way to query the db for discriminators that are already in use?
According to the Mongoose API docs the way to do this is to use Model.discriminators. So in the case above it would be
ReadUser.discriminators
or
WriteUser.discriminators
However this doesn't return anything for me. What does work is using
Object.keys(Model.discriminators)
As expected this gets you an array of strings of the discriminator names you've set previously.
If you want to use the existing discriminator model and know its name what you can do is use Model.discriminators.discriminatorName. In your example it would be:
let ReadSpecialUserDocument = new ReadUser.discriminators.SpecialUser({
key: value,
key: value,
});
ReadSpecialUserDocument.save()
This can be useful when you need to reuse the discriminator at different times, and its name is tied to your data in some way.
I'm using Sequelize.js for ORM and have a few associations (which actually doesn't matter now). My models get get and set methods from those associations. Like this (from docs):
var User = sequelize.define('User', {/* ... */})
var Project = sequelize.define('Project', {/* ... */})
// One-way associations
Project.hasOne(User)
/*
...
Furthermore, Project.prototype will gain the methods getUser and setUser
according to the first parameter passed to define.
*/
So now, I have Project.getUser(), which returns a Promise. But if I call this twice on the very same object, I get SQL query executed twice.
My question is - am I missing something out, or this is an expected behavior? I actually don't want to make additional queries each time I call the same method on this object.
If this is expected - should I use custom getters with member variables which I manually populate and return if present? Or there is something more clever? :)
Update
As from DeBuGGeR's answer - I understand I can use includes when making a query in order to eager load everything, but I simply don't need it, and I can't do it all the time. It's waste of resources and a big overhead if I load my entire DB at the beginning, just to understand (by some criteria) that I won't need it. I want to make additional queries depending on situation. But I also can't afford to destroy all models (DAO objects) that I have and create new ones, with all the info inside them. I should be able to update parts of them, which are missing (from relations).
If you use getUser() it will make the query call, it dosent give you access to the user. You can manually save it to project.user or project.users depending on the association.
But you can try Eager Loading
Project.find({
include: [
{ model: User, as: 'user' } // here you HAVE to specify the same alias as you did in your association
]
}).success(function(project){
project.user // contains the user
});
Also e.g of getUser(). Dont expect it to automatically cache user and dont override this cleverly as it will create side effects. getUser is expected to get from database and it should!
Project.getUser().then(function(user){
// user is available and is a sequelize object
project.user = user; // save project.user and use it till u want to
})
The first part of things is clear - every call to get[Association] (for example Project.getUser()) WILL result in database query.
Sequelize does not maintain any kind of state nor cache for the results. You can get user in the Promisified result of the call, but if you want it again - you will have to make another query.
What #DeBuGGeR said - about using accessors is also not true - accessors are present only immediately after a query, and are not preserved.
As sometimes this is not ok, you have to implement some kind of caching system by yourself. Here comes the tricky part:
IF you want to use the same get method Project.getUser(), you won't be able to do it, as Sequelize overrides your instanceMethods. For example, if you have the association mentioned above, this won't work:
instanceMethods: {
getUser: function() {
// check if you have it, otherwise make a query
}
}
There are few possible ways to fix it - either change Sequelize core a little (to first check if the method exists), or use some kind of wrapper to those functions.
More details about this can be found here: https://github.com/sequelize/sequelize/issues/3707
Thanks to mickhansen for the cooperation on how to understand what to do :)