Using Express.js 3 with database modules, where to init database client? - node.js

Knowing that Express.js pretty much leaves it to developer on deciding app structure, and after reading quite a few suggestions on SO (see link1 and link2 for example) as well as checking the example in official repo, I am still not sure if what I am doing is the best way forward.
Say I am using Redis extensively in my app, and that I have multiple "models" that require redis client to run query, would it be better to init redis client in the main app.js, like this:
var db = redis.createClient();
var models = require('./models')(db);
var routes = require('./controllers')(models);
or would it be better to just init redis in each model, then let each controller require models of interests?
The latter approach is what I am using, which looks less DRY. But is passing models instance around the best way? Note that I am loading multiple models/controllers here - I am not sure how to modify my setup to pass the redis client correctly to each models.
//currently in models/index.js
exports.home = require('./home.js');
exports.users = require('./user.js');
TL;DR, my questions are:
where best to init redis client in a MVC pattern app?
how to pass this redis client instance to multiple models with require('./models')(db)
Update:
I tried a different approach for index.js, use module.exports to return an object of models/controllers instead:
module.exports = function(models){
var routes = {};
routes.online = require('./home.js')(models);
routes.users = require('./user.js')(models);
return routes;
};
Seems like a better idea now?

Perhaps it's useful if I share how I recently implemented a project using Patio, a SQL ORM. A bit more background: the MVC-framework I was using was Locomotive, but that's absolutely not a requirement (Locomotive doesn't have an ORM and it leaves implementing how you handle models and databases to the developer, similar to Express).
Locomotive has a construct called 'initializers', which are just JS files which are loaded during app startup; what they do is up to the developer. In my project, one initializer configured the database.
The initializer established the actual database connection, also took care of loading all JS files in the model directory. In pseudocode:
registry = require('model_registry'); // see below
db = createDatabaseConnection();
files = fs.readDirSync(MODEL_DIRECTORY);
for each file in files:
if filename doesn't end with '.js':
continue
mod = require(path.join(MODEL_DIRECTORY, filename));
var model = mod(db);
registry.registerModel(model);
Models look like this:
// models/mymodel.js
module.exports = function(db ) {
var model = function(...) { /* model class */ };
model.modelName = 'MyModel'; // used by registry, see below
return model;
};
The model registry is a very simple module to hold all models:
module.exports = {
registerModel : function(model) {
if (! model.hasOwnProperty('modelName'))
throw Error('[model registry] models require a modelName property');
this[model.modelName] = model;
}
};
Because the model registry stores the model classes in this (which is module.exports), they can then be imported from other files where you need to access the model:
// mycontroller.js
var MyModel = require('model_registry').MyModel;
var instance = new MyModel(...);
Disclaimer: this worked for me, YMMV. Also, the code samples above don't take into account any asynchronous requirements or error handling, so the actual implementation in my case was a bit more elaborate.

Related

Sequelize: connect to database on run time based on the request

I am working on a node.js app where I need to connect to more than one databases. One of the database is central database which contains information common to all. And then there are country level databases where data is stored according to the countries.
I am using sequelize ORM in the app.
Database is postgresql.
Framework is express.
The problem is I want to decide on runtime based on the request which database to use and models should automatically connect to the appropriate database. I have seen this question but didn't found it helpful.
I have also checked in another forums but didn't find anything.
You need to create objects corresponding to each of your database, and at each this object you need to instantiate the Sequelize. Further, for each sequelize instance you need to import the models (assumming that all those databases have exactly the same tables and model representations).
import Sequelize from 'sequelize';
let connectionsArray = [
'postgres://user:pass#example.com:5432/country1',
'postgres://user:pass#example.com:5432/country2',
'postgres://user:pass#example.com:5432/country3',
];
let country1DB, country2DB, country3DB;
country1DB = country2DB = country3DB = {};
country1DB.Sequelize = country2DB.Sequelize = country3DB.Sequelize = Sequelize;
country1DB.sequelize = new Sequelize(connectionsArray[0]);
country2DB.sequelize = new Sequelize(connectionsArray[1]);
country3DB.sequelize = new Sequelize(connectionsArray[2]);
// here you need to access the models path, maybe with fs module
// iterate over every model and import it into every country sequelize instance
// let's assume that models' paths are in simple array
models.forEach(modelFile => {
let model1DB = country1DB.sequelize.import(modelFile);
let model2DB = country2DB.sequelize.import(modelFile);
let model3DB = country3DB.sequelize.import(modelFile);
country1DB[model1DB.name] = model1DB;
country2DB[model2DB.name] = model2DB;
country3DB[model3DB.name] = model3DB;
});
// now every country?DB object has it's own sequelize instance and all model definitions inside
export {
country1DB,
country2DB,
country3DB
};
This is just some example code, it would need refactor to be useful (introduce some loops etc.). It should just show you the idea of how to use multiple databases within single application. If you would like to use e.g. country1 database somewhere, you would simply do
import { country1DB } from './databases';
country1DB.User.findAll({...});
Above code would perform SELECT * FROM users in previously specified country1 database. Example express route could look like below:
import * as databases from './databases';
app.get('/:dbIndex/users', (req, res) => {
databases['country' + req.params.dbIndex + 'DB'].User.find().then(user => {
res.json(user.toJSON());
});
});
Or, even better, you could write some middleware function that would be run before every request and which would be responsible for choosing proper database for further operations.

NodeJS (Express) - project structure and mongo connection

I started a new project from scratch with ExpressJS.
Everything works fine but now I begin to have a dozen of 'app.get(....)' function and I need to give the project a structure.
What I have in mind is quite simple, it should have a folder named 'routes' containing a file such as 'module1.js', with all of the app.get related to that module. (like I've seen in many examples)
The issue is how to tell Express to route 'http://url/module1/' to that route file and how to pass it a param variable, containing for instance the mongodb connection.
what I tried is :
var params = {
db: myMongoConnection
};
var mod1 = require('routes/module1');
app.use('/module1', mod1);
but now I still miss the 'params'.
If I try to pass it as an argument to the require method i get an error saying it needs middleware.
Another issue is related to the fact that the myMongoConnection is valid in the connection callback, so I think i need to require and use the route.js inside the MongoClient connect callback.
Any idea?
thanks a lot
For custom modules, create a folder, call it modules
In its index.js, expose the modules that you need.
Something like,
var mods = [
'mod1',
'mod2',
];
function init() {
var expose = {};
var params = {
db: myMongoConnection
};
mods.forEach(mods, function (mod) {
expose[mod] = require('./' + mod)(params);
});
return expose;
}
// export init
module.exports = init;
In mod1.js, wrap the params
module.exports = function(params) {
// all your functions here will have access to params.
}
Then in, server/app.js, require this and set it in the app.
app.set('mods', require('path-to/modules'));
Now, you can access all your modules, using app.get('mods').moduleName.methodname

Placing Backbone Models in external files using node.js

I'm trying to find my way through node.js and backbone.js. My intention is to share at least all model-states with the server and browser. Right now I'm not quite sure if I really need to share views as well. I also want to handle all routes by express and nodejs and not backbone. To keep my code a wee bit more structured I was looking forward to keep each model in a separate *.js file. By doing so I'm running into following error message:
TypeError: Object #<Object> has no method 'extend'
I thought it might be a problem with underscore missing in the separate file, so here is my basemodel:
models/BaseModel.js
var Backbone = require('../node_modules/backbone'),
_ = require('../node_modules/underscore'),
var BaseModel = Backbone.Model.extend({
modelName: 'basemodel'
});
exports.BaseModel = BaseModel;
app.js
var BaseModel = require('./models/BaseModel');
var MyModel = BaseModel.extend({
// ... attributes, functions etc.
});
Does anyone have a hint what I'm doing wrong there?
Then try this code in app.js:
BaseModel = require('./models/BaseModel').BaseModel;
Or in models/BaseModel.js not of exports.BaseModel = BaseModel; use it - module.exports = BaseModel
I actually figured it out. The flaw was in referencing to the BaseModel, so
this
var MyModel = BaseModel.extend({
//
});
had to turn into:
var MyModel = BaseModel.BaseModel.extend({
//
};
because of
exports.BaseModel = BaseModel;
in BaseModel.js

Mongoose model testing require models

I have a problem testing my mongoose models
I have a structure like
app
models
Address
User
Organization
test
Both models User and Organization need to know the model Address. My models are structured like:
module.exports = function (mongoose, config) {
var organizationSchema = new mongoose.Schema({
name : {
type : String
},
addresses : {
type : [mongoose.model('Address')]
}
});
var Organization = mongoose.model('Organization', organizationSchema);
return Organization;
};
In my normal app i require Address before requiring User and Organization and everything is fine. I now wrote tests for User and Organization. In order to have the Address model registered i call require('../models/Address.js') This works fine if i run one test. But if i run all tests in a batch i get an error because i tried to register Address twice.
OverwriteModelError: Cannot overwrite Address model once compiled.
How do i solve this problem?
The problem is that you cant set mongoose model twice. The easiest way to solve your problem is to take advantage of node.js require function.
Node.js caches all calls to require to prevent your model from initializing twice. But you wrapping your models with functions. Unwrapping them will solve your problem:
var mongoose = require('mongoose');
var config = require('./config');
var organizationSchema = new mongoose.Schema({
name : {
type : String
},
addresses : {
type : [mongoose.model('Address')]
}
});
module.exports = mongoose.model('Organization', organizationSchema);
Alternative solution is to make sure that each model initialized only once. For example, you can initialize all you modules before running your tests:
Address = require('../models/Address.js');
User = require('../models/User.js');
Organization = require('../models/Organization.js');
// run your tests using Address, User and Organization
Or you can add try catch statement to your models to handle this special case:
module.exports = function (mongoose, config) {
var organizationSchema = new mongoose.Schema({
name : {
type : String
},
addresses : {
type : [mongoose.model('Address')]
}
});
try {
mongoose.model('Organization', organizationSchema);
} catch (error) {}
return mongoose.model('Organization');
};
Update: In our project we have /models/index.js file to handle everything. First, it calls mongoose.connect to establish connection. Then it requires every model in models directory and creates a dictionary of it. So, when we need some model (e.g. user) we requires it by calling require('/models').user.
Best solution (IMO):
try {
mongoose.model('config')
} catch (_) {
mongoose.model('config', schema)
}
This question already has an answer, but for a unique way to accomplish this check out https://github.com/fbeshears/register_models. This sample project uses a register_models.js that includes all models from an array of file names. It works really well and you end up with all your models pretty much wherever you need them. Keep in mind node.js's cache will cache objects in your project while it's running.
I use try/catch to tackle this problem and it works ok. But i think this not the best way of doing this.
try{
var blog = mongoose.model('blog', Article);
} catch (error) {}
I fixed this by patching r.js and making sure that all my modules used localRequire calls.
Checkout the thread here
https://github.com/jrburke/requirejs/issues/726

efficiency of mongodb/mongoskin access by multiple modules approach?

I'm developing an express app that provides a REST api, it uses mongodb through mongoskin. I wanted a layer that splits routing from db acess. I have seen an example that creates a database bridge by creating a module file, an example models/profiles.js:
var mongo = require('mongoskin'),
db = mongo.db('localhost:27017/profiler'),
profs = db.collection('profiles');
exports.examplefunction = function (info, cb) {
//code that acess the profs collection and do the query
}
later this module is required in the routing files.
My question is: If I use this aproach for creating one module for each collection, will it be efficient? Do I have an issue of connecting and disconnecting multiple(unnecessary) times from mongo by doing that?
I was thiking that maybe exporting the db variable from one module to the others that handle each collection would solve the suposed issue, but I'm not sure.
Use a single connection and then create your modules passing in the shared db instance. You want to avoid setting up separate db pools for each module. One of doing this is to construct the module as a class.
exports.build = function(db) {
return new MyClass(db);
}
var MyClass = function(db) {
this.db = db;
}
MyClass.doQuery = function() {
}

Resources