tl:dr;
I want to provide the database object to my models and controllers in ExpressJS. How should I do this?
Longer explanation:
My background is in working with Codeigniter and the active-record implementation it has. I have seen other example Node implementations (the mean-stack for instance), however, I was looking to get a site something simple, low-level and that I understood from the ground-up.
The problems is that I don't know express well enough to understand how I should be passing objects around. So I have rounting, db connections and pages rendering all well. However, I am unsure about DB connections being provided to the actual controllers or models.
So, the setup I presently have is:
App.js:
...
//Start the database
var Db = require('mysql-activerecord');
var db = new Db.Adapter({
server: 'localhost',
username: 'root',
password: '********',
database: 'test'
});
// }}}
...
require('./config/routes')(app);
And for config/routes.js
2 /*
3 Exporting variable:
4 #param app The express App
5 */
6
7
8 module.exports = function(app) {
9
10 //Home route
11 var index = require('../server/controllers/index');
12 app.get('/', index.render);
13
14 }
15
So...
The question is perhaps more stylistic than technical. My questions are:
Am I thinking about this the right way?
How would you recommend going about connecting this? Should I use 'require(../..)' in the controllers or is there a more elegant solution?
You can pass the db object to your routes.js (or any other required file you make) file as well:
require('./config/routes')({ app: app, db: db });
Related
I'm trying to create an API using nodeJS, express and azure-mobile-apps to do some data synchronisation between an Ionic3 mobile app (which use an SQLite local database) and a Microsoft SQL Database.
My API has to create a synchronisation connection for each mobile application. Each application will be linked to a distant database. For example, if user_01 wants to synchronise his data, he's going to be linked to his client_01 database. So each time it'll have to, the API will create a new process running on a different port.
here is an example : https://zupimages.net/up/19/36/szhu.png
The problem is that i'm not able to create more than one connection with azure-mobile-apps. The first one always works, but the second, third etc are still using the first connection that i have instantiated. I've looked into the app stack and everything seems fine.
Is that an issue with azure-mobile-app, or did I misunderstand something with express ?
Thanks for your responses !
var azureMobileApps = require('azure-mobile-apps');
var express = require('express');
module.exports = {
async createConnection(client) {
try {
let app = express();
mobileApp = azureMobileApps({
homePage: true,
swagger: true,
data: {
server: '****',
user: client.User,
password: client.Password,
port: '1443',
database: client.Database,
provider: 'mssql',
dynamicSchema: false,
options: {
encrypt: false
}
}
});
await mobileApp.tables.import('./tables');
await mobileApp.tables.initialize();
app.listen(global.portCounter);
app.use(mobileApp);
console.log(app._router.stack);
console.log('Listening on port ',global.portCounter);
global.portCounter++;
} catch (error) {
console.log(error);
}
}
}
It's working now. The thing is, it's impossible to do multiple connection with the azure-mobile-apps SDK for nodeJS.
I had to use worker-thread which seems to isolate the memory in a sub-proccess.
Hope it can help somebody one day
Node.js is my first backend language and I am at the point where I am asking myself "where do I put the database connection information?".
There is a lot of good information regarding this issue. Unfortunately for me all the examples are in PHP. I get the ideas but I am not confident enough to replicate it in Node.js.
In PHP you would put the information in a config file outside the web root, and include it when you need database data.
How would you do this in Node.js? using the Express.js framework.
So far I have this:
var express = require('express'), app = express();
var mysql = require('mysql');
app.get('/', function(req,res) {
var connection = mysql.createConnection({
host: 'localhost',
user: 'root',
password: 'password',
database: 'store'
});
var query = connection.query('SELECT * from customers where email = "deelo42#gmail.com"');
query.on('error', function(err) {
throw err;
});
query.on('fields', function(fields) {
console.log('this is fields');
});
query.on('result', function(row) {
var first = row.first_name;
var last = row.last_name;
res.render('index.jade', {
title: "My first name is " + first,
category: "My last name is " + last
});
});
});
app.listen(80, function() {
console.log('we are logged in');
});
As you can see I have a basic express application with 1 GET route. This route sets off the function to go to the database and pull out information based on an email address.
At the top of the GET route is the database connection information. Where do I put that? How do I call it? How do I keep it out of web root, and include it like PHP ? Can you please show me in a working example. Thanks!
I use the Express Middleware concept for same and that gives me nice flexibility to manage files.
I am writing a detailed answer, which includes how i am use the config params in app.js to connect to DB.
So my app structure looks something this:
How i connect to DB? (I am using MongoDB, mongoose is ORM, npm install mongoose)
var config = require('./config/config');
var mongoose = require("mongoose");
var connect = function(){
var options = {
server: {
socketOptions:{
keepAlive : 1
}
}
};
mongoose.connect(config.db,options);
};
connect();
under the config folder i also have 'env' folder, which stores the environment related configurations in separate files such as development.js, test.js, production.js
Now as the name suggests, development.js stores the configuration params related to my development environment and same applies to the case of test and production. Now if you wish you can have some more configuration setting such as 'staging' etc.
project-name/config/config.js
var path = require("path");
var extend = require("util")._extend;
var development = require("./env/development");
var test = require("./env/test");
var production = require("./env/production");
var defaults = {
root: path.normalize(__dirname + '/..')
};
module.exports = {
development: extend(development,defaults),
test: extend(test,defaults),
production: extend(production,defaults)
}[process.env.NODE_ENV || "development"]
project-name/config/env/test.js
module.exports = {
db: 'mongodb://localhost/mongoExpress_test'
};
Now you can make it even more descriptive by breaking the URL's into, username, password, port, database, hostname.
For For more details have a look at my repo, where you can find this implementation, in fact now in all of my projects i use the same configuration.
If you are more interested then have a look at Mean.js and Mean.io, they have some better ways to manage all such things. If you are beginner i would recommend to keep it simple and get things going, once you are comfortable, you can perform magic on your own. Cheers
I recommend the 12-factor app style http://12factor.net which keeps all of this in env vars. You never should have this kind of information hard-coded or in the app source-code / repo, so you can reuse it in different environments or even share it publicly without breaking security.
However, since there are lots of environment vars, I tend to keep them together in a single env.js like the previous responder wrote - although it is not in the source code repo - and then source it with https://www.npmjs.org/package/dotenv
An alternative is to do it manually and keep it in, e.g. ./env/dev.json and just require() the file.
Any of these works, the important point is to keep all configuration information separate from code.
I agree with the commenter, put it in a config file. There is no ultimate way, but nconf is also one of my favourites.
The important best practise is that you keep the config separate if you have a semi-public project, so your config file will not overwrite other developers.
config-sample.json (has to be renamed and is tracked with for example git)
config.json (not tracked / ignored by git)
I have an application where I need to limit the number of active connections, and the most logical way of doing this to me, would be to simply count the number of active sessions.
I have searched for "node express session count" but did not find anything useful.
Is it possible to get the number of open sessions in node express ?
The alternative would be to save the IP address of each connected client, but the problem with this method is that the ip address would need to be manually removed from the datastore.
I guess if I use redis as the data store, I could use expire to achieve something similar.
set ip.192.168.42.1 true
expire ip.192.168.42.1 60
ttl ip.192.168.42.1
etc etc
Or is there a better way ?
EDIT
I have tried the suggestion of using MemoryStore but I can't seem to figure out how to use it ??
var express = require("express");
var MemoryStore = express.session.MemoryStore;
MemoryStore.prototype.length = function(fn) {
fn(null, Object.keys(this.sessions).length);
};
MemoryStore.length(function(len) {
console.log('sessions:'+len);
});
Use the Store.length(fn) method. In the default Connect memory storage, it's written like this:
MemoryStore.prototype.length = function(fn) {
fn(null, Object.keys(this.sessions).length);
};
So you would call it like this:
store.length(function(len) {
// we have len number of sessions
});
I am well aware of how old this post is but I thought I would share my way, as this comes up as the top result as I was trying to figure it out.
This what I did and it seems to be working.
So if you are using MySQLStore & Express Session you can approach it like this,
Add activeUsers to your options(app/index.js):
var options = {
host: process.env.DB_HOST,
port: 3306,
user: process.env.DB_USER,
password: process.env.DB_PASS,
database: process.env.DB_NAME,
activeUsers: 0
};
On your login.js add:
req.sessionStore.options.userCount += 1
And on your logout:
req.sessionStore.options.userCount += 1
Now I am not sure if on session timeout if this will return back to 0 or not, but I would assume that it does. And then just create a app.local that refers to the userSession count
I'm building a REST API with the use of NodeJS (Mongoose & ExpressJS). I think I have a pretty good basic structure at the moment, but I'm wondering what the best practices are for this kind of project.
In this basic version, everything passes through the app.js file. Every HTTP method is then passed to the resource that has been requested. This allows me to dynamically add resources to the API and every request will be passed along accordingly. To illustrate:
// app.js
var express = require('express');
var mongoose = require('mongoose');
var app = express();
app.use(express.bodyParser());
mongoose.connect('mongodb://localhost/kittens');
var db = mongoose.connection;
var resources = [
'kitten'
];
var repositories = {};
for (var i = 0; i < resources.length; i++) {
var resource = resources[i];
repositories[resource] = require('./api/' + resource);
}
db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', function callback() {
console.log('Successfully connected to MongoDB.');
app.get('/:resource', function (req, res) {
res.type('application/json');
repositories[req.params.resource].findAll(res);
});
app.get('/:resource/:id', function (req, res) {
res.type('application/json');
repositories[req.params.resource].findOne(req, res);
});
app.listen(process.env.PORT || 4730);
});
-
// api/kitten.js
var mongoose = require('mongoose');
var kittenSchema = mongoose.Schema({
name: String
});
var Kitten = mongoose.model('Kitten', kittenSchema);
exports.findAll = function (res) {
Kitten.find(function (err, kittens) {
if (err) {
}
res.json(kittens);
});
};
exports.findOne = function (req, res) {
Kitten.findOne({ _id: req.params.id}, function (err, kitten) {
if (err) {
}
res.json(kitten);
});
};
Obviously, only a couple of methods have been implemented so far. What do you guys think of this approach? Anything I could improve on?
Also, a small side question: I have to require mongoose in every API resource file (like in api\kitten.js, is there a way to just globally require it in the app.js file or something?
Any input is greatly appreciated!
Well, you can separate out your routes, db models and templates in different files.
Have a directory structure something like this,
| your_app
| -- routes
| -- models
| -- templates
| -- static
| -- css
| -- js
| -- images
| -- config.js
| -- app.js
| -- url.js
For each Mongoose model have a separate file placed in your ./models
In templates directory place your jade files. (Assuming you are using jade as your template engine). Though it seems like you are only serving JSON, not HTML. Consider using Jade if you want to render HTML. (Here are few other template engines you can consider going with)
./static directory for static JS, CSS and XML files etc.
Things like db connections or 3rd party API keys and stuff can be put in config.js
In url.js have a procedure which take express app object as argument and extend upon app.get and app.post there in single place.
P.S. This is the approach I go with for a basic web app in express. I am in no way saying this the best way to follow, but it helps me maintain my code.
There is no right way, but I did create a seed application for my personal directory structure to help my roommate with this.
You can clone it: git clone https://github.com/hboylan/express-mongoose-api-seed.git
Or with npm: npm install express-mongoose-api-seed
As codemonger5 said there is no right way of organising directory structure.
However, you can use this boilerplate application for creating REST APIs using Express and mongoose using ES6. We use the same directory structure in our production API services.
git clone https://github.com/KunalKapadia/express-mongoose-es6-rest-api
cd express-mongoose-es6-rest-api
npm install
npm start
Knowing that Express.js pretty much leaves it to developer on deciding app structure, and after reading quite a few suggestions on SO (see link1 and link2 for example) as well as checking the example in official repo, I am still not sure if what I am doing is the best way forward.
Say I am using Redis extensively in my app, and that I have multiple "models" that require redis client to run query, would it be better to init redis client in the main app.js, like this:
var db = redis.createClient();
var models = require('./models')(db);
var routes = require('./controllers')(models);
or would it be better to just init redis in each model, then let each controller require models of interests?
The latter approach is what I am using, which looks less DRY. But is passing models instance around the best way? Note that I am loading multiple models/controllers here - I am not sure how to modify my setup to pass the redis client correctly to each models.
//currently in models/index.js
exports.home = require('./home.js');
exports.users = require('./user.js');
TL;DR, my questions are:
where best to init redis client in a MVC pattern app?
how to pass this redis client instance to multiple models with require('./models')(db)
Update:
I tried a different approach for index.js, use module.exports to return an object of models/controllers instead:
module.exports = function(models){
var routes = {};
routes.online = require('./home.js')(models);
routes.users = require('./user.js')(models);
return routes;
};
Seems like a better idea now?
Perhaps it's useful if I share how I recently implemented a project using Patio, a SQL ORM. A bit more background: the MVC-framework I was using was Locomotive, but that's absolutely not a requirement (Locomotive doesn't have an ORM and it leaves implementing how you handle models and databases to the developer, similar to Express).
Locomotive has a construct called 'initializers', which are just JS files which are loaded during app startup; what they do is up to the developer. In my project, one initializer configured the database.
The initializer established the actual database connection, also took care of loading all JS files in the model directory. In pseudocode:
registry = require('model_registry'); // see below
db = createDatabaseConnection();
files = fs.readDirSync(MODEL_DIRECTORY);
for each file in files:
if filename doesn't end with '.js':
continue
mod = require(path.join(MODEL_DIRECTORY, filename));
var model = mod(db);
registry.registerModel(model);
Models look like this:
// models/mymodel.js
module.exports = function(db ) {
var model = function(...) { /* model class */ };
model.modelName = 'MyModel'; // used by registry, see below
return model;
};
The model registry is a very simple module to hold all models:
module.exports = {
registerModel : function(model) {
if (! model.hasOwnProperty('modelName'))
throw Error('[model registry] models require a modelName property');
this[model.modelName] = model;
}
};
Because the model registry stores the model classes in this (which is module.exports), they can then be imported from other files where you need to access the model:
// mycontroller.js
var MyModel = require('model_registry').MyModel;
var instance = new MyModel(...);
Disclaimer: this worked for me, YMMV. Also, the code samples above don't take into account any asynchronous requirements or error handling, so the actual implementation in my case was a bit more elaborate.