I'm developing my first Node.js App with Socket.IO and everything is fine but now the app is slowly getting bigger and I'd like to divide the app-code into different files for better maintenance.
For example I'm defining all my mongoose schemas and the routings in the main file. Underneath are all the functions for the socket.IO connection. But now I want to have an extra file for the schemas, an extra file for routing and one for the functions.
Of course, I'm aware of the possibility to write my own module or load a file with require. That just does not make sense for me, because I can't work with the vars like app, io or db without making them global. And if I pass them to a function in my module, I can't change them. What am I missing? I'd like to see an example how this is done in practice without using global vars..
It sounds like you have a highly coupled application; it's difficult for you to split out your code into modules because pieces of the application that should not depend on each other do. Looking into the principles of OO design may help out here.
For example, if you were to split your dataabse logic out of the main application, you should be able to do so, as the database logic should not depend on app or io--it should be able to work on its own, and you require it into other pieces of your application to use it.
Here's a fairly basic example--it's more pseudocode than actual code, as the point is to demonstrate modularity by example, not to write a working application. It's also only one of many, many ways you may decide to structure your application.
// =============================
// db.js
var mongoose = require('mongoose');
mongoose.connect(/* ... */);
module.exports = {
User: require('./models/user');
OtherModel: require('./models/other_model');
};
// =============================
// models/user.js (similar for models/other_model.js)
var mongoose = require('mongoose');
var User = new mongoose.Schema({ /* ... */ });
module.exports = mongoose.model('User', User);
// =============================
// routes.js
var db = require('./db');
var User = db.User;
var OtherModel = db.OtherModel;
// This module exports a function, which we call call with
// our Express application and Socket.IO server as arguments
// so that we can access them if we need them.
module.exports = function(app, io) {
app.get('/', function(req, res) {
// home page logic ...
});
app.post('/users/:id', function(req, res) {
User.create(/* ... */);
});
};
// =============================
// realtime.js
var db = require('./db');
var OtherModel = db.OtherModel;
module.exports = function(io) {
io.sockets.on('connection', function(socket) {
socket.on('someEvent', function() {
OtherModel.find(/* ... */);
});
});
};
// =============================
// application.js
var express = require('express');
var sio = require('socket.io');
var routes = require('./routes');
var realtime = require('./realtime');
var app = express();
var server = http.createServer(app);
var io = sio.listen(server);
// all your app.use() and app.configure() here...
// Load in the routes by calling the function we
// exported in routes.js
routes(app, io);
// Similarly with our realtime module.
realtime(io);
server.listen(8080);
This was all written off the top of my head with minimal checking of the documentation for various APIs, but I hope it plants the seeds of how you might go about extracting modules from your application.
Related
I am new to NodeJS. So I am writing NodeJS application with socket.io.
I understand how to separate controllers. In my app I created:
controllers/userCtrl.js
controller/marketCtrl.js etc.
And in userCtrl.js controller I did like this:
exports.create = function(req, res) {
// Create user
}
// Other actions
In application I use it:
// ...
var userCtrl = require('./controllers/userCtrl');
app.post('/user', userCtrl.create);
// ...
With models the same. But I have a lot of socket.io related code in app.js and don't understand how remove it (like controllers) from app.js:
var frontend = io.of('/frontend');
frontend.on('connection', function (client) {
logger.info('Someone connected to frontend socket');
client.on('join', function (message) {
logger.info('In join event');
var token = message.token;
if (!token) {
logger.debug('No usertoken provided. Sending login required');
client.emit('join', {error: 103, message: 'Login required', data: null});
return;
}
//... etc..
My question is: How to split socket.io related code into files? What is best practice for it? Thank you!
Different files is still the way. Use exports or module.exports and then just require in your app.js.
Perhaps make a setup() function that takes in an app/http instance, or whatever else you need in your socket.io stuff, and then call that function at the right time in app.js.
-- socketSetup.js --
'use strict';
var io = require('socket.io');
function setup( app, logger, whatever ){
//do stuff here
}
module.exports = setup;
-- app.js --
'use strict';
var express = require('express');
var socketSetup = require('./socketSetup.js');
var app = express();
//other express things
//setup the socket stuff
socketSetup( app, logger );
The result is a shorter and more readable app.js file, and your socket setup is contained as well. Repeat as things grow in your socketSetup.js file as well.
I have a file where I call all the require modules called 'app.js'
var express = require('express');
...
var app = express();
...
var routes = require('ruotes/index.js)
use('/', routes);
module.exports = app;
I call the index.js where I have all the routes
routes/index.js
var express = require('express');
var app = express();
/* import controllers */
var indexController = require('../controllers/index');
app.route('/')
.get(indexController.index);
module.exports = app;
The problem is I have two variables called app and I call twice express(). Does it exist a more elegant way to handle this situation? Maybe I could call the variable in router file in another way?
You are currently using a technique that is known as submounting.
If you want to avoid creating different instances of Express, you will have to pass your unique instance of Express as a parameter to your router, meaning that your index.js needs to export a function that accepts your app as a parameter.
Something like :
//=== routes/index.js
/* import controllers */
var indexController = require('../controllers/index');
module.exports = function(app){
app.route('/').get(indexController.index);
};
Doing so will allow you to keep the same instance of Express in your entire Web App.
Since you exported a function in index.js, you need to call it in app.js.
var express = require('express');
...
var app = express();
...
var routes = require('routes/index.js)(app)
module.exports = app;
You can use the same technique for routes, modules, models etc...
This technique will allow you to avoid re-requiring modules, and instantiating them, but you will need to write extra-bits to make sure your variables are passed from one side of your app to the other. Because of this, some prefer using the submounting technique, and requiring modules only where they are needed.
This is not a concrete app/code question, it's just about common app architecture.
I'm trying to understand proper way to organize my mongoose application. As I'm new to mongoose, that's how I do it now:
core/settings.js
var mongoose = require('mongoose');
exports.mongoose = mongoose;
mongoose.connect('mongodb://localhost/blog');
exports.db = mongoose.connection;
core/models.js
settings = require("./settings");
// post schema
var postSchema = settings.mongoose.Schema({
header: String,
author: String,
text: String
})
//compiling our schema into a Model
exports.post = settings.mongoose.model('post', postSchema)
core/db-layer.js
settings = require("./core/settings");
models = require("./core/models");
exports.function = createAndWriteNewPost(function(callback) {
settings.db.on('error', console.error.bind(console, 'connection error:'));
settings.db.once('open', function callback() {
new models.post({
header: 'header',
author: "author",
text: "Hello"
}).save(function(err, post) {
callback('ok');
});
});
});
routes/post.js
db = reqiure("../core/db.js")
exports.get = function(req, res) {
db.createAndWriteNewPost(function(status){
res.render('add_material', {
//blah blah blah
});
});
};
app.js
var post = require ('routes/post.js')
...
app.get('/post', post.get);
So, this code was extremely simplified (even not tested) just to show my current architecture thoughts. It's not a concrete app, just something like creating an abstract blog post. So thats how it works:
app.js --> routes/post.js <--> core/db-layer.js
|
v
core/models.js <--> core/settings.js
It seems a bit over superfluous for me. Could you suggest more optimal app structure? Thanks.
When I first got into Node.js, Express and Mongoose I struggled with scaling my code.
The intention of my answer is to help someone who's working on more than just a simple blog, but to help with an even larger scalable project.
I am always connected to the database, I do not open and close connections when needed
I use index.js as the root file of a folder, just like we'd do in other languages
models are kept in their own documents, and require()d into the models/index.js file.
routes are similar to models, each route level has a folder, which has an index.js file in turn. So it's easy to arrange something like http://example.com/api/documents/:id. It also makes more sense when one goes through the file structure.
Here's the structure of what I use:
-- app.js
-- models/
---- index.js
---- blog.js
-- mongoose/
---- index.js
-- routes/
---- index.js
---- blog/index.js
-- public/
-- views/
---- index.{your layout engine} => I use Jade.lang
-- methods/
---- index.js => use if you'd rather write all your functions here
---- blog.js => can store more complex logic here
app.js
var db = require('./mongoose'),
express = require('express');
// note that I'm leaving out the other things like 'http' or 'path'
var app = express();
// get the routes
require('./routes')(app);
// I just require routes, without naming it as a var, & that I pass (app)
mongoose/index.js
// Mongoose connect is called once by the app.js & connection established
// No need to include it elsewhere
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/blog');
// I have just connected, and I'm not exporting anything from here
models/index.js
// Logic here is to keep a good reference of what's used
// models
Blog = require('./blog');
// User = require('./user');
// exports
exports.blogModel = Blog.blogModel;
// exports.userModel = User.userModel;
models/blog.js
So for every model that you work on you create a model.js document, and add it in the models/index.js above. As an example I've added a User model but commented it out.
// set up mongoose
var mongoose = require('mongoose');
var Schema = mongoose.Schema,
ObjectId = Schema.ObjectId;
var BlogSchema = Schema({
header: {type: String },
author: {type: String },
text: {type: String },
_id: { type: ObjectId } // not necessary, showing use of ObjectId
});
Blog = mongoose.model('Blog', BlogSchema);
// the above is necessary as you might have embedded schemas which you don't export
exports.blogModel = Blog;
routes/index.js
module.exports = function(app) {
app.get('/', function(req, res) {
// do stuff
});
require('./blog')(app);
// other routes entered here as require(route)(app);
// we basically pass 'app' around to each route
}
routes/blog/index.js
module.exports = function(app) {
app.get('/blog', function(req, res) {
// do stuff
});
require('./nested')(app);
// this is for things like http://example.com/blog/nested
// you would follow the same logic as in 'routes/index.js' at a nested level
}
suggested use
models: for creating the logic that deals with the documents, i.e. creating, updating, deleting, and searching.
routes: minimal coding, only where I need to parse http data, create instances of models, and then I send queries to the relevant model.
methods: for the more complex logic that doesn't directly involve models. As an example, I have an algorithms/ folder where I store all the algorithms that I use in my app.
Hope this provides more clarity. This structure is working wonders for me as I find it easy to follow.
That's pretty much how I go about it, with a few differences:
I don't think you can have the open listener inside your function in the db-layer. What I generally do when using a persistent connection like yours is start the application itself in the db open handler. If you don't want to use persistent connections, use createConnection in the db layer function, and make sure you close it before calling the callback. I am not sure if I am making myself clear. Let me know if you want a code example.
This is more of a general node.js tip, but I keep my database connection string and other configuration in a json file and require it wherever it is needed. You probably won't need another settings.js file after that.
You can also use schema functions (http://mongoosejs.com/docs/api.html#schema_Schema-method) to code some app functionality into your models itself.
I'm developing my first Node.js App with Socket.IO and everything is fine but now the app is slowly getting bigger and I'd like to divide the app-code into different files for better maintenance.
For example I'm defining all my mongoose schemas and the routings in the main file. Underneath are all the functions for the socket.IO connection. But now I want to have an extra file for the schemas, an extra file for routing and one for the functions.
Of course, I'm aware of the possibility to write my own module or load a file with require. That just does not make sense for me, because I can't work with the vars like app, io or db without making them global. And if I pass them to a function in my module, I can't change them. What am I missing? I'd like to see an example how this is done in practice without using global vars..
It sounds like you have a highly coupled application; it's difficult for you to split out your code into modules because pieces of the application that should not depend on each other do. Looking into the principles of OO design may help out here.
For example, if you were to split your dataabse logic out of the main application, you should be able to do so, as the database logic should not depend on app or io--it should be able to work on its own, and you require it into other pieces of your application to use it.
Here's a fairly basic example--it's more pseudocode than actual code, as the point is to demonstrate modularity by example, not to write a working application. It's also only one of many, many ways you may decide to structure your application.
// =============================
// db.js
var mongoose = require('mongoose');
mongoose.connect(/* ... */);
module.exports = {
User: require('./models/user');
OtherModel: require('./models/other_model');
};
// =============================
// models/user.js (similar for models/other_model.js)
var mongoose = require('mongoose');
var User = new mongoose.Schema({ /* ... */ });
module.exports = mongoose.model('User', User);
// =============================
// routes.js
var db = require('./db');
var User = db.User;
var OtherModel = db.OtherModel;
// This module exports a function, which we call call with
// our Express application and Socket.IO server as arguments
// so that we can access them if we need them.
module.exports = function(app, io) {
app.get('/', function(req, res) {
// home page logic ...
});
app.post('/users/:id', function(req, res) {
User.create(/* ... */);
});
};
// =============================
// realtime.js
var db = require('./db');
var OtherModel = db.OtherModel;
module.exports = function(io) {
io.sockets.on('connection', function(socket) {
socket.on('someEvent', function() {
OtherModel.find(/* ... */);
});
});
};
// =============================
// application.js
var express = require('express');
var sio = require('socket.io');
var routes = require('./routes');
var realtime = require('./realtime');
var app = express();
var server = http.createServer(app);
var io = sio.listen(server);
// all your app.use() and app.configure() here...
// Load in the routes by calling the function we
// exported in routes.js
routes(app, io);
// Similarly with our realtime module.
realtime(io);
server.listen(8080);
This was all written off the top of my head with minimal checking of the documentation for various APIs, but I hope it plants the seeds of how you might go about extracting modules from your application.
I'm writing a mudule express-based NodeJS app. Here are important parts of my app.js:
var express = require('express')
, routes = require('./routes')
, passport = require('passport')
, LocalStrategy = require('passport-local').Strategy;
var app = module.exports = express.createServer();
var ab = 'this is a test!';
// Configuration
app.configure(function(){
...
// Routes
app.get('/', routes.index);
app.listen(3000);
Routes.index - is a controller, that executes when '/' is requested. Here is the code:
exports.index = function(req, res){
passport.serializeUser(function(user, done) {
done(null, user.id);
});
...
res.render('index.ejs', {
title: ab
})
};
Techically, index.js - is separate file, located in '/routes' folder. So, when I launch my app, it crashed cause can't find passport var, declared in main app. Also, ab also can't be found, however it was declared. If I re-declate vars in index.js, JS will create new objects for me. How can I use my vars in every module of my app? I looked through a few topics on SO, however couldn't understand – is it a common problem or just a structure of my app is wrong? Thanks!
As you have discovered, variables declared in one module don't magically appear in other modules. When it comes to modules such as passport, usually people just require them again where they're needed, so:
app.js:
var passport = require('passport'),
index = require('./routes/index');
index.js:
var passport = require('passport');
Sometimes you want to pass parameters though -- I often do it for unit testing, if I can pass in dependencies I can also mock those dependencies. Or you have an app-wide configuration you want to pass around. I usually do this:
app.js:
var ab = "foo",
index = require('/routes/index')(ab);
index.js:
module.exports = function(ab) {
// Here I have access to ab and can to what I need to do
return {
index: function(req, res) { res.send(ab) }
}
}
Other people prefer a "singleton" pattern, where each time you require a module, you check if it's already been initialized and if so, pass that instance.