I'm developing my first Node.js App with Socket.IO and everything is fine but now the app is slowly getting bigger and I'd like to divide the app-code into different files for better maintenance.
For example I'm defining all my mongoose schemas and the routings in the main file. Underneath are all the functions for the socket.IO connection. But now I want to have an extra file for the schemas, an extra file for routing and one for the functions.
Of course, I'm aware of the possibility to write my own module or load a file with require. That just does not make sense for me, because I can't work with the vars like app, io or db without making them global. And if I pass them to a function in my module, I can't change them. What am I missing? I'd like to see an example how this is done in practice without using global vars..
It sounds like you have a highly coupled application; it's difficult for you to split out your code into modules because pieces of the application that should not depend on each other do. Looking into the principles of OO design may help out here.
For example, if you were to split your dataabse logic out of the main application, you should be able to do so, as the database logic should not depend on app or io--it should be able to work on its own, and you require it into other pieces of your application to use it.
Here's a fairly basic example--it's more pseudocode than actual code, as the point is to demonstrate modularity by example, not to write a working application. It's also only one of many, many ways you may decide to structure your application.
// =============================
// db.js
var mongoose = require('mongoose');
mongoose.connect(/* ... */);
module.exports = {
User: require('./models/user');
OtherModel: require('./models/other_model');
};
// =============================
// models/user.js (similar for models/other_model.js)
var mongoose = require('mongoose');
var User = new mongoose.Schema({ /* ... */ });
module.exports = mongoose.model('User', User);
// =============================
// routes.js
var db = require('./db');
var User = db.User;
var OtherModel = db.OtherModel;
// This module exports a function, which we call call with
// our Express application and Socket.IO server as arguments
// so that we can access them if we need them.
module.exports = function(app, io) {
app.get('/', function(req, res) {
// home page logic ...
});
app.post('/users/:id', function(req, res) {
User.create(/* ... */);
});
};
// =============================
// realtime.js
var db = require('./db');
var OtherModel = db.OtherModel;
module.exports = function(io) {
io.sockets.on('connection', function(socket) {
socket.on('someEvent', function() {
OtherModel.find(/* ... */);
});
});
};
// =============================
// application.js
var express = require('express');
var sio = require('socket.io');
var routes = require('./routes');
var realtime = require('./realtime');
var app = express();
var server = http.createServer(app);
var io = sio.listen(server);
// all your app.use() and app.configure() here...
// Load in the routes by calling the function we
// exported in routes.js
routes(app, io);
// Similarly with our realtime module.
realtime(io);
server.listen(8080);
This was all written off the top of my head with minimal checking of the documentation for various APIs, but I hope it plants the seeds of how you might go about extracting modules from your application.
Related
I am new to NodeJS. So I am writing NodeJS application with socket.io.
I understand how to separate controllers. In my app I created:
controllers/userCtrl.js
controller/marketCtrl.js etc.
And in userCtrl.js controller I did like this:
exports.create = function(req, res) {
// Create user
}
// Other actions
In application I use it:
// ...
var userCtrl = require('./controllers/userCtrl');
app.post('/user', userCtrl.create);
// ...
With models the same. But I have a lot of socket.io related code in app.js and don't understand how remove it (like controllers) from app.js:
var frontend = io.of('/frontend');
frontend.on('connection', function (client) {
logger.info('Someone connected to frontend socket');
client.on('join', function (message) {
logger.info('In join event');
var token = message.token;
if (!token) {
logger.debug('No usertoken provided. Sending login required');
client.emit('join', {error: 103, message: 'Login required', data: null});
return;
}
//... etc..
My question is: How to split socket.io related code into files? What is best practice for it? Thank you!
Different files is still the way. Use exports or module.exports and then just require in your app.js.
Perhaps make a setup() function that takes in an app/http instance, or whatever else you need in your socket.io stuff, and then call that function at the right time in app.js.
-- socketSetup.js --
'use strict';
var io = require('socket.io');
function setup( app, logger, whatever ){
//do stuff here
}
module.exports = setup;
-- app.js --
'use strict';
var express = require('express');
var socketSetup = require('./socketSetup.js');
var app = express();
//other express things
//setup the socket stuff
socketSetup( app, logger );
The result is a shorter and more readable app.js file, and your socket setup is contained as well. Repeat as things grow in your socketSetup.js file as well.
I'm developing my first Node.js App with Socket.IO and everything is fine but now the app is slowly getting bigger and I'd like to divide the app-code into different files for better maintenance.
For example I'm defining all my mongoose schemas and the routings in the main file. Underneath are all the functions for the socket.IO connection. But now I want to have an extra file for the schemas, an extra file for routing and one for the functions.
Of course, I'm aware of the possibility to write my own module or load a file with require. That just does not make sense for me, because I can't work with the vars like app, io or db without making them global. And if I pass them to a function in my module, I can't change them. What am I missing? I'd like to see an example how this is done in practice without using global vars..
It sounds like you have a highly coupled application; it's difficult for you to split out your code into modules because pieces of the application that should not depend on each other do. Looking into the principles of OO design may help out here.
For example, if you were to split your dataabse logic out of the main application, you should be able to do so, as the database logic should not depend on app or io--it should be able to work on its own, and you require it into other pieces of your application to use it.
Here's a fairly basic example--it's more pseudocode than actual code, as the point is to demonstrate modularity by example, not to write a working application. It's also only one of many, many ways you may decide to structure your application.
// =============================
// db.js
var mongoose = require('mongoose');
mongoose.connect(/* ... */);
module.exports = {
User: require('./models/user');
OtherModel: require('./models/other_model');
};
// =============================
// models/user.js (similar for models/other_model.js)
var mongoose = require('mongoose');
var User = new mongoose.Schema({ /* ... */ });
module.exports = mongoose.model('User', User);
// =============================
// routes.js
var db = require('./db');
var User = db.User;
var OtherModel = db.OtherModel;
// This module exports a function, which we call call with
// our Express application and Socket.IO server as arguments
// so that we can access them if we need them.
module.exports = function(app, io) {
app.get('/', function(req, res) {
// home page logic ...
});
app.post('/users/:id', function(req, res) {
User.create(/* ... */);
});
};
// =============================
// realtime.js
var db = require('./db');
var OtherModel = db.OtherModel;
module.exports = function(io) {
io.sockets.on('connection', function(socket) {
socket.on('someEvent', function() {
OtherModel.find(/* ... */);
});
});
};
// =============================
// application.js
var express = require('express');
var sio = require('socket.io');
var routes = require('./routes');
var realtime = require('./realtime');
var app = express();
var server = http.createServer(app);
var io = sio.listen(server);
// all your app.use() and app.configure() here...
// Load in the routes by calling the function we
// exported in routes.js
routes(app, io);
// Similarly with our realtime module.
realtime(io);
server.listen(8080);
This was all written off the top of my head with minimal checking of the documentation for various APIs, but I hope it plants the seeds of how you might go about extracting modules from your application.
Dumb/Newb question...
I am learning/working on an API in Node / Express4 and I would like to break my routes out into another module. I have it working with the following code, but it seems awkward to me to keep re-using the require('express') statement... Is there a way to move more of the code from the routes.js file into server.js and still keep my .get and .post statements in the routes module? Thanks in advance!
server.js:
'use strict';
var express = require('express');
var routes = require('./routes');
var app = express();
app.use('/api', routes);
app.listen(3000, function() {
console.log('Listening);
});
routes.js
var express = require('express'); // how do I get rid of this line?
var router = express.Router(); // can I move this to server.js?
var apiRoute = router.route('');
apiRoute.get(function (req, res) {
res.send('api GET request received');
});
module.exports = router;
Your on the right track. Its actually cool to reuse the var express = require('express'); statement each time you need it. Importing, ( requiring ), modules is a cornerstone of modular development and allows you to maintain a separation of concerns with in the files of your project.
As far as modularly adding routes is concerned: The issue is that routes.js is misleading.
In order to modularly separate out your routes you should use several modules named <yourResource>.js. Those modules would contain all of the routing code as well as any other configuration or necessary functions. Then you would attach them in app.js with:
var apiRoute = router.route('/api');
apiRoute.use('/<yourResource', yourResourceRouter);
For example, if you had a resource bikes:
In app.js or even a module api.js:
var apiRoute = router.route('/api')
, bikeRoutes = require('./bikes');
apiRoute.use('/bikes', bikeRoutes);
Then in bike.js:
var express = require('express');
var router = express.Router();
var bikeRoutes = router.route('/');
bikeRoutes.get(function (req, res) {
res.send('api GET request received');
});
module.exports = bikeRoutes;
From there its easy to see that you can build many different resources and continually nest them.
This is not a concrete app/code question, it's just about common app architecture.
I'm trying to understand proper way to organize my mongoose application. As I'm new to mongoose, that's how I do it now:
core/settings.js
var mongoose = require('mongoose');
exports.mongoose = mongoose;
mongoose.connect('mongodb://localhost/blog');
exports.db = mongoose.connection;
core/models.js
settings = require("./settings");
// post schema
var postSchema = settings.mongoose.Schema({
header: String,
author: String,
text: String
})
//compiling our schema into a Model
exports.post = settings.mongoose.model('post', postSchema)
core/db-layer.js
settings = require("./core/settings");
models = require("./core/models");
exports.function = createAndWriteNewPost(function(callback) {
settings.db.on('error', console.error.bind(console, 'connection error:'));
settings.db.once('open', function callback() {
new models.post({
header: 'header',
author: "author",
text: "Hello"
}).save(function(err, post) {
callback('ok');
});
});
});
routes/post.js
db = reqiure("../core/db.js")
exports.get = function(req, res) {
db.createAndWriteNewPost(function(status){
res.render('add_material', {
//blah blah blah
});
});
};
app.js
var post = require ('routes/post.js')
...
app.get('/post', post.get);
So, this code was extremely simplified (even not tested) just to show my current architecture thoughts. It's not a concrete app, just something like creating an abstract blog post. So thats how it works:
app.js --> routes/post.js <--> core/db-layer.js
|
v
core/models.js <--> core/settings.js
It seems a bit over superfluous for me. Could you suggest more optimal app structure? Thanks.
When I first got into Node.js, Express and Mongoose I struggled with scaling my code.
The intention of my answer is to help someone who's working on more than just a simple blog, but to help with an even larger scalable project.
I am always connected to the database, I do not open and close connections when needed
I use index.js as the root file of a folder, just like we'd do in other languages
models are kept in their own documents, and require()d into the models/index.js file.
routes are similar to models, each route level has a folder, which has an index.js file in turn. So it's easy to arrange something like http://example.com/api/documents/:id. It also makes more sense when one goes through the file structure.
Here's the structure of what I use:
-- app.js
-- models/
---- index.js
---- blog.js
-- mongoose/
---- index.js
-- routes/
---- index.js
---- blog/index.js
-- public/
-- views/
---- index.{your layout engine} => I use Jade.lang
-- methods/
---- index.js => use if you'd rather write all your functions here
---- blog.js => can store more complex logic here
app.js
var db = require('./mongoose'),
express = require('express');
// note that I'm leaving out the other things like 'http' or 'path'
var app = express();
// get the routes
require('./routes')(app);
// I just require routes, without naming it as a var, & that I pass (app)
mongoose/index.js
// Mongoose connect is called once by the app.js & connection established
// No need to include it elsewhere
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/blog');
// I have just connected, and I'm not exporting anything from here
models/index.js
// Logic here is to keep a good reference of what's used
// models
Blog = require('./blog');
// User = require('./user');
// exports
exports.blogModel = Blog.blogModel;
// exports.userModel = User.userModel;
models/blog.js
So for every model that you work on you create a model.js document, and add it in the models/index.js above. As an example I've added a User model but commented it out.
// set up mongoose
var mongoose = require('mongoose');
var Schema = mongoose.Schema,
ObjectId = Schema.ObjectId;
var BlogSchema = Schema({
header: {type: String },
author: {type: String },
text: {type: String },
_id: { type: ObjectId } // not necessary, showing use of ObjectId
});
Blog = mongoose.model('Blog', BlogSchema);
// the above is necessary as you might have embedded schemas which you don't export
exports.blogModel = Blog;
routes/index.js
module.exports = function(app) {
app.get('/', function(req, res) {
// do stuff
});
require('./blog')(app);
// other routes entered here as require(route)(app);
// we basically pass 'app' around to each route
}
routes/blog/index.js
module.exports = function(app) {
app.get('/blog', function(req, res) {
// do stuff
});
require('./nested')(app);
// this is for things like http://example.com/blog/nested
// you would follow the same logic as in 'routes/index.js' at a nested level
}
suggested use
models: for creating the logic that deals with the documents, i.e. creating, updating, deleting, and searching.
routes: minimal coding, only where I need to parse http data, create instances of models, and then I send queries to the relevant model.
methods: for the more complex logic that doesn't directly involve models. As an example, I have an algorithms/ folder where I store all the algorithms that I use in my app.
Hope this provides more clarity. This structure is working wonders for me as I find it easy to follow.
That's pretty much how I go about it, with a few differences:
I don't think you can have the open listener inside your function in the db-layer. What I generally do when using a persistent connection like yours is start the application itself in the db open handler. If you don't want to use persistent connections, use createConnection in the db layer function, and make sure you close it before calling the callback. I am not sure if I am making myself clear. Let me know if you want a code example.
This is more of a general node.js tip, but I keep my database connection string and other configuration in a json file and require it wherever it is needed. You probably won't need another settings.js file after that.
You can also use schema functions (http://mongoosejs.com/docs/api.html#schema_Schema-method) to code some app functionality into your models itself.
I'm writing a mudule express-based NodeJS app. Here are important parts of my app.js:
var express = require('express')
, routes = require('./routes')
, passport = require('passport')
, LocalStrategy = require('passport-local').Strategy;
var app = module.exports = express.createServer();
var ab = 'this is a test!';
// Configuration
app.configure(function(){
...
// Routes
app.get('/', routes.index);
app.listen(3000);
Routes.index - is a controller, that executes when '/' is requested. Here is the code:
exports.index = function(req, res){
passport.serializeUser(function(user, done) {
done(null, user.id);
});
...
res.render('index.ejs', {
title: ab
})
};
Techically, index.js - is separate file, located in '/routes' folder. So, when I launch my app, it crashed cause can't find passport var, declared in main app. Also, ab also can't be found, however it was declared. If I re-declate vars in index.js, JS will create new objects for me. How can I use my vars in every module of my app? I looked through a few topics on SO, however couldn't understand – is it a common problem or just a structure of my app is wrong? Thanks!
As you have discovered, variables declared in one module don't magically appear in other modules. When it comes to modules such as passport, usually people just require them again where they're needed, so:
app.js:
var passport = require('passport'),
index = require('./routes/index');
index.js:
var passport = require('passport');
Sometimes you want to pass parameters though -- I often do it for unit testing, if I can pass in dependencies I can also mock those dependencies. Or you have an app-wide configuration you want to pass around. I usually do this:
app.js:
var ab = "foo",
index = require('/routes/index')(ab);
index.js:
module.exports = function(ab) {
// Here I have access to ab and can to what I need to do
return {
index: function(req, res) { res.send(ab) }
}
}
Other people prefer a "singleton" pattern, where each time you require a module, you check if it's already been initialized and if so, pass that instance.