How to properly organize routes in ExpressJS? - node.js

I'm developing my first test application on NodeJS and encountered the following problem: i can't figure out how to properly organize routes in ExpressJS framework. For example i want to do registration, so i create route like:
app.get('/registration', function (request, response) {
if (request.body.user.email && request.body.user.password) {
var user = new User();
var result = user.createNew(request.body.user.email, request.body.user.email);
// do stuff...
}
response.render('registration.html', config);
});
User function looks like this (not the final):
function User() {
var userSchema = new mongoose.Schema({
'email': {
'type': String,
'required': true,
'lowercase': true,
'index': { 'unique': true }
},
'password': {
'type': String,
'required': true
}
});
var userModel = mongoose.model('users', userSchema);
this.createNew = function(email, password) {
var new_user = new users({'email': email, 'password': password});
new_user.save(function(err){
console.log('Save function');
if (err)
return false;
return true;
});
}
}
I try to do a bit of structured applications like MVC. The problem is that the save method is asynch and each time i registrate new user get registration.html without waiting for the result.
Basically i need to run route callback in save callback, but how to do this in the right way i can't figure out by myself...

this.createNew = function(email, password, callback) {
var new_user = new users({'email': email, 'password': password});
new_user.save(function(err){
console.log('Save function');
if (err)
// return false;
callback (false);
else
//return true;
callback (true);
});
}
I find that whenever I'm using some module (db for example) and it's using a callback, I will often have to also use a callback for any function I wrap around it (unless I don't care about the results).
Here:
app.get('/registration', function (request, response) {
if (request.body.user.email && request.body.user.password) {
var user = new User();
// var result = user.createNew(request.body.user.email, request.body.user.email);
user.createNew(request.body.user.email, request.body.user.email, function (results) {
// do something with results (will be true or false)
// not sure why you have this with the actual registration stuff,
// but it's your site. :)
response.render('registration.html', config);
});
}
});
Also, you might want to put your object methods in the prototype, instead of:
this.createNew = function (....) {}
Try:
User.prototype.createNew = function ( ... ) { }
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Closures#Performance_considerations for info on why.

For a starting point on organising your Node.js app, try reading through the source code on this demo express.js app. Click here for the link
It's a fairly popular repo, which I often find myself referencing when building Node.js apps from scratch.
It may be a good reference for your project given it's done in an the MVC style and it uses mongoose. The routes are organised into a single file which can be found in config/routes.js. You should also take a look at the models in app/models/ for an alternative way to organise your user model

Related

How to perform database related code in model

I am .NET developer and learning MEAN stack, developing an API for simple post and comment app. I am using MVC pattern without view as it is API. Here is how I am trying to create my model function :
const User = module.exports = mongoose.model('User',UserSchema);
module.exports.getUsers = function(callback){
User.find()
.exec(function(err,list_users){
if(err) { callback(); }
return list_users;
});
}
And here is the controller :
exports.user_list = function(req,res,next){
User.getUsers((err,list_users) => {
if(err){ next(err);}
res.json({user_list : list_users});
})
}
I am getting an error as getUsers is not getting imported. I am trying here to separate the data access code from controller and make controller only to manage request and response as I have been doing in .NET. Any help how can I achieve this?
I would look through the documentation to get familiar with how to use mongoose.
First thing I see is that you are not defining the methods correctly according to the documentation here http://mongoosejs.com/docs/guide.html#methods.
But you do not have to define a find() function. Mongoose already has it built in.
So the next step would be how are you creating the instance of the Model in your controller.
Your model should be:
const mongoose = require('mongoose'),
UserSchema = mongoose.Schema();
const User = new UserSchema({
// define user
});
module.exports = mongoose.model('User',UserSchema);
And the controller:
const User = require('path_to_User_model');
exports.user_list = function(req,res,next){
User.find((err,result)=>{
if(err)
// log err and send response
else
// send the result ex: res.json(result);
});
}
also read through the setup on the npmjs website. https://www.npmjs.com/package/mongoose
So to create have your data functions and have your controller use that function you may try
const mongoose = require('mongoose'),
UserSchema = mongoose.Schema();
const User = new UserSchema({
// define user
});
// data functions
User.methods.getUsers = function (){
User.find((err,result)=>{
if(err)
// log err and return something
return false;
else
return result;
});
}
module.exports = mongoose.model('User',UserSchema);
then the controller
const User = require('path_to_User_model');
exports.user_list = function(req,res,next){
var users = User.getUsers();
if (!users)
res.status(500).send('error_page');
else
res.json(users);
}

Asyncronous set method in mongoose

When registering a new user using Mongoose & MongoDB i would like to fetch values from another collection before saving the user instance. Currently I use the following solution, but this causes the user to get saved twice... Even worse, if the user does not pass the first validation the save() inside the set method is called anyway... usually resulting in an uncaught exception.
My current code works the following way:
UserSchema.path('address.postCode').set(function(newVal, cb) {
var that = this;
this.model('PostCode').findOne({ postCode: newVal }, function(err, postCode) {
if(postCode) {
that.longitude = postCode.longitude;
that.latitude = postCode.latitude;
} else {
that.longitude = undefined;
that.latitude = undefined;
}
that.save();
});
return newVal;
});
Anyone know a better way to do this?
The SchemaType#set function expects a synchronous return. You have an asynchronous call inside this function and therefore the return will be called before PostCode returns its results.
Also, you shouldn't be calling save() inside the setter, since it will be saved anyway.
Since you want to fetch data from another model, you should use a pre middleware. For example:
UserSchema.pre('save', true, function (next, done) {
var that = this;
if(this.isNew || this.isModified('address.postCode')) {
this.model('PostCode').findOne({ postCode: that.address.postCode }, function(err, postCode) {
if(postCode) {
that.longitude = postCode.longitude;
that.latitude = postCode.latitude;
} else {
that.longitude = undefined;
that.latitude = undefined;
}
done();
});
}
next();
});
The second parameter passed to pre (true) is what defines this is an asynchronous middleware.
For more information in middlewares, check the mongoose docs.

connect session save redundant?

How does connect's session work? I'm following this tutorial. I've noticed that even though i've commented out the session.save() call everything still works, that property still gets persisted to the session. Does Express automatically call session.save() every time a response is served up? If that's the case, what is the purpose of the save method?
var BaseController = require('./Base'),
View = require('../views/Base');
module.exports = BaseController.extend({
name: 'Admin',
username: 'dadams',
password: 'dadams',
run: function(req, res, next) {
if(this.authorize(req)) {
req.session.fastdelivery = true;
// req.session.save(function(err) {
var v = new View(res, 'admin');
v.render({
title: 'Administration',
content: 'Welcome to the control panel'
});
// });
} else {
var v = new View(res, 'admin-login');
v.render({
title: 'Please login'
});
}
},
authorize: function(req) {
return (
req.session &&
req.session.fastdelivery &&
req.session.fastdelivery === true
) || (
req.body &&
req.body.username === this.username &&
req.body.password === this.password
);
}
});
Connect’s session handling is automatic. Looking at the code, save gets called automatically on res.end (when your response is sent), so there’s no need to call it separately.
Consider it an implementation detail that’s exposed to you. I can’t think of many reasons why you would use it. Maybe if you are saving to Redis or a database and you want the session to be committed early for some reason, before calling end?

Asynchronous issues with node

I have a function running in a node app that I can't get to work due to my lack of understanding on how to write asynchronous code properly. Below is a function that takes in a profile with emails. I would like to loop through each email and check to see whether that user exists in my database. If they do, I would like to return the callback given and completely exist the function without doing anything else. If the user is not found, I would then like to create a new user based on the information given in the profile, and then return the same callback with the newly created user. As of now, the function works as intended, except that it creates a new user even when a user is already found in my database. (the 'User' variable is defined above and has the 'create' function. Also, I would like to avoid using the 'async' node module if at all possible)
function processProfile(profile, callback) {
var existingUser;
if (profile.emails) {
profile.emails.forEach(function(email) {
console.log("Searching for user with this email:" + email.value);
existingUser = findUserByEmail(email.value);
if (existingUser) {
console.log("Found the existing user");
return callback(null, existingUser);
}
});
if(!existingUser){
console.log("Creating new user");
var newUser = {
id: profile.id,
firstName: profile.name.givenName,
lastName: profile.name.familyName,
email: profile.emails[0].value
};
user.create(newUser, profile.provider, function(err, user) {
if (err) throw err;
return callback(null, user);
});
}
}
}
Is there something wrong with this?
function processProfile(profile, callback) {
var existingUser;
var index = 0;
function processNextEmail() {
if(index >= profile.emails.size()) return; //When we've popped nothing exit
var email = profile.emails[index++];
console.log("Searching for user with this email:" + email.value);
existingUser = findUserByEmail(email.value);
if (existingUser) {
console.log("Found the existing user");
callback(null, existingUser);
processEmail();//recursive call to prcess the next email
} else {
console.log("Creating new user");
var newUser = {
id: profile.id,
firstName: profile.name.givenName,
lastName: profile.name.familyName,
email: profile.emails[0].value
};
user.create(newUser, provider, function(err, user) {
if (err) throw err;
callback(null, user);
processNextEmail();//recursive call to process the next email after creating the user and adding it to the database.
});
}
}
processNextEmail();
}
If you need the recursive logic to not remove the emails, you can do a simple modification involving an indice within the scope of the processProfile() closure.
Also note, the return callback() lines, don't really do anything. Returning from functions that are happening asynchronously is a waste of time. Just call the callback, and then you can call an empty return to skip the rest of the function if you wish, but it is unecessary, unless the return effects the flow of logic.
EDIT: It turns out this is example is too simple to be much more interesting. The code below I used as an example for some people at work who were having trouble grasping async code. The one time I think that it is okay to use sync code in node is for gathering configuration data. Let's pretend that we have configuration stored in a file, that in turn gets the filename from that file, and gathers another layer of configuration data from another file. We can do this two ways, using readFileSyn, or using readFile. The asynchronous version is tricky, because we need to wait for the first step to complete, because we have to grab the filename from the first file, in order to know where the second file is stored. Below is the code for both the sync solution and async solution.
//The synchronous way
function useConfigurationData(configData) {
dosomethinginterestingwith(configData);
}
function getConfigurationData(fileName) {
var fileName2 = fs.readFileSync(fileName);
var configurationData = fs.readFileSync(fileName2);
return configurationData;
}
var fileData = getConfigurationData('someFile');
useConfigurationData(fileData);
//The equivalent async way
function getConfigurationData(fileName, useConfigDataCallBack) {
fs.readFile(fileName, getConfigDataStepTwo);
function getConfigDataStepTwo(err, fileName2) {
fs.readFile(fileName2, getConfigDataStepThree);
}
function getConfigDataStepThree(err, fileData) {
useConfigDataCallBack(fileData);
}
}
getConfigurationData('someFile', useConfigurationData);
Notice that the callback we supply to getConfigurationData is the last step. We could also just rely on the globally defined getConfigurationData function, but passing it in as a callback is better style.
What I like about this syntax, is the code withing the second getConfigurationData function reads in order, very synchronously. But if you follow the flow of logic, it is all being run async. It's easy to read and follows nodes async I/O model. IN the case of configuration data, I think the synchronous option is acceptable, but this is still a good demo of how to get synchronous behavior and syntax(ish) from asynchronous callbacks.

Making Mocha test work for my db module

I am new to Mocha, and only a little experience with Node/Express. My DbProvider module works perfectly (mongodb) when I am access it through my Express app. And now I want to test it. I have read the Mocha site and some tutorials I could find. But I have big trouble of finding an real-world example out there that I could follow (any links much appreciated!).
Here is my unsuccessful attempt to write a testfile:
var DbProvider = require('../db').DbProvider;
var assert = require('assert');
var dbProvider = new DbProvider('localhost', 27017, 'mydb');
var util = require('util');
console.log(util.inspect(dbProvider));
describe('DbProvider', function(){
describe('findAllNotes', function(){
it('should return some notes', function(){
dbProvider.findAllNotes({}, function (err, result){
assert(result.length > 0);
});
})
})
})
The output I get is this:
$ mocha
{}
✖ 1 of 1 test failed:
1) DbProvider findAllNotes should return some notes:
TypeError: Cannot call method 'collection' of undefined
at DbProvider.doOperation (/Users/frode/Node/json/db.js:46:11)
at DbProvider.findAllNotes (/Users/frode/Node/json/db.js:56:8)
at Context.<anonymous> (/Users/frode/Node/json/test/test.js:15:18)
(cutting out the rest)
It seems that I am unsuccessful to create the dbProvider. This works perfectly in my app... How can I make this work? (And perhaps also: Is the way I have set it up in general ok?)
Edit: Here is the db.js file:
// Database related
'use strict';
var MongoClient = require('mongodb').MongoClient;
var BSON = require('mongodb').BSONPure;
var ObjectID = require('mongodb').ObjectID;
var checkForHexRegExp = new RegExp("^[0-9a-fA-F]{24}$");
var Validator = require('validator').Validator
var fieldMaxLength = 1024;
//var util = require('util');
var DbProvider = function(host, port, database) {
var dbUrl = "mongodb://"+host+":"+port+"/"+database;
var self = this;
MongoClient.connect(dbUrl, function(err, db) {
self.db = db;
});
};
// Do some basic validation on the data we get from the client/user
var validateParams = function(params, callback) {
// Let´ do a quick general sanity check on the length on all fields
for(var key in params) {
if(params[key].length > fieldMaxLength) callback(new Error('Field ' + key + ' is too long.'));
}
// and the let us check some specific fields better
if (params._id) {
if(checkForHexRegExp.test(params._id)) {
// In case of '_id' we also need to convert it to BSON so that mongodb can use it.
params._id = new BSON.ObjectID(params._id);
} else {
var err = {error: 'Wrong ID format'};
}
}
if(err) callback(err);
}
// Generalized function to operations on the database
// Todo: Generalize even more when user authenication is implemented
DbProvider.prototype.doOperation = function(collection, operation, params, callback) {
validateParams(params, callback);
var operationCallback = function(err, result) {
callback(err, result);
};
this.db.collection(collection, function(err, collection) {
if(operation==='find') {
collection.find().toArray(operationCallback);
} else {
collection[operation](params, operationCallback);
}
});
}
DbProvider.prototype.findAllNotes = function(params, callback) {
this.doOperation('notes', 'find', params, callback);
};
DbProvider.prototype.findNoteById = function(params, callback) {
this.doOperation('notes', 'findOne', params, callback);
};
DbProvider.prototype.saveNote = function(params, callback) {
params.created_at = new Date();
this.doOperation('notes', 'save', params, callback);
};
DbProvider.prototype.deleteNote = function(params, callback) {
this.doOperation('notes', 'remove', params, callback);
};
DbProvider.prototype.findUser = function(params, callback) {
this.doOperation('users', 'findOne', params, callback);
};
exports.DbProvider = DbProvider;
SOLUTION:
After Benjamin told me to handle the async nature of mongodb connecting to the database, and inspired by his suggestion on how to adapt the code, I split the constructor function DbProvider into two parts. The first part, the constructor DbProvider now just saves the db-parameters into a variable. The second part, a new function, DbProvider.connect does the actual async connection. See below.
var DbProvider = function(host, port, database) {
this.dbUrl = "mongodb://"+host+":"+port+"/"+database;
};
DbProvider.prototype.connect = function(callback) {
var self = this;
MongoClient.connect(this.dbUrl, function(err, db) {
self.db = db;
callback();
});
};
So I can now make a Mocha test like this (and async tests also need the "Done" included, like you see in the code below):
var assert = require('assert');
var DbProvider = require('../db').DbProvider;
var dbProvider = new DbProvider('localhost', 27017, 'nki');
describe('DbProvider', function(){
describe('findAllNotes', function(){
it('should return some notes', function(done){
dbProvider.connect(function(){
dbProvider.findAllNotes({}, function (err, result){
assert(result.length > 0);
done();
});
});
})
})
})
Note that the acutal test ("should return some notes") is nothing to be proud of. What I wanted here was to get set up so I am able to test something. Now that I finally acutally can do that, I need to write good tests (something in the the line of having a test database, clear it, test insert a document, test search for a document, and so on...).
And in my Express app, I used to set up the database like this:
var DbProvider = require('./db').DbProvider;
// Setup db instance
var dbProvider = new DbProvider(
process.env.mongo_host || 'localhost',
process.env.mongo_port || 27017,
process.env.mongo_db || 'nki'
);
Now I do the same, but in addition, I call the new connect-function:
// Connect to db. I use (for now) 1 connection for the lifetime of this app.
// And I do not use a callback when connecting here (we do in the testing)
dbProvider.connect(function(){});
Benjamin actually pointed out that it may be ok but not the best practice to have the database set up like this in an Express app. But until I figure out what the best practice really is, I will leave this code as it is. Here is a couple of links reagarding the subject I found (but I have still not concluded of how I will solve it myself):
What's the best practice for MongoDB connections on Node.js? and
[node-mongodb-native] MongoDB Best practices for beginner
If you like, you are very welcome to follow/fork/whatever this project on github. My goal is to get it as production ready I can. The link is
https://github.com/frodefi/node-mongodb-json-server
MongoClient.connect is asynchronous.
From the docs:
callback (function) – this will be called after executing this method. The first parameter will contain the Error object if an error occured, or null otherwise. While the second parameter will contain the initialized db object or null if an error occured.
That means DbProvider.db isn't set yet in the test which is why you're getting undefined.
In here:
MongoClient.connect(dbUrl, function(err, db) {
self.db = db;
});
You're telling it "update self.db after the connection happened", which is at least one event loop tick after this one (but may be more). In your mocha code you're executing your .describe and .it methods right after creating your DbProvider instance which means it was not initialized yet.
I suggest that you re-factor DbProvider to return a callback instead of being a constructor function. Maybe something along the lines of:
var getDbProvider = function(host, port, database,callback) {
var dbUrl = "mongodb://"+host+":"+port+"/"+database;
MongoClient.connect(dbUrl, function(err, db) {
self.db = db;
callback(db);
});
};
Which also means moving all the DBProvider methods to an object (maybe the callback will return a dbprovider object and not just a db?).
Another bug solved by using Unit Tests :)
This is what I used: https://github.com/arunoda/mocha-mongo
It has set of testing helpers for mongodb

Resources