ExpressJS & Mongoose REST API structure: best practices? - node.js

I'm building a REST API with the use of NodeJS (Mongoose & ExpressJS). I think I have a pretty good basic structure at the moment, but I'm wondering what the best practices are for this kind of project.
In this basic version, everything passes through the app.js file. Every HTTP method is then passed to the resource that has been requested. This allows me to dynamically add resources to the API and every request will be passed along accordingly. To illustrate:
// app.js
var express = require('express');
var mongoose = require('mongoose');
var app = express();
app.use(express.bodyParser());
mongoose.connect('mongodb://localhost/kittens');
var db = mongoose.connection;
var resources = [
'kitten'
];
var repositories = {};
for (var i = 0; i < resources.length; i++) {
var resource = resources[i];
repositories[resource] = require('./api/' + resource);
}
db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', function callback() {
console.log('Successfully connected to MongoDB.');
app.get('/:resource', function (req, res) {
res.type('application/json');
repositories[req.params.resource].findAll(res);
});
app.get('/:resource/:id', function (req, res) {
res.type('application/json');
repositories[req.params.resource].findOne(req, res);
});
app.listen(process.env.PORT || 4730);
});
-
// api/kitten.js
var mongoose = require('mongoose');
var kittenSchema = mongoose.Schema({
name: String
});
var Kitten = mongoose.model('Kitten', kittenSchema);
exports.findAll = function (res) {
Kitten.find(function (err, kittens) {
if (err) {
}
res.json(kittens);
});
};
exports.findOne = function (req, res) {
Kitten.findOne({ _id: req.params.id}, function (err, kitten) {
if (err) {
}
res.json(kitten);
});
};
Obviously, only a couple of methods have been implemented so far. What do you guys think of this approach? Anything I could improve on?
Also, a small side question: I have to require mongoose in every API resource file (like in api\kitten.js, is there a way to just globally require it in the app.js file or something?
Any input is greatly appreciated!

Well, you can separate out your routes, db models and templates in different files.
Have a directory structure something like this,
| your_app
| -- routes
| -- models
| -- templates
| -- static
| -- css
| -- js
| -- images
| -- config.js
| -- app.js
| -- url.js
For each Mongoose model have a separate file placed in your ./models
In templates directory place your jade files. (Assuming you are using jade as your template engine). Though it seems like you are only serving JSON, not HTML. Consider using Jade if you want to render HTML. (Here are few other template engines you can consider going with)
./static directory for static JS, CSS and XML files etc.
Things like db connections or 3rd party API keys and stuff can be put in config.js
In url.js have a procedure which take express app object as argument and extend upon app.get and app.post there in single place.
P.S. This is the approach I go with for a basic web app in express. I am in no way saying this the best way to follow, but it helps me maintain my code.

There is no right way, but I did create a seed application for my personal directory structure to help my roommate with this.
You can clone it: git clone https://github.com/hboylan/express-mongoose-api-seed.git
Or with npm: npm install express-mongoose-api-seed

As codemonger5 said there is no right way of organising directory structure.
However, you can use this boilerplate application for creating REST APIs using Express and mongoose using ES6. We use the same directory structure in our production API services.
git clone https://github.com/KunalKapadia/express-mongoose-es6-rest-api
cd express-mongoose-es6-rest-api
npm install
npm start

Related

NodeJS (Express) - project structure and mongo connection

I started a new project from scratch with ExpressJS.
Everything works fine but now I begin to have a dozen of 'app.get(....)' function and I need to give the project a structure.
What I have in mind is quite simple, it should have a folder named 'routes' containing a file such as 'module1.js', with all of the app.get related to that module. (like I've seen in many examples)
The issue is how to tell Express to route 'http://url/module1/' to that route file and how to pass it a param variable, containing for instance the mongodb connection.
what I tried is :
var params = {
db: myMongoConnection
};
var mod1 = require('routes/module1');
app.use('/module1', mod1);
but now I still miss the 'params'.
If I try to pass it as an argument to the require method i get an error saying it needs middleware.
Another issue is related to the fact that the myMongoConnection is valid in the connection callback, so I think i need to require and use the route.js inside the MongoClient connect callback.
Any idea?
thanks a lot
For custom modules, create a folder, call it modules
In its index.js, expose the modules that you need.
Something like,
var mods = [
'mod1',
'mod2',
];
function init() {
var expose = {};
var params = {
db: myMongoConnection
};
mods.forEach(mods, function (mod) {
expose[mod] = require('./' + mod)(params);
});
return expose;
}
// export init
module.exports = init;
In mod1.js, wrap the params
module.exports = function(params) {
// all your functions here will have access to params.
}
Then in, server/app.js, require this and set it in the app.
app.set('mods', require('path-to/modules'));
Now, you can access all your modules, using app.get('mods').moduleName.methodname

NodeJS Express Dependency Injection and Database Connections

Coming from a non Node background, my first instinct is to define my service as such
MyService.js
module.exports = new function(dbConnection)
{
// service uses the db
}
Now, I want one open db connection per request, so I define in middleware:
res.locals.db = openDbConnection();
And in some consuming Express api code:
api.js
var MyService = require(./services/MyService')
...
router.get('/foo/:id?', function (req, res) {
var service = new MyService(res.locals.db);
});
Now, being that Node's preferred method of dependency injection is via the require(...) statement, it seems that I shouldn't be using the constructor of MyService for injection of the db.
So let's say I want to have
var db = require('db');
at the top of MyService and then use somehow like db.current....but how would I tie the db to the current res.locals object now that db is a module itself? What's a recommended way of handling this kind of thin in Node?
Updated Answer: 05/02/15
If you want to attach a DB connection to each request object, then use that connection in your service, the connection will have to be passed to myService some how. The example below shows one way of doing that. If we try to use db.current or something to that effect, we'll be storing state in our DB module. In my experience, that will lead to trouble.
Alternatively, I lay out the approach I've used (and still use) in this previous answer. What this means for this example is the following:
// api.js
var MyService = require(./services/MyService')
...
router.get('/foo/:id?', function (req, res) {
MyService.performTask(req.params.id);
});
// MyService.js
var db = require('db');
module.exports = {
performTask: function(id)
{
var connection = db.getOpenConnection();
// Do whatever you want with the connection.
}
}
With this approach, we've decoupled the DB module from the api/app/router modules and only the module that actually uses it will know it exists.
Previous Answer: 05/01/15
What you're talking about could be done using an express middleware. Here's what it might look like:
var db = require('db');
// Attach a DB connection to each request coming in
router.use(req, res, next){
req.locals.db = db.getOpenConnection();
next();
}
// Later on..
router.get('/foo/:id?', function (req, res) {
// We should now have something attached to res.locals.db!
var service = new MyService(res.locals.db);
});
I personally have never seen something like new MyService before in express applications. That doesn't mean it can't be done, but you might consider an approach like this
// router.js
var MyService = require('MyService');
router.get('/foo/:id?', function (req, res) {
MyService.foo(res.locals.db);
});
// MyService.js
module.exports.foo(connection){
// I have a connection!
}

Where do I put database connection information in a Node.js app?

Node.js is my first backend language and I am at the point where I am asking myself "where do I put the database connection information?".
There is a lot of good information regarding this issue. Unfortunately for me all the examples are in PHP. I get the ideas but I am not confident enough to replicate it in Node.js.
In PHP you would put the information in a config file outside the web root, and include it when you need database data.
How would you do this in Node.js? using the Express.js framework.
So far I have this:
var express = require('express'), app = express();
var mysql = require('mysql');
app.get('/', function(req,res) {
var connection = mysql.createConnection({
host: 'localhost',
user: 'root',
password: 'password',
database: 'store'
});
var query = connection.query('SELECT * from customers where email = "deelo42#gmail.com"');
query.on('error', function(err) {
throw err;
});
query.on('fields', function(fields) {
console.log('this is fields');
});
query.on('result', function(row) {
var first = row.first_name;
var last = row.last_name;
res.render('index.jade', {
title: "My first name is " + first,
category: "My last name is " + last
});
});
});
app.listen(80, function() {
console.log('we are logged in');
});
As you can see I have a basic express application with 1 GET route. This route sets off the function to go to the database and pull out information based on an email address.
At the top of the GET route is the database connection information. Where do I put that? How do I call it? How do I keep it out of web root, and include it like PHP ? Can you please show me in a working example. Thanks!
I use the Express Middleware concept for same and that gives me nice flexibility to manage files.
I am writing a detailed answer, which includes how i am use the config params in app.js to connect to DB.
So my app structure looks something this:
How i connect to DB? (I am using MongoDB, mongoose is ORM, npm install mongoose)
var config = require('./config/config');
var mongoose = require("mongoose");
var connect = function(){
var options = {
server: {
socketOptions:{
keepAlive : 1
}
}
};
mongoose.connect(config.db,options);
};
connect();
under the config folder i also have 'env' folder, which stores the environment related configurations in separate files such as development.js, test.js, production.js
Now as the name suggests, development.js stores the configuration params related to my development environment and same applies to the case of test and production. Now if you wish you can have some more configuration setting such as 'staging' etc.
project-name/config/config.js
var path = require("path");
var extend = require("util")._extend;
var development = require("./env/development");
var test = require("./env/test");
var production = require("./env/production");
var defaults = {
root: path.normalize(__dirname + '/..')
};
module.exports = {
development: extend(development,defaults),
test: extend(test,defaults),
production: extend(production,defaults)
}[process.env.NODE_ENV || "development"]
project-name/config/env/test.js
module.exports = {
db: 'mongodb://localhost/mongoExpress_test'
};
Now you can make it even more descriptive by breaking the URL's into, username, password, port, database, hostname.
For For more details have a look at my repo, where you can find this implementation, in fact now in all of my projects i use the same configuration.
If you are more interested then have a look at Mean.js and Mean.io, they have some better ways to manage all such things. If you are beginner i would recommend to keep it simple and get things going, once you are comfortable, you can perform magic on your own. Cheers
I recommend the 12-factor app style http://12factor.net which keeps all of this in env vars. You never should have this kind of information hard-coded or in the app source-code / repo, so you can reuse it in different environments or even share it publicly without breaking security.
However, since there are lots of environment vars, I tend to keep them together in a single env.js like the previous responder wrote - although it is not in the source code repo - and then source it with https://www.npmjs.org/package/dotenv
An alternative is to do it manually and keep it in, e.g. ./env/dev.json and just require() the file.
Any of these works, the important point is to keep all configuration information separate from code.
I agree with the commenter, put it in a config file. There is no ultimate way, but nconf is also one of my favourites.
The important best practise is that you keep the config separate if you have a semi-public project, so your config file will not overwrite other developers.
config-sample.json (has to be renamed and is tracked with for example git)
config.json (not tracked / ignored by git)

How to pass a variable into a Kraken.js model?

I have a variable that holds my Sequelize connection, like so:
var sequelize = new Sequelize('database', 'username');
With Kraken.js, how do I pass this into a module? I don't see where I could add this in the index.js configuration file...
Thanks!
(I can't tag this as krakenjs because I don't have enough karma.)
It's a pretty similar setup to what's shown in this guide:
http://sequelizejs.com/articles/express
You would create the model file, (under ./models). You'd need to register this model using the associate function (See previous link).
And to actually use the model, you could replicate the same usage as in the Kraken Shopping cart example https://github.com/lmarkus/Kraken_Example_Shopping_Cart/
eg: (After you've registered your models)
'use strict';
var Product = require('../models/productModel');
module.exports = function (server) {
/**
* Display a list of the products.
*/
server.get('/', function (req, res) {
Product.find(function (err, prods) {
res.render('index', products);
});
});
};
You can add the connection in separate file and export it (http://openmymind.net/2012/2/3/Node-Require-and-Exports/). I store such files in a dir called lib/
Then in the index.js (the starting point of the app) import the file using
require("./lib/..").
And then invoke it in index.js as
app.configure = function configure(nconf, next) {...};
(http://krakenjs.com/#structure_of_a_project)
A good sample app is https://github.com/lmarkus/Kraken_Example_Shopping_Cart

Connect and Express utils

I'm new in the world of Node.js
According to this topic: What is Node.js' Connect, Express and “middleware”?
I learned that Connect was part of Express
I dug a little in the code, and I found two very interesting files :
./myProject/node_modules/express/lib/utils.js
and better :
./myProject/node_modules/express/node_modules/connect/lib/utils.js
These two files are full of useful functions and I was wondering how to invoke them correctly.
As far, in the ./myProject/app.js, that's what I do:
var express = require('express')
, resource = require('express-resource')
, mongoose = require('mongoose')
, expresstUtils =
require('./node_modules/express/lib/utils.js');
, connectUtils =
require('./node_modules/express/node_modules/connect/lib/utils.js');
But I found it a little clumsy, and what about my others files?
e.g., here is one of my routes:
myResources = app.resource(
'myresources',
require('./routes/myresources.js'));
and here is the content of myresources.js:
exports.index = function(req, res)
{
res.render('./myresources.jade', { title: 'My Resources' });
};
exports.show = function(req, res)
{
fonction resourceIsWellFormatted(param)
{
// Here is some code to determine whether the resource requested
// match with the required format or not
// return true if the format is ok
// return false if not
}
if (resourceIsWellFormatted(req.params['myresources']))
{
// render the resource
}
else
{
res.send(400); // HEY! what about the nice Connect.badRequest in its utils.js?
}
};
As you can see in the comment after the res.send(400), I ask myself if it is possible to use the badRequest function which is in the utils.js file of the Connect module.
What about the nice md5 function in the same file?
Do I have to place this hugly call at the start of my myresources.js to use them?:
var connectUtils =
require('../node_modules/express/node_modules/connect/lib/utils.js');
or, is there a more elegant solution (even for the app.js)?
Thank you in advance for your help!
the only more elegant way i came up with is (assuming express is inside your root "node_modules" folder):
require("express/node_modules/connect/lib/utils");
the node installation is on windows, node version 0.8.2
and a bit of extra information:
this way you don't need to know where you are in the path and be forced to use relative paths (./ or ../), this can be done on any file nesting level.
i put all my custom modules inside the root "node_modules" folder (i named my folder "custom_modules") and call them this way at any level of nesting:
require("custom_modules/mymodule/something")
If you want to access connect directly, I suggest you install connect as a dependency of your project, along with express. Then you can var utils = require('connect').utils.

Resources