I'm going all in and doing a project using only node. It's been a lot of fun, but sometimes I get a little lost in it and I want to try to gain understanding as I get confused so I build it correctly and don't get too overwhelmed. Anyway, here's the problem:
I have REST API that's using Express and mysql. I set up mysql:
app.js
//Variables, move these to env
var dbOptions = {
host: config.db_config.host,
user: config.db_config.user,
password: config.db_config.password,
port: config.db_config.port,
database: config.db_config.database
};
app.use(myConnection(mysql, dbOptions, 'single'));
and then I include my routes, passing the routes the app and log middleware so I can use them in the route:
app.js cont.
var userRoute = require('./routes/users.js')(app,log);
app.use('/users', userRoute);
That actually was a little confusing, but now I get it, I have to pass them into module in order for the module to get the data.
Then in my route file I want to use an object so that I can use the same functionality in other routes, but in the user file in order to use the same connection pool that everything else is using, or at least to not have to set up the connection again I have to pass it the response and request? There's gotta be a better way to do that. It's really ugly that way. Here's the relevant part of
routes/users.js
var User = require('../controllers/User.js');
module.exports = (function(app,log) {
var userR = express.Router();
userR.post('/register', function(req,res){
var email = req.body.email;
var password = req.body.password;
var firstName = req.body.first_name;
var lastName = req.body.last_name;
var userId;
try {
var query;
var status = 200;
var response = '';
var newUser = {
email: email,
password:password,
first_name: firstName,
last_name: lastName,
password: password
};
var user = new User(req,res);
user.register(newUser);
...
};
controllers/User.js
module.exports = function User(req,res) {
this.id = 0;
this.register = function(newUser){
var _this = this;
var deferred = q.defer();
req.getConnection(function(err,connection){
...
There must be a pattern I'm missing here. I should just be able to pass the app or something and have access to the req.getConnection etc.
Thanks.
There's kind of a lot going on here, but I'll take a crack at this one.
My first recommendation would be to try to keep your routers pretty slim. It's not labeled, but I'm assuming the largest snippet of code you've provided is your router. There are lots of opinions of course, but if I were doing this here's what my router would look like. I'm assuming you're using Express 4x.
routes/users.js
var User = require('../controllers/user.js');
var userRouter = express.Router();
userRouter.post("/register", User.register);
module.exports = userRouter;
So what we've done here is eliminated the dependency on your app/log middleware.
This means your main file (what I usually call app.js), will look like this:
app.js
var userRouter = require('./routes/users.js');
app.use('/users', userRouter);
This is also where you can put any middleware you chose (logging, body-parsing, error-handling, etc).
In the later versions of Express the above code actually mounts our userRouter to "/users". In this case you will now have a "/users/register" route.
So now since we've pulled out some of the logic of the router, we have to put that somewhere. Typically a router will talk to a controller so let's take a look:
controllers/user.js
var User = require("../models/user.js")
var register = function(req, res, next){
var email = req.body.email;
var password = req.body.password;
var firstName = req.body.first_name;
var lastName = req.body.last_name;
var userId;
var params = {
email: email,
password:password,
first_name: firstName,
last_name: lastName,
password: password
};
var newUser = new User(params);
try {
newUser.register();
// do other things...
}
};
module.exports = {register: register};
The first thing you'll notice is that I would have a UserModel file. By doing this, we've de-coupled our model object from this route. Let's say for instance we have a new register route (maybe one of them has an email + pw, the other registers through FB and we have to store different things). We should be able to use the same function (user.register in this case) specified in our model without having to change a whole bunch of things!
Now, here's what a UserModel might look like:
/models/user.js
var connection = require("../lib/connection.js");
var User = function(params){
this.email = params.email;
// ...etc
};
User.prototype.register = function(newUser){
connection.getConnection(function(error, connection){
//connection.doWhatever();
});
};
module.exports = User;
Now we've finally gotten to the core of your question. At the top, you'll see we have a connection file. This is where we're going to put all of our DB-related logic like our connection pools.
/lib/connection.js
/*
This will be in some JSON config we'll say
var dbOptions = {
host: config.db_config.host,
user: config.db_config.user,
password: config.db_config.password,
port: config.db_config.port,
database: config.db_config.database
};
*/
//This will depend on which version/module/db you're using, but here's what mine looks like
var MySQL = require("mysql");
var config = require("../config/db.json");
connectionPool = MySQL.createPool({host: config.db_config.host, ...});
var getConnection = function(done){
connectionPool.getConnection(done);
};
module.exports = {getConnection: getConnection};
So, in conclusion, instead of passing your connection along with your request object, we now simply include our short connection module in whichever model file we're in, kindly ask for a connection, and do whatever processing we need to. Also note you'll have to release the connections back into the pool, but I'll leave that as an exercise for you :).
Also, I usually write CoffeeScript, so please excuse any small errors. Let me know if you have need for further clarification.
Cheers,
Brennan
Related
I've posted an approximation of my node web application below. The original problem I had is that I want to be able to let the client know on post to createentity whether the insert was successful and the id of the inserted entity. However, connection.query having a callback rather than running synchronously, I can't use the entityservice how I'd expect in another language, ie simply returning the result synchronously. There are several solutions, and I'm curious which is the best/common practice re node.js or if there is another I'm not thinking of.
Passing res down to the service, and responding within a callback; seems poor practice
Similarly, passing functions to execute after success/failure to the service; also seems poor practice
Returning a promise from the service and setting res based on resolution/failure; seems like services shouldn't return promises but I'm new to node
Some better method using appropriate features of node.js of which I'm unaware
trying to change the service such that it runs synchronously and just returning the result Other questions/answers have made me leery that this is possible or wise
structure the application some other way
something else?
//app.js
var express = require('express');
var bodyParser = require('body-parser')
var path = require('path');
var EntityService = require('./entityService.js');
var app = express();
var urlencodedParser = bodyParser.urlencoded({ extended: true })
app.post('/createentity', urlencodedParser, function(req, res){
EntityService.createEntity(req.body.name);
res.status(200).json(null);
});
app.listen(3000);
//entityService.js
var mysql = require('mysql');
EntityService = function(){
var connection = mysql.createConnection({
host : CONNECTION_IP,
user : 'root',
password : 'password',
database : 'entitydb'
});
connection.connect();
this.createEntity = function(name){
var record = {name: 'name'};
var query = connection.query('INSERT INTO entity set ?', record, function(error, results, fields ){
//want to return the results.insertId from the service
});
}
}
module.exports = new EntityService();
The correct approach here is option 3 - have your service return a Promise
this.createEntity = name => new Promise((resolve, reject) => {
const query = connection.query('...', { name }, (err, results) => {
if (err) return reject(err);
return resolve(results.map(r => r.insertId));
});
})
If you're on the latest version of Node, I'd go with the asynch/await syntax.
Your service returns a promise,then your calling code can do:
app.post('/createentity', urlencodedParser, async function(req, res){
const entity = await EntityService.createEntity(req.body.name);
res.status(200).json(entity);
});
I'm trying to write a small api with Facebook authentication and as I'm new to node.js I'm just a little confused as to how I should structure this.
I have a route called auth.js that looks like this:
var express = require('express');
var crypto = require('crypto');
var rp = require('request-promise');
var router = express.Router();
router.post('/', function(req, res) {
rp('https://graph.facebook.com/me?access_token=' + req.body.fbAccessToken).then(function(body) {
var json = JSON.parse(body);
if(json.error) { res.status(403).send(json.error.message); }
var user = new User({
userId: json.id,
fbAccessToken: req.body.fbAccessToken,
apiAccessToken: crypto.randomBytes(64).toString('hex'),
firstName: json.first_name,
lastName: json.last_name,
email: json.email
});
user.save(function(err) {
if (err) throw err;
return user.userId;
});
}).then(function(userId) {
res.status(201).send('something');
}).catch(function(err) {
res.status(403).send(err);
});
});
module.exports = router;
When the route recieves a post it takes the Facebook token from the request and checks if it is legit using the Facebook graph API. (Apologies if the Promise stuff looks a little funky too, I'm trying to learn that as well).
Anyway, I have a User model and a user.js route too. What I want to know is if the User creation that happens in my auth route here should be moved to the User model and the model functions called from here somehow? Via the user route maybe?
Cheers
Yes and yes. I think user stuff can go under /user and it is cleaner if the user creation is in its own module.
BTW did you see fbgraph?
Nothing important
My first question here on stackoverflow. I've used it for years to find answers, but now I need a bit of guidance. I'm new to node and express and the async way of structuring an app.
Goal - A REST interface with validation and neDB database
I got the following code working. POST a new user is the only route. It's based on many answers and tuts mixed together. I find it hard to scaffold out the logic, to get a structure you can build on.
I'm not sure at all whether this structure is crap or not. Any advice would be appreciated.
Main file is initializing the database, middleware validator, and starting the app.
// rest.js
var express = require('express'),
bodyParser = require('body-parser'),
validator = require('express-validator'),
db = require('./database/db'),
userRouter = require('./routers/users');
db.init();
var app = express();
app.use(bodyParser.urlencoded({ extended: true }));
app.use(bodyParser.json());
app.use(validator());
app.use('/api/users', userRouter);
var port = process.env.PORT || 8080;
app.listen(port);
Database
This question/answer made me create the small database module with alterations.
How do you pass objects around node express application?
It doesn't have much attention. Maybe because it's very obvious or maybe not a good answer.
The idea is that whem i get multiple collections, they all get initialized on startup, but I can request a single collection if that's all the module needs, or I can get the entire db object back if another module would require that.
// database/db.js
var nedb = require('nedb');
var db = {};
db.init = function() {
db.users = new nedb({ filename: './database/data/users', autoload: true });
db.users.ensureIndex({ fieldName: 'username', unique: true }, function (err) {});
db.users.ensureIndex({ fieldName: 'email', unique: true }, function (err) {});
};
db.get = function(collection) {
if (collection && db[collection])
return db[collection];
return db;
}
module.exports = db;
Router
I require the User Model here and use the express-validator and sanitizes the request before passing it on to the model, based on a minimalist key schema in the model. I don't have any controllers. If I had (or when I do), I would put the validation there. The router is supposed to send the response and status right?
// routers/users.js
var express = require('express'),
_ = require('lodash'),
User = require('../models/user');
var userRouter = express.Router();
userRouter.route('/')
.post(function(req, res) {
req.checkBody('username', 'Username must be 3-20 chars').len(3,20);
req.checkBody('email', 'Not valid email').isEmail();
req.checkBody('password', 'Password must be 6-20 chars').len(6,20);
var err = req.validationErrors();
if (err) {
res.status(422).send(err);
return;
}
var data = _.pick(req.body, _.keys(User.schema));
User.create(data, function (err, newData) {
if (err) {
res.status(409).send(err);
} else {
res.status(201).send(newData);
}
});
});
module.exports = userRouter;
Model
The model requires the database module and gets the "connection". Is this OK?
// models/user.js
var db = require('../database/db');
var User = function (data) {
this.data = data;
};
User.schema = {
_id: null,
username: null,
email: null,
password: null
};
User.create = function (data, callback) {
db.get('users').insert(data, callback);
};
module.exports = User;
Thanks for reading this far. Now, my question is:
Is there something fundamentally wrong with this setup, concerning the database usage and the validation logic. I know the model looks stupid :)
I am trying to create a login system where an admin can log in and look at data that is not available to the public. Right now I'm just trying to find my admin instance in my database. I have two node files: app.js and account_manager.js
app.js
//creates node app
var express = require('express');
var app = express();
var mongoose = require('mongoose');
var AM = require('./account_manager');
//stuff...
/***************************************
LOGIN
***************************************/
app.post('/login', function(req, res){
AM.manualLogin(req.param('username'), req.param('password'), function(e, o){
if (!o){
res.send(e, 400);
} else{
req.session.user = o;
res.send(o, 200);
}
});
});
account_manger.js is required in app.js and is stored in AM
account_manager.js:
var express = require('express');
var app = express();
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost');
var admin_db = mongoose.connection;
admin_db.on('error', console.error.bind(console, 'connection error: '));
//connection open?
admin_db.once('open', function callback(){
console.log("User Database connection open!\n");
});
var User_Schema = new mongoose.Schema({username: String, password: String});
var Admin = mongoose.model('Admin', User_Schema);
exports.manualLogin = function(user, pass, callback)
{
admin_db.find({username: user},function(err,docs){ //error is here
if(docs){
var x = 0;
var flag = false;
while(docs[x]){ //goes through all the admins
if (docs[x].param("username") == user){ //if it's a match
callback(null, docs);
flag = true;
break;
}
x+=1;
}
if (flag == false){
callback('invalid-password/username');
}
}
});
}
I get a TypeError that says:
Object #<NativeConnection> has no method 'find'
what's my problem?
I'm still quite new to node.js myself, but I'll try and answer anyway.
It looks like you've properly built your connection to mongodb through mongoose, but you have not created a schema. While mongodb doesn't have schemas, mongoose does.
What you'll need to do is create a schema (UserSchema) that matches the construction of a user document in your users collection, then create an instance of that schema (User) which is now your model, then call .find on that model.
The Mongoose Quick Start guide goes through this process:
http://mongoosejs.com/docs/index.html
EDIT after update:
You are currently calling admin_db.find. This does not exist. This is the error you are getting.
You need to change that to Admin.find. You also need to understand what the difference is.
EDIT once more:
You're not properly using the admin_db.once callback.
I suggest you go back and reread the Mongoose Quick Start guide I linked. It's quite short and goes through all of this.
I see you're rolling your own system here, but I thought I'd share a link to Drywall in case you could gain some inspiration or knowledge from it.
Drywall - A website and user system for Node.js: http://jedireza.github.io/drywall/
I hope this is helpful. Feel free to open issues in GitHub if you run into any road blocks.
I'm developing my first Node.js App with Socket.IO and everything is fine but now the app is slowly getting bigger and I'd like to divide the app-code into different files for better maintenance.
For example I'm defining all my mongoose schemas and the routings in the main file. Underneath are all the functions for the socket.IO connection. But now I want to have an extra file for the schemas, an extra file for routing and one for the functions.
Of course, I'm aware of the possibility to write my own module or load a file with require. That just does not make sense for me, because I can't work with the vars like app, io or db without making them global. And if I pass them to a function in my module, I can't change them. What am I missing? I'd like to see an example how this is done in practice without using global vars..
It sounds like you have a highly coupled application; it's difficult for you to split out your code into modules because pieces of the application that should not depend on each other do. Looking into the principles of OO design may help out here.
For example, if you were to split your dataabse logic out of the main application, you should be able to do so, as the database logic should not depend on app or io--it should be able to work on its own, and you require it into other pieces of your application to use it.
Here's a fairly basic example--it's more pseudocode than actual code, as the point is to demonstrate modularity by example, not to write a working application. It's also only one of many, many ways you may decide to structure your application.
// =============================
// db.js
var mongoose = require('mongoose');
mongoose.connect(/* ... */);
module.exports = {
User: require('./models/user');
OtherModel: require('./models/other_model');
};
// =============================
// models/user.js (similar for models/other_model.js)
var mongoose = require('mongoose');
var User = new mongoose.Schema({ /* ... */ });
module.exports = mongoose.model('User', User);
// =============================
// routes.js
var db = require('./db');
var User = db.User;
var OtherModel = db.OtherModel;
// This module exports a function, which we call call with
// our Express application and Socket.IO server as arguments
// so that we can access them if we need them.
module.exports = function(app, io) {
app.get('/', function(req, res) {
// home page logic ...
});
app.post('/users/:id', function(req, res) {
User.create(/* ... */);
});
};
// =============================
// realtime.js
var db = require('./db');
var OtherModel = db.OtherModel;
module.exports = function(io) {
io.sockets.on('connection', function(socket) {
socket.on('someEvent', function() {
OtherModel.find(/* ... */);
});
});
};
// =============================
// application.js
var express = require('express');
var sio = require('socket.io');
var routes = require('./routes');
var realtime = require('./realtime');
var app = express();
var server = http.createServer(app);
var io = sio.listen(server);
// all your app.use() and app.configure() here...
// Load in the routes by calling the function we
// exported in routes.js
routes(app, io);
// Similarly with our realtime module.
realtime(io);
server.listen(8080);
This was all written off the top of my head with minimal checking of the documentation for various APIs, but I hope it plants the seeds of how you might go about extracting modules from your application.