it is a naive question.
Say you have a config file storing all you need for making a connection to your db.
If you are reading asynchronously, is the connection done in the callback?
fs.readFile(pathToConfigFile, function (err, data){ createConnection(data);});
Does it look ugly, suspicious and bit dangerous, doesn't it?
(This is an example, but I'd like to hear some opinions about it, if and why it is worng doing that and so on)
In my case I am using mongoose, I query the db on the schema object (i.e. User.find()...) so I simply get null results because the connection doesn't happen synchronously I guess.
Thank you very much
(here is my app-structure)
app.js
db.js (read the config file and make the connection)
config.json (info of db and others)
user.js (model for users)
home.js (query all the users and list them on .get '/')
It's ok to so sync operations during app start up. However, do not do any sync stuff once the app has started, especially while handling requests.
Your app should be configured to start IF the database connectivity was successful.
// set up your app
var express = require('express')
var app = express()
....
// set up database connectiveity
var db = mongoose.connection;
db.on('error', function() {
console.log('DB connectivity error')
process.exit()
});
db.once('open', function() {
// start your app now
app.listen()
});
create config.json like this:
config.json
module.exports = {
dbUrl:"mongodb://localhost/testDB",
serverPort: 9999
// add you other config key here
}
and require config file wherever you want to use like:
var config = require("./config");
console(config, config.severPort)
Suggestion: Use config module. It will read your config file as per the environment or default config file
Related
I'm currently developing a web app. I just started the development and i try to setup the routes after the db (mongoose Schemas and connection) is ready because i need the models in my router.js file. My Code looks like this right now:
File: server.js
//configuration here...
mongoose.connect( "mongodb://" + config.database.host + "/" + config.database.dbName );
var db = mongoose.connection;
db.on( "error", console.error.bind(console, "connection error:" ));
db.once( "open", function () {
var allModels = models.getAll( mongoose ); //retreive models from different files
router.setupRoutes(express, app, allModels); //setup all the routes also specified in other files
console.log("connected to database and created the routes!");
});
app.listen(port, host, function() {
console.log("");
console.log("Server started at: " + "http://" + host + ":" + port);
console.log("Press 'STRG+C' to stop the Server");
console.log("");
});
My Question is:
Is that the right way to do that? I call the app.listen function not inside this once("open") function (app.listen is at the bottom of this file and gets called before the routes are set up) My test routes are working but im not accesing the mongoose models right now. Should i call app.listen in the mongoose callback or like now at the end of my file?
i just want to make sure that this is the right way to do this. it seems to be the right way. mongoose said so... Look here
I wouldn't say that there's a right or a wrong way. It depends what you're trying to do. Here are my thoughts:
Do you want to have your server running if the database connection isn't working? Maybe because some routes don't involve the database? If that's what you're aiming for, you'd have to set up the routes outside the db connection callback. Something like this:
db.once('open', function() {
// set up routes that involve db connection
});
// set up routes that don't involve db connection
app.listen(port, host, function() {
});
If all of your routes involve database communication, I don't see a purpose to having app.listen outside of the db connection callback. Because the fact that the server is listening wouldn't be useful if the database connection is broken.
I would like to point to specific case. when we put the connection in the server.js The app will try to connect when it is lunched. And If error has been occured it will throw the error and never look back. However, when we put the connection in the routes, the app will try to reach the db only when it needs to, and every time it needs to.The thing that make us able to reach db related end points and none related end points and open to db status changements. So If the db has a problem and we fix it, no need to restart the server.
Coming from a non Node background, my first instinct is to define my service as such
MyService.js
module.exports = new function(dbConnection)
{
// service uses the db
}
Now, I want one open db connection per request, so I define in middleware:
res.locals.db = openDbConnection();
And in some consuming Express api code:
api.js
var MyService = require(./services/MyService')
...
router.get('/foo/:id?', function (req, res) {
var service = new MyService(res.locals.db);
});
Now, being that Node's preferred method of dependency injection is via the require(...) statement, it seems that I shouldn't be using the constructor of MyService for injection of the db.
So let's say I want to have
var db = require('db');
at the top of MyService and then use somehow like db.current....but how would I tie the db to the current res.locals object now that db is a module itself? What's a recommended way of handling this kind of thin in Node?
Updated Answer: 05/02/15
If you want to attach a DB connection to each request object, then use that connection in your service, the connection will have to be passed to myService some how. The example below shows one way of doing that. If we try to use db.current or something to that effect, we'll be storing state in our DB module. In my experience, that will lead to trouble.
Alternatively, I lay out the approach I've used (and still use) in this previous answer. What this means for this example is the following:
// api.js
var MyService = require(./services/MyService')
...
router.get('/foo/:id?', function (req, res) {
MyService.performTask(req.params.id);
});
// MyService.js
var db = require('db');
module.exports = {
performTask: function(id)
{
var connection = db.getOpenConnection();
// Do whatever you want with the connection.
}
}
With this approach, we've decoupled the DB module from the api/app/router modules and only the module that actually uses it will know it exists.
Previous Answer: 05/01/15
What you're talking about could be done using an express middleware. Here's what it might look like:
var db = require('db');
// Attach a DB connection to each request coming in
router.use(req, res, next){
req.locals.db = db.getOpenConnection();
next();
}
// Later on..
router.get('/foo/:id?', function (req, res) {
// We should now have something attached to res.locals.db!
var service = new MyService(res.locals.db);
});
I personally have never seen something like new MyService before in express applications. That doesn't mean it can't be done, but you might consider an approach like this
// router.js
var MyService = require('MyService');
router.get('/foo/:id?', function (req, res) {
MyService.foo(res.locals.db);
});
// MyService.js
module.exports.foo(connection){
// I have a connection!
}
Node.js is my first backend language and I am at the point where I am asking myself "where do I put the database connection information?".
There is a lot of good information regarding this issue. Unfortunately for me all the examples are in PHP. I get the ideas but I am not confident enough to replicate it in Node.js.
In PHP you would put the information in a config file outside the web root, and include it when you need database data.
How would you do this in Node.js? using the Express.js framework.
So far I have this:
var express = require('express'), app = express();
var mysql = require('mysql');
app.get('/', function(req,res) {
var connection = mysql.createConnection({
host: 'localhost',
user: 'root',
password: 'password',
database: 'store'
});
var query = connection.query('SELECT * from customers where email = "deelo42#gmail.com"');
query.on('error', function(err) {
throw err;
});
query.on('fields', function(fields) {
console.log('this is fields');
});
query.on('result', function(row) {
var first = row.first_name;
var last = row.last_name;
res.render('index.jade', {
title: "My first name is " + first,
category: "My last name is " + last
});
});
});
app.listen(80, function() {
console.log('we are logged in');
});
As you can see I have a basic express application with 1 GET route. This route sets off the function to go to the database and pull out information based on an email address.
At the top of the GET route is the database connection information. Where do I put that? How do I call it? How do I keep it out of web root, and include it like PHP ? Can you please show me in a working example. Thanks!
I use the Express Middleware concept for same and that gives me nice flexibility to manage files.
I am writing a detailed answer, which includes how i am use the config params in app.js to connect to DB.
So my app structure looks something this:
How i connect to DB? (I am using MongoDB, mongoose is ORM, npm install mongoose)
var config = require('./config/config');
var mongoose = require("mongoose");
var connect = function(){
var options = {
server: {
socketOptions:{
keepAlive : 1
}
}
};
mongoose.connect(config.db,options);
};
connect();
under the config folder i also have 'env' folder, which stores the environment related configurations in separate files such as development.js, test.js, production.js
Now as the name suggests, development.js stores the configuration params related to my development environment and same applies to the case of test and production. Now if you wish you can have some more configuration setting such as 'staging' etc.
project-name/config/config.js
var path = require("path");
var extend = require("util")._extend;
var development = require("./env/development");
var test = require("./env/test");
var production = require("./env/production");
var defaults = {
root: path.normalize(__dirname + '/..')
};
module.exports = {
development: extend(development,defaults),
test: extend(test,defaults),
production: extend(production,defaults)
}[process.env.NODE_ENV || "development"]
project-name/config/env/test.js
module.exports = {
db: 'mongodb://localhost/mongoExpress_test'
};
Now you can make it even more descriptive by breaking the URL's into, username, password, port, database, hostname.
For For more details have a look at my repo, where you can find this implementation, in fact now in all of my projects i use the same configuration.
If you are more interested then have a look at Mean.js and Mean.io, they have some better ways to manage all such things. If you are beginner i would recommend to keep it simple and get things going, once you are comfortable, you can perform magic on your own. Cheers
I recommend the 12-factor app style http://12factor.net which keeps all of this in env vars. You never should have this kind of information hard-coded or in the app source-code / repo, so you can reuse it in different environments or even share it publicly without breaking security.
However, since there are lots of environment vars, I tend to keep them together in a single env.js like the previous responder wrote - although it is not in the source code repo - and then source it with https://www.npmjs.org/package/dotenv
An alternative is to do it manually and keep it in, e.g. ./env/dev.json and just require() the file.
Any of these works, the important point is to keep all configuration information separate from code.
I agree with the commenter, put it in a config file. There is no ultimate way, but nconf is also one of my favourites.
The important best practise is that you keep the config separate if you have a semi-public project, so your config file will not overwrite other developers.
config-sample.json (has to be renamed and is tracked with for example git)
config.json (not tracked / ignored by git)
I'm building a REST API with the use of NodeJS (Mongoose & ExpressJS). I think I have a pretty good basic structure at the moment, but I'm wondering what the best practices are for this kind of project.
In this basic version, everything passes through the app.js file. Every HTTP method is then passed to the resource that has been requested. This allows me to dynamically add resources to the API and every request will be passed along accordingly. To illustrate:
// app.js
var express = require('express');
var mongoose = require('mongoose');
var app = express();
app.use(express.bodyParser());
mongoose.connect('mongodb://localhost/kittens');
var db = mongoose.connection;
var resources = [
'kitten'
];
var repositories = {};
for (var i = 0; i < resources.length; i++) {
var resource = resources[i];
repositories[resource] = require('./api/' + resource);
}
db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', function callback() {
console.log('Successfully connected to MongoDB.');
app.get('/:resource', function (req, res) {
res.type('application/json');
repositories[req.params.resource].findAll(res);
});
app.get('/:resource/:id', function (req, res) {
res.type('application/json');
repositories[req.params.resource].findOne(req, res);
});
app.listen(process.env.PORT || 4730);
});
-
// api/kitten.js
var mongoose = require('mongoose');
var kittenSchema = mongoose.Schema({
name: String
});
var Kitten = mongoose.model('Kitten', kittenSchema);
exports.findAll = function (res) {
Kitten.find(function (err, kittens) {
if (err) {
}
res.json(kittens);
});
};
exports.findOne = function (req, res) {
Kitten.findOne({ _id: req.params.id}, function (err, kitten) {
if (err) {
}
res.json(kitten);
});
};
Obviously, only a couple of methods have been implemented so far. What do you guys think of this approach? Anything I could improve on?
Also, a small side question: I have to require mongoose in every API resource file (like in api\kitten.js, is there a way to just globally require it in the app.js file or something?
Any input is greatly appreciated!
Well, you can separate out your routes, db models and templates in different files.
Have a directory structure something like this,
| your_app
| -- routes
| -- models
| -- templates
| -- static
| -- css
| -- js
| -- images
| -- config.js
| -- app.js
| -- url.js
For each Mongoose model have a separate file placed in your ./models
In templates directory place your jade files. (Assuming you are using jade as your template engine). Though it seems like you are only serving JSON, not HTML. Consider using Jade if you want to render HTML. (Here are few other template engines you can consider going with)
./static directory for static JS, CSS and XML files etc.
Things like db connections or 3rd party API keys and stuff can be put in config.js
In url.js have a procedure which take express app object as argument and extend upon app.get and app.post there in single place.
P.S. This is the approach I go with for a basic web app in express. I am in no way saying this the best way to follow, but it helps me maintain my code.
There is no right way, but I did create a seed application for my personal directory structure to help my roommate with this.
You can clone it: git clone https://github.com/hboylan/express-mongoose-api-seed.git
Or with npm: npm install express-mongoose-api-seed
As codemonger5 said there is no right way of organising directory structure.
However, you can use this boilerplate application for creating REST APIs using Express and mongoose using ES6. We use the same directory structure in our production API services.
git clone https://github.com/KunalKapadia/express-mongoose-es6-rest-api
cd express-mongoose-es6-rest-api
npm install
npm start
I am trying to set up logging for the native mongo driver for node. I've got the following snippet set up as a demonstration for what I am trying to do. Unfortunately nothing is being emitted on the console. Any ideas?
var express = require('express') ;
var app = express();
var http = require('http');
var mongod = require('mongodb');
var server_conf = new mongod.Server('localhost', 27017, {auto_reconnect:true});
//dummy logger
var logger = {
error:function(message, object) {console.log('anything')},
log:function(message, object) {console.log('anything')},
debug:function(message, object) {console.log('anything')}}
var db_container = {db: new mongod.Db('test', server_conf,
{w:1,journal:true, native_parser:true, logger: logger})}
app.use(express.bodyParser());
app.use(app.router);
db_container.db.open(function(err, index_info){
if(err) throw err;
var testcol = db_container.db.collection('testcol');
app.get('/', function(request, res){
testcol.insert({hello:"moto"}, function(err,doc){
if(err){
throw err;
}
testcol.find({}).toArray(function(err,docs){
res.send(docs);
});
});
});
http.createServer(app).listen(3000, function () {
console.log('Express server listening on port ' + '3000');
});
});
Replicated below is: a copy of the answer to this question I posted on the node-mongodb-native google group
After looking at the most recent version of the code (commit 0fd78b3278da8deac980cb876fcb2bf72eaafb79) it looks like logging isn't really implemented yet.
First some ground rules: If you create a server config object (http://mongodb.github.io/node-mongodb-native/api-generated/server.html?highlight=server#Server) and set the logger object there that's ok. If you create a Db() object and set a logger in its options then it will override any logger you have set its server config. If you create a MongoClient() object this is equivalent to new Db('test', serverOptions, options) so you can set your logger object in either serverOptions or options. I'm not going to cover MongoClient.Connect and what happens there because I don't care to look over that portion of the code.
I could only find usage of the logger object happening in connection.js where it's revealed that you actually need a few additional properties to get logging working.
i.e line 69
if(this.logger != null && this.logger.doDebug){
this.logger.debug("opened connection", this.socketOptions);
}
or line 307
if(self.logger != null && self.logger.doError) self.logger.error("parseError", errorObject);
There are many more doDebug/doError property lookups in connection.js that require logging to work. There is a very minimal amount of logging happening from the looks of it but if you want to enable it you need to set the doError/doLog/doDebug properties on your logger as well. I actually haven't tested this out since I don't have the proper setup here but from looking over the code this seems to be the case.