I want to create a module for NodeJS to connecto to MongoDB. I've seen that the new, better approach is to use MongoClient, but I can't get to know how can I make concurrent operations on the database. The goal I want to achieve is to have functions to abstract the database, like the following:
exports.insertItem(item){
//Whatever
}
According to the docs, I am supposed to connect to the database this way:
MongoClient.connect("mongodb://localhost:27017/integration_test", function(err, db) {
//Do stuff on the db object
});
The problem is how I am supposed to reuse the db object if it's not in a scope I can use to export functions in node? Am I supposed to make a MongoClient.connect() on every function that deals with the DB?
You make a single db connection and reuse it everywhere
A typical pattern for modules is
export.myinsert = function(db) {
return function(whatever) {
}
}
and then do
require('mymodule')(db)
Have a look at an example
https://github.com/christkv/tic-tac-toe-steps
Here is a stripped down version of some code that I use to show the overall concept...Hopefully this helps you get started.
var mongodb = require('mongodb'),
MongoClient = mongodb.MongoClient;
var db;
// Initialize our connection to MongoDB once
MongoClient.connect("mongodb://localhost:27017/integration_test", function(err, database){
if(err){
console.log('MongoClient connect failed');
console.log(err);
}
db = database;
});
exports.Connect = function (callback) {
callback(db);
};
exports.MongoClient = MongoClient;
exports.ObjectID = mongodb.ObjectID;
To use it
var Connect = require('myMongo').Connect;
Connect(function(db){
// then use your db.collection() stuff here
})
Related
I'm working on a project that will be a multi-tenant Saas application, and am having difficulty implementing a way to log into various databases depending on the user login info. Right now, I just want to split traffic between a Sandbox database (for demo purposes, and will be wiped on a regular basis), and an Alpha database (for current client testing and development). I have written the middleware below, config.js, that detects the user ID on login and assigns a database object using mongoose.createConnection(). This key-value pair is then added to a store using memory-cache. Here is the config.js code:
var mcache = require('memory-cache'),
Promise = require("bluebird"),
mongoose = require('mongoose');
Promise.promisifyAll(require("mongoose"));
(function () {
'use strict';
var dbSand = mongoose.createConnection(process.env.DB_SAND);
var dbAlpha = mongoose.createConnection(process.env.DB_ALPHA);
function dbPathConfigMiddlewareWrapper (){
return function setDbPath(req, res, next){
if ( req ){
if (!mcache.get(req.session.id) && req.body.email){
var login = req.body.email;
if (login === 'demo#mysite.com'){
mcache.put(req.session.id, dbSand);
} else {
mcache.put(req.session.id, dbAlpha);
}
}
req.dbPath = mcache.get(req.session.id);
next();
}
};
}
module.exports = dbPathConfigMiddlewareWrapper;
}());
So far so good. But I have been unsuccessful in calling the correct database in my routes. When I was just using a single database, I could easily use this:
var connStr = process.env.DBPATH;
if(mongoose.connection.readyState === 0){
mongoose.connect(connStr, function(err) {
if (err) throw err;
console.log('Successfully connected to MongoDB');
});
}
Now, I'm trying this to no avail:
var connStr = req.dbPath; //where req.dbPath is assigned in the config middleware above.
if(connStr.connection.readyState === 0){
mongoose.connect(req.dbPath, function(err) {
if (err) throw err;
console.log('Successfully connected to MongoDB');
});
}
Any guidance here would be greatly appreciated. This seems like it should be much more straightforward, and the documentation alludes to it but does not elaborate.
Here, I think, the problem is you are saving a database object to your key value storage. mcache.put(req.session.id, dbSand);. Which caused error in if(connStr.connection.readyState === 0).
You can stringify your object. mcache.put(req.session.id, JSON.stringify(dbSand));. And get the object's string and parse it into JSON like var connStr = JSON.parse(req.dbPath);.
You don't call mongoose.connect() if you're manually creating connections.
Instead, you have to register your models for each connection, which is a bit of a PITA but as far as I know there's no way around that. It may require some restructuring of your code.
Here's some untested code on how you could set something like that up.
Your middleware file:
// Create connections
const registerModels = require('./register-models');
let dbSand = mongoose.createConnection(process.env.DB_SAND);
let dbAlpha = mongoose.createConnection(process.env.DB_ALPHA);
// Register your models for each connection.
registerModels(dbSand);
registerModels(dbAlpha);
function dbPathConfigMiddlewareWrapper() { ... }
register-models.js:
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
let UserSchema = new mongoose.Schema(...);
module.exports = function(conn) {
conn.model('User', UserSchema);
};
This does mean that you can't use User.find(...) in your routes, because that only works when you're using a single connection (the default one that gets created with mongoose.connect(), which you're not using).
Instead, you should use something like this in your routes:
req.dbPath.model('User').find(...);
I am creating an API with Node.js and Express. I'm using Postgres as DB.
I would like to create a "global object", called DBConn or something, that I can access from everywhere in the App. This object would have the functions for inserting, updating, validating, etc.
How would be the general architecture in Node and Express for this to work? Does it make sense in Node to instantiate it just once and keep the communication with the DB open, or should I instantiate it everytime I want to perform a DB action?
Here's everything that you are looking for, using pg-promise:
// Users repository:
var repUsers = function (obj) {
return {
add: function (name) {
return obj.none("insert into users(name) values($1)", name);
},
delete: function (id) {
return obj.none("delete from users where id=$1", id);
}
// etc...
};
};
var options = {
extend: function () {
// extending the protocol:
this.users = repUsers(this);
}
};
var pgp = require('pg-promise')(options);
var cn = "postgres://username:password#host:port/database";
var db = pgp(cn); // your global database instance;
db.users.add("John")
.then(function () {
// success;
})
.catch(function (error) {
// error;
});
This will also manage your database connection automatically, you will just keep using variable db throughout your application.
And setting up a repository is optional, you can always use in-line queries instead. See the library for details and more examples.
I don't know Postgres at all,but maybe you can try this:
Create a file named 'DBConn.js' in YourApp/common directory.
DBConn.js:
var DBConn = exports = modules.exports = {}
//code to connect to database
....
//insert update detele select
DBConn.insert = function(arguments) {
//some code
}
.....
DBConn.update = function(arguments) {
//some code
}
Then you can require it in any other controller like YouApp/controller/UserController.js
UserController.js:
var DBConn = require('../common/DBConn.js')
Example of module cache
index.js:
require('./DB.js');
require('./DB.js');
DB.js
var DB = exports = module.exports = {}
function connect() {
//connect to database code
console.log('Connect!');
}
connect();
//other code
Then node index.js,we can see that 'Connect!' only logged once.
Because when we first require('DB.js'),node.js put it to the module cache,and when we require DB.js again,we get DB.js from cache.
I would like to connect to mongodb first, then run everything else in my application.
To do it I have to write something like:
MongoClient.connect("mongodb://localhost/test", function(err, connection) {
if (err) { console.error(err); }
db = connection;
var app = express();
// Include API V1
require("./apiv1.js")(app, db);
app.listen(3000, function(err) {
if (err) { console.error(err); } else { console.log("Started on *:3000"); }
});
});
This makes my app to be completely indented inside the .connect function... Which looks ugly and takes space while I work on my project.
I think the best solution would be have the MongoDB connection synchronous (even because witout the DB connection my app cannot work so why should I do something while it's connecting?) and then run the rest of my code.
How can I do?
You can't connect to MongoDB synchronously, but you may get rid of this ugly callback from your code.
The best way to do it is to adopt some wrapper around node-mongodb-native driver.
Take a look at the following modules.
mongojs
var mongojs = require('mongojs');
var db = mongojs('localhost/test');
var mycollection = db.collection('mycollection');
mongoskin
var mongo = require('mongoskin');
var db = mongo.db("mongodb://localhost:27017/test", {native_parser:true});
monk
var monk = require('monk');
var db = monk('localhost/test');
var users = db.get('users')
Of course, internally all of them are establishing MongoDB connection asynchronously.
Using the async library, you can aleve some of these issues.
For example in my server startup I do the following :
async.series([
function(callback){
// Initialize the mongodb connection and callback on completion in init.
db.init(function(){
callback();
});
},
function(callback){
// Listen on requests etc.
webServer.init(function(){
callback();
});
},
function(callback){
// Set up anything else that I need
callback();
}
]);
If you are using Node 6 and up versions, you can do something like this:
const MongoClient = require('mongodb').MongoClient;
const url = 'mongodb://localhost:27017/mydb';
let db = null;
getdb();
//your code
async function getdb() {
db = await MongoClient.connect(url);
}
Bring the mongodb library.
Declare the url constant .
Declare the variable db as null.
Call the getdb function.
Create the getdb function which has firt the async word
Assign to the db variable the result of the connection with the key word await.
You can do it with thunky, thunky executes an async function once and caches it, the subsequent calls are returned from the cache.
const MongoClient = require('mongodb').MongoClient;
const thunky = require('thunky');
var connect = thunky(function(cb){
let url = 'mongodb://localhost:27017/test';
MongoClient.connect(url, function(err, client){
console.log('connecting')
cb(err, client);
})
})
connect( (err, client) => {
console.log('connection 1')
})
connect( (err, client) => {
console.log('connection 2')
})
connect( (err, client) => {
console.log('connection 3')
console.log('closing')
client.close();
})
*Note: I am using latest 3.x mongodb driver
Context
If I have say 2 classes e.g. User.js class and Log.js class which both access a database in their own unique methods, and I have installed a MySQL database module e.g. db-mysql.
Question
How can I make it so the same (one) database instance can be used across both JS files?
Thoughts
The two methods I can think of at the moment are not very memory conscious I guess:
pass db parameter in
function(db){
this.db = db;
}
create an instance of it inside every class
function(){
this.db = require(moduleName);
}
I’m just looking for the best way and need a bit of guidance.
Create a separate file where you connect to db. You keep that connection in that modules' closure, and when you later require that module, from any other file, it will use that same connection.
The simple example is something like this: lib/db.js
var mysql = require('db-mysql');
var settings = { // import these from ENV is a good pattern
hostname: 'localhost'
, user: 'user'
, pw: '****'
, database: 'base'
}
var db =new mysql.Database(settings).on('error', function(error) {
console.log('ERROR: ' + error);
}).on('ready', function(server) {
console.log('Connected to ' + server.hostname + ' (' + server.version + ')');
}).connect();
module.exports = db;
Then use this in other files:
var db = require('../lib/db');
db.query(...)
You can even abstract some basic queries in the db.js, something like:
function getUsers(filterString, callback) {
db.query().select('id, username, email').from('users').where('username like', filterString)
.execute(callback);
}
module.exports.getUsers = getUsers
Then in other files:
var db = require('lib/db');
db.getUsers('mike', function(err, rows, cols) {
if(err) throw err;
return [rows, cols];
});
Pass the DB Parameter in.
Also create a new JS file called DBConfig.js where you can store the credentials for the MySQL DB. Use this javascript object to initiate your db. For Ex:
var db = require('DBConfig);
Inside your DBConfig.js, you can write
module.exports = {
host:'<host_url>',
username: root,
password: '',
database: '<database-name>'
}
In this manner you can use the same config accross the JS files.
Hi is there a way to switch database with mongoose?
I thought I could do it like that:
mongoose.disconnect();
mongoose.connect('localhost',db);
but it does not work I receive this error:
Error: Trying to open unclosed connection.
I do not know if it is because is asynchronous
As already stated you could do it using useDb function :
Example code :
async function myDbConnection() {
const url = 'mongodb+srv://username:password#cluster0-pauvx.mongodb.net/test?retryWrites=true&w=majority';
try {
await mongoose.connect(url, { useNewUrlParser: true });
console.log('Connected Successfully')
// Here from above url you've already got connected to test DB,
So let's make a switch as needed.
mongoose.connection.useDb('myDB'); // Switching happens here..
/**
* Do some DB transaction with mongoose models as by now models has already been registered to created DB connection
*/
} catch (error) {
console.log('Error connecting to DB ::', error);
}
}
Or if you wanted to create a complete new connection then you've to try mongoose.createConnection(). Just for reference in case of mongoDB driver you would use ::
mongodb.MongoClient.connect(mongourl, function(err, primaryDB) {
// open another database over the same connection
const secondaryDB = primaryDB.db(SECONDARY_DATABASE_NAME);
// now you can use both `database` and `database2`
...
});
Ref : mongoose multiple different connections, mongoose useDb(), mongoDB driver switch connections
It is asynchronous. If you pass a callback function to disconnect and try to connect to the next database in that callback, it will work.
Ex.
var mongoose = require('mongoose')
mongoose.connect('mongodb://localhost/test1', function() {
console.log('Connected to test 1')
mongoose.disconnect(connectToTest2)
})
function connectToTest2() {
mongoose.connect('mongodb://localhost/test2', function() {
console.log('Connected to test 2')
process.exit()
})
}
The top-voted answer had thrown me in a loop for a few hours. To help the OP and others who may have the same issue, here's my take on the solution:
Suppose you have two databases with the same schema and you want to switch between them on the fly. Let's call them DB1 and DB2. Here's how you can go about doing that:
(async () => {
const connection = await mongoose.createConnection(url, options);
const getModel = (database) => {
const dbConnection = connection.useDb(database);
return dbConnection.model('Model', modelSchema);
};
const Db1Model = getModel('DB1');
const Db2Model = getModel('DB2');
})();
Tested on Node.js v12 and Mongoose v5.
One way you can achieve this is by appending the database name with the database URL. For eg: if you are working with localhost
mongoose.connect('mongodb://localhost:portNumber/xyz_db');
When you connect like this, all your models will be saved in the xyz_db under your model as a collection.
You should use the useDb function like so:
mongoose.connect(process.env.MONGO_URI, {
useNewUrlParser: true,
useUnifiedTopology: true
});
mongoose.connection.useDb('Users'); # Change the string to the name of the database you want to use