I'm writing a Node module which whould perform queries to MongoDB.
My module should take as parameter (when I init it with new MyModule(db)) the MongoDB connection and use it inside it.
I was using standard MongoDB without any NPM module and I was passing in my db variable the connection to MongoDB. Now I'm switching to Mongoose and I can't find a way to pass the Mongoose connection to my module.
I don't want to initialize the Mongoose connection inside my module because I want to share it with my tests and with other modules.
How can I do? I've tried passing mongoose to my module but it doesn't work "is not a function".
Edit:
After reading the reply of #Neil Lunn I'm posting this sample of my module:
(function () {
"use strict";
/**
* various requires
*/
function TopicManager(dbURI) {
if (!(this instanceof TopicManager)) { return new TopicManager(dbURI); }
mongoose.connect(dbURI);
}
TopicManager.prototype.save = function (topics, done) {
var Topic = new mongoose.schema(
{
title: { type: String },
slug: { type: String, index: { unique: true } }
},
{collection : "topics"}
);
/**
* Use monguurl on "slug"
*/
mongoose.update(
{title: topic.title},
{upsert: true},
function (err, numberAffected, raw) {
if (err) { return done(err); }
return done(null, raw);
}
);
};
module.exports = TopicManager;
})();
It doesn't work because when I use it I get undefined is not a function when it runs new mongoose.
Generally speaking you don't do this. The mindset is a little different with mongoose than working with the native driver in it's raw form, and there are plenty of things helping under the hood to make things work a bit more seamlessly without diving into the gory details.
So the basic approach is when you define your "schema" and "models", these are bound to the default connection instance. Unless you have a specific reason for binding to another connection, this is what you should follow:
So you would have a Schema and model definition:
var mySchema = new Schema({
"name": String
});
module.exports = mongoose.model( "Model", mySchema, "collection" )
Where the "collection" part is optional otherwise the "model" name in the first argument is put to standard rules, usually lowercase and pluralized.
Then in your other code listing, you pull this in with require:
var Model = require('./models/mymodel.js');
And use your model objects as mongoose permits:
Model.find({ "field": "name"}, function(err,models) {
});
So it allows a bit more abstract handling than is done with the basic driver as the "models" themselves know how to bind to connections, or are otherwise explicitly bound to the connection you want as an optional parameter.
You can create a default Mongoose connection:
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost:27017/test');
var db = mongoose.connection;
If you want to open multiple connections you can use createConnection:
var dbconnection = mongoose.createConnection ('uri,uri');
This connection object can then be used for creating/retrieving models that are scoped only to this connection.
The Mongoose connection object has multiple events on which you attach handlers. You should check out the documentation for the full list of handlers you can use.
To get you started, a basic approach would be:
// log connection errors
db.on('error', console.error.bind(console, 'connection error:'));
// wait for opening the connection
db.once('open', function () {
// do something once connection to the MongoDB is open
});
If you open your connection using createConnection, you need to use it when creating models:
// dbconnection is created using createConnection
var MyModel = dbconnection.model('MyModel', MyModelSchema);
Related
I'm doing a Node.js project that contains sub projects. One sub project will have one Mongodb database and Mongoose will be use for wrapping and querying db. But the problem is
Mongoose doesn't allow to use multiple databases in single mongoose instance as the models are build on one connection.
To use multiple mongoose instances, Node.js doesn't allow multiple module instances as it has caching system in require(). I know disable module caching in Node.js but I think it is not the good solution as it is only need for mongoose.
I've tried to use createConnection() and openSet() in mongoose, but it was not the solution.
I've tried to deep copy the mongoose instance (http://blog.imaginea.com/deep-copy-in-javascript/) to pass new mongoose instances to the sub project, but it throwing RangeError: Maximum call stack size exceeded.
I want to know is there anyways to use multiple database with mongoose or any workaround for this problem? Because I think mongoose is quite easy and fast. Or any other modules as recommendations?
According to the fine manual, createConnection() can be used to connect to multiple databases.
However, you need to create separate models for each connection/database:
var conn = mongoose.createConnection('mongodb://localhost/testA');
var conn2 = mongoose.createConnection('mongodb://localhost/testB');
// stored in 'testA' database
var ModelA = conn.model('Model', new mongoose.Schema({
title : { type : String, default : 'model in testA database' }
}));
// stored in 'testB' database
var ModelB = conn2.model('Model', new mongoose.Schema({
title : { type : String, default : 'model in testB database' }
}));
I'm pretty sure that you can share the schema between them, but you have to check to make sure.
Pretty late but this might help someone. The current answers assumes you are using the same file for your connections and models.
In real life, there is a high chance that you are splitting your models into different files. You can use something like this in your main file:
mongoose.connect('mongodb://localhost/default');
const db = mongoose.connection;
db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', () => {
console.log('connected');
});
which is just how it is described in the docs. And then in your model files, do something like the following:
import mongoose, { Schema } from 'mongoose';
const userInfoSchema = new Schema({
createdAt: {
type: Date,
required: true,
default: new Date(),
},
// ...other fields
});
const myDB = mongoose.connection.useDb('myDB');
const UserInfo = myDB.model('userInfo', userInfoSchema);
export default UserInfo;
Where myDB is your database name.
One thing you can do is, you might have subfolders for each projects. So, install mongoose in that subfolders and require() mongoose from own folders in each sub applications. Not from the project root or from global. So one sub project, one mongoose installation and one mongoose instance.
-app_root/
--foo_app/
---db_access.js
---foo_db_connect.js
---node_modules/
----mongoose/
--bar_app/
---db_access.js
---bar_db_connect.js
---node_modules/
----mongoose/
In foo_db_connect.js
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/foo_db');
module.exports = exports = mongoose;
In bar_db_connect.js
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/bar_db');
module.exports = exports = mongoose;
In db_access.js files
var mongoose = require("./foo_db_connect.js"); // bar_db_connect.js for bar app
Now, you can access multiple databases with mongoose.
As an alternative approach, Mongoose does export a constructor for a new instance on the default instance. So something like this is possible.
var Mongoose = require('mongoose').Mongoose;
var instance1 = new Mongoose();
instance1.connect('foo');
var instance2 = new Mongoose();
instance2.connect('bar');
This is very useful when working with separate data sources, and also when you want to have a separate database context for each user or request. You will need to be careful, as it is possible to create a LOT of connections when doing this. Make sure to call disconnect() when instances are not needed, and also to limit the pool size created by each instance.
Mongoose and multiple database in single node.js project
use useDb to solve this issue
example
//product databse
const myDB = mongoose.connection.useDb('product');
module.exports = myDB.model("Snack", snackSchema);
//user databse
const myDB = mongoose.connection.useDb('user');
module.exports = myDB.model("User", userSchema);
A bit optimized(for me atleast) solution. write this to a file db.js and require this to wherever required and call it with a function call and you are good to go.
const MongoClient = require('mongodb').MongoClient;
async function getConnections(url,db){
return new Promise((resolve,reject)=>{
MongoClient.connect(url, { useUnifiedTopology: true },function(err, client) {
if(err) { console.error(err)
resolve(false);
}
else{
resolve(client.db(db));
}
})
});
}
module.exports = async function(){
let dbs = [];
dbs['db1'] = await getConnections('mongodb://localhost:27017/','db1');
dbs['db2'] = await getConnections('mongodb://localhost:27017/','db2');
return dbs;
};
I have been using this method and it works great for me until now.
const mongoose = require('mongoose');
function makeNewConnection(uri) {
const db = mongoose.createConnection(uri, {
useNewUrlParser: true,
useUnifiedTopology: true
});
db.on('error', function (error) {
console.log(`MongoDB :: connection ${this.name} ${JSON.stringify(error)}`);
db.close().catch(() => console.log(`MongoDB :: failed to close connection ${this.name}`));
});
db.on('connected', function () {
mongoose.set('debug', function (col, method, query, doc) {
console.log(`MongoDB :: ${this.conn.name} ${col}.${method}(${JSON.stringify(query)},${JSON.stringify(doc)})`);
});
console.log(`MongoDB :: connected ${this.name}`);
});
db.on('disconnected', function () {
console.log(`MongoDB :: disconnected ${this.name}`);
});
return db;
}
// Use
const db1 = makeNewConnection(MONGO_URI_DB1);
const db2 = makeNewConnection(MONGO_URI_DB2);
module.exports = {
db1,
db2
}
I'm doing a Node.js project that contains sub projects. One sub project will have one Mongodb database and Mongoose will be use for wrapping and querying db. But the problem is
Mongoose doesn't allow to use multiple databases in single mongoose instance as the models are build on one connection.
To use multiple mongoose instances, Node.js doesn't allow multiple module instances as it has caching system in require(). I know disable module caching in Node.js but I think it is not the good solution as it is only need for mongoose.
I've tried to use createConnection() and openSet() in mongoose, but it was not the solution.
I've tried to deep copy the mongoose instance (http://blog.imaginea.com/deep-copy-in-javascript/) to pass new mongoose instances to the sub project, but it throwing RangeError: Maximum call stack size exceeded.
I want to know is there anyways to use multiple database with mongoose or any workaround for this problem? Because I think mongoose is quite easy and fast. Or any other modules as recommendations?
According to the fine manual, createConnection() can be used to connect to multiple databases.
However, you need to create separate models for each connection/database:
var conn = mongoose.createConnection('mongodb://localhost/testA');
var conn2 = mongoose.createConnection('mongodb://localhost/testB');
// stored in 'testA' database
var ModelA = conn.model('Model', new mongoose.Schema({
title : { type : String, default : 'model in testA database' }
}));
// stored in 'testB' database
var ModelB = conn2.model('Model', new mongoose.Schema({
title : { type : String, default : 'model in testB database' }
}));
I'm pretty sure that you can share the schema between them, but you have to check to make sure.
Pretty late but this might help someone. The current answers assumes you are using the same file for your connections and models.
In real life, there is a high chance that you are splitting your models into different files. You can use something like this in your main file:
mongoose.connect('mongodb://localhost/default');
const db = mongoose.connection;
db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', () => {
console.log('connected');
});
which is just how it is described in the docs. And then in your model files, do something like the following:
import mongoose, { Schema } from 'mongoose';
const userInfoSchema = new Schema({
createdAt: {
type: Date,
required: true,
default: new Date(),
},
// ...other fields
});
const myDB = mongoose.connection.useDb('myDB');
const UserInfo = myDB.model('userInfo', userInfoSchema);
export default UserInfo;
Where myDB is your database name.
One thing you can do is, you might have subfolders for each projects. So, install mongoose in that subfolders and require() mongoose from own folders in each sub applications. Not from the project root or from global. So one sub project, one mongoose installation and one mongoose instance.
-app_root/
--foo_app/
---db_access.js
---foo_db_connect.js
---node_modules/
----mongoose/
--bar_app/
---db_access.js
---bar_db_connect.js
---node_modules/
----mongoose/
In foo_db_connect.js
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/foo_db');
module.exports = exports = mongoose;
In bar_db_connect.js
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/bar_db');
module.exports = exports = mongoose;
In db_access.js files
var mongoose = require("./foo_db_connect.js"); // bar_db_connect.js for bar app
Now, you can access multiple databases with mongoose.
As an alternative approach, Mongoose does export a constructor for a new instance on the default instance. So something like this is possible.
var Mongoose = require('mongoose').Mongoose;
var instance1 = new Mongoose();
instance1.connect('foo');
var instance2 = new Mongoose();
instance2.connect('bar');
This is very useful when working with separate data sources, and also when you want to have a separate database context for each user or request. You will need to be careful, as it is possible to create a LOT of connections when doing this. Make sure to call disconnect() when instances are not needed, and also to limit the pool size created by each instance.
Mongoose and multiple database in single node.js project
use useDb to solve this issue
example
//product databse
const myDB = mongoose.connection.useDb('product');
module.exports = myDB.model("Snack", snackSchema);
//user databse
const myDB = mongoose.connection.useDb('user');
module.exports = myDB.model("User", userSchema);
A bit optimized(for me atleast) solution. write this to a file db.js and require this to wherever required and call it with a function call and you are good to go.
const MongoClient = require('mongodb').MongoClient;
async function getConnections(url,db){
return new Promise((resolve,reject)=>{
MongoClient.connect(url, { useUnifiedTopology: true },function(err, client) {
if(err) { console.error(err)
resolve(false);
}
else{
resolve(client.db(db));
}
})
});
}
module.exports = async function(){
let dbs = [];
dbs['db1'] = await getConnections('mongodb://localhost:27017/','db1');
dbs['db2'] = await getConnections('mongodb://localhost:27017/','db2');
return dbs;
};
I have been using this method and it works great for me until now.
const mongoose = require('mongoose');
function makeNewConnection(uri) {
const db = mongoose.createConnection(uri, {
useNewUrlParser: true,
useUnifiedTopology: true
});
db.on('error', function (error) {
console.log(`MongoDB :: connection ${this.name} ${JSON.stringify(error)}`);
db.close().catch(() => console.log(`MongoDB :: failed to close connection ${this.name}`));
});
db.on('connected', function () {
mongoose.set('debug', function (col, method, query, doc) {
console.log(`MongoDB :: ${this.conn.name} ${col}.${method}(${JSON.stringify(query)},${JSON.stringify(doc)})`);
});
console.log(`MongoDB :: connected ${this.name}`);
});
db.on('disconnected', function () {
console.log(`MongoDB :: disconnected ${this.name}`);
});
return db;
}
// Use
const db1 = makeNewConnection(MONGO_URI_DB1);
const db2 = makeNewConnection(MONGO_URI_DB2);
module.exports = {
db1,
db2
}
So I have this schema:
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var TreeSchema = new Schema({
}, { collection: 'treeLocations' });
var TreeDetailsSchema = new Schema({
}, { collection: 'treeInfo' });
module.exports = mongoose.model('Tree', TreeSchema);
module.exports = mongoose.model('TreeDetail', TreeDetailsSchema, "treeInfo");
And I am calling by ID like this:
var TreeDetails = require('./app/models/tree').model('TreeDetail');
router.route('/api/trees/:tree_id')
.get(function(req, res) {
TreeDetails.findById(req.params.tree_id, function(err, treedetail) {
if (err)
res.send(err);
res.json(treedetail);
});
});
For some reason - http://localhost:3000/api/trees/5498517ab68ca1ede0612d0a which is a real tree, is returning null
Something that might help you help me:
I was following this tutorial: https://scotch.io/tutorials/build-a-restful-api-using-node-and-express-4
The only thing I can think of that changed is that I have a collection name. Might that be it?
The step that I don't see is how you actually connect to MongoDB and after that, how you get the Model from the connection.
// connect to MongoDB
var db = mongoose.createConnection('mongodb://user:pass#host:port/database');
// now obtain the model through db, which is the MongoDB connection
TreeDetails = db.model('TreeDetails');
This last step is how you associate your model with the connected mongo database.
More info on Mongoose.model
There are several ways to establish a connection to MongoDB with mongoose, the tutorial uses:
mongoose.connect('mongodb://node:node#novus.modulusmongo.net:27017/Iganiq8o');
(Personally I prefer the more explicit mongoose.createConnection as shown in the example)
(I used mongoose 4.3.1 for this example)
My steps to reproduce, in order to provide a working example (without creating a webservice for it):
var mongoose = require('mongoose'),
TreeDetails, db;
// create the model schema
mongoose.model('TreeDetails', mongoose.Schema({
// .. your field definitions
}, {collection: 'treeInfo'}));
db = mongoose.createConnection('mongodb://user:pass#host/example');
TreeDetails = db.model('TreeDetails');
TreeDetails.findById('5671ac9217fb1730bb69e8bd', function(error, document) {
if (error) {
throw new Error(error);
}
console.log(document);
});
Instead of:
var TreeDetails = require('./app/models/tree').model('TreeDetail');
try:
var mongoose = require('mongoose'),
TreeDetails = mongoose.model('TreeDetail');
Defining the collection name shouldn't give you any issues. It's just what the collection will be called in the database / when using the mongo shell.
And just to be sure, try logging req.params.tree_id before calling findById to make sure it's coming through as you suspect.
I have an Node.js Express web app running.
I have two database connections, the system_db is created with "Mongoose.connect", the user_db is created with "Mongoose.createConnection". This seems to make them separate and have different connection pools. Although this could be the crux of the problem.
I have code dealing with with a Mongoose model
in the model file I have:
var authSchema = mongoose.Schema({
teamName: String,
league: String,
players: []
});
var Team = module.exports = mongoose.model('teams',authSchema);
in my main file I have two connections:
one is a system database
connection_uri = "mongodb://localhost:27017/";
system_db = mongoose.connect(connection_uri.concat("sparks"), {
native_parser : true
}, function(err){
if (err) throw err;
});
the other is a user database
user_db = mongoose.createConnection(connection_uri.concat(user_id));
then I have this piece of code in my main file which finds a Team object based off id:
app.param('team_id', function(req, res, next, team_id) {
Team.findOne(team_id, function(err, team) {
if (err) {
return next(err);
}
if (!team) {
throw new Error("no team matched");
}
req.team = team;
next();
});
});
the problem is that the app.param function above is not finding any matches for teams, even though they exist in a collection in the user_db database. this means, that I am thinking my model is pointing to the wrong database somehow? Somehow it must be pointing to this system_db instead of the user_db. Could this be right? How would I fix that?
The below method opens a connection to the system_db and is binded to the mongoose object,.i.e the current mongoose object is modified. The return type is not a Connection.
system_db = mongoose.connect(connection_uri.concat("sparks")...
Now when you again do:
user_db = mongoose.createConnection(connection_uri.concat(user_id));
This creates a new Connection to the user database and returns a connection, but does not modify the mongoose instance. The mongoose instance is still binded to the system_db database connection.
Since Team model is obtained from the same mongoose instance,
var Team = mongoose.model('teams',authSchema);
whatever operation is done on this model effectively occurs in the connection that the mongoose instance holds, and that is of the system_db database.
So you need to obtain the model from the user_db connection :
var user_db = mongoose.createConnection(..);
// retrieve the Team model
var Team= user_db.model('teams'); // not from mongoose instance.
use createConnection wherever you want to obtain a connection.
I read an article in this link http://theholmesoffice.com/mongoose-and-node-js-tutorial/
here there is a code:
exports.teamlist = function(gname,callback){
db.once('open', function(){
var teamSchema = new mongoose.Schema({
country: String,
GroupName: String
});
var Team = db.model('Team', teamSchema);
Team.find({'GroupName':gname}, function (err, teams) {
if(err){
onErr(err,callback);
}else{
mongoose.connection.close();
console.log(teams);
callback("",teams);
}
})// end Team.find
});// end db.once open
};
Here it calls db.once method whereas in other places its used like this
var mongoose = require('mongoose')
,Schema = mongoose.Schema
,ObjectId = Schema.ObjectId;
var postSchema = new Schema({
thread: ObjectId,
date: {type: Date, default: Date.now},
author: {type: String, default: 'Anon'},
post: String
});
module.exports = mongoose.model('Post', postSchema);
In the router part its used like this
exports.show = (function(req, res) {
Thread.findOne({title: req.params.title}, function(error, thread) {
var posts = Post.find({thread: thread._id}, function(error, posts) {
res.send([{thread: thread, posts: posts}]);
});
})
});
And in the app.js there is
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/norum');
I dont understand why these two have different approach and which one is the better one and why? Can Anyone please help me. One thing that I have observed is that the second approach is the one most used. SO please help me as to which one is a better approach.
I know this is mainly concerned regarding creation of the schemes in Mongodb, and so the once method looks little better. But still I am not at all sure. Please help.
Basically, in the first approach - it mearly shows you what you can do with the mongoose. It presumes that somewhere you were opening a connection to the database (presumably a few string of code back). Then it listens to the event and does all the job later. It's only there to show you what you can do.
For intance, this creates a mongoose model (it doesnt rely on the database connection being open) - hence it could be included from a separate file with ease and generally be written as a module as it is in the second approach
var teamSchema = new mongoose.Schema({
country: String,
GroupName: String
});
var Team = db.model('Team', teamSchema);
Later on in the first approach you close the connection to the database, but thats not the thing you would want to do in the real application. You have a connection pool, and use it to frequently query the database, closing the connection is only needed when you plan to stop the application from running.
Team.find({'GroupName':gname}, function (err, teams) {
if(err){
onErr(err,callback);
}else{
mongoose.connection.close();
console.log(teams);
callback("",teams);
}
First line - you perform find on the database, if no err was returned - you close the connection, log the results, etc..
Anyway, to sum up - first approach is generally a show-off, second - is what its like in a real world - where you separate logic, make models reusable and includable, where you have router logic