I am working on nodejs mongoose for connecting to MongoDB, but I need 2 different connections to 2 different MongoDBs sitting on two different locations, what I did is as below :
global.otherDB = mongooseGlobal.createConnection(uri_1/DB1, { db: { safe: true });
global.mongoose = mongooseGlobal.connect(uri_2/DB2, { db: { safe: true });
Now where ever I need to connect to uri_2/DB2, I use mongoose and where ever I need to connect to uri_1/DB1, I use otherDB, now here is what is happening surprisingly,
Whenever I try to write or access DB1, it is creating or accessing collections available at uri_2/DB1 instead of uri_1/DB1, which is very surprising.
I didn't have any path of uri_2/DB1 while creating any mongo connection, how is it possible that DB1 is getting created at uri_2 instead of uri_1
Please give inputs.
Related
I was trying to understand how to use the passport.js in Mongodb native driver,Express now the problem is that all the reference or tutorials are showing the LOCAL-STRATEGY by using mongoose which creates a schema or model........so now iam STUCK
Take a look at the mongodb documentation for their Nodejs driver.
mongoDB Node Driver
Sorry for being here for a little bit late, but maybe my answer would be helpful to others who seeking answer for this kind of question.
I assume that you were struggling with these problems:
How to reuse database connection among your project
You can define your MongoClient object once and reuse this across multiple modules in your project as follow:
dbUtil.js hold definition of MongoClient object:
const MongoClient = require('mongodb').MongoClient;
const SERVER_URI = // link to your server, for example: 'http://localhost:27017';
const DB_NAME = // database name, for example: 'test';
/* #WHY?:
(option 1?) : to let the server assign objectId instead of Node driver,
(option 2 & 3?) : to get rid of deprecation warnings
*/
const clientObj = new MongoClient(`${SERVER_URI}/${DB_NAME}`, {
forceServerObjectId: true,
useNewUrlParser: true,
useUnifiedTopology: true
});
module.exports = {
client: clientObj,
dbName: DB_NAME
}
In another module where you need to use the defined connection:
const { client, dbName } = require('dbUtil');
// Because client.connect() return a Promise, you should wrap everything
// inside an immediately-invoked expression like this
(async () => {
await client.connect(); // at first you need to open the connection client
const dbO = await client.db(dbName); // get the connection to database
/* perform database operations, for example:
dbO.collection(users).insertOne({ name:'mongoadmin' });
*/
client.close(); // remember to close the connection when you're done
})();
So instead of the Mongoose way of using User.find().exec(), in Mongo native driver you have to activate connection to Client first and then use client.dbO.collection('users') (which return a Promise).
What the heck is Passport and why it's needed for your project
Passport is authentication middleware for Express that support authentication from Facebook, Google, JWT,... and many other authentication strategies. It can be helpful when you need to you want to support authentication from multiple authentication portal. However, it's not a must-have.
Sometimes applying another layer of abstraction from third-party libraries not only bring no sense to you & your project, but also over-complicate your existed code base. You'd chose not to use Mongoose and adapted MongoDb native driver instead, stated that you didn't need schema & model stuffs. For the same logic, I don't see any necessity of adapting Passport. This link can be helpful to you in some way: another Stackoverflow post
To apply authentication using JSON web token to your Express routes, you need to do these following steps:
Generate token for user signed in
Verify token
Define protected routes and write middlewares for those
All these tasks can be done without any third-party modules/libraries!
I believe your question stems from using mongodb schema validation instead of mongoose schema. You can use another means of authentication like JWT which does not directly need models for its authentication.
I am trying to implement a feature which a user can decide on login to which DB to connect. As it is a web-app, running on a server which all the clients approach, how can I implement this feature without changing every client DB?
At our company we are using mongoose as the MongoDB API.
I read all the docs, and didn't notice any functionality for using multiple connections to different DB's on different hosts within the same App at once - without damaging other's client work.
The most valuable thing I have accomplished is to open few connections based on multiple mongoose instances, based on this post:
Mongoose and multiple database in single node.js project
I have created few files for example:
var mongoose = require('mongoose');
mongoose.createConnection('mongodb://10.20.100.71:27017/DB_NAME');
module.exports = exports = mongoose;
And then I required them:
let stageAccess = require('./databsesConnections/stageAccess');
let prodAccess = require('./databsesConnections/prodAccess');
I debugged the files and checked the connections are establishing.
Further more I checked in the mongoose docs and concluded that I can choose which connection is the default connection, as the docs state:
"Mongoose creates a default connection when you call mongoose.connect(). You can access the default connection using mongoose.connection."
So I tried:
mongoose.connection = mongoose.connections[1];
And it works fine.
So the actual question is, what will happen if client 1 approach the app, select to connect dbNum1 and starts to work,
then client 2 approach the app and select to connect to dbNum2?
I have a new sandbox cluster on MongoDB Atlas that I am connecting to with mongoose on a Node.js server. This is the code I'm using to connect:
const mongoDbUrl =
`mongodb+srv://${username}:${password}#mydb-sandbox-12345.mongodb.net/testdb`
mongoose.connect(mongoDbUrl)
const connection = mongoose.connection
connection.on('connected', () => {
console.log('Connected to mongodb')
})
In the Atlas dashboard I have a readWriteAnyDatabase user that I am authenticating with. Authentication works as expected. I am able to connect to the database and no error is thrown on the connection. I can confirm this by removing a character in the password - authentication fails and I'm unable to connect.
The problem is when I try to insert documents.
const UserModel = require('./models/User')
UserModel.create({
name: 'Hello Atlas'
})
I get the following error:
MongoError: not authorized on admin to execute command {
insert: "users",
documents: [
[{
name Hello Atlas
} {
_id ObjectIdHex("5aa17933d72d25730a340611")
} {
__v 0
}]
],
ordered: false
}
As far as I know the user I'm authenticating with should have permission to read and write on the database I'm connecting to. The other part I don't understand is that the error shows that it's trying to write to admin even though my url is connecting to testdb.
Not sure if you have seen this post, but it could be because you are on a free cluster? Hope this helps.
UPDATE
I looked into the problem further and reproduced it on my own. I got the same error. However, I noticed that at one point Atlas provided me with a choice of connection strings. I went back to that page and chose I am using driver 3.4 or earlier.
The connection string looks like this:
const mongoDbUrl = `mongodb://${username}:${password}#cluster0-shard-00-00-1wntz.mongodb.net:27017,cluster0-shard-00-01-1wntz.mongodb.net:27017,cluster0-shard-00-02-1wntz.mongodb.net:27017/testdb?ssl=true&replicaSet=Cluster0-shard-0&authSource=admin`;
It worked with that connection string.
It looks like the free version of MongoDB Atlas launches with v3.4
If you are using free cluster. change 'admin' to 'test' in the path:
mongodb+srv://username:password#cluster0-yauj8.mongodb.net/test?retryWrites=true&w=majority
This worked for me
Going off #rithesh-mys 's answer. Replacing it with "test" is specifying the db. So you have to change it to the db name that you would use.
I had the same problem. I was trying a lot of connection strings, but the one for olders mongo shell (3.4 or earlier) worked for me.
mongodb://my_username:my_password#cluster0-shard-00-00.osobw.mongodb.net:27017,cluster0-shard-00-01.osobw.mongodb.net:27017,cluster0-shard-00-02.osobw.mongodb.net:27017/my_database?ssl=true&replicaSet=atlas-xw3rfy-shard-0&authSource=admin
I think that newer versions of connection strings don't work with mongoose, at least with free clusters.
Make sure the user you have created has the write and read privileges.
I have a user with the role: {role: 'root', db:'admin'} which should have access to all the databases in my mongo instance. I'm using mongoskin in node.js to interact with mongo. My problem is that it isn't correctly accessing my databases. If I authenticate with
mongodb://owner:mylocalpassword#localhost:27017/mydatabase
It simply gives me MongoError: Authentication failed. If I instead auth with:
mongodb://owner:mylocalpassword#localhost:27017/admin
it authenticates, but I can't access mydatabase.
Here's my connection code:
var connection = mongoskin.db("mongodb://owner:mylocalpassword#localhost:27017/admin", {journal:true, auto_reconnect:true})
I assume that since I'm accessing the admin database there, that's the only one it interacts with. So I tried do then do this:
var mydatabaseConnection = connection.db('mydatabase')
But when I use that, my app is returning no results for queries on collections that I know have data. What am I doing wrong here? How do I get this user to access a database other than admin?
Ok, so I found out that mongoskin's db method simply doesn't work. Finally I'm forced to completely remove mongoskin from my codebase. The real answer here is don't use mongoskin.
This code worked with mongo native:
MongoClient.connect("mongodb://owner:mylocalpassword#localhost:27017/admin", {journal: true, auto_reconnect:true}).then(function(db) {
console.log("Connected!")
var mydb = db.db('mydatabase')
var User = mydb.collection('User')
return User.find({}).toArray().then(function(users) {
console.log(users)
db.close()
})
}).catch(function(e) {
console.log(e.stack)
})
I am using mongodb-native-driver in express.js app. I have around 6 collections in the database, so I have created 6 js files with each having a collection as a javascript object (e.g function collection(){}) and the prototypes functions handling all the manipulation on those collections. I thought this would be a good architecture.
But the problem I am having is how to connect to the database? Should I create a connection in each of this files and use them? I think that would be an overkill as the connect in mongodb-native-driver creates a pool of connections and having several of them would not be justified.
So how do I create a single connection pool and use it in all the collections.js files? I want to have the connection like its implemented in mongoose. Let me know if any of my thought process in architecture of the app is wrong.
Using Mongoose would solve these problems, but I have read in several places thats it slower than native-driver and also I would prefer a schema-less models.
Edit: I created a module out of models. Each collection was in a file and it took the database as an argument. Now in the index.js file I called the database connection and kept a variable db after I got the database from the connection. (I used the auto-reconnect feature to make sure that the connection wasn't lost). In the same index.js file I exported each of the collections like this
exports.model1 = require('./model1').(db)
exprorts.model2 = require('./model2').(db)
This ensured that the database part was handled in just one module and the app would just call function that each model.js file exported like save(), fincdbyid() etc (whatever you do in the function is upto you to implement).
how to connect to the database?
In order to connect using the MongoDB native driver you need to do something like the following:
var util = require('util');
var mongodb = require('mongodb');
var client = mongodb.MongoClient;
var auth = {
user: 'username',
pass: 'password',
host: 'hostname',
port: 1337,
name: 'databaseName'
};
var uri = util.format('mongodb://%s:%s#%s:%d/%s',
auth.user, auth.pass, auth.host, auth.port, auth.name);
/** Connect to the Mongo database at the URI using the client */
client.connect(uri, { auto_reconnect: true }, function (err, database) {
if (err) throw err;
else if (!database) console.log('Unknown error connecting to database');
else {
console.log('Connected to MongoDB database server at:');
console.log('\n\t%s\n', uri);
// Create or access collections, etc here using the database object
}
});
A basic connection is setup like this. This is all I can give you going on just the basic description of what you want. Post up some code you've got so far to get more specific help.
Should I create a connection in each of this files and use them?
No.
So how do I create a single connection pool and use it in all the collections.js files?
You can create a single file with code like the above, lets call it dbmanager.js connecting to the database. Export functions like createUser, deleteUser, etc. which operate on your database, then export functionality like so:
module.exports = {
createUser: function () { ; },
deleteUser: function () { ; }
};
which you could then require from another file like so:
var dbman = require('./dbmanager');
dbman.createUser(userData); // using connection established in `dbmanager.js`
EDIT: Because we're dealing with JavaScript and a single thread, the native driver indeed automatically handles connection pooling for you. You can look for this in the StackOverflow links below for more confirmation of this. The OP does state this in the question as well. This means that client.connect should be called only once by an instance of your server. After the database object is successfully retrieved from a call to client.connect, that database object should be reused throughout the entire instance of your app. This is easily accomplished by using the module pattern that Node.JS provides.
My suggestion is to create a module or set of modules which serves as a single point of contact for interacting with the database. In my apps I usually have a single module which depends on the native driver, calling require('mongodb'). All other modules in my app will not directly access the database, but instead all manipulations must be coordinated by this database module.
This encapsulates all of the code dealing with the native driver into a single module or set of modules. The OP seems to think there is a problem with the simple code example I've posted, describing a problem with a "single large closure" in my example. This is all pretty basic stuff, so I'm adding clarification as to the basic architecture at work here, but I still do not feel the need to change any code.
The OP also seems to think that multiple connections could possibly be made here. This is not possible with this setup. If you created a module like I suggest above then the first time require('./dbmanager') is called it will execute the code in the file dbmanager.js and return the module.exports object. The exports object is cached and is also returned on each subsequent call to require('./dbmanager'), however, the code in dbmanager.js will only be executed the first require.
If you don't want to create a module like this then the other option would be to export only the database passed to the callback for client.connect and use it directly in different places throughout your app. I recommend against this however, regardless of the OPs concerns.
Similar, possibly duplicate Stackoverflow questions, among others:
How to manage mongodb connections in nodejs webapp
Node.JS and MongoDB, reusing the DB object
Node.JS - What is the right way to deal with MongoDB connections
As accepted answer says - you should create only one connection for all incoming requests and reuse it, but answer is missing solution, that will create and cache connection. I wrote express middleware to achieve this - express-mongo-db. At first sight this task is trivial, and most people use this kind of code:
var db;
function createConnection(req, res, next) {
if (db) { req.db = db; next(); }
client.connect(uri, { auto_reconnect: true }, function (err, database) {
req.db = db = databse;
next();
});
}
app.use(createConnection);
But this code lead you to connection-leak, when multiple request arrives at the same time, and db is undefined. express-mongo-db solving this by holding incoming clients and calling connect only once, when module is required (not when first request arrives).
Hope you find it useful.
I just thought I would add in my own method of MongoDB connection for others interested or having problems with different methods
This method assumes you don't need authentication(I use this on localhost)
Authentication is still easy to implement
var MongoClient = require('mongodb').MongoClient;
var Server = require('mongodb').Server;
var client = new MongoClient(new Server('localhost',27017,{
socketOptions: {connectTimeoutMS: 500},
poolSize:5,
auto_reconnect:true
}, {
numberOfRetries:3,
retryMilliseconds: 500
}));
client.open(function(err, client) {
if(err) {
console.log("Connection Failed Via Client Object.");
} else {
var db = client.db("theDbName");
if(db) {
console.log("Connected Via Client Object . . .");
db.logout(function(err,result) {
if(!err) {
console.log("Logged out successfully");
}
client.close();
console.log("Connection closed");
});
}
}
});
Credit goes to Brad Davley which goes over this method in his book (page 231-232)