Simplest way to create a connection and execute a query on replicaset with mongoose? - node.js

I'm trying to execute a specific query on replica set... but without success.
What is the simplest way to create a connection and execute a query on replicaset with mongoose?
My code-1:
var mongoose = require('mongoose');
mongoose.connect('mongodb://user:pass#host:port/db,user:pass#host2:port/db');
var someSchema = new mongoose.Schema({...}, { read: 'secondary' });
var some = mongoose.model('Some', someSchema);
some.find(...) -> err [Error: not authorized for query on bd.collection]
My code-2:
mongoose.connect('mongodb://user:pass#host:port/db,user:pass#host2:port/db?slaveOk=true');
some.find(...) -> err [Error: not authorized for query on bd.collection]
My code-3:
mongoose.connect('mongodb://user:pass#host:port/db,mongodb://user:pass#host2:port/db?slaveOk=true');
some.find(...) -> err [Error: not authorized for query on bd.collection]
My code-4:
mongoose.connect('mongodb://user:pass#end:port/db,mongodb://user:pass#end2:port/db);
some.find(...) -> err [Error: not authorized for query on bd.collection]
My code-5:
mongoose.connect('mongodb://user:pass#end:port/db,mongodb://user:pass#end2:port/db?slaveOk=true);
var someSchema = new mongoose.Schema({...});
var some = mongoose.model('Some', someSchema);
some.find(...).slaveOk() -> query executed on master (apparently)
some.find(...).read('secondary') -> query executed on master (apparently - no change on mongostat of replicaset. I'm mongoHQ user and I don't trust in mongostat from dashboard.)
SO, if I connect with mongo hostReplica:port/db -u user -p pass, and exec db.getMongo().setSlaveOk(), I can exec db.collection.find without problem.
What I'm doing wrong?
Thanks

Connecting to mongo client with readPreference option set to secondary:
mongodb://username:password#host:27017/database?readPreference=secondary
Not sure about slaveOk being deprecated, but readPreference always works.

I followed this doc and it works fine for me. My connection string for Mongoose looks like this:
mongodb://user:pass#host1,user:pass#host2/db?replicaSet=myReplica&readPreference=secondaryPreferred
I think that slaveOk is deprecated, and we should now use the readPreference option.
As I use Nodejs/Mongoose for a read-only api, I want to read from the slave(s), and keep the master for insertions mainly (which are done outside of Nodejs). That's why I use secondaryPreferred.

Related

MongoError - Authentication error (password with #)

Here is my code in my NodeJS application, to connect to my MongoDB engine :
const collection = 'mynewcollection';
const password = 'passwordwithan#';
const mongoUrl = `mongodb://admin:${encodeURIComponent(password)}#mymongobase.net/${collection}`;
// Connect using the connection string
MongoClient.connect(mongoUrl, {useNewUrlParser: true}, function(err, db) {
console.log(err.toString())
});
I get an authentication error. I tried several things to handle the '#' character and reading the documentation I thought that it was the good one...
But it is still failing even if the user and password are the good one.
Is the API correctly used ? Do you understand what is wrong ?
Thanks in advance.
OK I found the solution.
If you use :
const mongoUrl = `mongodb://admin:${encodeURIComponent(password)}#mymongobase.net/${collection}`;
Mongo will try to connect with admin/password defined on the collection.
If you don't put the collection in the mongo url :
const mongoUrl = `mongodb://admin:${encodeURIComponent(password)}#mymongobase.net`;
Then you can use the super user defined on all the mongo engine.

Mongodb Client Connection TypeError NULL issue

Working on a simple nodejs express app using Mongodb. I am getting a typeerror cannot read from null when I try to work with the connection object that is returned. The connection object isn't null so I am not clear on what the error message is actually trying to point out.
the line of code that is generating the error is:
const silos = conn.db('toolkit').collection('silos')
I have a debug console.log right before that of:
console.log("Connection: ",conn)
I am checking for an error on the connect callback and the error is null. Prior to this issue I had an issue with the connection and username/authentication so I know the error checking works as before it was triggering on bad logins.
The error is:
TypeError: Cannot read property 'db' of null
at mongoClient.connect (/var/projects/drake/Routes/Silos.js:22:28)
at err (/var/projects/drake/node_modules/mongodb/lib/utils.js:415:14)
at executeCallback (/var/projects/drake/node_modules/mongodb/lib/utils.js:404:25)
at executeOperation (/var/projects/drake/node_modules/mongodb/lib/utils.js:422:7)
at MongoClient.connect (/var/projects/drake/node_modules/mongodb/lib/mongo_client.js:168:10)
at getSilos (/var/projects/drake/Routes/Silos.js:18:17)
at Layer.handle [as handle_request] (/var/projects/drake/node_modules/express/lib/router/layer.js:95:5)
at next (/var/projects/drake/node_modules/express/lib/router/route.js:137:13)
at Route.dispatch (/var/projects/drake/node_modules/express/lib/router/route.js:112:3)
at Layer.handle [as handle_request] (/var/projects/drake/node_modules/express/lib/router/layer.js:95:5)
the console.log line generators this message:
Connection: MongoClient {
Here is the code that I use for the Mongo connection. This is in a file and the MongoClient object is exported that is used in other files to connect to the database.
const url = `mongodb://${process.env.MONGO_HOST}:${process.env.MONGO_PORT}`
const opts = {
useNewUrlParser: true,
authMechanism: process.env.MONGO_MECH,
authSource: process.env.MONGO_SRC,
auth: {
user: process.env.MONGO_USER,
password: process.env.MONGO_PWD
}
}
const mongoClient = require('mongodb').MongoClient;
const objectID = require('mongodb').ObjectID
const mongo = new mongoClient(url,opts)
module.exports.mongoClient = mongo
Here is the where I pull in that code and call the connect.
Importing the code:
const { mongoClient } = require('../mongo')
Using the imported code:
mongoClient.connect( (err, conn) => {
if (err) err(res,err, "Database Connection Error", 500)
console.log("Connection: ",conn)
const silos = conn.db('toolkit').collection('silos')
this last line is the one that gives the error.
Results of the console.log
Connection: MongoClient {
One common cause of this is when you try to interact with the mongodb server before a connection is actually made. I might be wrong but I suspect you might be calling your db operations in the wrong order. To avoid this, make sure you have actually successfully connected to the mongodb server before running queries on the collection. If you are using mongoose, then
mongoose.connect('/a/valid/mongo/uri')
will make
mongoose.connection
a valid object that can be interacted with.
Could you please update your question with your connection handling code?
No idea what this was about.
I appreciate the feedback that I got to this.
I spent a weekend rewriting the code from scratch and it worked this time. I did the same thing I believe. There must of been a typo that I was overlooking.
For what it's worth I want to share my experiences with this problem. I was using Mongoose to make the connection but it would crash on either a "replica set" or an "auth error". Eventually I decided to move to MongoClient, just like you did. This is because "Mongoose will not throw any errors by default if you use a model without connecting.".
Moving to Mongo the connection error also returned 'null'. I searched and searched and tried several things, including copying your code and adding the 'auth' option, etc. I eventually switched back to Mongoose (though that shouldn't matter for what solved it for me) and did this:
Create a new user with ReadWrite abilities (didn't help immediately but is definitely needed, since it gave an error when deploying to Heroku without the correct 'new' user.). I now have two users in my MongoDB account.
Add the following to my (Mongoose) Schema:
///Already had this:
let testSchema = new Schema {
foo: String,
bar: Number
}
///Added:
,{
bufferCommands: false,
autoCreate: false
});
Since the DB was still empty I added the following code after creating the model.
///Already had this:
let Test = mongoose.model('Test', testSchema);
///Added:
Test.createCollection();
I believe it then initiated the collection 'for real'. In the mongo shell I know that a collection is not visible until there is an item in there. Now that the app.js didn't crash upon starting I managed to add a document to the database using a form in my HTML and it worked. I now have a working DB connected to my app. Hope this could help anyone also landing on this page.

Mongoose with ReplicaSet on Atlas

I have a replica set on MongoDB Atlas and this is my mongo shell connection string which connects perfectly:
$ mongo "mongodb://MY_SERVER-shard-00-00-clv3h.mongodb.net:27017,MY_SERVER-shard-00-01-clv3h.mongodb.net:27017,MY_SERVER-shard-00-02-clv3h.mongodb.net:27017/MY_DATABASE?replicaSet=MY_REPLICASET-NAME-shard-0" --ssl --username MY_USERNAME --password MY_PASSWORD --authenticationDatabase MY_ADMIN_DATABASE
How Can I convert it to use in mongoose? How Can I build my uri and options variable?
I tried the following without success:
// connection string using mongoose:
var uri = 'mongodb://MY_USER:MY_PASSWORD#' +
'MY_SERVER-shard-00-00-clv3h.mongodb.net:27017,' +
'MY_SERVER-shard-00-01-clv3h.mongodb.net:27017,' +
'MY_SERVER-shard-00-02-clv3h.mongodb.net:27017/MY_DATABASE';
var options = {
replset: {
ssl: true,
authSource: 'MY_ADMIN_DATABASE',
rs_name: 'MY_REPLICASET_NAME-shard-0'
}
};
mongoose.connect(uri, options);
var db = mongoose.connection;
I've tried including user: and pass: on options, removing MY_USER:MY_PASSWORD# from uri, change rs_name to replicaSet, every unsuccessful attempt. It seems that mongoose is not considering the authSource option.
Using the mongojs, it works fine with the following code:
// connection string using mongojs:
var uri = 'mongodb://MY_USER:MY_PASSWORD#' +
'MY_SERVER-shard-00-00-clv3h.mongodb.net:27017,' +
'MY_SERVER-shard-00-01-clv3h.mongodb.net:27017,' +
'MY_SERVER-shard-00-02-clv3h.mongodb.net:27017/MY_DATABASE';
var options = {
ssl: true,
authSource: 'MY_ADMIN_DATABASE',
replicaSet: 'MY_REPLICASET_NAME-shard-0'
};
var db = mongojs(uri,'', options);
But, I need to use mongoose because the ODM in my project.
How can I build my uri and options variable using mongoose?
ON MONGODB 3.4.x
I resolved this issue putting the 'options' value directly in 'uri' string, according to documentation (http://mongoosejs.com/docs/connections.html) on 'Replica Set Connections' section.
// connection string using mongoose:
var uri = 'mongodb://MY_USER:MY_PASSWORD#' +
'MY_SERVER-shard-00-00-clv3h.mongodb.net:27017,' +
'MY_SERVER-shard-00-01-clv3h.mongodb.net:27017,' +
'MY_SERVER-shard-00-02-clv3h.mongodb.net:27017/MY_DATABASE' +
'ssl=true&replicaSet=MY_REPLICASET_NAME-shard-0&authSource=MY_ADMIN_DATABASE';
mongoose.connect(uri);
var db = mongoose.connection;
Now, it is working fine!
NOTICE WITH MONGODB 3.6
On MongoDB Atlas using the version 3.6.x, the connection string changed to use a DNS server making the link shorter.
mongodb+srv://MY_USER:MY_PASSWORD#MY_SERVER.mongodb.net/MY_DATABASE
...if you use this connection string in your application, this will connect with success but it will be able to read and write only with atlas users with higher privilegies access (atlasAdmin, readWriteAnyDatabase...).
To you work with an specific user with privilege only to readWrite your database, you will need to keep the same connection string used in MongoDB 3.4 because the mongoose not recognized the DNS option (mongodb+srv).
P.S. all the new resources from MongoDB 3.6.x will continue working normally!
Add username and password to database connection
mongodb://[username:password#]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]]
Standard Connection String Format

mongodb not erroring with wrong uri

I am having big problems trying to wrap my head around mongodb.
First of all, I tried using mongoose, but I really don't like its schemas, I want to have my own classes, and perform CRUD operations in a classic style.
So I moved on and started using mongodb. I think I already gonna give up and use MySQL, because I find their documentation very very obscure, and also no public issue tracker on github.
Anyway, my problem is that if I connect to a non existing db or to a non existing host, mongodb isn't showing any error.
My code:
const mongodb = require( 'mongodb' ),
MongoClient = mongodb.MongoClient;
let db_uri = `mongodb://${config.host}:${config.port}/${config.name}`;
MongoClient.connect( db_uri, function( err, mongoclient ) {
console.log( err );
});
Assume that config.host or config.port are not correct.
err is alway null.
Why is this ?
Mongodb does this great(and in some cases really annoying thing if you have fat fingers) where if you attempt to connect a database that doesn't exist, it will create it for you. I knew it did this in command line, but it appears it does it with this code as well. Using the insert code from the npm node instructions I was able to test and verify this.
Edit- removing example and adding something actually helpful
It may be cumbersome, but there is a way to test if a collection exists. You ask if an index exists.
const mongodb = require( 'mongodb' ),
MongoClient = mongodb.MongoClient;
var db_uri = "mongodb://localhost:27017/newDb";
MongoClient.connect( db_uri, function( err, db ) {
var collection = db.collection('documents');
collection.indexExists(0, function(err, indexExists){
});
});

Node.js and node-sqlanywhere - No Connection Available

I have been trying to use node-sqlanywhere to query my Sybase database.
However, I cannot seem to be able to connect to the database.
var sqlanywhere = require('sqlanywhere');
var sqlanywhere_conn = sqlanywhere.createConnection();
var sqlanywhere_conn_params = {
DatabaseFile: 'C:\Users\support\Dropbox\database.db',
AutoStart: 'YES',
UserId: 'user_id',
Password: 'password'
};
sqlanywhere_conn.connect(sqlanywhere_conn_params, function(){
sqlanywhere_conn.prepare('SELECT * FROM some_table WHERE some_column = ?', function (err, stmt){
if (err) throw err;
// do something with the statement
});
});
Whether I am trying to connect to the database using Server attribute or the DatabaseFile attribute, I always get error code -2005 No Connection Available.
That being said, I am able to connect to my database using Interactive SQL by either specifying the server or the database file.
So.. I must be missing something here and I can't figure it out !
Thanks !!
Try doubling up the backslash characters, like DatabaseFile: 'C:\\Users\\support\\Dropbox\\database.db'.
FYI you can also ask SQL Anywhere-specific questions on http://sqlanywhere-forum.sap.com.
Full disclosure: I work for SAP in SQL Anywhere engineering. I am responsible for both the node.js driver and the sqlanywhere-forum.sap.com site.

Resources