We are using a node stack to connect to a mongo replica set. Because our replicas are geographically distributed, we would like to use the readPreference option in the URI and set it to nearest. But when we do so, while performance is greatly improved, we start getting "not master" errors.
Am I misunderstanding the use of the flag?
We are using mongo 2.6.3 and we are using version 2.0.24 of the mongodb node library.
The URI for the connection is:
mongodb://mongo-1:27017,mongo-2:27017,mongo-3:27017,mongo-4:27017,mongo-5:27017/db?replicaSet=main&readPreference=nearest
Burc
Option 1:
You could append slaveOk to end of connection URI. readPreference tells mongodb that how you'd like to read data and slaveOk instructs that it's OK to read from secondaries (bit duplicate) but works.
e.g.
mongodb://mongo-1:27017,mongo-2:27017,mongo-3:27017,mongo-4:27017,mongo-5:27017/db?replicaSet=main&readPreference=nearest&slaveOk=true
please see &slaveOk=true and end of URI.
https://mongodb.github.io/node-mongodb-native/driver-articles/mongoclient.html#read-preference
Option 2:
if above solution is not working, you'll need to modify code:
var client = require('mongodb').MongoClient;
var uri= "mongodb://mongo-1:27017,mongo-2:27017,mongo-3:27017,mongo-4:27017,mongo-5:27017/db?replicaSet=main";
Please note that I have modified connection uri. Instead of setting readPrefference in Uri, I moved it to as db option in MongoClient.connect.
var options = {db:{readPreference:"ReadPreference.NEAREST"}};
client.connect(uri, options, function(err, db){
if(err){
console.log(err);
return;
}
db = db.collection('data');
db.findOne({}, function(err, result){
console.log(result);
});
});
I have tested in nodejs driver 2.2 and hopping it should work in 2.0 version too.
It seems like there was a bug in the driver which was fixed in 2.0.28, where findAndModify used the readPreference setting. Upgrading the driver to the latest release seemed to have fixed the problem.
2.0.x drivers history
Related
In the MongoDB Nodejs driver, I see some confusion in what the connection URI can be.
On one hand, in the page describing the URI (https://docs.mongodb.com/manual/reference/connection-string/) it says the the path parameter is the "authentication database".
On the other hand, in many of the official examples (http://mongodb.github.io/node-mongodb-native/driver-articles/mongoclient.html#mongoclient-connect) it seems they are using the path parameter as the active database to use (they call db.collection() straight away, without calling .database().
Am I missing something?
TL;DR:
Calling db.collection() immediately after connection only works in versions of the driver less than 3.0.
Details:
Firstly, the official examples you sighted were from MongoDB driver at version 1.4.9, the driver is now at version 3.5.8, I would suggest you check out the latest documentation and examples here.
To clarify the confusion, the database path specified in the connection URI is the authentication database i.e the database used to log in, this is true even for the 1.4.9 version of the driver - reference.
However, the reason for the difference you mentioned, i.e being able to call db.collection() immediately after a connection in some cases is a result of the change in the MongoClient class in version 3 of the driver - reference.
Before version 3, MongoClient.connect would return a DB instance to its call back function and this instance would be referencing the database specified in the path of the connection URI, so you could call db.collection() straight away:
MongoClient.connect("<connection_URI>", function(err, db) {
// db is a DB instance, so I can access my collections straight away:
db.collection('sample_collection').find();
});
However, an update was made at version 3 such that, MongoClient.connect now returns a MongoClient instance not a DB instance anymore - reference:
MongoClient.connect("<connection_URI>", function(err, client) {
// client is a MongoClient instance, you would have to call
// the Client.db() method to access your database
const db = client.db('sample_database');
// Now you can access your collections
db.collection('sample_collection').find();
});
What is the difference between these two packages to use thrift on NodeJS:
- thrift
- thrift-http
At first glance it appears thrift-http would be needed to use thrift over http but I installed the thrift package and it does have a createHttpConnection method defined in it. Here is sample code I wrote:
var thrift = require('thrift');
var myService = require('./gen-nodejs/MyService');
var transport = thrift.TBufferedTransport;
var protocol = thrift.TBinaryProtocol;
// connect to proxy on 127.0.0.1:8888
var connection = thrift.createHttpConnection('127.0.0.1', 8888, {
transport : transport,
protocol : protocol,
path: 'http://localhost:5331/myhandler',
});
connection.on('error', function(err) {
console.log(err);
});
// Create a client with the connection
var client = thrift.createHttpClient(myService, connection);
console.log('calling getTotalJobCount...');
client.getTotalJobCount(function(count)
{
console.log('total job count = ' + count);
});
But when I run it I get this:
total job count = NULL
This is not as expected as the service returns an integer which cannot appear as NULL in the client callback. I can't find the link now, but yesterday I read somewhere that thrift only supports TJSONProtocol over http. Is that correct even now? And is that the reason why I am seeing a NULL? Also where can I see the source code for thrift in nodejs? This is the first link that appears in Google when searching for "thrift nodejs" and it seems the code is completely outdated.
The thrift package is the one provided by the Apache Thrift project team.
The other one is someone elses package, not related to the "official" sources in any way. After a quick check against the relevant commits, the version of the files seems to be 0.9.3 as well, which is the most recent official release. However, I can't say whether all these files are entirely original, or if they have been modified. You will have to verify that on your own (or just trust that guy).
I am making an application in nodejs in which i have to create cassandra database through CQL.
I have tried to use helenus,cassandra-driver but didn't able to create keyspace through CQL.
I have also tried apollo-cassandra and also able to connect to DB and create keyspace and enter tuples but when we fire queries again after restarting the application then it is showing schime mis-match error.
Which nodejs driver for cassandra is simple and fully supported for CQL?
Datastax has an official driver in the companys github repo.
https://github.com/datastax/nodejs-driver
The driver is for Apache Cassandra. This driver works exclusively with the Cassandra Query Language version 3 (CQL3) and Cassandra's native protocol.
High level features:
Automatic node discovery
Simple, prepared and batch statement support
Load balancing policies
Failover with retry and reconnection policies
SASL auth support
It works really well and is actively mantained.
Another route (if you prefer not writing the query statement outright in your code) is CassanKnex, it uses the datastax driver to execute queries - usage:
var cassanKnex = require("cassanknex")({
connection: { // default is 'undefined'
contactPoints: ["10.0.0.2"]
},
exec: { // default is '{}'
prepare: false // default is 'true'
}
});
cassanKnex.on("ready", function (err) {
var qb.createKeyspace("cassanKnexy")
.withSimpleStrategy(1);
.exec(function(err, res) {
if (err)
console.error("error", err);
else
console.log("res", res);
});
}
});
I read :
How do I manage MongoDB connections in a Node.js web application?
http://mongodb.github.io/node-mongodb-native/driver-articles/mongoclient.html
How can I set up MongoDB on a Node.js server using node-mongodb-native in an EC2 environment?
And I am really confused. How I should work with mongoDB from node.js? I’m a rookie, and my question may look stupid.
var db = new db.MongoClient(new db.Server('localhost', 27017));
db.open(function(err, dataBase) {
//all code here?
dataBase.close();
});
Or every time when I needing something from db I need call:
MongoClient.connect("mongodb://localhost:27017/myDB", function(err, dataBase) {
//all code here
dataBase.close();
});
What is the difference betwen open and connect? I read in the manual that open: Initialize and second connect. But what exactly does that mean? I assume that both do the same, but in the other way, so when should I use one instead the other?
I also wanna ask it's normal that mongoClient needing 4 socket? I running two myWEbServer at the same time, here’s picture:
http://i43.tinypic.com/29mlr14.png
EDIT:
I wanna mention that this isn't a problem ( rather doubt :D), my server works perfect. I ask because I wanna know if I am using mongoDB driver correctly.
Now/Actually I use first option,init mongo dirver at the beginning and inside load put all code.
I'd recommend trying the MongoDB tutorial they offer. I was in the same boat, but this breaks it down nicely. In addition, there's this article on github that explains the basics of DB connection.
In short, it does look like you're doing it right.
MongoClient.connect("mongodb://localhost:27017/myDB", function(err, dataBase) {
//all code here
var collection = dataBase.collection('users');
var document1 = {'name':'John Doe'};
collection.insert(document1, {w:1}, function(err,result){
console.log(err);
});
dataBase.close();
});
You still can sign up for a free course M101JS: MongoDB for Node.js Developers, provided by MongoDB guys
Here is short description:
This course will go over basic installation, JSON, schema design,
querying, insertion of data, indexing and working with language
drivers. In the course, you will build a blogging platform, backed by
MongoDB. Our code examples will be in Node.js.
I had same question. I couldn't find any proper answer from mongo documentation.
All document say is to prefer new db connection and then use open (rather than using connect() )
http://docs.mongodb.org/manual/reference/method/connect/
I'm going through a few error scenarios, trying to understand how to handle those.
In the case where there is no database connection, a Mongoose Model.find(...) call seems to hang. Below the example code. I would have assumed that the callback is invoked with an err object, but it is not.
How can I prevent the model call to hang? Do I manually have to check the readyState each time I access a model?
// app.js
// Let's use a non-existing host so connecting fails:
// (callback is invoked with err object)
mongoose.connect('mongodb://localhostXXX/blog', function(err){ ... });
BlogPost = mongoose.model('BlogPost', BlogPostSchema);
// api.js
exports.list_posts = function(req, res) {
// Ready state is '0' = disconnected (since we used a wrong hostname)
console.log('DB ready state: ' + BlogPost.db.readyState);
// This will not invoke the callback:
BlogPost.find(function(err, threads) {
// Never called...
});
}
It is not an answer, but hope it will help you to find solution. Had very similar issue with
mongoose.createConnection
while using passport module, found out that it works fine with
mongoose.connect
Since you're already using an error handler in the connect call, a sensible thing would be to quit your app when the DB is not up, or activate some middleware that responds with a pretty 500 Internal Server Error.
Mongoose uses node-mongodb-native under the hood for connections to mongodb, you might find other useful connection options in there. :)
EDIT: try setting socketOptions.socketTimeoutMS. Apparently there is no default timeout set. See http://mongodb.github.com/node-mongodb-native/api-generated/server.html.
I don't have node on my work machine to try out the exact syntax for you, but you will probably have to use mongoose.Connection, which has an open() method that accepts options to pass through to node-mongodb-native. I don't think mongoose.connect() accepts these options, but I could be wrong.
In order to solve this problem you need to do 3 tasks:
Configure the bufferMaxEntries:0 in the options.db section (for more details see here)
So when disable bufferMaxEntries this causes mongoose to stop buffer commands and retrying to send them when the server is down.
Configure the autoReconnect:false in the options.db section
disable autoReconnet in the db level. (see more information here)
if you are working with mongodb replicaset then you need to disable bufferCommands in the schema level (for each schema you create)
var schema = new Schema({..}, { bufferCommands: false });