I'm using Mongodb with only one shard and 3 members in a replica set. I have each replica set member on a different machine. Two members each have a mongos router, one member (that has a router) has a config server, and all three have a shard server. I have one hostname that resolves to the two ip addresses of each one of the mongodb instances that has a mongos router. I'm using mongoose v.2.5.10 to connect to mongodb from my node.js application with the following code:
var mongoose = require('mongoose');
mongoose.connection.on('error', function(err){
console.error('Mongodb connection error: ', err);
});
var db = mongoose.connect('mongodb://username:password#' + mongoHost + '/database');
I simulate what would happen in a failover scenario by terminating one of the mongodb instances that has the mongos router, and when this happens, I detect that the instance is down and prune the instance's ip address from the DNS record. However, my application does not seem to correctly reconnect to mongodb, as the error event of the mongoose connection is not being emitted, and my application hangs until I restart my node server.
What might be going wrong here?
UPDATE:
I've just confirmed that the net client that the mongodb node.js client is using to connect (this.connection = net.createConnection(this.socketOptions.port, this.socketOptions.host);) is not receiving the close or error event when I manually terminate the Amazon instance. So this seems like this could be an internal node issue...
Related
I'm running a replicated mongoDB and I can connect to the master DB in the set no problem using mongo:
bash-4.2$ mongo --port 25023
MongoDB shell version: 3.2.6
rs0:PRIMARY>
But when using mongoose like this:
mongoose.connect('mongodb://127.0.0.1:25023/XXX', { useNewUrlParser: true });
I get:
XXX/node_modules/mongoose/lib/utils.js:452
throw err;
^
MongoError: not master and slaveOk=false
at queryCallback (/XXX/node_modules/mongodb-core/lib/cursor.js:247:25)
The suggestion was to do this:
mongodb://user:password#host:port,replicaSetHost:replicaSetPort/database?replicaSet=rs-someServer.
But that is a bit unwieldly. Is there no way to tell mongoose we are connecting to a master server and not a slave?
Second problem ... Even listing the other hosts didn't work:
mongoose.connect('mongodb://localhost:P,S1:P,S2:P/XXX?replicaSet=rs0', { useNewUrlParser: true });
I get this error even though I supplied the replicaSet:
(node:13757) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): MongoError: seed list contains no mongos proxies, replicaset connections requires the parameter replicaSet to be supplied in the URI or options object, mongodb://server:port/db?replicaSet=name
Here is where it gets strange: I CAN make this work. If I use this line:
mongoose.connect('mongodb://127.0.0.1:25023/XXX', { useNewUrlParser: true });
And let it fail, and then touch a JS file forcing a NodeJS restart, then it seems to start working?
Same issue, basically I wanted to connect to the primary node in a replica set (write concern), but I was intermittently connecting to a secondary node (race condition).
Solution / TLDR;
Add ?replicaSet=test&w=majority to your MONGO_URL connection string.
From the documentation...
Replica Set with a High Level of Write Concern
The following connects to a replica set with write concern configured to wait for replication to succeed across a majority of the data-bearing voting members, with a two-second timeout.
NOTE
For a replica set, specify the hostname(s) of the mongod instance(s) as listed in the replica set configuration.
mongodb://example1.com,example2.com,example3.com/?replicaSet=test&w=majority&wtimeoutMS=2000
I have mongodb in AWS EC2 instance.
I am trying to connec to mongodb instance from nodejs server using mongoClient. My node application is deployed on PCF platform.
When i try to connect to mongodb
it throws an error "502 Bad Gateway:
Registered endpoint failed to handle the request". My node application is working fine, one the connection with mongodb is causing the problem.
conn_str += req.params.dbname;
mongoClient.connect(conn_str, function(err, db) {
if(err) {
res.end(err);
} else {
res.end(db);
}
});
Help much appreciated.
Ok. I had faced a similar issue while I was trying to connect to my mongo cluster which was deployed on Google Compute Engine. The below checklist helped me resolve the issue
Check if your firewall and make sure that the port 27017 is open for external requests
Check if your MongoDB is up and running (just do a curl localhost:27017 from terminal) if it's running you should get the protocol error
Check if your MongoDB is configured to accept requests from applications that is outside it's network (if you're running on different VPC) -- By default, MongoDB doesn't allow external requests
I have a mongodb database hosted on an Atlas MongoDB Cloud cluster. I'm currently accessing the database in my node.js application using mongoose:
mongoose.connect("mongodb://user:pw#cluster0-shard-00-00-***.mongodb.net:***,cluster0-shard-00-01-***.mongodb.net:***,cluster0-shard-00-02-***.mongodb.net:***/admin?ssl=true&replicaSet=Cluster0-shard-0&authSource=admin");
Because Atlas MongoDB Cloud have a whitelist, and Heroku doesn't provide the possibility to obtain a fixed IP address for my application, I'm using Fixie add-on. Basically Fixie acts as a proxy for outbound traffic.
This way, I can request resources via HTTP, which are tunneled through a fixed IP address provided by Fixie. But what I do need is to connect to the Atlas Cloud cluster using Fixie's proxy, in order to retrieve and modify data from the database.
Can it be done using mongoose?
The mongoose.connect function accepts an option parameter, but I couldn't find any option regarding the establishment of a connection through a proxy.
Just got a reply from Fixie's team:
Fixie is an http/https proxy, so it won't work for lower-level TCP connections like you'd need for your mongodb connection
When I asked about the possibility of using SOCKS for this case, they replied:
It looks like mongoose does not natively support socks proxies, and it does not accept a custom socket (which is how Node database drivers for MySQL and Postgres support it).
So apparently, in my case, there is no way to establish a connection to MongoDB Atlas cluster using mongoose through the proxy solution offered by Fixie (Heroku Add-on).
A lot has changed since this question was originally asked in 2017. Since then, Mongoose has added support for proxy options, and Fixie released Fixie Socks, a SOCKS5 proxy designed for proxying database connections.
To connect to a MongoDB server (including MondoDB Atlas) through Fixie Socks, you can do the following:
const mongoose = require('mongoose');
const fixieData = process.env.FIXIE_SOCKS_HOST.split(new RegExp('[/(:\\/#/]+'));
mongoose.connect(process.env.DB_CONNECTION,
{
proxyUsername: fixieData[0],
proxyPassword: fixieData[1],
proxyHost: fixieData[2],
proxyPort: fixieData[3]
},
(error) => {
if(error){
console.log(error);
} else {
console.log('Connected to database');
}
}
);
I have an Express App which connects to a MongoDB server at startup and serves requests on-demand (I don't disconnect - it's a single threaded server so no pooling - fairly simple stuff)
Problem is that it's possible the MongoDB server will be unavailable for periods of time (it's not on-site) and whilst the Express App doesn't crash, it seems that any requests made to the server will run indefinately until the connection is restored!
I'd like to limit that (e.g. throw an error back after a period of time) but I can't seem to make that happen...
I'm using connect options "{server: {auto_reconnect: true}}" which seems to ensure that once the MongoDB server reappears, requests complete (without it, requests made during downtime seem to run forever...) - and I don't have access to the client code so I can't fix it there...
I'd assumed a combination of 'connectTimeoutMS' or 'socketTimeoutMS' would allow me to terminate requests when MongoDB is unavailable for longer periods, but I just can't get those to work (I've tried them as connect options, passing them in the URI etc. etc.)
Any attempt to open a Collection and Find/Insert/Update just 'hangs' until the MongoDB reappears - I've left it over 30 mins and everything was just sitting these (and completed AOK when the network was restored!)
What's the best way around this? Should I open a connection specifically for each request (not really a performance issue - it's not a high volume app) or is there something else I'm missing?
Updated to add the connect code
var myDB
var mongodb = require('mongodb')
var uri = // some env vars and stuff
mongodb.MongoClient.connect(uri, {server: {auto_reconnect: true}}, function (err, db) {
myDB = db
})
myDB is then used elsewhere to open collections - and the handle from that is used to find/insert etc.
If the connection to the DB is interrupted, myDB.collection() calls (or calls to find/insert on their handles) will simply hang until the connection is restored - nothing I've tried will cause them to 'time out' sooner!?
I assume that you are using mongoose as a driver.
You'd catch the error by this.
var db = require('domain').create();
db.on('error', function(err) {
console.log('DB got a problem');
});
db.run(function() {
mongoose.connect(config, options);
});
or you can directly access
mongoose.connection.readyState
to check the statement of your DB.
Connection ready state
0 = disconnected
1 = connected
2 = connecting
3 = disconnecting
Each state change emits its associated event name.
http://mongoosejs.com/docs/api.html
I am running NginX, Node and Mongodb. And it seems that I can't acces the same database from a second app I am running. For example, I don't get anything back when I do:
collection.findOne({
name: someName
}, function(err, results){
// Returns no errors or results. Just stops working.
});
I can access the database perfectly fine from my first app, but not the second one.
This is the code I use to connect to the database in both apps.
Server = require('mongodb').Server,
Db = require('mongodb').Db,
db = new Db('database', new Server('localhost', 27017, { auto_reconnect: true }), { w: true });
Anyone know what the problem might be?
Edit: Does it have something to do with the subdomain or ports? Too many connections?
Edit 2 (more info):
I run mongodb with service mongodb start.
In my /etc/mongodb.conf I have bind_ip = 127.0.0.1 and dbpath=/var/lib/mongodb (rest is default)
In both my apps I run the same code to establish a connection to the database, but only the first one works (I know that because I am able to retrieve information from the database in my first app).
The apps are running on different ports. The first one is running on port 1337 and the second one runs on 3000.
You are using 'localhost' as the host name to connect to this server.
This means you will only be able to connect from the same machine that mongod is running on with that hostname.
Unless all your apps run on the same server as mongod you will need to change your connect code to use the actual hostname of the mongod server.