I'm building a node.js app that talks to MongoDb using Q promises. I'm trying to make it more fault tolerant - so that if MongoDb server goes down - my app detects it and attempts reconnect
the connection is initialized as
this.db = Q.nfcall(Mongodb.connect.bind(Mongodb), config.mongoServer).catch(function(){
console.log('could not connect..');
});
and then occasional reads or writes are made chaining no to this.db promise
this.db
.then(function(db) {
var collection = db.collection(pair);
return Q.nfcall(collection.mapReduce.bind(collection), map, reduce, { out: { inline: 1 }, query: query });
});
if MongoDb server goes down after connection promise is fulfilled - it never goes into catch clause. and even if I add another catch to further promises no disconnection error is cought
Are there any suggestions?
Related
When I start my server, an error was shown like this:
Error in DB connection : Error: queryTxt ETIMEOUT yky-api-ajgvf.mongodb.net
I tried many times to work properly in order to fix this problem. I couldn't even deploy my project on heroku properly yet. I love nodejs so much but now I just want to give up continuously learning Nodejs for now.
Someone help me to solve my problem if possible. I'm the one who is learning Node.Js. It's been around 4 or 5 months. Before I decided to leran PHP, but now I'm still trying to learn nodejs.
I'm a newbie. :)
This is my config database code in my small project.
const mongoose = require('mongoose')
const log = console.log
mongoose.connect('mongodb+srv://chanlay:chanlay123#yky-api-ajgvf.mongodb.net/yky-blog-api?retryWrites=true&w=majority', {
useNewUrlParser: true
}, (err) => {
if (!err) {
log('MongoDB Connection Succeeded.')
} else {
log('Error in DB connection : ' + err)
}
});
require('../models/posts')
mongoose.connect() can look like this (docs reference):
mongoose.connect(uri, options, function(error) {
// Check error in initial connection. There is no 2nd param to the callback.
});
// Or using promises
mongoose.connect(uri, options).then(
() => { /** ready to use. The `mongoose.connect()` promise resolves to mongoose instance. */ },
err => { /** handle initial connection error */ }
);
Before demonstrating further, first of all, this block of yours is wrong:
if (!err) {
log('MongoDB Connection Succeeded.')
}
error handling doesn't work that way, this condition will never happen, as when there's no error, there is no reason to call error callback function which checks this condition.
Instead use .then() block, as a successful connection returns a Promise:
mongoose.connect(uri, options).then(
() => { console.log('MongoDB Connection Succeeded.') },
err => { console.log(err) }
);
// OR use .catch() after .then()
mongoose.connect(uri, options).then(() => console.log('MongoDB Connection Succeeded.')).catch( err => console.log(err))
Your answer: set socketTimeoutMS to 60000 or 90000.
If still didn't work, add connectTimeoutMS=90000 and reconnectTries=1000 to options too.
Declare a new variable named options (the name is arbitrary, but better follow the convention), and you put your useNewUrlParser: true also there (reference):
const options = {
useNewUrlParser: true,
socketTimeoutMS: 60000 //or 90000
}
Tip: You can also specify driver options in your connection string as parameters in the query string portion of the URI (reference):
{main-url}?socketTimeoutMS=90000, that is:
const uri = mongodb+srv://chanlay:chanlay123#yky-api-ajgvf.mongodb.net/yky-blog-api?retryWrites=true&w=majority?socketTimeoutMS=90000
but stick with options variable as it's cleaner, more readable, and easier to maintain (otherwise you would put everything in a string which makes it harder for you to read and debug).
Please let me know by leaving to my answer a comment either as a feedback or reporting the persistent error.
I'd like my Node JS app to exit immediately if it can't connect to Mongo. I'm using the mongodb node library.
I've reduced the code down to
const {MongoClient} = require('mongodb');
MongoClient.connect('mongodb://localhost:27017');
If Mongo is not running, I get an UnhandledPromiseRejectionWarning with ECONNREFUSED, which I fully expect, but then the program hangs and never exits. This is with Node version 10.0.0.
Since the connection never succeeded I don't have a connection handle to close. I've tried various ways to catch the rejected promise, but I have been unsuccessful in getting the program to exit.
What do I need to do to shut down the MongoClient and make the program exit in this case?
Your application is remaining alive because it is trying to reconnect. You can try disabling the recconect:
MongoClient.connect('mongodb://localhost:27017', {
autoReconnect: false
}, (err, client) => {
if (client) client.close();
});
Or, you can terminate the process using process.exit(1) to kill the program.
const {
MongoClient
} = require('mongodb');
// Callback syntax
MongoClient.connect('mongodb://localhost:27017', (err, db) => {
if (err) process.exit(1);
});
// Promise syntax
MongoClient
.connect('mongodb://localhost:27017')
.catch(err => {
process.exit(1);
});
// Async/await syntax
(async function() {
let db;
try {
db = await MongoClient.connect('mongodb://localhost:27017');
} catch (err) {
process.exit(1);
}
}());
I'm trying to get the MongoDB Native Driver to work with Kontainer-di. I want to add the connected client (returned from the connect method) to the container so that I can inject it into the controllers/services directly.
There is an option to use a start function which returns a promise which I thought would work with the mongo native connect function. The database is connected inside the then. My issue is that I'm not sure how I can access the connected database client to add the session to the container.
My code so far looks like:
var mongoClient = require('mongodb').MongoClient;
var promise = require('bluebird');
var mongoFactory = function(config) {
function start() {
return mongoClient.connect("mongodb://127.0.0.1:27017/test", {promiseLibrary: promise})
.then(function(database) {
console.log('mongo connection initialised');
})
.catch(function(err) {
console.error('Error: ', err);
});
}
function stop() {
db.close();
}
return {
start: start,
stop: stop
}
}
module.exports = mongoFactory;
In case somebody else has the same issue as me. I ended up going with the mongojs library instead which doesn't use promises for the connection so the active connection could easily be added to the container.
I still wanted to use promises rather than callbacks for the queries so I used bluebird and it's promisifyAll method.
I have a script that I use to save all my models (for ex to reindex). I am getting socket closing errors after a couple of hundred saves (Ihave 900 total). I recently upgraded to Mongoose 4.2.3 from 3.x.x and started seeing these errors. I am not sure what else to go on.
Errors:
{ [MongoError: server ds0133252-a0.mongolab.com:133252 sockets closed]
name: 'MongoError',
message: 'server ds051252-a0.mongolab.com:51252 sockets closed' }
{ [MongoError: server ds0133252-a0.mongolab.com:133252 sockets closed]
name: 'MongoError',
....
The script is pretty basic:
var mongoose = require(mongoose),
Product = require('../models/product'),
config = require('config');
mongoose.connect(config.db.mongo.connection, config.db.mongo.options);
Product.find(function(err, products) {
products.forEach(function(p) {
p.markModified('description');
p.save(function(e, product) {
if(e) console.log(e);
console.log(product.id);
});
});
});
The model is pretty complex but hasn't changed in a while. I have disabled the "save" middleware with same errors so it should be pretty standard.
Suggestions?
You can use an async flow control library like async to use an async iterator that lets you limit the number of concurrent save operations.
In this case, async.eachLimit would be a good fit (doc link is to each, scroll down to see the eachLimit variant). For example, to limit the iteration to no more than 5 concurrent saves:
Product.find(function(err, products) {
async.eachLimit(products, 5, function(p, callback) {
p.markModified('description');
p.save(function(e, product) {
if(e) console.log(e);
console.log(product.id);
callback(err);
});
});
});
Note that the callback parameter of eachLimit must be called when the save completes so that the library knows that particular iteration is complete.
I want my application (lets say a simple node file for now) to work as it is even if redis is not available. I'm not able to do it the correct way. This is what I've tried.
var redis = require('redis');
var redisClient = null;
var getRedisClient = function(){
if(redisClient){
return redisClient;
}
try {
redisClient = redis.createClient({connect_timeout : 5000, max_attempts : 1});
redisClient.on("error", function(err) {
console.error("Error connecting to redis", err);
redisClient = null;
});
return redisClient;
} catch(ex){
console.log("error initialising redis client " + ex);
return null;
}
};
try {
var client = getRedisClient();
console.log("done!");
} catch (ex){
console.log("Exception");
}
However, with this code my application exits if redis is not available (it shouldn't because i've not given a process.exit() command).
How can I solve this?
Checking for Successful Connection on Start
Using a promise, you could guarantee that at least initially, you were able to connect to redis without error within a specified time period:
const redis = require('redis');
const Promise = require('bluebird');
function getRedisClient(timeoutMs){
return new Promise((resolve, reject) => {
const redisClient = redis.createClient();
const timer = setTimeout(() => reject('timeout'), timeoutMs);
redisClient.on("ready", () => {
clearTimeout(timer);
resolve(redisClient);
});
redisClient.on("error", (err) => {
clearTimeout(timer);
reject(err);
});
});
};
const redisReadyTimeoutMs = 10000;
getRedisClient(redisReadyTimeoutMs)
.then(redisClient => {
// the client has connected to redis sucessfully
return doSomethingUseful();
}, error => {
console.log("Unable to connect to redis", error);
});
You Need Proper Error Handling
The redis client being non-null does NOT guarantee using it won't throw an error.
you could experience infrastructure misfortune e.g. crashed redis process, out of memory or network being down.
a bug in your code could cause an error e.g. invalid or missing arguments to a redis command.
You should be handling redis client errors as a matter of course.
DON'T null the Redis Client on Error
It won't give you much but it will force you to check for null every time you try and use it.
The redis client also has inbuilt reconnect and retry mechanisms that you'll miss out on if you null it after the first error. See the redis package docs, look for retry_strategy.
DO Wrap your redis client code with try .. catch ... or use .catch in your promise chain.
DO Make use of a retry_strategy.