I'm able to connect to my primary DB no problem, but when I try to connect to my replica set, I get the TTL error. I've done my best to include all relevant code examples, but please ask if you need to see something that's not included. This is driving me bananas. The DB is at mongoHQ.
So, the issue:
I can connect to my primary set (workingDB)
I cannot connect to my replica set (failingDB)
I cannot connect when trying to connect to both(mongoHQ).
Code example
mongoHQ = "mongodb://<user>:<password>#candidate
.14.mongolayer.com:10120/dbName,mongodb://<user>:<password>#candidate
.15.mongolayer.com:10120"
failingDB = "mongodb://<user>:<password>#candidate
.14.mongolayer.com:10120/dbName"
workingDB = "mongodb://<user>:<password>#candidate
.15.mongolayer.com:10120/dbName"
# DB Options
opts =
mongos: true
server:
auto_reconnect: true
# Connect to DB
mongoose.connect mongoHQ, opts
# express/mongo session storage
app.use express.session(
secret: "Secrets are for children"
cookie:
maxAge: process.env.SESSION_TTL * 3600000
httpOnly: false
store: new mongoStore(
url: mongoHQ
collection: "sessions"
, ->
console.log "We're connected to the session store"
return
)
)
# Error: Error setting TTL index on collection : sessions
# * Connecting to "workingDB" works as expected.
# * Connecting to "failingDB" throws the same TTL Error
# * candidate.14 is the primary set, candidate.15 is the replica set
Perhaps a bit late, but I was today getting similar errors with MongoHQ and Mongoose. I solved it by removing the option mongos: true (at least for the moment, fingers crossed). I think that option is not really needed for replica sets (only when using mongos servers):
http://mongoosejs.com/docs/connections.html#replicaset_connections
http://support.mongohq.com/languages/mongoose.html
Also, it's better to wait for the connection being established before trying to set up MongoStore, for example:
mongoose.connect(mongoHQ);
var db = mongoose.connection;
db.on('error', function () {
// Panic
});
db.on('connected', function() {
var app = express();
app.use(express.session(sessionOptions));
// etc...
app.listen(config.port, function() {
console.log('Listening on port ', config.port);
});
});
I was able to resolve the issue by simply removing the reference to the replica set in the URI
mongoHQ = "mongodb://<user>:<password>#candidate.15.mongolayer.com:10120/dbName";
Related
When i try to connect my Nodsjs application to RedisCloud on Heroku I am getting the following error
Redis::CannotConnectError: Error connecting to Redis on 127.0.0.1:6379 (ECONNREFUSED)
I have even tried to directly set the redis URL and port in the code to test it out as well. But still, it tried to connect to the localhost on Heroku instead of the RedisCloud URL.
const {Queue} = require('bullmq');
const Redis = require('ioredis');
const conn = new Redis(
'redis://rediscloud:mueSEJFadzE9eVcjFei44444RIkNO#redis-15725.c9.us-east-1-4.ec2.cloud.redislabs.com:15725'
// Redis Server Connection Configuration
console.log('\n==================================================\n');
console.log(conn.options, process.env.REDISCLOUD_URL);
const defaultQueue = () => {
// Initialize queue instance, by passing the queue-name & redis connection
const queue = new Queue('default', {conn});
return queue;
};
module.exports = defaultQueue;
Complete Dump of the Logs https://pastebin.com/N9awJYL9
set REDISCLOUD_URL on .env file as follows
REDISCLOUD_URL =redis://rediscloud:password#hostname:port
import * as Redis from 'ioredis';
export const redis = new Redis(process.env.REDISCLOUD_URL);
I just had a hard time trying to find out how to connect the solution below worked for me.
Edit----
Although I had been passed the parameters to connect to the Redis cloud, it connected to the local Redis installed in my machine. Sorry for that!
I will leave my answer here, just in case anyone need to connect to local Redis.
let express = require('express');
var redis = require('ioredis');
pwd = 'your_pwd'
url = 'rediss://host'
port = '1234'
redisConfig = `${url}${pwd}${port}`
client = redis.createClient({ url: redisConfig })
client.on('connect', function() {
console.log('-->> CONNECTED');
});
client.on("error", function(error) {
console.error('ERRO DO REDIS', error);
});
Just wanted to post my case in case someone has the same problem like me.
In my situation I was trying to use Redis with Bull, so i need it the url/port,host data to make this happened.
Here is the info:
https://devcenter.heroku.com/articles/node-redis-workers
but basically you can start your worker like this:
let REDIS_URL = process.env.REDISCLOUD_URL || 'redis://127.0.0.1:6379';
//Once you got Redis info ready, create your task queue
const queue = new Queue('new-queue', REDIS_URL);
In the case you are using local, meaning 'redis://127.0.0.1:6379' remember to run redis-server:
https://redis.io/docs/getting-started/
I have a replica set on MongoDB Atlas and this is my mongo shell connection string which connects perfectly:
$ mongo "mongodb://MY_SERVER-shard-00-00-clv3h.mongodb.net:27017,MY_SERVER-shard-00-01-clv3h.mongodb.net:27017,MY_SERVER-shard-00-02-clv3h.mongodb.net:27017/MY_DATABASE?replicaSet=MY_REPLICASET-NAME-shard-0" --ssl --username MY_USERNAME --password MY_PASSWORD --authenticationDatabase MY_ADMIN_DATABASE
How Can I convert it to use in mongoose? How Can I build my uri and options variable?
I tried the following without success:
// connection string using mongoose:
var uri = 'mongodb://MY_USER:MY_PASSWORD#' +
'MY_SERVER-shard-00-00-clv3h.mongodb.net:27017,' +
'MY_SERVER-shard-00-01-clv3h.mongodb.net:27017,' +
'MY_SERVER-shard-00-02-clv3h.mongodb.net:27017/MY_DATABASE';
var options = {
replset: {
ssl: true,
authSource: 'MY_ADMIN_DATABASE',
rs_name: 'MY_REPLICASET_NAME-shard-0'
}
};
mongoose.connect(uri, options);
var db = mongoose.connection;
I've tried including user: and pass: on options, removing MY_USER:MY_PASSWORD# from uri, change rs_name to replicaSet, every unsuccessful attempt. It seems that mongoose is not considering the authSource option.
Using the mongojs, it works fine with the following code:
// connection string using mongojs:
var uri = 'mongodb://MY_USER:MY_PASSWORD#' +
'MY_SERVER-shard-00-00-clv3h.mongodb.net:27017,' +
'MY_SERVER-shard-00-01-clv3h.mongodb.net:27017,' +
'MY_SERVER-shard-00-02-clv3h.mongodb.net:27017/MY_DATABASE';
var options = {
ssl: true,
authSource: 'MY_ADMIN_DATABASE',
replicaSet: 'MY_REPLICASET_NAME-shard-0'
};
var db = mongojs(uri,'', options);
But, I need to use mongoose because the ODM in my project.
How can I build my uri and options variable using mongoose?
ON MONGODB 3.4.x
I resolved this issue putting the 'options' value directly in 'uri' string, according to documentation (http://mongoosejs.com/docs/connections.html) on 'Replica Set Connections' section.
// connection string using mongoose:
var uri = 'mongodb://MY_USER:MY_PASSWORD#' +
'MY_SERVER-shard-00-00-clv3h.mongodb.net:27017,' +
'MY_SERVER-shard-00-01-clv3h.mongodb.net:27017,' +
'MY_SERVER-shard-00-02-clv3h.mongodb.net:27017/MY_DATABASE' +
'ssl=true&replicaSet=MY_REPLICASET_NAME-shard-0&authSource=MY_ADMIN_DATABASE';
mongoose.connect(uri);
var db = mongoose.connection;
Now, it is working fine!
NOTICE WITH MONGODB 3.6
On MongoDB Atlas using the version 3.6.x, the connection string changed to use a DNS server making the link shorter.
mongodb+srv://MY_USER:MY_PASSWORD#MY_SERVER.mongodb.net/MY_DATABASE
...if you use this connection string in your application, this will connect with success but it will be able to read and write only with atlas users with higher privilegies access (atlasAdmin, readWriteAnyDatabase...).
To you work with an specific user with privilege only to readWrite your database, you will need to keep the same connection string used in MongoDB 3.4 because the mongoose not recognized the DNS option (mongodb+srv).
P.S. all the new resources from MongoDB 3.6.x will continue working normally!
Add username and password to database connection
mongodb://[username:password#]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]]
Standard Connection String Format
I'm currently developing a web app. I just started the development and i try to setup the routes after the db (mongoose Schemas and connection) is ready because i need the models in my router.js file. My Code looks like this right now:
File: server.js
//configuration here...
mongoose.connect( "mongodb://" + config.database.host + "/" + config.database.dbName );
var db = mongoose.connection;
db.on( "error", console.error.bind(console, "connection error:" ));
db.once( "open", function () {
var allModels = models.getAll( mongoose ); //retreive models from different files
router.setupRoutes(express, app, allModels); //setup all the routes also specified in other files
console.log("connected to database and created the routes!");
});
app.listen(port, host, function() {
console.log("");
console.log("Server started at: " + "http://" + host + ":" + port);
console.log("Press 'STRG+C' to stop the Server");
console.log("");
});
My Question is:
Is that the right way to do that? I call the app.listen function not inside this once("open") function (app.listen is at the bottom of this file and gets called before the routes are set up) My test routes are working but im not accesing the mongoose models right now. Should i call app.listen in the mongoose callback or like now at the end of my file?
i just want to make sure that this is the right way to do this. it seems to be the right way. mongoose said so... Look here
I wouldn't say that there's a right or a wrong way. It depends what you're trying to do. Here are my thoughts:
Do you want to have your server running if the database connection isn't working? Maybe because some routes don't involve the database? If that's what you're aiming for, you'd have to set up the routes outside the db connection callback. Something like this:
db.once('open', function() {
// set up routes that involve db connection
});
// set up routes that don't involve db connection
app.listen(port, host, function() {
});
If all of your routes involve database communication, I don't see a purpose to having app.listen outside of the db connection callback. Because the fact that the server is listening wouldn't be useful if the database connection is broken.
I would like to point to specific case. when we put the connection in the server.js The app will try to connect when it is lunched. And If error has been occured it will throw the error and never look back. However, when we put the connection in the routes, the app will try to reach the db only when it needs to, and every time it needs to.The thing that make us able to reach db related end points and none related end points and open to db status changements. So If the db has a problem and we fix it, no need to restart the server.
I made a simple NodeJS App, with Mongoose as MongoDB Driver. And connected to a mongodb replica set. The App is working fine until I shut down the current primary, When the PRIMARY is down, the replica set automatically elected a new PRIMARY. But, after that the node application doesn't seems to be responding for DB queries.
CODE: DB Connection
var options = {
server: {
socketOptions: {
keepAlive: 1,
connectTimeoutMS:30000,
socketTimeoutMS:90000 }
},
replset: {
socketOptions: {
keepAlive: 1,
connectTimeoutMS : 30000 ,
socketTimeoutMS: 90000
},
rs_name: 'rs0'
} };
var uri = "mongodb://xxx.xxx.xxx.xxx:27017,xxx.xxx.xxx.xxx:27017,xxx.xxx.xxx.xxx:27017/rstest";
mongoose.connect(uri,options);
CODE: DB Query
router.('/test',function(req,res){
var testmodel = new testModel('test') ;
testmodel.save(function (err, doc,numberAffected) {
if (err) {
console.log("ERROR: "+ err);
res.status = 404;
res.end;
}else{
console.log("Response sent ");
res.status = 200;
res.end;
}
});
});
Steps Followed
Created a MongoDB replica set in three VMs.
Created a simple nodeJS App (Express + Mongoose) with a test API as above
Sent GET request to 'test' continuously with some time interval to the app from a local system.
Took the PRIMARY instance down
Console will log "ERROR: Error: connection closed"
APPLICATION STOPPED RESPONDING TO REQUESTS
Varsions:
"express": "4.10.6",
"mongodb": "1.4.23",
"mongoose": "3.8.21",
A sample app that I have done for debugging this issue is available at https://melvingeorge#bitbucket.org/melvingeorge/nodejsmongorssample.git
I am not sure if this is a bug or some mis-configuration from my end. How to solve this issue ?
Write operations are made only on master instance. It will take some time for replica set to select a new primary server.
from http://docs.mongodb.org/manual/faq/replica-sets/
How long does replica set failover take?
It varies, but a replica set will select a new primary within a
minute.
It may take 10-30 seconds for the members of a replica set to declare
a primary inaccessible. This triggers an election. During the
election, the cluster is unavailable for writes.
The election itself may take another 10-30 seconds.
check your code with read operations (find/count)
as long as there is not a master instance, you can't do write operations
The 'rs_name' in replset options is necessary to specify a replicaSet. You can use mongoose.createConnection(uri, conf, callback) and get final conf in callback.
It looks like this got fixed in NODE-818 / 2.2.10.
But I am using 2.2.22 and still have a problem like that.
Upon reconnect, the mongo client reconnects to a a secondary instead of a newly selected primary which then is, I cannot write to the database.
My connection string is like mongodb://mongo1,mongo2,mongo3/db?replicaSet=rs
I have nodejs/express/redis/express-session in use in my nodejs application (express 4.x)
The redis initializing is done by connect-redis/session framework under hood. So far it works. Now I need to use redis to store other data in addition to session, and world like to have a new store other than the session store. Is it just one store possible?
And is it possible to use the redis client initialized by connect-redis/session if only one store is possible? how to get it?
Thanks for the help!
The code now is:
var express = require('express');
var session = require('express-session');
// pass the express to the connect redis module
// allowing it to inherit from session.Store
var RedisStore = require('connect-redis')(session);
....
// Populates req.session
app.use(session({
resave: false, // don't save session if unmodified
saveUninitialized: false, // don't create session until something stored
secret: 'keyboard cat',
store: new RedisStore
}));
You'll actually want to initialize a new client for anything else, as the session library is handling it's own client under the hood.
You should most likely import the redis library itself, make your own client, and use that for all future requests / etc.
When I put following code after the code above, I got error
"myRedis Error-> Error: Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED". It seems it is not allowed to init a new Redis instance.
So is there a way to have new client from connect-redis?
var myRedis = require('redis'); // additional redis store
var myRedisCli = myRedis.createClient();
myRedisCli.on('error', function (err) {
console.log('myRedis Error-> ' + err);
});