There are tons of examples and resources for connecting to a redis "server" but not to a redis "cluster".
What I currently have is:
const redis = require("redis");
const client = redis.createClient({
port : 6379, // replace with your port
host : '10.0.0.100', // replace with your hostanme or IP address
password : 'Notforyou', // replace with your password
// optional, if using SSL
// use `fs.readFile[Sync]` or another method to bring these values in
tls : {
key : stringValueOfKeyFile,
cert : stringValueOfCertFile,
ca : [ stringValueOfCaCertFile ]
}
});
But this is for a single redis server not a cluster. How would I go about connecting to my cluster?
The Readme tab in redis npm package has this link(Clustering Guide) explaining how to connect to redis cluster with node under heading 'Clustering'. It's very simple, create a cluster instead of creating client and configure the nodes url as given in this example
Check out this library ioredis
I think that this will help you with your issue.
Related
To antecipate the question: do I need to get SSL support on Heroku in order to establish a connection between Heroku and Atlas MongoDB Cloud using SSL? (TSL/SSL connection is a requirement to access Atlas MongoDB Cloud service).
I am trying to connect my Heroku App, written in node.js, to a cluster hosted at Atlas MongoDB Cloud.
My current database is hosted at mLab (as a Heroku Add-on), and the MongoDB URI used to access the cluster through mongoose is (using xxx to omit confidential info):
MONGODB_URI="mongodb://xxx:xxx#xxx-a0.mlab.com:23266,xxx-a1.mlab.com:xxx/xxx?replicaSet=rs-xxx"
Now that I've migrated my data from mLab to Atlas MongoDB Cloud, I am currently accessing the cluster using the URI:
MONGODB_URI="mongodb://xxx:xxx#cluster0-shard-xxx.mongodb.net:xxx,cluster0-shard-xxx.mongodb.net:xxx,cluster0-shard-xxx.mongodb.net:xxx/xxx?replicaSet=xxx&ssl=true&authSource=admin"
When running my Heroku App locally in my machine I can access the database with no problem. I'm also able to connect to the cluster using mongo shell.
However, when running the App in Heroku, the connection cannot be established. In the Browser JS console, I get the 503 service unavailable message. In heroku, I get the error:
no primary found in replica set
I am aware that Atlas MongoDB Cloud requires SSL connection, differently from mLab. In my local machine, I suppose a self signed certificate is being used to connect successfully to the cluster.
My question is: do I need to get SSL support in Heroku in order to be able to access establish the secure connection between Heroku and MongoDB Atlas? Or the SSL suport in Heroku is only required to client/Heroku secure connection?
What I think might fix your problem
Disclaimer: I have used neither Heroku nor MongoDB Atlas but I am looking into them.
According to a Github issue I found, you will get that error message if you haven't whitelisted the server IP addresses in MongoDB Atlas.
Reading the MongoDB Atlas docs, the only way I see to do this in combination with Heroku dynos is to add 0.0.0.0/0 (i.e. all addresses) to your MongoDB Atlas whitelist.
Give that a try and please report back whether you can instantiate a connection.
On SSL
Trying to reply to the SSL question, I do not think that you need to enable it on Heroku based on what I read, although I am not totally sure.
If the MongoDB server performed certificate validation, the Node.js code for connecting to it would have to look like the following (taken from the Node.js driver documentation):
var MongoClient = require('mongodb').MongoClient,
f = require('util').format,
fs = require('fs');
// Read the certificates
var ca = [fs.readFileSync(__dirname + "/ssl/ca.pem")];
var cert = fs.readFileSync(__dirname + "/ssl/client.pem");
var key = fs.readFileSync(__dirname + "/ssl/client.pem");
// Connect validating the returned certificates from the server
MongoClient.connect("mongodb://localhost:27017/test?ssl=true", {
server: {
sslValidate:true
, sslCA:ca
, sslKey:key
, sslCert:cert
, sslPass:'10gen'
}
}, function(err, db) {
db.close();
});
If the MongoDB server does not check for any SSL certificates, you can simply use code like the following (also taken from the Node.js driver documentation):
var MongoClient = require('mongodb').MongoClient;
MongoClient.connect("mongodb://localhost:27017/test?ssl=true", function(err, db) {
db.close();
});
Given that the Atlas documentation contains the following example code for connecting to it from Node.js, I think that you do not have to enable SSL on Heroku:
var MongoClient = require('mongodb').MongoClient;
var uri = "mongodb://kay:myRealPassword#mycluster0-shard-00-00-wpeiv.mongodb.net:27017,mycluster0-shard-00-01-wpeiv.mongodb.net:27017,mycluster0-shard-00-02-wpeiv.mongodb.net:27017/admin?ssl=true&replicaSet=Mycluster0-shard-0&authSource=admin";
MongoClient.connect(uri, function(err, db) {
db.close();
});
You can find all IP ranges for Heroku with this command:
HEROKU_REGION=eu; sudo apt -qqy install curl jq 2>/dev/null 1>/dev/null; heroku regions --json 2>/dev/null | jq ".[] | select(.name==\"$HEROKU_REGION\") | .provider.region" | (REGION=$(cat); curl -s https://ip-ranges.amazonaws.com/ip-ranges.json | jq ".prefixes[] | select(.region==$REGION) | .ip_prefix")
Also had to add 0.0.0.0/0 to the Mongo IP whitelist AND redeploy my app on Heroku for it to finally work (before changing IP, a CORS error was thrown).
very simple solution! just add to the white list IP in mongo atlas the adress "0.0.0.0/0"
it will open the mongo atlas to all the world..... so it os not for production but it helps for small tests
I solved this by installing an addon(i used Fixie Socks) for Static IP addresses for database requests and other TCP connections.
More options here: https://elements.heroku.com/addons#network
Since allowing access from anywhere is not secure and IP ranges could change, I ended up installing add-on QuotaGuard Static IP's (it provides 2 IP addresses for IP whitelist) so SOCKS5 Proxy can be used with QGTunnel.
QGTunnel should be downloaded and included in the codebase
curl https://s3.amazonaws.com/quotaguard/qgtunnel-latest.tar.gz | tar xz
Procfile should be updated
web: bin/qgtunnel npm start
Let’s say you want to access a replicated MongoDB cluster using QGTunnel with 3 replicas located on the hosts: rs01.mongodb.net:52115, rs02.mongodb.net:52115, and rs1.mongodb.net:52115. For this configuration, you will need to create 3 separate tunnels for each host on port 52115 in transparent mode. Once this is done, QGTunnel will alter the DNS resolution process to resolve these hostnames to the appropriate loopback address and auto-discovery for your replicated cluster should work as intended.
To antecipate the question: do I need to get SSL support on Heroku in order to establish a connection between Heroku and Atlas MongoDB Cloud using SSL? (TSL/SSL connection is a requirement to access Atlas MongoDB Cloud service).
I am trying to connect my Heroku App, written in node.js, to a cluster hosted at Atlas MongoDB Cloud.
My current database is hosted at mLab (as a Heroku Add-on), and the MongoDB URI used to access the cluster through mongoose is (using xxx to omit confidential info):
MONGODB_URI="mongodb://xxx:xxx#xxx-a0.mlab.com:23266,xxx-a1.mlab.com:xxx/xxx?replicaSet=rs-xxx"
Now that I've migrated my data from mLab to Atlas MongoDB Cloud, I am currently accessing the cluster using the URI:
MONGODB_URI="mongodb://xxx:xxx#cluster0-shard-xxx.mongodb.net:xxx,cluster0-shard-xxx.mongodb.net:xxx,cluster0-shard-xxx.mongodb.net:xxx/xxx?replicaSet=xxx&ssl=true&authSource=admin"
When running my Heroku App locally in my machine I can access the database with no problem. I'm also able to connect to the cluster using mongo shell.
However, when running the App in Heroku, the connection cannot be established. In the Browser JS console, I get the 503 service unavailable message. In heroku, I get the error:
no primary found in replica set
I am aware that Atlas MongoDB Cloud requires SSL connection, differently from mLab. In my local machine, I suppose a self signed certificate is being used to connect successfully to the cluster.
My question is: do I need to get SSL support in Heroku in order to be able to access establish the secure connection between Heroku and MongoDB Atlas? Or the SSL suport in Heroku is only required to client/Heroku secure connection?
What I think might fix your problem
Disclaimer: I have used neither Heroku nor MongoDB Atlas but I am looking into them.
According to a Github issue I found, you will get that error message if you haven't whitelisted the server IP addresses in MongoDB Atlas.
Reading the MongoDB Atlas docs, the only way I see to do this in combination with Heroku dynos is to add 0.0.0.0/0 (i.e. all addresses) to your MongoDB Atlas whitelist.
Give that a try and please report back whether you can instantiate a connection.
On SSL
Trying to reply to the SSL question, I do not think that you need to enable it on Heroku based on what I read, although I am not totally sure.
If the MongoDB server performed certificate validation, the Node.js code for connecting to it would have to look like the following (taken from the Node.js driver documentation):
var MongoClient = require('mongodb').MongoClient,
f = require('util').format,
fs = require('fs');
// Read the certificates
var ca = [fs.readFileSync(__dirname + "/ssl/ca.pem")];
var cert = fs.readFileSync(__dirname + "/ssl/client.pem");
var key = fs.readFileSync(__dirname + "/ssl/client.pem");
// Connect validating the returned certificates from the server
MongoClient.connect("mongodb://localhost:27017/test?ssl=true", {
server: {
sslValidate:true
, sslCA:ca
, sslKey:key
, sslCert:cert
, sslPass:'10gen'
}
}, function(err, db) {
db.close();
});
If the MongoDB server does not check for any SSL certificates, you can simply use code like the following (also taken from the Node.js driver documentation):
var MongoClient = require('mongodb').MongoClient;
MongoClient.connect("mongodb://localhost:27017/test?ssl=true", function(err, db) {
db.close();
});
Given that the Atlas documentation contains the following example code for connecting to it from Node.js, I think that you do not have to enable SSL on Heroku:
var MongoClient = require('mongodb').MongoClient;
var uri = "mongodb://kay:myRealPassword#mycluster0-shard-00-00-wpeiv.mongodb.net:27017,mycluster0-shard-00-01-wpeiv.mongodb.net:27017,mycluster0-shard-00-02-wpeiv.mongodb.net:27017/admin?ssl=true&replicaSet=Mycluster0-shard-0&authSource=admin";
MongoClient.connect(uri, function(err, db) {
db.close();
});
You can find all IP ranges for Heroku with this command:
HEROKU_REGION=eu; sudo apt -qqy install curl jq 2>/dev/null 1>/dev/null; heroku regions --json 2>/dev/null | jq ".[] | select(.name==\"$HEROKU_REGION\") | .provider.region" | (REGION=$(cat); curl -s https://ip-ranges.amazonaws.com/ip-ranges.json | jq ".prefixes[] | select(.region==$REGION) | .ip_prefix")
Also had to add 0.0.0.0/0 to the Mongo IP whitelist AND redeploy my app on Heroku for it to finally work (before changing IP, a CORS error was thrown).
very simple solution! just add to the white list IP in mongo atlas the adress "0.0.0.0/0"
it will open the mongo atlas to all the world..... so it os not for production but it helps for small tests
I solved this by installing an addon(i used Fixie Socks) for Static IP addresses for database requests and other TCP connections.
More options here: https://elements.heroku.com/addons#network
Since allowing access from anywhere is not secure and IP ranges could change, I ended up installing add-on QuotaGuard Static IP's (it provides 2 IP addresses for IP whitelist) so SOCKS5 Proxy can be used with QGTunnel.
QGTunnel should be downloaded and included in the codebase
curl https://s3.amazonaws.com/quotaguard/qgtunnel-latest.tar.gz | tar xz
Procfile should be updated
web: bin/qgtunnel npm start
Let’s say you want to access a replicated MongoDB cluster using QGTunnel with 3 replicas located on the hosts: rs01.mongodb.net:52115, rs02.mongodb.net:52115, and rs1.mongodb.net:52115. For this configuration, you will need to create 3 separate tunnels for each host on port 52115 in transparent mode. Once this is done, QGTunnel will alter the DNS resolution process to resolve these hostnames to the appropriate loopback address and auto-discovery for your replicated cluster should work as intended.
I am using node-redis and having a hard time connecting to external redis instance. I tried with redis-cli and it worked. However with node I am not able to figure out how to properly give the url and port.
With Redis-cli-
redis-cli -h mydomain.something.something.cache.amazonaws.com -p 6379
However with nodejs
Below didn't work
var client = redis.createClient('redis://mydomain.something.something.cache.amazonaws.com:6379'),
neither
var client = redis.createClient({host:'redis://mydomain.something.something.cache.amazonaws.com', port: 6379});
How do I configure it. Please help.
Following should work with node.js -
var client = require('redis').createClient(6379, 'elastichache endpoint string', {
no_ready_check: true
});
Also, make sure that your security group on AWS allows you to access the database.
var client = require('redis').createClient(6379, 'elastichache endpoint string', {
no_ready_check: true
});
With the above code, it was always trying to connect with localhost,
Below code worked for me.
var client = require('redis').createClient(
{
url: `redis://${elasticCacheConnectionString}`,
}
);
Please note, i have appended redis:// as communication protocol before actual connection string.
FYI: I am using redis#4.1.0 version.
To antecipate the question: do I need to get SSL support on Heroku in order to establish a connection between Heroku and Atlas MongoDB Cloud using SSL? (TSL/SSL connection is a requirement to access Atlas MongoDB Cloud service).
I am trying to connect my Heroku App, written in node.js, to a cluster hosted at Atlas MongoDB Cloud.
My current database is hosted at mLab (as a Heroku Add-on), and the MongoDB URI used to access the cluster through mongoose is (using xxx to omit confidential info):
MONGODB_URI="mongodb://xxx:xxx#xxx-a0.mlab.com:23266,xxx-a1.mlab.com:xxx/xxx?replicaSet=rs-xxx"
Now that I've migrated my data from mLab to Atlas MongoDB Cloud, I am currently accessing the cluster using the URI:
MONGODB_URI="mongodb://xxx:xxx#cluster0-shard-xxx.mongodb.net:xxx,cluster0-shard-xxx.mongodb.net:xxx,cluster0-shard-xxx.mongodb.net:xxx/xxx?replicaSet=xxx&ssl=true&authSource=admin"
When running my Heroku App locally in my machine I can access the database with no problem. I'm also able to connect to the cluster using mongo shell.
However, when running the App in Heroku, the connection cannot be established. In the Browser JS console, I get the 503 service unavailable message. In heroku, I get the error:
no primary found in replica set
I am aware that Atlas MongoDB Cloud requires SSL connection, differently from mLab. In my local machine, I suppose a self signed certificate is being used to connect successfully to the cluster.
My question is: do I need to get SSL support in Heroku in order to be able to access establish the secure connection between Heroku and MongoDB Atlas? Or the SSL suport in Heroku is only required to client/Heroku secure connection?
What I think might fix your problem
Disclaimer: I have used neither Heroku nor MongoDB Atlas but I am looking into them.
According to a Github issue I found, you will get that error message if you haven't whitelisted the server IP addresses in MongoDB Atlas.
Reading the MongoDB Atlas docs, the only way I see to do this in combination with Heroku dynos is to add 0.0.0.0/0 (i.e. all addresses) to your MongoDB Atlas whitelist.
Give that a try and please report back whether you can instantiate a connection.
On SSL
Trying to reply to the SSL question, I do not think that you need to enable it on Heroku based on what I read, although I am not totally sure.
If the MongoDB server performed certificate validation, the Node.js code for connecting to it would have to look like the following (taken from the Node.js driver documentation):
var MongoClient = require('mongodb').MongoClient,
f = require('util').format,
fs = require('fs');
// Read the certificates
var ca = [fs.readFileSync(__dirname + "/ssl/ca.pem")];
var cert = fs.readFileSync(__dirname + "/ssl/client.pem");
var key = fs.readFileSync(__dirname + "/ssl/client.pem");
// Connect validating the returned certificates from the server
MongoClient.connect("mongodb://localhost:27017/test?ssl=true", {
server: {
sslValidate:true
, sslCA:ca
, sslKey:key
, sslCert:cert
, sslPass:'10gen'
}
}, function(err, db) {
db.close();
});
If the MongoDB server does not check for any SSL certificates, you can simply use code like the following (also taken from the Node.js driver documentation):
var MongoClient = require('mongodb').MongoClient;
MongoClient.connect("mongodb://localhost:27017/test?ssl=true", function(err, db) {
db.close();
});
Given that the Atlas documentation contains the following example code for connecting to it from Node.js, I think that you do not have to enable SSL on Heroku:
var MongoClient = require('mongodb').MongoClient;
var uri = "mongodb://kay:myRealPassword#mycluster0-shard-00-00-wpeiv.mongodb.net:27017,mycluster0-shard-00-01-wpeiv.mongodb.net:27017,mycluster0-shard-00-02-wpeiv.mongodb.net:27017/admin?ssl=true&replicaSet=Mycluster0-shard-0&authSource=admin";
MongoClient.connect(uri, function(err, db) {
db.close();
});
You can find all IP ranges for Heroku with this command:
HEROKU_REGION=eu; sudo apt -qqy install curl jq 2>/dev/null 1>/dev/null; heroku regions --json 2>/dev/null | jq ".[] | select(.name==\"$HEROKU_REGION\") | .provider.region" | (REGION=$(cat); curl -s https://ip-ranges.amazonaws.com/ip-ranges.json | jq ".prefixes[] | select(.region==$REGION) | .ip_prefix")
Also had to add 0.0.0.0/0 to the Mongo IP whitelist AND redeploy my app on Heroku for it to finally work (before changing IP, a CORS error was thrown).
very simple solution! just add to the white list IP in mongo atlas the adress "0.0.0.0/0"
it will open the mongo atlas to all the world..... so it os not for production but it helps for small tests
I solved this by installing an addon(i used Fixie Socks) for Static IP addresses for database requests and other TCP connections.
More options here: https://elements.heroku.com/addons#network
Since allowing access from anywhere is not secure and IP ranges could change, I ended up installing add-on QuotaGuard Static IP's (it provides 2 IP addresses for IP whitelist) so SOCKS5 Proxy can be used with QGTunnel.
QGTunnel should be downloaded and included in the codebase
curl https://s3.amazonaws.com/quotaguard/qgtunnel-latest.tar.gz | tar xz
Procfile should be updated
web: bin/qgtunnel npm start
Let’s say you want to access a replicated MongoDB cluster using QGTunnel with 3 replicas located on the hosts: rs01.mongodb.net:52115, rs02.mongodb.net:52115, and rs1.mongodb.net:52115. For this configuration, you will need to create 3 separate tunnels for each host on port 52115 in transparent mode. Once this is done, QGTunnel will alter the DNS resolution process to resolve these hostnames to the appropriate loopback address and auto-discovery for your replicated cluster should work as intended.
I am involved in a development project of a chat where we are using node.js, socket.io (rooms) and mongodb. We are at the stage of performance testing and we are very concerned if the system needs a load balance.
How can we develop if our project needs it? J'a researched on NGINX looks cool, but we are in doubt whether solves our problem as how the system will be a chat, we fear the servers are not ~talking~ with each other correctly ...
Where do we go if we need a load balancing?
To ensure that we can scale to multiple nodes but keep up interconnectivity between different clients and different servers, I use redis. It's actually very simple to use and set up.
What this does is creates a pub/sub system between your servers to keep track of your different socket clients.
var io = require('socket.io')(3000),
redis = require('redis'),
redisAdapter = require('socket.io-redis'),
port = 6379,
host = '127.0.0.1',
pub = redis.createClient(port, host),
sub = redis.createClient(port, host, {detect_buffers: true}),
server = http(),
socketServer = io(server, {adapter: redisAdapter({pubClient: pub, subClient: sub})});
read more here: socket.io-redis
As far as handling the different node servers, there are different approaches.
AWS ELB(elastic load balancer)
Nginx
Apache
HAProxy
Among others...
Check out the NPM package mong.socket.io . It has the ability to save socket.io data to mongoDB like below;
{
"_id" : ObjectId("54b901332e2f73f5594c6267"),
"event" : "join",
"message" : {
"name" : "join",
"nodeId" : 426506139219,
"args" : "[\"URAiA6mO6VbCwquWKH0U\",\"/54b6821asdf66asdasd2f0f9cd2997413780273376\"]"
}}
Or you may use the redis adapter as mentioned there;
Socket.IO Using multiple nodes
Then just use the NGINX reverse proxy and all of the node processes should share Socket.IO events with each other.