Problem in connecting node.js api on heroku with mongoDb atlas [duplicate] - node.js

To antecipate the question: do I need to get SSL support on Heroku in order to establish a connection between Heroku and Atlas MongoDB Cloud using SSL? (TSL/SSL connection is a requirement to access Atlas MongoDB Cloud service).
I am trying to connect my Heroku App, written in node.js, to a cluster hosted at Atlas MongoDB Cloud.
My current database is hosted at mLab (as a Heroku Add-on), and the MongoDB URI used to access the cluster through mongoose is (using xxx to omit confidential info):
MONGODB_URI="mongodb://xxx:xxx#xxx-a0.mlab.com:23266,xxx-a1.mlab.com:xxx/xxx?replicaSet=rs-xxx"
Now that I've migrated my data from mLab to Atlas MongoDB Cloud, I am currently accessing the cluster using the URI:
MONGODB_URI="mongodb://xxx:xxx#cluster0-shard-xxx.mongodb.net:xxx,cluster0-shard-xxx.mongodb.net:xxx,cluster0-shard-xxx.mongodb.net:xxx/xxx?replicaSet=xxx&ssl=true&authSource=admin"
When running my Heroku App locally in my machine I can access the database with no problem. I'm also able to connect to the cluster using mongo shell.
However, when running the App in Heroku, the connection cannot be established. In the Browser JS console, I get the 503 service unavailable message. In heroku, I get the error:
no primary found in replica set
I am aware that Atlas MongoDB Cloud requires SSL connection, differently from mLab. In my local machine, I suppose a self signed certificate is being used to connect successfully to the cluster.
My question is: do I need to get SSL support in Heroku in order to be able to access establish the secure connection between Heroku and MongoDB Atlas? Or the SSL suport in Heroku is only required to client/Heroku secure connection?

What I think might fix your problem
Disclaimer: I have used neither Heroku nor MongoDB Atlas but I am looking into them.
According to a Github issue I found, you will get that error message if you haven't whitelisted the server IP addresses in MongoDB Atlas.
Reading the MongoDB Atlas docs, the only way I see to do this in combination with Heroku dynos is to add 0.0.0.0/0 (i.e. all addresses) to your MongoDB Atlas whitelist.
Give that a try and please report back whether you can instantiate a connection.
On SSL
Trying to reply to the SSL question, I do not think that you need to enable it on Heroku based on what I read, although I am not totally sure.
If the MongoDB server performed certificate validation, the Node.js code for connecting to it would have to look like the following (taken from the Node.js driver documentation):
var MongoClient = require('mongodb').MongoClient,
f = require('util').format,
fs = require('fs');
// Read the certificates
var ca = [fs.readFileSync(__dirname + "/ssl/ca.pem")];
var cert = fs.readFileSync(__dirname + "/ssl/client.pem");
var key = fs.readFileSync(__dirname + "/ssl/client.pem");
// Connect validating the returned certificates from the server
MongoClient.connect("mongodb://localhost:27017/test?ssl=true", {
server: {
sslValidate:true
, sslCA:ca
, sslKey:key
, sslCert:cert
, sslPass:'10gen'
}
}, function(err, db) {
db.close();
});
If the MongoDB server does not check for any SSL certificates, you can simply use code like the following (also taken from the Node.js driver documentation):
var MongoClient = require('mongodb').MongoClient;
MongoClient.connect("mongodb://localhost:27017/test?ssl=true", function(err, db) {
db.close();
});
Given that the Atlas documentation contains the following example code for connecting to it from Node.js, I think that you do not have to enable SSL on Heroku:
var MongoClient = require('mongodb').MongoClient;
var uri = "mongodb://kay:myRealPassword#mycluster0-shard-00-00-wpeiv.mongodb.net:27017,mycluster0-shard-00-01-wpeiv.mongodb.net:27017,mycluster0-shard-00-02-wpeiv.mongodb.net:27017/admin?ssl=true&replicaSet=Mycluster0-shard-0&authSource=admin";
MongoClient.connect(uri, function(err, db) {
db.close();
});

You can find all IP ranges for Heroku with this command:
HEROKU_REGION=eu; sudo apt -qqy install curl jq 2>/dev/null 1>/dev/null; heroku regions --json 2>/dev/null | jq ".[] | select(.name==\"$HEROKU_REGION\") | .provider.region" | (REGION=$(cat); curl -s https://ip-ranges.amazonaws.com/ip-ranges.json | jq ".prefixes[] | select(.region==$REGION) | .ip_prefix")

Also had to add 0.0.0.0/0 to the Mongo IP whitelist AND redeploy my app on Heroku for it to finally work (before changing IP, a CORS error was thrown).

very simple solution! just add to the white list IP in mongo atlas the adress "0.0.0.0/0"
it will open the mongo atlas to all the world..... so it os not for production but it helps for small tests

I solved this by installing an addon(i used Fixie Socks) for Static IP addresses for database requests and other TCP connections.
More options here: https://elements.heroku.com/addons#network

Since allowing access from anywhere is not secure and IP ranges could change, I ended up installing add-on QuotaGuard Static IP's (it provides 2 IP addresses for IP whitelist) so SOCKS5 Proxy can be used with QGTunnel.
QGTunnel should be downloaded and included in the codebase
curl https://s3.amazonaws.com/quotaguard/qgtunnel-latest.tar.gz | tar xz
Procfile should be updated
web: bin/qgtunnel npm start
Let’s say you want to access a replicated MongoDB cluster using QGTunnel with 3 replicas located on the hosts: rs01.mongodb.net:52115, rs02.mongodb.net:52115, and rs1.mongodb.net:52115. For this configuration, you will need to create 3 separate tunnels for each host on port 52115 in transparent mode. Once this is done, QGTunnel will alter the DNS resolution process to resolve these hostnames to the appropriate loopback address and auto-discovery for your replicated cluster should work as intended.

Related

NodeJS MongoDB API in Docker Container can't connect to Database running on Host

I have a MongoDB Database set up on my host server and a docker container running on that host with a NodeJS application. In that application, I try to connect to the Database on the host but the request always times out.
I set the network to "host" and used the adress "host.docker.internal". I also tried installing mongodb-shell and it actually works! So basically I can connect to the database with the shell in the container but not with the MongoDB NodeJS API.
Code of the NodeJS application:
const mongodb = require('mongodb');
mongodb.MongoClient.connect("mongodb://host.docker.internal:27017/");
using mongo mongodb://host.docker.internal:27017/ in the shell works and I connect correctly.
I finally found out the issue.
I had to bind it to the correct IPs / in my config I used the argument --bind_ip_all.
Thanks a lot for the help!

Mongoose wont connect to MongoDB on Heroku (Works on local dev env)

I get the following error message when trying to connect my NodeJS Express app to my MongoDB server hosted on Heroku. It works fine on my local dev env.
MongoNetworkError: failed to connect to server [cluster0-shard-00-00-hl87n.mongodb.net:27017] on first connect [MongoNetworkError: connection 4 to cluster0-shard-00-00-hl87n.mongodb.net:27017 closed]
I am getting the connect string the following way.
const server = process.env.MONGODB_URI;
const database = process.env.MONGODB_DB;
const user = process.env.MONGODB_USER;
const password = process.env.MONGODB_PASSWORD;
mongoose.connect(`mongodb+srv://${user}:${password}#${server}/${database}`, { useNewUrlParser: true });
The enviroment variables are active on my Heroku instance, I confirmed this by doing heroku config, and also with a console log that correctly prints out all of the above information.
Any idea what could be pointing to this issue, Im out of trails to follow.
Mongodb Cloud requires that you Whitelist the IP that the connections are coming from. Unfourtunatly Heroku doesn't supply you with an IP address so you need to allow access from all IP addresses if you want it to work.
Incase anyone else falls upon this issue that was the fix.

MongoClient must be connected before calling MongoClient.prototype.db on Heroku [duplicate]

To antecipate the question: do I need to get SSL support on Heroku in order to establish a connection between Heroku and Atlas MongoDB Cloud using SSL? (TSL/SSL connection is a requirement to access Atlas MongoDB Cloud service).
I am trying to connect my Heroku App, written in node.js, to a cluster hosted at Atlas MongoDB Cloud.
My current database is hosted at mLab (as a Heroku Add-on), and the MongoDB URI used to access the cluster through mongoose is (using xxx to omit confidential info):
MONGODB_URI="mongodb://xxx:xxx#xxx-a0.mlab.com:23266,xxx-a1.mlab.com:xxx/xxx?replicaSet=rs-xxx"
Now that I've migrated my data from mLab to Atlas MongoDB Cloud, I am currently accessing the cluster using the URI:
MONGODB_URI="mongodb://xxx:xxx#cluster0-shard-xxx.mongodb.net:xxx,cluster0-shard-xxx.mongodb.net:xxx,cluster0-shard-xxx.mongodb.net:xxx/xxx?replicaSet=xxx&ssl=true&authSource=admin"
When running my Heroku App locally in my machine I can access the database with no problem. I'm also able to connect to the cluster using mongo shell.
However, when running the App in Heroku, the connection cannot be established. In the Browser JS console, I get the 503 service unavailable message. In heroku, I get the error:
no primary found in replica set
I am aware that Atlas MongoDB Cloud requires SSL connection, differently from mLab. In my local machine, I suppose a self signed certificate is being used to connect successfully to the cluster.
My question is: do I need to get SSL support in Heroku in order to be able to access establish the secure connection between Heroku and MongoDB Atlas? Or the SSL suport in Heroku is only required to client/Heroku secure connection?
What I think might fix your problem
Disclaimer: I have used neither Heroku nor MongoDB Atlas but I am looking into them.
According to a Github issue I found, you will get that error message if you haven't whitelisted the server IP addresses in MongoDB Atlas.
Reading the MongoDB Atlas docs, the only way I see to do this in combination with Heroku dynos is to add 0.0.0.0/0 (i.e. all addresses) to your MongoDB Atlas whitelist.
Give that a try and please report back whether you can instantiate a connection.
On SSL
Trying to reply to the SSL question, I do not think that you need to enable it on Heroku based on what I read, although I am not totally sure.
If the MongoDB server performed certificate validation, the Node.js code for connecting to it would have to look like the following (taken from the Node.js driver documentation):
var MongoClient = require('mongodb').MongoClient,
f = require('util').format,
fs = require('fs');
// Read the certificates
var ca = [fs.readFileSync(__dirname + "/ssl/ca.pem")];
var cert = fs.readFileSync(__dirname + "/ssl/client.pem");
var key = fs.readFileSync(__dirname + "/ssl/client.pem");
// Connect validating the returned certificates from the server
MongoClient.connect("mongodb://localhost:27017/test?ssl=true", {
server: {
sslValidate:true
, sslCA:ca
, sslKey:key
, sslCert:cert
, sslPass:'10gen'
}
}, function(err, db) {
db.close();
});
If the MongoDB server does not check for any SSL certificates, you can simply use code like the following (also taken from the Node.js driver documentation):
var MongoClient = require('mongodb').MongoClient;
MongoClient.connect("mongodb://localhost:27017/test?ssl=true", function(err, db) {
db.close();
});
Given that the Atlas documentation contains the following example code for connecting to it from Node.js, I think that you do not have to enable SSL on Heroku:
var MongoClient = require('mongodb').MongoClient;
var uri = "mongodb://kay:myRealPassword#mycluster0-shard-00-00-wpeiv.mongodb.net:27017,mycluster0-shard-00-01-wpeiv.mongodb.net:27017,mycluster0-shard-00-02-wpeiv.mongodb.net:27017/admin?ssl=true&replicaSet=Mycluster0-shard-0&authSource=admin";
MongoClient.connect(uri, function(err, db) {
db.close();
});
You can find all IP ranges for Heroku with this command:
HEROKU_REGION=eu; sudo apt -qqy install curl jq 2>/dev/null 1>/dev/null; heroku regions --json 2>/dev/null | jq ".[] | select(.name==\"$HEROKU_REGION\") | .provider.region" | (REGION=$(cat); curl -s https://ip-ranges.amazonaws.com/ip-ranges.json | jq ".prefixes[] | select(.region==$REGION) | .ip_prefix")
Also had to add 0.0.0.0/0 to the Mongo IP whitelist AND redeploy my app on Heroku for it to finally work (before changing IP, a CORS error was thrown).
very simple solution! just add to the white list IP in mongo atlas the adress "0.0.0.0/0"
it will open the mongo atlas to all the world..... so it os not for production but it helps for small tests
I solved this by installing an addon(i used Fixie Socks) for Static IP addresses for database requests and other TCP connections.
More options here: https://elements.heroku.com/addons#network
Since allowing access from anywhere is not secure and IP ranges could change, I ended up installing add-on QuotaGuard Static IP's (it provides 2 IP addresses for IP whitelist) so SOCKS5 Proxy can be used with QGTunnel.
QGTunnel should be downloaded and included in the codebase
curl https://s3.amazonaws.com/quotaguard/qgtunnel-latest.tar.gz | tar xz
Procfile should be updated
web: bin/qgtunnel npm start
Let’s say you want to access a replicated MongoDB cluster using QGTunnel with 3 replicas located on the hosts: rs01.mongodb.net:52115, rs02.mongodb.net:52115, and rs1.mongodb.net:52115. For this configuration, you will need to create 3 separate tunnels for each host on port 52115 in transparent mode. Once this is done, QGTunnel will alter the DNS resolution process to resolve these hostnames to the appropriate loopback address and auto-discovery for your replicated cluster should work as intended.

Connecting Heroku App to Atlas MongoDB Cloud service

To antecipate the question: do I need to get SSL support on Heroku in order to establish a connection between Heroku and Atlas MongoDB Cloud using SSL? (TSL/SSL connection is a requirement to access Atlas MongoDB Cloud service).
I am trying to connect my Heroku App, written in node.js, to a cluster hosted at Atlas MongoDB Cloud.
My current database is hosted at mLab (as a Heroku Add-on), and the MongoDB URI used to access the cluster through mongoose is (using xxx to omit confidential info):
MONGODB_URI="mongodb://xxx:xxx#xxx-a0.mlab.com:23266,xxx-a1.mlab.com:xxx/xxx?replicaSet=rs-xxx"
Now that I've migrated my data from mLab to Atlas MongoDB Cloud, I am currently accessing the cluster using the URI:
MONGODB_URI="mongodb://xxx:xxx#cluster0-shard-xxx.mongodb.net:xxx,cluster0-shard-xxx.mongodb.net:xxx,cluster0-shard-xxx.mongodb.net:xxx/xxx?replicaSet=xxx&ssl=true&authSource=admin"
When running my Heroku App locally in my machine I can access the database with no problem. I'm also able to connect to the cluster using mongo shell.
However, when running the App in Heroku, the connection cannot be established. In the Browser JS console, I get the 503 service unavailable message. In heroku, I get the error:
no primary found in replica set
I am aware that Atlas MongoDB Cloud requires SSL connection, differently from mLab. In my local machine, I suppose a self signed certificate is being used to connect successfully to the cluster.
My question is: do I need to get SSL support in Heroku in order to be able to access establish the secure connection between Heroku and MongoDB Atlas? Or the SSL suport in Heroku is only required to client/Heroku secure connection?
What I think might fix your problem
Disclaimer: I have used neither Heroku nor MongoDB Atlas but I am looking into them.
According to a Github issue I found, you will get that error message if you haven't whitelisted the server IP addresses in MongoDB Atlas.
Reading the MongoDB Atlas docs, the only way I see to do this in combination with Heroku dynos is to add 0.0.0.0/0 (i.e. all addresses) to your MongoDB Atlas whitelist.
Give that a try and please report back whether you can instantiate a connection.
On SSL
Trying to reply to the SSL question, I do not think that you need to enable it on Heroku based on what I read, although I am not totally sure.
If the MongoDB server performed certificate validation, the Node.js code for connecting to it would have to look like the following (taken from the Node.js driver documentation):
var MongoClient = require('mongodb').MongoClient,
f = require('util').format,
fs = require('fs');
// Read the certificates
var ca = [fs.readFileSync(__dirname + "/ssl/ca.pem")];
var cert = fs.readFileSync(__dirname + "/ssl/client.pem");
var key = fs.readFileSync(__dirname + "/ssl/client.pem");
// Connect validating the returned certificates from the server
MongoClient.connect("mongodb://localhost:27017/test?ssl=true", {
server: {
sslValidate:true
, sslCA:ca
, sslKey:key
, sslCert:cert
, sslPass:'10gen'
}
}, function(err, db) {
db.close();
});
If the MongoDB server does not check for any SSL certificates, you can simply use code like the following (also taken from the Node.js driver documentation):
var MongoClient = require('mongodb').MongoClient;
MongoClient.connect("mongodb://localhost:27017/test?ssl=true", function(err, db) {
db.close();
});
Given that the Atlas documentation contains the following example code for connecting to it from Node.js, I think that you do not have to enable SSL on Heroku:
var MongoClient = require('mongodb').MongoClient;
var uri = "mongodb://kay:myRealPassword#mycluster0-shard-00-00-wpeiv.mongodb.net:27017,mycluster0-shard-00-01-wpeiv.mongodb.net:27017,mycluster0-shard-00-02-wpeiv.mongodb.net:27017/admin?ssl=true&replicaSet=Mycluster0-shard-0&authSource=admin";
MongoClient.connect(uri, function(err, db) {
db.close();
});
You can find all IP ranges for Heroku with this command:
HEROKU_REGION=eu; sudo apt -qqy install curl jq 2>/dev/null 1>/dev/null; heroku regions --json 2>/dev/null | jq ".[] | select(.name==\"$HEROKU_REGION\") | .provider.region" | (REGION=$(cat); curl -s https://ip-ranges.amazonaws.com/ip-ranges.json | jq ".prefixes[] | select(.region==$REGION) | .ip_prefix")
Also had to add 0.0.0.0/0 to the Mongo IP whitelist AND redeploy my app on Heroku for it to finally work (before changing IP, a CORS error was thrown).
very simple solution! just add to the white list IP in mongo atlas the adress "0.0.0.0/0"
it will open the mongo atlas to all the world..... so it os not for production but it helps for small tests
I solved this by installing an addon(i used Fixie Socks) for Static IP addresses for database requests and other TCP connections.
More options here: https://elements.heroku.com/addons#network
Since allowing access from anywhere is not secure and IP ranges could change, I ended up installing add-on QuotaGuard Static IP's (it provides 2 IP addresses for IP whitelist) so SOCKS5 Proxy can be used with QGTunnel.
QGTunnel should be downloaded and included in the codebase
curl https://s3.amazonaws.com/quotaguard/qgtunnel-latest.tar.gz | tar xz
Procfile should be updated
web: bin/qgtunnel npm start
Let’s say you want to access a replicated MongoDB cluster using QGTunnel with 3 replicas located on the hosts: rs01.mongodb.net:52115, rs02.mongodb.net:52115, and rs1.mongodb.net:52115. For this configuration, you will need to create 3 separate tunnels for each host on port 52115 in transparent mode. Once this is done, QGTunnel will alter the DNS resolution process to resolve these hostnames to the appropriate loopback address and auto-discovery for your replicated cluster should work as intended.

Connecting to mongodb using mongoose and Fixie (Heroku add-on)

I have a mongodb database hosted on an Atlas MongoDB Cloud cluster. I'm currently accessing the database in my node.js application using mongoose:
mongoose.connect("mongodb://user:pw#cluster0-shard-00-00-***.mongodb.net:***,cluster0-shard-00-01-***.mongodb.net:***,cluster0-shard-00-02-***.mongodb.net:***/admin?ssl=true&replicaSet=Cluster0-shard-0&authSource=admin");
Because Atlas MongoDB Cloud have a whitelist, and Heroku doesn't provide the possibility to obtain a fixed IP address for my application, I'm using Fixie add-on. Basically Fixie acts as a proxy for outbound traffic.
This way, I can request resources via HTTP, which are tunneled through a fixed IP address provided by Fixie. But what I do need is to connect to the Atlas Cloud cluster using Fixie's proxy, in order to retrieve and modify data from the database.
Can it be done using mongoose?
The mongoose.connect function accepts an option parameter, but I couldn't find any option regarding the establishment of a connection through a proxy.
Just got a reply from Fixie's team:
Fixie is an http/https proxy, so it won't work for lower-level TCP connections like you'd need for your mongodb connection
When I asked about the possibility of using SOCKS for this case, they replied:
It looks like mongoose does not natively support socks proxies, and it does not accept a custom socket (which is how Node database drivers for MySQL and Postgres support it).
So apparently, in my case, there is no way to establish a connection to MongoDB Atlas cluster using mongoose through the proxy solution offered by Fixie (Heroku Add-on).
A lot has changed since this question was originally asked in 2017. Since then, Mongoose has added support for proxy options, and Fixie released Fixie Socks, a SOCKS5 proxy designed for proxying database connections.
To connect to a MongoDB server (including MondoDB Atlas) through Fixie Socks, you can do the following:
const mongoose = require('mongoose');
const fixieData = process.env.FIXIE_SOCKS_HOST.split(new RegExp('[/(:\\/#/]+'));
mongoose.connect(process.env.DB_CONNECTION,
{
proxyUsername: fixieData[0],
proxyPassword: fixieData[1],
proxyHost: fixieData[2],
proxyPort: fixieData[3]
},
(error) => {
if(error){
console.log(error);
} else {
console.log('Connected to database');
}
}
);

Resources