How to connect to a MongoDB running on an aws ec2 host (linux) out of docker - node.js

Situation:
I prototyped a small web (node.js) app and dockerized it for deployment and replicability purposes.
The app speaks with a MongoDB running directly on the host.
Problem:
On the server (AWS EC2 instance, only Port 80 and 443 open), I am not able to interact with MongoDB and I am wondering why.
docker run --net="host" -e 'NODE_ENV=production' -e 'MONGO_URI=mongodb://USER:PASSWORD!#172.31.32.1:27017/test_db' DOCKER_IMAGE
MongoDB connected ...
Warning: connect.session() MemoryStore is not
designed for a production environment, as it will leak
memory, and will not scale past a single process.
HTTP Server started on port 80
(node:1) DeprecationWarning: current Server Discovery and Monitoring engine is deprecated, and will be removed in a future version. To use the new Server Discover and Monitoring engine, pass option { useUnifiedTopology: true } to the MongoClient constructor.
MongoNetworkError: failed to connect to server [172.31.32.1:27017] on first connect [MongoNetworkError: connection timed out
at connectionFailureError (/app/server/node_modules/mongodb/lib/core/connection/connect.js:377:14)
at Socket.<anonymous> (/app/server/node_modules/mongodb/lib/core/connection/connect.js:287:16)
at Object.onceWrapper (events.js:284:20)
at Socket.emit (events.js:196:13)
at Socket._onTimeout (net.js:432:8)
at listOnTimeout (internal/timers.js:531:17)
at processTimers (internal/timers.js:475:7) {
name: 'MongoNetworkError',
[Symbol(mongoErrorContextSymbol)]: {}
}]
at Pool.<anonymous> (/app/server/node_modules/mongodb/lib/core/topologies/server.js:433:11)
at Pool.emit (events.js:196:13)
at /app/server/node_modules/mongodb/lib/core/connection/pool.js:571:14
at /app/server/node_modules/mongodb/lib/core/connection/pool.js:994:11
at /app/server/node_modules/mongodb/lib/core/connection/connect.js:40:11
at callback (/app/server/node_modules/mongodb/lib/core/connection/connect.js:262:5)
at Socket.<anonymous> (/app/server/node_modules/mongodb/lib/core/connection/connect.js:287:7)
at Object.onceWrapper (events.js:284:20)
at Socket.emit (events.js:196:13)
at Socket._onTimeout (net.js:432:8)
at listOnTimeout (internal/timers.js:531:17)
at processTimers (internal/timers.js:475:7) {
name: 'MongoNetworkError',
[Symbol(mongoErrorContextSymbol)]: {}
}
I initially tried localhost instead of the IP address, but that does not work. It throws an authentication error (which is somewhat strange). Since there is no host.docker.internal for Linux I had to (temporarily) resolve to the explicit IP address. The IP address that I am using I got via:
netstat -nr | grep '^0\.0\.0\.0' | awk '{print $2}'
What I find very strange is that I am not getting an Authentication error but a Timeout error, so to me it seems like the app is able to connect to Mongo. Also, the "MonogDB connected ..." would indicate that, as it is produced by the following line in my server script.
mongoose
.connect(DB, { useNewUrlParser: true })
.then(console.log("MongoDB connected ..."))
.catch(err => console.log(err));
For completeness sake, the same setup (i.e. dockerized app and running MongoDB directly on the host) worked without a problem locally.
Also, I am able to enter the Mongo Shell on the server via mongo.
Any explanation or tip is appreciated!

If mongo DB is running outside of the Docker, you can use the private IP of the EC2 instance to connect with Mongo DB (running on Host) from inside Docker.
EC2_PRIVATE_IP=$(curl http://169.254.169.254/latest/meta-data/local-ipv4)
docker run --net="host" -e 'NODE_ENV=production' -e 'MONGO_URI=mongodb://USER:PASSWORD!#$EC_PRIVATE_IP:27017/test_db' DOCKER_IMAGE

Related

Unable to use MongoDB on GCP cloud run and connect it to a NodeJS

I am testing a nodejs app on GCP functions and a MongoDB instance in GCP cloud run(I know this is not a good idea) but again this is a test. I am able to get the mongo image running on cloud run and I am given a service endpoint as https://mongodb.foo.bar.run.app and I have the container port in this cloud run service as 27017.
When I try and edit the mongoose.connect() method to include this service uri I am not able to connect with the app. I have the mongodb cloud run instance open to receive traffic from all traffic(Again not a good idea), but I still can't get the node app to connect.
I have setup the connection in node as mongoose.connect('mongodb://mongodb.foo.bar.run.app:27017/test');
but I get this error
Error: connect ETIMEDOUT <IP ADDRESS GOES HERE>:27017
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
at TCPConnectWrap.callbackTrampoline (internal/async_hooks.js:129:14)
Emitted 'error' event on NativeConnection instance at:
at NativeConnection.Connection.error (/Users/foo/nodejs/bar/node_modules/mongoose/lib/connection.js:443:8)
at /Users/foo/nodejs/bar/node_modules/mongoose/lib/connection.js:472:15
at /Users/foo/nodejs/bar/node_modules/mongoose/lib/drivers/node-mongodb-native/connection.js:59:21
at /Users/foo/nodejs/bar/node_modules/mongodb/lib/db.js:232:14
at Server.<anonymous> (/Users/foo/nodejs/bar/node_modules/mongodb/lib/server.js:240:9)
at Object.onceWrapper (events.js:421:26)
at Server.emit (events.js:314:20)
at Pool.<anonymous> (/Users/foo/nodejs/bar/node_modules/mongodb-core/lib/topologies/server.js:308:68)
at Pool.emit (events.js:314:20)
at Connection.<anonymous> (/Users/foo/nodejs/bar/node_modules/mongodb-core/lib/connection/pool.js:115:12)
at Object.onceWrapper (events.js:421:26)
at Connection.emit (events.js:314:20)
at Socket.<anonymous> (/Users/foo/nodejs/bar/node_modules/mongodb-core/lib/connection/connection.js:144:49)
at Object.onceWrapper (events.js:421:26)
at Socket.emit (events.js:314:20)
at emitErrorNT (internal/streams/destroy.js:100:8) {
name: 'MongoError'
}
I tried to curl the service uri as
curl https://mongodb.foo.bar.run.app
and I get a 200 response in the mongo logs GET 200 466B but if I
curl mongodb.foo.bar.run.app
I get a 302 response code GET 302 0B.
I looked at the mongoose documentation but I couldn't figure what I was doing wrong. I know the documentation states user and password on the uri string, but when I run it locally on my docker engine I did not use a user and password and it worked.
As mentioned in the comments, it's not possible to run MongoDB on Cloud Run since as mentioned here:
Cloud Run can only receive HTTP requests or Pub/Sub push events.
Connecting to MongoDB requires a TCP protocol rather than HTTP, that's why every call to it would fail even if the port is reachable.
Cloud Run is in this sense more similar to AWS Lambda than it's to AWS Fargate as it's even billed on a per request basis rather than on resources consumed.

Connecting database (MongoDB) and backend (nodejs) running in docker containers [duplicate]

This question already has answers here:
MongoDB on with Docker "failed to connect to server [localhost:27017] on first connect "
(4 answers)
Closed 2 years ago.
First, I looked through several discussion with similar problems and it still did not work.
I have a mongodb docker container running, I did port forwarding with -p command
to be exact this is the command I ran:
sudo docker run -t -d -p 27017:27017 --name mongo mongo-0000
docker ps shows container running
9d9040a7bd66 mongo-0000 "docker-entrypoint.s…" 4 minutes ago Up 4 minutes 0.0.0.0:27017->27017/tcp mongo
as in another thread it was suggested to change mongodb bindip from 127.0.0.1 to 0.0.0.0 , which I also did (I tried both ways).
then I am trying to start up a backend app container with nodejs express backend app, I have had them working fine together on a VM, not on docker jet.
And i get following error
sudo docker run conduit-backend
Listening on port 3000
/ConduitReactApp/src/node_modules/mongodb/lib/server.js:261
process.nextTick(function() { throw err; })
^
Error [MongoError]: failed to connect to server [localhost:27017] on first connect
at Pool.<anonymous> (/ConduitReactApp/src/node_modules/mongodb-core/lib/topologies/server.js:313:35)
at Pool.emit (node:events:378:20)
at Connection.<anonymous> (/ConduitReactApp/src/node_modules/mongodb-core/lib/connection/pool.js:260:12)
at Object.onceWrapper (node:events:485:26)
at Connection.emit (node:events:378:20)
at Socket.<anonymous> (/ConduitReactApp/src/node_modules/mongodb-core/lib/connection/connection.js:162:49)
at Object.onceWrapper (node:events:485:26)
at Socket.emit (node:events:378:20)
at emitErrorNT (node:internal/streams/destroy:188:8)
at emitErrorCloseNT (node:internal/streams/destroy:153:3)
at processTicksAndRejections (node:internal/process/task_queues:81:21)
Emitted 'error' event on NativeConnection instance at:
at /ConduitReactApp/src/node_modules/mongoose/lib/connection.js:288:17
at NativeConnection.Connection.error (/ConduitReactApp/src/node_modules/mongoose/lib/connection.js:489:12)
at /ConduitReactApp/src/node_modules/mongoose/lib/connection.js:520:15
at /ConduitReactApp/src/node_modules/mongoose/lib/drivers/node-mongodb-native/connection.js:69:21
at /ConduitReactApp/src/node_modules/mongodb/lib/db.js:229:14
at Server.<anonymous> (/ConduitReactApp/src/node_modules/mongodb/lib/server.js:259:9)
at Object.onceWrapper (node:events:485:26)
at Server.emit (node:events:378:20)
at Pool.<anonymous> (/ConduitReactApp/src/node_modules/mongodb-core/lib/topologies/server.js:313:21)
at Pool.emit (node:events:378:20)
[... lines matching original stack trace ...]
at Socket.emit (node:events:378:20)
Also, inside app.js (in the backend app) for connecting to mongoDB it reads so
if(isProduction){
mongoose.connect(process.env.MONGODB_URI);
} else {
mongoose.connect('mongodb://localhost/conduit');
mongoose.set('debug', true);
}
What is still wrong here ?
Firstly, you should run your mongodb image after that you should mount mongo container to app container while run your app container. For example,
docker run -p xxxx:xxxx --link mongo:mongo <image-name>
And to connect mongodb like this, you should use a connection string like below,
'mongodb://mongo:27017/<db-name>'

MongoNetworkError on first connect using Docker and NodeJS

I'm still incredibly new to coding. I'm trying to launch an app using docker, everything goes well until I get this error:
{ MongoNetworkError: failed to connect to server [localhost:27017] on first connect [MongoNetworkError: connect ECONNREFUSED 127.0.0.1:27017]
at Pool.<anonymous> (/app/node_modules/mongodb/lib/core/topologies/server.js:433:11)
at Pool.emit (events.js:198:13)
at createConnection (/app/node_modules/mongodb/lib/core/connection/pool.js:577:14)
at connect (/app/node_modules/mongodb/lib/core/connection/pool.js:1007:11)
at makeConnection (/app/node_modules/mongodb/lib/core/connection/connect.js:31:7)
at callback (/app/node_modules/mongodb/lib/core/connection/connect.js:247:5)
at Socket.err (/app/node_modules/mongodb/lib/core/connection/connect.js:276:7)
at Object.onceWrapper (events.js:286:20)
at Socket.emit (events.js:198:13)
at emitErrorNT (internal/streams/destroy.js:91:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
at process._tickCallback (internal/process/next_tick.js:63:19)
name: 'MongoNetworkError',
[Symbol(mongoErrorContextSymbol)]: {} }
[nodemon] clean exit - waiting for changes before restart
I have mongodb running as a server on Windows 10, and I also have it running in another window, so I'm not sure what could be causing it. I've tried all solutions on this page, but to no avail. Thanks in advance!
Localhost is the container, not your machine. When working with containers, you need to think of container as a separate machine and use a full network adressess to communicate between your machine and container.
I was using a similar concept like yours. Try to run both node-app and Mongo service under the same network.
In Mongo connection string, the Hostname should be the same as your service name mentioned in docker-compose.yml file.

Error connecting web server on Elastic Beanstalk with database server on MongoDB Atlas

I am trying to connect my web servers on Elastic Beanstalk with my database servers on MongoDB Atlas, but somehow I get the following error
Listening at http://:::3001
Unable to connect to the mongodb instance. Error: { MongoNetworkError: failed to connect to server [<server>.mongodb.net:27017] on first connect [MongoNetworkError: connection 5 to <server>.mongodb.net:27017 closed]
at Pool.<anonymous> (/var/app/current/node_modules/mongodb-core/lib/topologies/server.js:431:11)
at Pool.emit (events.js:198:13)
at connect (/var/app/current/node_modules/mongodb-core/lib/connection/pool.js:557:14)
at callback (/var/app/current/node_modules/mongodb-core/lib/connection/connect.js:109:5)
at runCommand (/var/app/current/node_modules/mongodb-core/lib/connection/connect.js:129:7)
at Connection.errorHandler (/var/app/current/node_modules/mongodb-core/lib/connection/connect.js:321:5)
at Object.onceWrapper (events.js:286:20)
at Connection.emit (events.js:198:13)
at TLSSocket.<anonymous> (/var/app/current/node_modules/mongodb-core/lib/connection/connection.js:350:12)
at Object.onceWrapper (events.js:286:20)
at TLSSocket.emit (events.js:198:13)
at _handle.close (net.js:606:12)
at TCP.done (_tls_wrap.js:388:7)
name: 'MongoNetworkError',
errorLabels: [ 'TransientTransactionError' ],
[Symbol(mongoErrorContextSymbol)]: {} }
Listening at http://:::3001
(node:11482) DeprecationWarning: current Server Discovery and Monitoring engine is deprecated, and will be removed in a future version. To use the new Server Discover and Monitoring engine, pass option { useUnifiedTopology: true } to the MongoClient constructor.
Listening at http://:::3001
I am using mongoose as follows
mongoose.connect('mongodb+srv://<UserName>:<Password>#<cluster>.mongodb.net/test?retryWrites=true&w=majority',
{ useNewUrlParser: true, useUnifiedTopology: true })
.catch(function (reason) {
console.log('Unable to connect to the mongodb instance. Error: ', reason);
});
The problem is not in the IP Whitelist on MongoDB Atlas, since I already ssh'd into the server on elastic beanstalk and from there connected to the Mongo Servers on Atlas directly. The problem is in the code, although I can't seem to see it.
I had the same issue. Add 0.0.0.0/0 to your IP whitelist.

mongoose wont connect to ec2 mongodb instance

i'm trying to create a node.js server that connect to my mongodb hosted on ec2 instance. I'm doing like this :
mongoose.connect("mongodb://username:password#ec2-xx-xx-xxx-xxx.eu-central-1.compute.amazonaws.com:27017/dbname");
db.on('error', err => {
console.log('Mongoose default connection error:', err);
db.close();
});
db.once('open', () => {
console.log(`Mongoose default connection open`);
});
But it keep saying me this error :
Mongoose default connection error: { MongoError: failed to connect to server [xxx] on first
connect [MongoError: connect ETIMEDOUT xxx:27017]
at Pool.<anonymous> (/Users/jmisiti/hm-landing/node_modules/mongodb-core/lib/topologies/server.js:336:35)
at emitOne (events.js:115:13)
at Pool.emit (events.js:210:7)
at Connection.<anonymous> (/Users/jmisiti/hm-landing/node_modules/mongodb-core/lib/connection/pool.js:280:12)
at Object.onceWrapper (events.js:318:30)
at emitTwo (events.js:125:13)
at Connection.emit (events.js:213:7)
at Socket.<anonymous> (/Users/jmisiti/hm-landing/node_modules/mongodb-core/lib/connection/connection.js:189:49)
at Object.onceWrapper (events.js:316:30)
at emitOne (events.js:115:13)
at Socket.emit (events.js:210:7)
at emitErrorNT (internal/streams/destroy.js:62:8)
at _combinedTickCallback (internal/process/next_tick.js:102:11)
at process._tickCallback (internal/process/next_tick.js:161:9)
name: 'MongoError',
message: 'failed to connect to server [xxx] on first connect [MongoError: connect ETIMEDO
UT xxx:27017]' }
My connection string is 100% sure it works. My mongoose version work great with my mongodb version. Any idea ? :)
PS : Ports are open on 27017 and there's already one of my aws instance using my mongodb. So i'm sure it work.
Make sure that you have registered 27017 port in Outbound ports In the EC2 security group.
Then try to access the DB. and if you have already added port and still experiencing the error then check if you MongoDB process is running as a service(i.e. in the background). If not then follow this procedure
Got it. I can't connect from my localhost to my mongodb server (Don't know why) but when i'm connecting to my mongodb server from my ec2 instance it work. So it's fine :)

Resources