I am currently trying to deploy my app with docker but I have some problem with mongodb.
My app is not succeeding to connect to mongodb in another container.
Here is my docker-compose.yml file:
version: "3.7"
services:
mqtt_broker:
build:
context: .
dockerfile: Dockerfile
container_name: mqtt_broker
ports:
- "4040:4040"
links:
- mqtt_db:mqtt_db
depends_on:
- mqtt_db
mqtt_db:
build:
context: ./mongo
dockerfile: Dockerfile
container_name: mqtt_db
ports:
- "27017:27017"
Here is the Dockerfile for the mongo container:
FROM mongo:latest
COPY init_docker.js /docker-entrypoint-initdb.d/
CMD mongod --replSet "rs0" --bind_ip_all
Here is how I connect to mongo in the app container:
const client = new MongoClient(config.get("dbURL"));
const mongo = await client.connect();
const db = client.db(config.get("dbName"));
with the configuration file:
{
"cors": {
"origin": "*"
},
"clientPort": 4040,
"mqttPort": 1883,
"dbURL": "mongodb://mqtt_db:27017",
"dbName": "mqtt_proxy"
}
Here the error I get with the mongo node client:
MongoServerSelectionError: Server selection timed out after 30000 ms
at Timeout._onTimeout (/broker/node_modules/mongodb/lib/sdam/topology.js:305:38)
at listOnTimeout (node:internal/timers:559:17)
at processTimers (node:internal/timers:502:7) {
reason: TopologyDescription {
type: 'Unknown',
servers: Map(1) {
'mqtt_db:27017' => ServerDescription {
_hostAddress: HostAddress { isIPv6: false, host: 'mqtt_db', port: 27017 },
address: 'mqtt_db:27017',
type: 'RSGhost',
hosts: [],
passives: [],
arbiters: [],
tags: {},
minWireVersion: 0,
maxWireVersion: 13,
roundTripTime: 35.24000000000001,
lastUpdateTime: 13896371,
lastWriteDate: 0,
topologyVersion: {
processId: ObjectId { [Symbol(id)]: [Buffer [Uint8Array]] },
counter: 0
},
logicalSessionTimeoutMinutes: 30
}
},
stale: false,
compatible: true,
heartbeatFrequencyMS: 10000,
localThresholdMS: 15,
commonWireVersion: 13,
logicalSessionTimeoutMinutes: undefined
},
code: undefined,
[Symbol(errorLabels)]: Set(0) {}
}
I can actually ping mqtt_db, the ip is resolved, but I don't see it in /etc/hosts.
I also tried to add networks in my docker-compose but it translated mqtt_db in localhost and it does not work.
Here is the repo of my project: https://gitlab.com/mqtt-broker-sniffer/sniffer
EDIT
I had to specify ?directConnection=true when I was connecting to the db.
Crédit: https://stackoverflow.com/a/70204195/19127936
const client = new MongoClient(
`${config.get("dbURL")}/${config.get("dbName")}?directConnection=true`
);
this.client = await client.connect();
this.db = client.db(config.get("dbName"));
Related
I am attempting to dockerize a nodeJs application, however I cannot connect using mongoose, Mongodb Compass works (from host OS)
The app connects to a Mongodb server ( also on docker) using the following code:
const mongoUri: string = "mongodb://mongo:27017?replicaSet=rs0";
mongoose
.connect(mongoUri)
.then(() => console.log("Connected to database")).catch(err => console.log(err.reason));
I am using the following docker-compose file:
version: "3.8"
services:
mongo:
image: mongo:4.4.0
volumes:
- ./mongo:/etc/mongo
- ./data/db:/data/db
entrypoint: [/etc/mongo/start.sh]
ports:
- 27017:27017
hostname: mongo
backend:
depends_on:
- mongo
build: ./sp_backend
ports:
- 8080:8080
- 8883:8883
- 8884:8884
hostname: backend
The start.sh is setting a custom mongod.conf as following:
mongod --config /etc/mongo/mongod.conf
and in the mongod.conf the replication is set (to enable change streams)
...
replication:
replSetName: "rs0"
...
I also ran rs.initiate() on the mongo instance at the first startup, and I can connect from the host to the mongo server using Mongodb Compass, however mongoose spits out the following error:
TopologyDescription {
type: 'ReplicaSetNoPrimary',
servers: Map(1) {
'127.0.0.1:27017' => ServerDescription {
address: '127.0.0.1:27017',
type: 'Unknown',
hosts: [],
passives: [],
arbiters: [],
tags: {},
minWireVersion: 0,
maxWireVersion: 0,
roundTripTime: -1,
lastUpdateTime: 25342884,
lastWriteDate: 0,
error: [MongoNetworkError],
topologyVersion: null,
setName: null,
setVersion: null,
electionId: null,
logicalSessionTimeoutMinutes: null,
primary: null,
me: null,
'$clusterTime': null
}
},
stale: false,
compatible: true,
heartbeatFrequencyMS: 10000,
localThresholdMS: 15,
setName: 'rs0',
maxElectionId: new ObjectId("7fffffff0000000000000001"),
maxSetVersion: 1,
commonWireVersion: 0,
logicalSessionTimeoutMinutes: null
}
My thinking is that the servers map should not contain 127.0.0.1 as a server since the mongo runs on a sepparate server. I also checked to ping the database container from the nodejs container and it works.
Later update:
It seems I was using the an old mongo docker image, after upgrading (mongo:4.4.0) to mongo:latest fixed the problem.
I'm getting an error when my server is connecting to the MongoDB database. Both the server and mongoDB are in containers
The docker-compose files are as follows:
docker-compose.yml
version: '3'
services:
client:
build:
context: ./client
dockerfile: Dockerfile
ports:
- 3001:3001
networks:
- mern-network
volumes:
- ./client/src:/usr/app/src
- ./client/public:/usr/app/public
depends_on:
- server
environment:
- REACT_APP_SERVER=http://localhost:5001
# - CHOKIDAR_USEPOLLING=true
# container 1: backend node server
server:
build:
context: ./server
dockerfile: Dockerfile
ports:
- '5001:5001'
networks:
- mern-network
environment:
- PORT=5001
depends_on:
- mongo
# container 2: database
mongo:
image: mongo:4.4.13
networks:
- mern-network
environment:
- MONGO_INITDB_ROOT_USERNAME=docker_mongo
- MONGO_INITDB_ROOT_PASSWORD=password
volumes:
- mongo-db:/data/db
networks:
mern-network:
driver: bridge
volumes:
mongo-db:
docker-compose.dev.yml
version: '3'
services:
# container 3: client, dev settings
client:
command: npm start
# container 1: server, dev setting
server:
build:
args:
BUILD_ENV: development
volumes:
- ./server/:/app
# - ./server/app/node_modules
environment:
- BUILD_ENV=development
- MONGO_USER=docker_mongo
- MONGO_PASSWORD=password
- SESSION_SECRET=secret123456890qwerty
command: npm run dev
# container 2: database, dev settings
mongo:
environment:
- MONGO_INITDB_ROOT_USERNAME=docker_mongo
- MONGO_INITDB_ROOT_PASSWORD=password
I have no problems accessing into the mongodb database when I inspect the container using docker exec -it but error when I connect with node. I've tried many versions of mongoose still gives me same error.
MongoServerSelectionError: getaddrinfo EAI_AGAIN mongo
at Timeout._onTimeout (/home/user/git_folder/project/server/node_modules/mongodb/lib/sdam/topology.js:312:38)
at listOnTimeout (node:internal/timers:559:17)
at processTimers (node:internal/timers:502:7) {
reason: TopologyDescription {
type: 'Unknown',
servers: Map(1) {
'mongo:27017' => ServerDescription {
_hostAddress: HostAddress { isIPv6: false, host: 'mongo', port: 27017 },
address: 'mongo:27017',
type: 'Unknown',
hosts: [],
passives: [],
arbiters: [],
tags: {},
minWireVersion: 0,
maxWireVersion: 0,
roundTripTime: -1,
lastUpdateTime: 53541928,
lastWriteDate: 0,
error: MongoNetworkError: getaddrinfo EAI_AGAIN mongo
at connectionFailureError (/home/user/git_folder/project/server/node_modules/mongodb/lib/cmap/connect.js:375:20)
at Socket.<anonymous> (/home/user/git_folder/project/server/node_modules/mongodb/lib/cmap/connect.js:295:22)
at Object.onceWrapper (node:events:640:26)
at Socket.emit (node:events:520:28)
at emitErrorNT (node:internal/streams/destroy:157:8)
at emitErrorCloseNT (node:internal/streams/destroy:122:3)
at processTicksAndRejections (node:internal/process/task_queues:83:21)
}
},
stale: false,
compatible: true,
heartbeatFrequencyMS: 10000,
localThresholdMS: 15,
logicalSessionTimeoutMinutes: undefined
},
code: undefined
}
I am running mongodb replicaSet inside docker containers in Windows.
This is docker compose file
version: '3'
services:
rs0:
image: mongo:4.4
ports:
- "27018:27017"
command: mongod --replSet rsnmbp
volumes:
- rs0_data:/data/db
- ./nmbprsdata0:/nmbpdata
rs1:
image: mongo:4.4
ports:
- "27019:27017"
command: mongod --replSet rsnmbp
volumes:
- rs1_data:/data/db
- ./nmbprsdata1:/nmbpdata
rs2:
image: mongo:4.4
ports:
- "27020:27017"
command: mongod --replSet rsnmbp
volumes:
- rs2_data:/data/db
- ./nmbprsdata2:/nmbpdata
rs3:
image: mongo:4.4
ports:
- "27021:27017"
command: mongod --replSet rsnmbp
volumes:
- rs3_data:/data/db
- ./nmbprsdata3:/nmbpdata
rs4:
image: mongo:4.4
ports:
- "27022:27017"
command: mongod --replSet rsnmbp
volumes:
- rs4_data:/data/db
- ./nmbprsdata4:/nmbpdata
volumes:
rs0_data:
rs1_data:
rs2_data:
rs3_data:
rs4_data:
Replica set is configured via
rsconf = {
_id: "rsnmbp",
members: [
{
_id: 0,
host: "rs0:27017"
},
{
_id: 1,
host: "rs1:27017"
},
{
_id: 2,
host: "rs2:27017"
},
{
_id: 3,
host: "rs3:27017"
},
{
_id: 4,
host: "rs4:27017"
},
]
}
rs.initiate(rsconf)
I am trying to connect to replica set via mongoose in node.js
const DB_URI = 'mongodb://localhost:27018,localhost:27019,localhost:27020,localhost:27021,localhost:27022/test'
mongoose.connect(DB_URI)
.then((result) =>console.log ("connected to database"))
.catch((err) =>console.log (err))
but I am receiving following error
MongooseServerSelectionError: getaddrinfo ENOTFOUND rs0
...
reason: TopologyDescription {
type: 'ReplicaSetNoPrimary',
servers: Map(5) {
'rs0:27017' => [ServerDescription],
'rs1:27017' => [ServerDescription],
'rs2:27017' => [ServerDescription],
'rs3:27017' => [ServerDescription],
'rs4:27017' => [ServerDescription]
},
stale: false,
compatible: true,
heartbeatFrequencyMS: 10000,
localThresholdMS: 15,
setName: 'rsnmbp',
maxSetVersion: 1,
maxElectionId: new ObjectId("7fffffff0000000000000003"),
commonWireVersion: 9,
logicalSessionTimeoutMinutes: undefined
}
}
I have added following lines to etc/hosts on Windows
127.0.0.1 rs0
127.0.0.1 rs1
127.0.0.1 rs2
127.0.0.1 rs3
127.0.0.1 rs4
and changed const DB_URI to
const DB_URI = 'mongodb://rs0:27018,rs1:27019,rs2:27020,rs3:27021,rs4:27022/test'
but now I am receiving following error
MongooseServerSelectionError: Server selection timed out after 30000 ms
...
reason: TopologyDescription {
type: 'ReplicaSetNoPrimary',
servers: Map(0) {},
stale: false,
compatible: true,
heartbeatFrequencyMS: 10000,
localThresholdMS: 15,
setName: 'rsnmbp',
maxSetVersion: 1,
maxElectionId: new ObjectId("7fffffff0000000000000003"),
commonWireVersion: 9,
logicalSessionTimeoutMinutes: undefined
}
}
How can I connect to this replicaSet with mongoose in node.js
Thanks in advance.
You must use one Replica set & a Replica set can have multiple members.
and make sure that you have added Replica set nodes in the host machine in etc/hosts file.
Just like the below example -
127.0.0.1 mongoset1 mongoset2 mongoset3
Note - 127.0.0.1 is your host machine and mongoset1, mongoset2 and mongoset3 are the nodes (members) of the replicaset.
I have some issue on connect to mongodb with docker and NodeJS driver.
Here is my docker-compose.yml file:
version: "3.3"
services:
mongodb:
image: mongo:4.0.20
ports:
- 27017:27017
command: mongod --replSet rs0
deploy:
replicas: 1
restart_policy:
condition: "on-failure"
And I launch my service with
docker stack deploy -c ./docker-compose.yml --with-registry-auth XXX
Then, when I'm sure mongo is well started, I run:
docker exec $(docker ps -q -f name=mongodb) mongo local --eval "rs.initiate()"
which initialize the replicat set.
Here is my nodejs test file:
const MongoClient = require('mongodb');
(async () => {
let db;
try {
db = (await MongoClient(
'mongodb://localhost:27017/qos',
{
useNewUrlParser: true,
useUnifiedTopology: true
}
)).db();
console.log('connected');
} catch (error) {
console.trace(error)
}
})()
But when I run the test.js file, I see:
Trace: MongoServerSelectionError: getaddrinfo ENOTFOUND 272b05abb632
at Timeout._onTimeout (node_modules/mongodb/lib/core/sdam/topology.js:438:30)
at listOnTimeout (internal/timers.js:554:17)
at processTimers (internal/timers.js:497:7) {
reason: TopologyDescription {
type: 'ReplicaSetNoPrimary',
setName: 'rs0',
maxSetVersion: 1,
maxElectionId: 7fffffff0000000000000001,
servers: Map { 'bcc9e46ea248:27017' => [ServerDescription] },
stale: false,
compatible: true,
compatibilityError: null,
logicalSessionTimeoutMinutes: null,
heartbeatFrequencyMS: 10000,
localThresholdMS: 15,
commonWireVersion: 7
}
}
at test.js:15:13
I check that 272b05abb632 is the good container id,
and on my docker process I see:
2020-11-10T16:34:49.019360315Z XXX_mongodb.1.awsb46tji1lg#docker-desktop | 2020-11-10T16:34:49.018+0000 I NETWORK [listener] connection accepted from 10.0.0.2:49082 #4 (2 connections now open)
2020-11-10T16:34:49.027564962Z XXX_mongodb.1.awsb46tji1lg#docker-desktop | 2020-11-10T16:34:49.027+0000 I NETWORK [conn4] received client metadata from 10.0.0.2:49082 conn4: { driver: { name: "nodejs", version: "3.6.3" }, os: { type: "Darwin", name: "darwin", architecture: "x64", version: "19.6.0" }, platform: "'Node.js v12.19.0, LE (unified)" }
2020-11-10T16:34:49.033185739Z XXX_mongodb.1.awsb46tji1lg#docker-desktop | 2020-11-10T16:34:49.032+0000 I NETWORK [conn4] end connection 10.0.0.2:49082 (1 connection now open)
I don't understand why with useUnifiedTopology, it's not work :(
I request your help !
I also tried to add mongod.conf in my container via docker-compose, but it doesn't work too:
version: "3.3"
services:
mongodb:
image: mongo:4.0.20
ports:
- 27017:27017
command: mongod --config /etc/mongod.conf
volumes:
- "./mongod.conf:/etc/mongod.conf"
deploy:
replicas: 1
restart_policy:
condition: "on-failure"
security:
authorization: 'enabled'
net:
port: 27017
bindIp: 0.0.0.0 #default value is 127.0.0.1
replication:
replSetName: rs0
My postgres pool is not connecting to the client and i'm almost sure that i passed the right properties to it.
That's the pool instance:
const { Pool } = require('pg')
module.exports = new Pool({
host: 'localhost',
port: 5432,
user: 'laura',
password: 123,
database: 'launchstore'
})
And i created the postgres service with docker-compose:
version: '3.3'
services:
postgres:
container_name: postgres
image: postgres
environment:
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_DB: ${POSTGRES_DATABASE}
ports:
- "5432:5432"
restart: unless-stopped
My env file:
POSTGRES_DATABASE=launchstore
POSTGRES_PASSWORD=123
POSTGRES_USER=laura
When i enter the page, it keeps loading and that's the result why i try to log the pool instance:
Pool {
_events: [Object: null prototype] {},
_eventsCount: 0,
_maxListeners: undefined,
options: {
Client: [Function: Client] { Query: [Function: Query] },
host: 'localhost',
port: 5432,
user: 'laura',
password: 123,
database: 'postgres',
max: 10,
idleTimeoutMillis: 10000
},
log: [Function (anonymous)],
Client: [Function: Client] { Query: [Function: Query] },
Promise: [Function: Promise],
_clients: [],
_idle: [],
_pendingQueue: [],
_endCallback: undefined,
ending: false,
ended: false,
[Symbol(kCapture)]: false
}
If i'm passing the right properties and my container is running, why my client can't connect to the pool?
The PG dependency wasn't installed correctly and that's why the pool wasn't working too. So it wasn't related to any configuration error.