Digital ocean mongo DB docker compose thorws "MongoServerError: Authentication failed." - node.js

Wrapping up a docker tutorial. When it comes to docker containers on my laptop, it works fine no Mongo DB issue.
But ever since I put it on digital ocean(Ubuntu 20.04 LTS x64, 2GB memory, 10GB disk) the issue occured below:
MongoServerError: Authentication failed.
at Connection.onMessage (/app/node_modules/mongodb/lib/cmap/connection.js:207:30)
at MessageStream.<anonymous> (/app/node_modules/mongodb/lib/cmap/connection.js:60:60)
at MessageStream.emit (node:events:513:28)
at processIncomingData (/app/node_modules/mongodb/lib/cmap/message_stream.js:132:20)
at MessageStream._write (/app/node_modules/mongodb/lib/cmap/message_stream.js:33:9)
at writeOrBuffer (node:internal/streams/writable:392:12)
at _write (node:internal/streams/writable:333:10)
at Writable.write (node:internal/streams/writable:337:10)
at Socket.ondata (node:internal/streams/readable:766:22)
at Socket.emit (node:events:513:28) {
ok: 0,
code: 18,
codeName: 'AuthenticationFailed',
connectionGeneration: 0,
[Symbol(errorLabels)]: Set(2) { 'HandshakeError', 'ResetPool' }
codes:
package.json
"dependencies": {
"bcryptjs": "^2.4.3",
"connect-redis": "^6.1.3",
"cors": "^2.8.5",
"express": "^4.18.2",
"express-session": "^1.17.3",
"mongoose": "^6.7.2",
"redis": "^4.5.0"
},
index.js
const {
MONGO_USER,
MONGO_PASSWORD,
MONGO_IP,
MONGO_PORT,
REDIS_URL,
REDIS_PORT,
SESSION_SECRET,
} = require("./config/config");
let redisClient = redis.createClient({
legacyMode: true,
socket: {
host: REDIS_URL,
port: REDIS_PORT,
},
});
redisClient
.connect()
.then(() => console.log("redis connected"))
.catch((e) => console.error("redis error", e));
const postRouter = require("./routes/postRoutes");
const userRouter = require("./routes/userRoutes");
const app = express();
const connectWithRetry = () => {
mongoose
.connect(
`mongodb://${MONGO_USER}:${MONGO_PASSWORD}#${MONGO_IP}:${MONGO_PORT}/?authSource=admin`
)
.then(() => console.log("successfully connected to DB"))
.catch((e) => {
console.error(e);
setTimeout(connectWithRetry, 5000);
});
};
connectWithRetry();
app.enable("trust proxy");
app.use(cors({}));
...
docker-compose.yml
version: "3"
services:
nginx:
image: nginx:stable-alpine
ports:
- "3000:80"
volumes:
- ./nginx/default.conf:/etc/nginx/conf.d/default.conf:ro
depends_on:
- node-app
node-app:
build: .
environment:
- PORT=3000
depends_on:
- mongo
mongo:
image: mongo
environment:
- MONGO_INITDB_ROOT_USERNAME=snap
- MONGO_INITDB_ROOT_PASSWORD=mypassword
volumes:
- mongo-db:/data/db
redis:
image: redis
volumes:
mongo-db:
docker-compose.prod.yml
version: "3"
services:
nginx:
ports:
- "80:80"
node-app:
build:
context: .
args:
NODE_ENV: production
environment:
- NODE_ENV=production
- MONGO_USER=${MONGO_USER}
- MONGO_PASSWORD=${MONGO_PASSWORD}
- SESSION_SECRET=${SESSION_SECRET}
command: node index.js
mongo:
environment:
- MONGO_INITDB_ROOT_USERNAME=${MONGO_INITDB_ROOT_USERNAME}
- MONGO_INITDB_ROOT_PASSWORD=${MONGO_INITDB_ROOT_PASSWORD}
The environment variables stored in .env file which located in the root directory of Ubuntu droplet. I tried hard coding the mongo uri instead of using environment variables but got the same result so pretty sure it may not be a cause.
Here's a log of db users below:
use admin
switched to db admin
admin> db.system.users.find()
[
{
_id: 'admin.snap',
userId: new UUID("4e79e245-894e-4d9a-9d76-802c970d9129"),
user: 'snap',
db: 'admin',
...
},
roles: [ { role: 'root', db: 'admin' } ]
},
{
_id: 'admin.admin',
userId: new UUID("0a8a471e-0e70-4e1c-b9ed-d99ede9ad0bd"),
user: 'admin',
db: 'admin',
...
},
roles: [ { role: 'root', db: 'admin' } ]
},
{
_id: 'test.snap',
userId: new UUID("4282b24f-9b15-4956-865f-ef9b206a499d"),
user: 'snap',
db: 'test',
...
roles: [ { role: 'readWrite', db: 'test' } ]
}
]
Thanks for your help.

Related

Docker Node MongoDB AuthenticationFailed: SCRAM-SHA-1 authentication failed, storedKey mismatch

I am posting it here as I am run out of options.
I am trying to make a connection from my node app to the mongodb.
I am getting AuthenticationFailed: SCRAM-SHA-1 authentication failed, storedKey mismatch
My local environment works 100%. I dumped my local MongoDB (the app database and admin too) into the docker container.
I created my docker-compose.yml as below:
version: "1.0"
services:
mongodb:
image: mongo:3.4.7
container_name: MongoDB2
restart: unless-stopped
ports:
- '27017:27017'
app:
links:
- mongodb
depends_on:
- mongodb
image: eamello/gsd:myCore
ports:
- '8087:8087'
stdin_open: true
tty: true
volumes:
db:
networks:
node-webapp-network:
driver: bridge
My config.json file, which has the database connection details:
"myCore": {
"database": {
"url": "mongodb://mongodb:27017/myCore",
"options": {
"db": {
"native_parser": true
},
"server": {
"poolSize": 100,
"socketOptions": {
"keepAlive": 1000,
"connectTimeoutMS": 30000
}
},
"replset": {},
"user": "myAdmin",
"pass": "/WnUU5Jqithypb9970AfIQ==",
"auth": {
"authdb": "admin"
},
"queryLevel":{
"common":{
"maxTimeMS": 15000
}
}
}
},
I am 100% sure the user created in my admin database has the same password.
I check and rechecked several times.
I also tried to add my user via javascript file... it looks like the javascript was never executed.
db.createUser(
{
user: "myAdmin",
pwd: "/WnUU5Jqithypb9970AfIQ==",
roles: [
{
role: "userAdmin", db: "myCore"
},
{
role: "readWrite", db: "myCore"
}
]
}
);
As I can manage my MongoDB via Compass, I left this javascript aside.
Does anyone have any clue why I am getting AuthenticationFailed: SCRAM-SHA-1 authentication failed, storedKey mismatch?
I changed some names above as this is a company issue. Thanks.

MongoClient cannot connect to mongo in another Docker container

I am currently trying to deploy my app with docker but I have some problem with mongodb.
My app is not succeeding to connect to mongodb in another container.
Here is my docker-compose.yml file:
version: "3.7"
services:
mqtt_broker:
build:
context: .
dockerfile: Dockerfile
container_name: mqtt_broker
ports:
- "4040:4040"
links:
- mqtt_db:mqtt_db
depends_on:
- mqtt_db
mqtt_db:
build:
context: ./mongo
dockerfile: Dockerfile
container_name: mqtt_db
ports:
- "27017:27017"
Here is the Dockerfile for the mongo container:
FROM mongo:latest
COPY init_docker.js /docker-entrypoint-initdb.d/
CMD mongod --replSet "rs0" --bind_ip_all
Here is how I connect to mongo in the app container:
const client = new MongoClient(config.get("dbURL"));
const mongo = await client.connect();
const db = client.db(config.get("dbName"));
with the configuration file:
{
"cors": {
"origin": "*"
},
"clientPort": 4040,
"mqttPort": 1883,
"dbURL": "mongodb://mqtt_db:27017",
"dbName": "mqtt_proxy"
}
Here the error I get with the mongo node client:
MongoServerSelectionError: Server selection timed out after 30000 ms
at Timeout._onTimeout (/broker/node_modules/mongodb/lib/sdam/topology.js:305:38)
at listOnTimeout (node:internal/timers:559:17)
at processTimers (node:internal/timers:502:7) {
reason: TopologyDescription {
type: 'Unknown',
servers: Map(1) {
'mqtt_db:27017' => ServerDescription {
_hostAddress: HostAddress { isIPv6: false, host: 'mqtt_db', port: 27017 },
address: 'mqtt_db:27017',
type: 'RSGhost',
hosts: [],
passives: [],
arbiters: [],
tags: {},
minWireVersion: 0,
maxWireVersion: 13,
roundTripTime: 35.24000000000001,
lastUpdateTime: 13896371,
lastWriteDate: 0,
topologyVersion: {
processId: ObjectId { [Symbol(id)]: [Buffer [Uint8Array]] },
counter: 0
},
logicalSessionTimeoutMinutes: 30
}
},
stale: false,
compatible: true,
heartbeatFrequencyMS: 10000,
localThresholdMS: 15,
commonWireVersion: 13,
logicalSessionTimeoutMinutes: undefined
},
code: undefined,
[Symbol(errorLabels)]: Set(0) {}
}
I can actually ping mqtt_db, the ip is resolved, but I don't see it in /etc/hosts.
I also tried to add networks in my docker-compose but it translated mqtt_db in localhost and it does not work.
Here is the repo of my project: https://gitlab.com/mqtt-broker-sniffer/sniffer
EDIT
I had to specify ?directConnection=true when I was connecting to the db.
Crédit: https://stackoverflow.com/a/70204195/19127936
const client = new MongoClient(
`${config.get("dbURL")}/${config.get("dbName")}?directConnection=true`
);
this.client = await client.connect();
this.db = client.db(config.get("dbName"));

MongoDB replica set, error when connecting from mongoose node.js

I am running mongodb replicaSet inside docker containers in Windows.
This is docker compose file
version: '3'
services:
rs0:
image: mongo:4.4
ports:
- "27018:27017"
command: mongod --replSet rsnmbp
volumes:
- rs0_data:/data/db
- ./nmbprsdata0:/nmbpdata
rs1:
image: mongo:4.4
ports:
- "27019:27017"
command: mongod --replSet rsnmbp
volumes:
- rs1_data:/data/db
- ./nmbprsdata1:/nmbpdata
rs2:
image: mongo:4.4
ports:
- "27020:27017"
command: mongod --replSet rsnmbp
volumes:
- rs2_data:/data/db
- ./nmbprsdata2:/nmbpdata
rs3:
image: mongo:4.4
ports:
- "27021:27017"
command: mongod --replSet rsnmbp
volumes:
- rs3_data:/data/db
- ./nmbprsdata3:/nmbpdata
rs4:
image: mongo:4.4
ports:
- "27022:27017"
command: mongod --replSet rsnmbp
volumes:
- rs4_data:/data/db
- ./nmbprsdata4:/nmbpdata
volumes:
rs0_data:
rs1_data:
rs2_data:
rs3_data:
rs4_data:
Replica set is configured via
rsconf = {
_id: "rsnmbp",
members: [
{
_id: 0,
host: "rs0:27017"
},
{
_id: 1,
host: "rs1:27017"
},
{
_id: 2,
host: "rs2:27017"
},
{
_id: 3,
host: "rs3:27017"
},
{
_id: 4,
host: "rs4:27017"
},
]
}
rs.initiate(rsconf)
I am trying to connect to replica set via mongoose in node.js
const DB_URI = 'mongodb://localhost:27018,localhost:27019,localhost:27020,localhost:27021,localhost:27022/test'
mongoose.connect(DB_URI)
.then((result) =>console.log ("connected to database"))
.catch((err) =>console.log (err))
but I am receiving following error
MongooseServerSelectionError: getaddrinfo ENOTFOUND rs0
...
reason: TopologyDescription {
type: 'ReplicaSetNoPrimary',
servers: Map(5) {
'rs0:27017' => [ServerDescription],
'rs1:27017' => [ServerDescription],
'rs2:27017' => [ServerDescription],
'rs3:27017' => [ServerDescription],
'rs4:27017' => [ServerDescription]
},
stale: false,
compatible: true,
heartbeatFrequencyMS: 10000,
localThresholdMS: 15,
setName: 'rsnmbp',
maxSetVersion: 1,
maxElectionId: new ObjectId("7fffffff0000000000000003"),
commonWireVersion: 9,
logicalSessionTimeoutMinutes: undefined
}
}
I have added following lines to etc/hosts on Windows
127.0.0.1 rs0
127.0.0.1 rs1
127.0.0.1 rs2
127.0.0.1 rs3
127.0.0.1 rs4
and changed const DB_URI to
const DB_URI = 'mongodb://rs0:27018,rs1:27019,rs2:27020,rs3:27021,rs4:27022/test'
but now I am receiving following error
MongooseServerSelectionError: Server selection timed out after 30000 ms
...
reason: TopologyDescription {
type: 'ReplicaSetNoPrimary',
servers: Map(0) {},
stale: false,
compatible: true,
heartbeatFrequencyMS: 10000,
localThresholdMS: 15,
setName: 'rsnmbp',
maxSetVersion: 1,
maxElectionId: new ObjectId("7fffffff0000000000000003"),
commonWireVersion: 9,
logicalSessionTimeoutMinutes: undefined
}
}
How can I connect to this replicaSet with mongoose in node.js
Thanks in advance.
You must use one Replica set & a Replica set can have multiple members.
and make sure that you have added Replica set nodes in the host machine in etc/hosts file.
Just like the below example -
127.0.0.1 mongoset1 mongoset2 mongoset3
Note - 127.0.0.1 is your host machine and mongoset1, mongoset2 and mongoset3 are the nodes (members) of the replicaset.

Connecting to MongoDB - Command 'createIndexes' requires authentication (Maybe Agenda.js related?)

As I'm introducing authentication for my MongoDB instance in Docker, I ran into problems which are probably related to the way agenda.js tries to connect to MongoDB, as the connection string invokes successful logs for mongoose connecting to the DB, therefore I assume the string should be valid.
Everything worked until I changed the connection string to use authentication. I verified the users are properly created in the database and also tried variations of the connection string and deleting/installing node modules.
Following output I get upon running docker-compose up:
server | /server/node_modules/agenda/node_modules/mongodb/lib/utils.js:133
server | throw err;
server | ^
server |
server | MongoError: command createIndexes requires authentication
server | at Connection.<anonymous> (/server/node_modules/agenda/node_modules/mongodb/lib/core/connection/pool.js:466:61)
server | at Connection.emit (events.js:314:20)
server | at processMessage (/server/node_modules/agenda/node_modules/mongodb/lib/core/connection/connection.js:384:10)
server | at Socket.<anonymous> (/server/node_modules/agenda/node_modules/mongodb/lib/core/connection/connection.js:553:15)
server | at Socket.emit (events.js:314:20)
server | at addChunk (_stream_readable.js:297:12)
server | at readableAddChunk (_stream_readable.js:272:9)
server | at Socket.Readable.push (_stream_readable.js:213:10)
server | at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
server | Emitted 'error' event on Agenda instance at:
server | at /server/node_modules/agenda/lib/agenda/db-init.js:23:14
server | at /server/node_modules/agenda/node_modules/mongodb/lib/operations/execute_operation.js:76:14
server | at /server/node_modules/agenda/node_modules/mongodb/lib/operations/execute_operation.js:63:27
server | at ClientSession.endSession (/server/node_modules/agenda/node_modules/mongodb/lib/core/sessions.js:135:41)
server | at executeCallback (/server/node_modules/agenda/node_modules/mongodb/lib/operations/execute_operation.js:59:17)
server | at handleCallback (/server/node_modules/agenda/node_modules/mongodb/lib/utils.js:129:55)
server | at /server/node_modules/agenda/node_modules/mongodb/lib/operations/create_index.js:85:14
server | at handleCallback (/server/node_modules/agenda/node_modules/mongodb/lib/utils.js:129:55)
server | at /server/node_modules/agenda/node_modules/mongodb/lib/operations/command.js:113:23
server | at /server/node_modules/agenda/node_modules/mongodb/lib/core/connection/pool.js:420:18
server | at processTicksAndRejections (internal/process/task_queues.js:79:11) {
server | ok: 0,
server | errmsg: 'command createIndexes requires authentication',
server | code: 13,
server | codeName: 'Unauthorized',
server | [Symbol(mongoErrorContextSymbol)]: {}
server | }
package.json
"dependencies": {
"agenda": "^4.1.2",
"axios": "^0.21.1",
"bcryptjs": "^2.4.3",
"body-parser": "latest",
"express": "^4.17.1",
"infinite-timeout": "^0.1.0",
"jsonwebtoken": "^8.5.1",
"method-override": "^3.0.0",
"mongoose": "^5.12.4",
"mosca": "^2.8.3",
"passport": "^0.4.1",
"passport-jwt": "^4.0.0",
"websocket": "^1.0.34",
"webstorm": "^1.0.0",
"ws": "latest"
},
"devDependencies": {
"#shelf/jest-mongodb": "^1.1.3",
"#types/express": "^4.17.1",
"cors": "^2.8.5",
"jest": "^24.9.0"
},
"engines": {
"node": "12.x"
}
docker-compose.yml
version: "3.8"
services:
server:
container_name: server
build: .
restart: unless-stopped
ports:
- "9000:9000"
- "1883:1883"
links:
- mongo
networks:
- app-network
command: npm start
mongo:
container_name: mongo
image: mongo:4.4.4
environment:
- MONGO_INITDB_ROOT_USERNAME=root
- MONGO_INITDB_ROOT_PASSWORD=superSafePassword
ports:
- "27317:27017"
volumes:
- dbdata:/data/db
- "./init.js/:/docker-entrypoint-initdb.d/init.js:ro"
restart: unless-stopped
networks:
- app-network
command: mongod
networks:
app-network:
driver: bridge
volumes:
dbdata:
node_modules:
Script to create users init.js
db = db.getSiblingDB("dbName");
db.createUser(
{
user: "dbUser",
pwd: "dbPassword",
roles: [
{
role: "readWrite",
db: "dbName"
}
]
}
);
Connection string
// mongo refers to the docker container
connectionString : 'mongodb://dbUser:dbPassword#mongo:27317/dbName',
How I initialize agenda:
class mScheduler {
constructor() {
if(typeof mScheduler.instance === 'object')
return mScheduler.instance;
this.agenda = new Agenda({ db: {address: config.connectionString, collection: 'agendaJobs', options: { useNewUrlParser: true }}, processEvery: '2 seconds', maxConcurrency : MAX_CONC});
process.on('SIGTERM', this.endJobs);
process.on('SIGINT', this.endJobs);
setImmediate(async ()=> await this.agenda.start());
mScheduler.instance = this;
return this;
}
someOtherMethods()
}
module.exports = new mScheduler();
try this
workaround these deprecation warnings with Mongoose provided options
const connectionOptions = {
useNewUrlParser: true,
useCreateIndex: true,
useFindAndModify: false,
useUnifiedTopology: true,
autoIndex: false
};
this.agenda = new Agenda({ db: {address: config.connectionString, collection: 'agendaJobs', options: connectionOptions}, processEvery: '2 seconds', maxConcurrency : MAX_CONC});

MongoServerSelectionError: getaddrinfo ENOTFOUND

I have some issue on connect to mongodb with docker and NodeJS driver.
Here is my docker-compose.yml file:
version: "3.3"
services:
mongodb:
image: mongo:4.0.20
ports:
- 27017:27017
command: mongod --replSet rs0
deploy:
replicas: 1
restart_policy:
condition: "on-failure"
And I launch my service with
docker stack deploy -c ./docker-compose.yml --with-registry-auth XXX
Then, when I'm sure mongo is well started, I run:
docker exec $(docker ps -q -f name=mongodb) mongo local --eval "rs.initiate()"
which initialize the replicat set.
Here is my nodejs test file:
const MongoClient = require('mongodb');
(async () => {
let db;
try {
db = (await MongoClient(
'mongodb://localhost:27017/qos',
{
useNewUrlParser: true,
useUnifiedTopology: true
}
)).db();
console.log('connected');
} catch (error) {
console.trace(error)
}
})()
But when I run the test.js file, I see:
Trace: MongoServerSelectionError: getaddrinfo ENOTFOUND 272b05abb632
at Timeout._onTimeout (node_modules/mongodb/lib/core/sdam/topology.js:438:30)
at listOnTimeout (internal/timers.js:554:17)
at processTimers (internal/timers.js:497:7) {
reason: TopologyDescription {
type: 'ReplicaSetNoPrimary',
setName: 'rs0',
maxSetVersion: 1,
maxElectionId: 7fffffff0000000000000001,
servers: Map { 'bcc9e46ea248:27017' => [ServerDescription] },
stale: false,
compatible: true,
compatibilityError: null,
logicalSessionTimeoutMinutes: null,
heartbeatFrequencyMS: 10000,
localThresholdMS: 15,
commonWireVersion: 7
}
}
at test.js:15:13
I check that 272b05abb632 is the good container id,
and on my docker process I see:
2020-11-10T16:34:49.019360315Z XXX_mongodb.1.awsb46tji1lg#docker-desktop | 2020-11-10T16:34:49.018+0000 I NETWORK [listener] connection accepted from 10.0.0.2:49082 #4 (2 connections now open)
2020-11-10T16:34:49.027564962Z XXX_mongodb.1.awsb46tji1lg#docker-desktop | 2020-11-10T16:34:49.027+0000 I NETWORK [conn4] received client metadata from 10.0.0.2:49082 conn4: { driver: { name: "nodejs", version: "3.6.3" }, os: { type: "Darwin", name: "darwin", architecture: "x64", version: "19.6.0" }, platform: "'Node.js v12.19.0, LE (unified)" }
2020-11-10T16:34:49.033185739Z XXX_mongodb.1.awsb46tji1lg#docker-desktop | 2020-11-10T16:34:49.032+0000 I NETWORK [conn4] end connection 10.0.0.2:49082 (1 connection now open)
I don't understand why with useUnifiedTopology, it's not work :(
I request your help !
I also tried to add mongod.conf in my container via docker-compose, but it doesn't work too:
version: "3.3"
services:
mongodb:
image: mongo:4.0.20
ports:
- 27017:27017
command: mongod --config /etc/mongod.conf
volumes:
- "./mongod.conf:/etc/mongod.conf"
deploy:
replicas: 1
restart_policy:
condition: "on-failure"
security:
authorization: 'enabled'
net:
port: 27017
bindIp: 0.0.0.0 #default value is 127.0.0.1
replication:
replSetName: rs0

Resources