Mongoose Failing to connect to MongoDB replicaSet - node.js

I have a replicaSet running in k8s, while trying to connect to the replicaSet from nodejs using mongoose, the connection fails with this logs:
MongooseServerSelectionError: connection timed out
at NativeConnection.Connection.openUri (/app/node_modules/mongoose/lib/connection.js:807:32)
at /app/node_modules/mongoose/lib/index.js:340:10
at promiseOrCallback (/app/node_modules/mongoose/lib/helpers/promiseOrCallback.js:10:12)
at Mongoose._promiseOrCallback (/app/node_modules/mongoose/lib/index.js:1140:10)
at Mongoose.connect (/app/node_modules/mongoose/lib/index.js:339:20)
at /app/src/connect.ts:21:14
at Promise._execute (/app/node_modules/bluebird/js/release/debuggability.js:313:9)
at Promise._resolveFromExecutor (/app/node_modules/bluebird/js/release/promise.js:488:18)
at new Promise (/app/node_modules/bluebird/js/release/promise.js:79:10)
at connect (/app/src/connect.ts:20:10)
at initialiseDatabase (/app/src/connect.ts:8:5)
at Object.<anonymous> (/app/src/mongoose.ts:36:19)
at Module._compile (internal/modules/cjs/loader.js:1085:14)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
at Module.load (internal/modules/cjs/loader.js:950:32)
at Function.Module._load (internal/modules/cjs/loader.js:790:12)
at Module.require (internal/modules/cjs/loader.js:974:19)
at require (internal/modules/cjs/helpers.js:101:18)
at Object.<anonymous> (/app/src/user/operations/find-by-id.ts:1:1)
at Module._compile (internal/modules/cjs/loader.js:1085:14)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
at Module.load (internal/modules/cjs/loader.js:950:32)
at Function.Module._load (internal/modules/cjs/loader.js:790:12)
at Module.require (internal/modules/cjs/loader.js:974:19)
at require (internal/modules/cjs/helpers.js:101:18)
at Object.<anonymous> (/app/src/middleware/authenticate.ts:7:1)
at Module._compile (internal/modules/cjs/loader.js:1085:14)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
at Module.load (internal/modules/cjs/loader.js:950:32)
at Function.Module._load (internal/modules/cjs/loader.js:790:12)
at Module.require (internal/modules/cjs/loader.js:974:19)
at require (internal/modules/cjs/helpers.js:101:18)
at Object.<anonymous> (/app/src/app.ts:8:1)
at Module._compile (internal/modules/cjs/loader.js:1085:14)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
at Module.load (internal/modules/cjs/loader.js:950:32)
at Function.Module._load (internal/modules/cjs/loader.js:790:12)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:75:12)
at internal/main/run_main_module.js:17:47 {
reason: TopologyDescription {
type: 'ReplicaSetNoPrimary',
servers: Map(3) {
'mongodb-0.mongodb-headless.staging.svc.cluster.local:27017' => [ServerDescription],
'mongodb-1.mongodb-headless.staging.svc.cluster.local:27017' => [ServerDescription],
'mongodb-2.mongodb-headless.staging.svc.cluster.local:27017' => [ServerDescription]
},
And this is my connection string:
mongodb://username:pw#mongodb-0.mongodb-headless.staging.svc.cluster.local:27017,mongodb-1.mongodb-headless.staging.svc.cluster.local:27017,mongodb-2.mongodb-headless.staging.svc.cluster.local:27017/labxpert?authSource=admin&serverSelectionTimeoutMS=50000&replicaSet=rs0
When I test the connection with the mongo Shell, it works just fine.
What could be causing the MongooseServerSelectionError: connection timed out.

For anyone who may face this problem with mongoose v6+ and nodejs and white listing your ip has not solved it or changing from localhost to 127.0.0.1.
Solution
Set the following options for the mongoose client connection options:
mongoose.connect(url, {
maxIdleTimeMS: 80000,
serverSelectionTimeoutMS: 80000,
socketTimeoutMS: 0,
connectTimeoutMS: 0
}
These options stop the mongoose client from timining out the connection before it selects a primary from the DB cluster.
Please remember to set any other options relevant to your setup like if you are running a cluster, make sure to set the replicaSet option or if you have authentication enabled, set the authSource option or the database you are connecting to dbName.
Some of these options can be set in the connection string.
The full list of options can be found here: https://mongodb.github.io/node-mongodb-native/4.2/interfaces/MongoClientOptions.html#authSource

Related

How can I connect MongoDB with Nodejs using Mongoose

I'm watching a two year-old tutorial from YouTube, and I stuck at here.
throw error;
^
SetOptionError: mongodb+srv://BilalAhmed:password/Test: "mongodb+srv://BilalAhmed:password/Test" is not a valid option to set
at Mongoose.set (F:\Webdevelopment\Node JS\node_modules\mongoose\lib\index.js:249:17)
at Object.<anonymous> (F:\Webdevelopment\Node JS\NodeWithMongoDB.js:5:10)
at Module._compile (node:internal/modules/cjs/loader:1159:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1213:10)
at Module.load (node:internal/modules/cjs/loader:1037:32)
at Module._load (node:internal/modules/cjs/loader:878:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
at node:internal/main/run_main_module:23:47 {
errors: {
'mongodb+srv://BilalAhmed:password/Test': SetOptionInnerError [MongooseError]: "mongodb+srv://BilalAhmed:password/Test" is not a valid option to set
at Mongoose.set (F:\Webdevelopment\Node JS\node_modules\mongoose\lib\index.js:251:33)
at Object.<anonymous> (F:\Webdevelopment\Node JS\NodeWithMongoDB.js:5:10)
at Module._compile (node:internal/modules/cjs/loader:1159:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1213:10)
at Module.load (node:internal/modules/cjs/loader:1037:32)
at Module._load (node:internal/modules/cjs/loader:878:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
at node:internal/main/run_main_module:23:47
}
}
const mongoose = require('mongoose')
mongoose.set('mongodb+srv://BilalAhmed:password/Test',
{
useNewUrlParser: false,
useUnifiedTopology: true
}).then(() => { console.warn("Mongodb connected") })
I just wanted to connect with Mongo DB server and perform CRUD operation.

I have tried to connect mongodb with my server. But the following error shows

ReferenceError: client is not defined
at run (C:\Users\DELOWAR\Desktop\Assignment 11\inventory-management-server\index.js:21:5)
at Object.<anonymous> (C:\Users\DELOWAR\Desktop\Assignment 11\inventory-management-server\index.js:38:1)
at Module._compile (internal/modules/cjs/loader.js:1072:14)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1101:10)
at Module.load (internal/modules/cjs/loader.js:937:32)
at Function.Module._load (internal/modules/cjs/loader.js:778:12)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:76:12)
at internal/main/run_main_module.js:17:47
First console the Uri
const client = new MongoClient(uri, {
useNewUrlParser: true,
useUnifiedTopology: true,
serverApi: ServerApiVersion.v1,

Error creating database in sequelize Dialect needs to be explicitly supplied as of v4.0.0

The error statement:
Error: Dialect needs to be explicitly supplied as of v4.0.0 at new Sequelize (D:\tutorial\node_modules\sequelize\lib\sequelize.js:181:13) at Object. (D:\tutorial\util\database.js:3:19) at Module._compile (internal/modules/cjs/loader.js:689:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10) at Module.load (internal/modules/cjs/loader.js:599:32) at tryModuleLoad (internal/modules/cjs/loader.js:538:12) at Function.Module._load (internal/modules/cjs/loader.js:530:3) at Module.require (internal/modules/cjs/loader.js:637:17) at require (internal/modules/cjs/helpers.js:22:18) at Object. (D:\tutorial\app.js:9:19) at Module._compile (internal/modules/cjs/loader.js:689:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10) at Module.load (internal/modules/cjs/loader.js:599:32) at tryModuleLoad (internal/modules/cjs/loader.js:538:12) at Function.Module._load (internal/modules/cjs/loader.js:530:3) at Function.Module.runMain (internal/modules/cjs/loader.js:742:12) at startup (internal/bootstrap/node.js:283:19) at bootstrapNodeJSCore (internal/bootstrap/node.js:743:3)
My database request is:
const Sequelize = require('sequelize');
const sequelize = new Sequelize('node_complete', 'root', 'Kornienko130', 'nodecomplete', {
dialect: 'mysql',
host: 'localhost',
});
module.exports = sequelize;
The dialect is explicitly stated, I don't know what's wrong, I googled it so many times and none of the solutions seems to work.

Mongoose Error must pass in valid bson parser

So I have a docker container (running a testing suite environment) that has stopped work on me. I'm using Mongoose 4.13.17. The problem I'm running into is every time I try to run my test cases it tries to connect to a testing db container but I get this error message.
Error: must pass in valid bson parser
at new Pool (/srv/app/node_modules/mongoose/node_modules/mongodb-core/lib/connection/pool.js:105:13)
at Server.connect (/srv/app/node_modules/mongoose/node_modules/mongodb-core/lib/topologies/server.js:373:17)
at Server.connect (/srv/app/node_modules/mongoose/node_modules/mongodb/lib/server.js:368:17)
at open (/srv/app/node_modules/mongoose/node_modules/mongodb/lib/db.js:229:19)
at Db.open (/srv/app/node_modules/mongoose/node_modules/mongodb/lib/db.js:252:44)
at createServer (/srv/app/node_modules/mongoose/node_modules/mongodb/lib/mongo_client.js:391:6)
at /srv/app/node_modules/mongoose/node_modules/mongodb/lib/mongo_client.js:512:14
at parseHandler (/srv/app/node_modules/mongoose/node_modules/mongodb/lib/url_parser.js:117:38)
at module.exports (/srv/app/node_modules/mongoose/node_modules/mongodb/lib/url_parser.js:97:5)
at connect (/srv/app/node_modules/mongoose/node_modules/mongodb/lib/mongo_client.js:485:3)
at Function.MongoClient.connect (/srv/app/node_modules/mongoose/node_modules/mongodb/lib/mongo_client.js:250:3)
at /srv/app/node_modules/mongoose/lib/connection.js:820:25
at Promise._execute (/srv/app/node_modules/bluebird/js/release/debuggability.js:313:9)
at Promise._resolveFromExecutor (/srv/app/node_modules/bluebird/js/release/promise.js:483:18)
at new Promise (/srv/app/node_modules/bluebird/js/release/promise.js:79:10)
at NativeConnection.Connection.openUri (/srv/app/node_modules/mongoose/lib/connection.js:819:17)
at Mongoose.connect (/srv/app/node_modules/mongoose/lib/index.js:262:17)
at new App (/srv/app/src/app.ts:34:5)
at Object.<anonymous> (/srv/app/src/app.ts:80:16)
at Module._compile (internal/modules/cjs/loader.js:689:30)
at Module.replacementCompile (/srv/app/node_modules/nyc/node_modules/append-transform/index.js:58:13)
at Module.m._compile (/srv/app/node_modules/ts-node/src/index.ts:439:23)
at module.exports (/srv/app/node_modules/nyc/node_modules/default-require-extensions/js.js:7:9)
at /srv/app/node_modules/nyc/node_modules/append-transform/index.js:62:4
at require.extensions.(anonymous function) (/srv/app/node_modules/ts-node/src/index.ts:442:12)
at Object.<anonymous> (/srv/app/node_modules/nyc/node_modules/append-transform/index.js:62:4)
at Module.load (internal/modules/cjs/loader.js:599:32)
at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
at Function.Module._load (internal/modules/cjs/loader.js:530:3)
at Module.require (internal/modules/cjs/loader.js:637:17)
at require (internal/modules/cjs/helpers.js:20:18)
at Object.<anonymous> (/srv/app/src/server.ts:6:1)
at Module._compile (internal/modules/cjs/loader.js:689:30)
at Module.replacementCompile (/srv/app/node_modules/nyc/node_modules/append-transform/index.js:58:13)
at Module.m._compile (/srv/app/node_modules/ts-node/src/index.ts:439:23)
at module.exports (/srv/app/node_modules/nyc/node_modules/default-require-extensions/js.js:7:9)
at /srv/app/node_modules/nyc/node_modules/append-transform/index.js:62:4
at require.extensions.(anonymous function) (/srv/app/node_modules/ts-node/src/index.ts:442:12)
at Object.<anonymous> (/srv/app/node_modules/nyc/node_modules/append-transform/index.js:62:4)
at Module.load (internal/modules/cjs/loader.js:599:32)
at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
at Function.Module._load (internal/modules/cjs/loader.js:530:3)
at Module.require (internal/modules/cjs/loader.js:637:17)
at require (internal/modules/cjs/helpers.js:20:18)
at Object.<anonymous> (/srv/app/test/unit/rest/Audio.test.ts:3:1)
at Module._compile (internal/modules/cjs/loader.js:689:30)
at Module.replacementCompile (/srv/app/node_modules/nyc/node_modules/append-transform/index.js:58:13)
at Module.m._compile (/srv/app/node_modules/ts-node/src/index.ts:439:23)
at module.exports (/srv/app/node_modules/nyc/node_modules/default-require-extensions/js.js:7:9)
at /srv/app/node_modules/nyc/node_modules/append-transform/index.js:62:4
at require.extensions.(anonymous function) (/srv/app/node_modules/ts-node/src/index.ts:442:12)
at Object.<anonymous> (/srv/app/node_modules/nyc/node_modules/append-transform/index.js:62:4)
at Module.load (internal/modules/cjs/loader.js:599:32)
at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
at Function.Module._load (internal/modules/cjs/loader.js:530:3)
at Module.require (internal/modules/cjs/loader.js:637:17)
at require (internal/modules/cjs/helpers.js:20:18)
at /srv/app/node_modules/mocha/lib/mocha.js:250:27
at Array.forEach (<anonymous>)
at Mocha.loadFiles (/srv/app/node_modules/mocha/lib/mocha.js:247:14)
at Mocha.run (/srv/app/node_modules/mocha/lib/mocha.js:576:10)
at Object.<anonymous> (/srv/app/node_modules/mocha/bin/_mocha:637:18)
at Module._compile (internal/modules/cjs/loader.js:689:30)
at Module.replacementCompile (/srv/app/node_modules/nyc/node_modules/append-transform/index.js:58:13)
at Module._extensions..js (internal/modules/cjs/loader.js:700:10)
at Object.<anonymous> (/srv/app/node_modules/nyc/node_modules/append-transform/index.js:62:4)
at Module.load (internal/modules/cjs/loader.js:599:32)
at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
at Function.Module._load (internal/modules/cjs/loader.js:530:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:742:12)
at runMain (/root/.node-spawn-wrap-45-4803909aaae7/node:68:10)
at Function.<anonymous> (/root/.node-spawn-wrap-45-4803909aaae7/node:171:5)
at Object.<anonymous> (/srv/app/node_modules/nyc/bin/wrap.js:23:4)
at Module._compile (internal/modules/cjs/loader.js:689:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
at Module.load (internal/modules/cjs/loader.js:599:32)
at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
at Function.Module._load (internal/modules/cjs/loader.js:530:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:742:12)
at /root/.node-spawn-wrap-45-4803909aaae7/node:178:8
at Object.<anonymous> (/root/.node-spawn-wrap-45-4803909aaae7/node:181:3)
at Module._compile (internal/modules/cjs/loader.js:689:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
at Module.load (internal/modules/cjs/loader.js:599:32)
at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
at Function.Module._load (internal/modules/cjs/loader.js:530:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:742:12)
at startup (internal/bootstrap/node.js:279:19)
at bootstrapNodeJSCore (internal/bootstrap/node.js:696:3)
I've looked all around for a solution but it doesn't seem like this is an extremely common problem.
This is what my class looks like:
import * as bluebird from 'bluebird';
import { Application } from 'express';
import { connect, connection, disconnect, mongo } from 'mongoose';
import * as mongoose from 'mongoose';
import 'reflect-metadata'; // this shim is required for routing-controllers
import { createExpressServer, useContainer } from 'routing-controllers';
import { Container } from 'typedi';
import { getEnv } from './Service/Env';
// middleware
import { AuthenticateTokenMiddleware } from './Middleware/AuthenticateTokenMiddleware';
import { CorsHeadersMiddleware } from './Middleware/CorsHeadersMiddleware';
import { DisableAllExceptGetMiddleware } from './Middleware/DisableAllExceptGetMiddleware';
import { ParseTokenMiddleware } from './Middleware/ParseTokenMiddleware';
// controllers
import { init } from 'ooyala-express';
import { AudioController } from './Controller/V1/AudioController';
import { MovieController } from './Controller/V1/MovieController';
import { RootController } from './Controller/V1/RootController';
import { Indexer } from './indexer';
class App {
public express: Application;
constructor() {
(<any> mongoose).Promise = bluebird;
connect(getEnv('MONGODB_URI'), { useMongoClient: true });
useContainer(Container);
Indexer.ensureIndexes();
init({
ooyala_api_host: getEnv('ooyala_api_host'),
ooyala_cdn_host: getEnv('ooyala_cdn_host'),
ooyala_rights_locker_host: getEnv('ooyala_rights_locker_host'),
ooyala_player_host: getEnv('ooyala_player_host'),
player_expiration_window: getEnv('player_expiration_window'),
});
this.express = createExpressServer({
// Disable so we don't get a nasty serialized mongoose document.
// If they fix the issue with Expose/Exclude not being respected then
// perhaps this could be set back to true
classTransformer: false,
controllers: [
RootController,
AudioController,
MovieController,
],
middlewares: [
// Order is important. Middlewares are executed in index order.
CorsHeadersMiddleware,
DisableAllExceptGetMiddleware,
ParseTokenMiddleware,
AuthenticateTokenMiddleware,
],
});
}
}
export default new App().express;

AWS Lambda (NodeJS) throws error when connecting RDS by Sequelize

Base on aws-serverless-express repository on Github. I've made a connection to AWS RDS in order to CRUD data. I'm using Sequelize for ORM.
I've tested on local, actions work perfectly. But when it's deployed on AWS Lambda the error was thrown at Sequelize initialization (without it works normally)
Sequelize initial
const database = new Sequelize('databasename', 'root', 'password', {
host: 'XXXXXX.XXXXXX.us-east-1.rds.amazonaws.com',
port: 3306,
logging: console.log,
maxConcurrentQueries: 100,
dialect: 'mysql',
dialectOptions: {
ssl: 'Amazon RDS',
},
pool: { maxConnections: 5, maxIdleTime: 30 },
language: 'en',
});
Error logs was got from CloudWatch:
module initialization error: Error
at Object.fs.openSync (fs.js:646:18)
at Object.fs.readFileSync (fs.js:551:33)
at Object.Module._extensions..js (module.js:662:20)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/var/task/mysql2/lib/packets/ssl_request.js:3:23)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at /var/task/mysql2/lib/packets/index.js:4:16
at Array.forEach (<anonymous>)
at Object.<anonymous> (/var/task/mysql2/lib/packets/index.js:3:4)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
I fear error throws cause SSL connection problem, but I don't know how to figure out.
Below is my config on RDS MySql instance:
Network & Security
Security group: default
Inbound:
Type: All trafic
Protocol: All
Port Range: All
Source: sg-3c941877 (default)
Outbound:
Type: All trafic
Protocol: All
Port Range: All
Destination: 0.0.0.0/0
Public accessibility: Yes
Database options
IAM DB authentication: Disabled
In AWS Lambda I also added AWSLambdaVPCAccessExecutionRole Policy
Thanks in advance.

Resources