How to fix this MongoClient connection? - node.js

I'm trying to connect a node.js app (written in TS) to MongoDB at Yandex Cloud. I have successfully connected there via mongosh:
mongosh "mongodb://<user>:<pass>#<host>:<port>/?replicaSet=<rs>&authSource=<db>&ssl=true" \
--tls --tlsCAFile ./YandexInternalRootCA.crt
where YandexInternalRootCA.crt is the downloaded certificate. Now I'm trying to do the same via MongoClient like this (the code is adapted from their examples; node v15.14.0, mongodb ^4.1.2):
import { MongoClient, Db } from 'mongodb'
import fs from 'fs'
const connnectionString = '<same connection string as the above argument of mongosh>'
const options = {
useNewUrlParser: true,
replSet: {
sslCA: fs.readFileSync('./YandexInternalRootCA.crt')
},
//tlsInsecure: true,
}
const getStorage = async (): Promise<Db> => {
// ts-ignore here is due to some typing problem: once you use 2 arguments
// in .connect, TS shows that it promises void (which is not true)
// #ts-ignore
return (await MongoClient.connect(connnectionString, options)).db()
}
Unexectedly, this results in
MongooseServerSelectionError: self signed certificate in certificate chain
I've tried to add tlsInsecure where it is show commented out (from suggestion for Mongoose), but it doesn't make a difference. What can be the cause and how can I fix this?
PS I've also tried various things like
const getStorage = async (): Promise<Db> => {
return (await MongoClient.connect(config.mongo.connectionUri, {
tls: true,
//sslCA: fs.readFileSync('./YandexInternalRootCA.crt'),
tlsCertificateFile: './YandexInternalRootCA.crt',
tlsInsecure: true,
})).db()
}
which still gives the same result.

If you use mongodb npm package version 4 or higher, you should pass TLS options like this:
const options = {
tls: true,
tlsCAFile: './YandexInternalRootCA.crt'
}

Related

AWS Redis Cluster MOVED Error using redis node library

I have created a Redis MemoryDB cluster with 2 nodes in AWS:
I connect to it using redis node library v4.0.0 like this:
import { createCluster } from 'redis';
(async () => {
const REDIS_USERNAME = 'test-username';
const REDIS_PASSWORD = 'test-pass';
const cluster = createCluster({
rootNodes: [
{
url: `rediss://node1.amazonaws.com:6379`,
},
{
url: `rediss://node2.amazonaws.com:6379`,
},
],
defaults: {
url: `rediss://cluster.amazonaws.com:6379`,
username: REDIS_USERNAME,
password: REDIS_PASSWORD,
}
});
cluster.on('error', (err) => console.log('Redis Cluster Error', err));
await cluster.connect();
console.log('connected to cluster...');
await cluster.set('key', 'value');
const value = await cluster.get('key');
console.log('Value', value);
await cluster.disconnect();
})();
But sometimes I get the error ReplyError: MOVED 12539 rediss://node2.amazonaws.com:6379 and I cannot get the value from the key.
Do you have any idea if there is something wrong with the configuration of the cluster or with the code using redis node library?
Edit:
I tried it with ioredis library and it works, so it's something wrong with the redis library.
Node.js Version: 16
Redis Server Version: 6
I had created an issue to redis library, so it's going to be solved soon with this PR.

Mongodb useUnifiedTopology automatically create a new connection [duplicate]

I am having a kind of strange problem when I trying to establish a single mongodb connection to a db with the mongodb nodejs native driver in the version 3.6.0.
My idea is to have just one connection opened through all the client session and reuse it in the different routes I have in my express server, when I hit it the first time the getDatabase function it creates two connections and after that one of it closes and one stands idle, but when I use a route, the second get opened again (it is like one connection just stays there and do nothing).
I just want to have one connection opened in my pool.
If you see the commented code i was testing with those options but none of them worked for me.
Pd: when I set the socketTimeoutMS to 5000ms just one connection is created but it auto-closes and reopen each 5000ms, which is weird (it reopen itself even when I don't use the connection).
All of this problem happen when I set the useUnifiedTopology to true (I can't set it to false because is deprecated and the other topologies will be removed in the next version of mdb ndjs driver)
Here is an image with the strange behaviour
The code is:
import { MongoClient, Db } from 'mongodb';
import { DB_URI } from '../config/config';
// This mod works as DataBase Singleton
let db: Db;
export const getDataBase = async (id: string) => {
try {
if (db) {
console.log('ALREADY CREATED');
return db;
} else {
console.log('CREATING');
let client: MongoClient = await MongoClient.connect(`${DB_URI}DB_${id}`, {
useUnifiedTopology: true,
/* minPoolSize: 1,
maxPoolSize: 1,
socketTimeoutMS: 180000,
keepAlive: true,
maxIdleTimeMS:10000
useNewUrlParser: true,
keepAlive: true,
w: 'majority',
wtimeout: 5000,
serverSelectionTimeoutMS: 5000,
connectTimeoutMS: 8000,
appname: 'myApp',
*/
});
db = client.db();
return db;
}
} catch (error) {
console.log('DB Connection error', error);
}
};
The driver internally creates one connection per known server for monitoring purposes. This connection is not used for application operations.
Hence, it is expected that you would get two connections established.

AWS Lambda - Error: unable to get local issuer certificate

I am trying to connect to an amazon postgreSQL RDS using a NodeJS lambda.
The lambda is in the same VPC as the RDS instance and as far as I can tell the security groups are set up to give the lambda access to the RDS. The lambda is called through API gateway and I'm using knex js as a query builder. When the lambda attempts to connect to the database it throws an "unable to get local issuer certificate" error, but the connection parameters are what I expect them to be.
I know this connection is possible as I've already implemented it in a different environment, without receiving the certificate issue. I've compared the two environments but cannot find any immediate differences.
The connection code looks like this:
import AWS from 'aws-sdk';
import { types } from 'pg';
import { Moment } from 'moment';
import knex from 'knex';
const TIMESTAMP_OID = 1114;
// Example value string: "2018-10-04 12:30:21.199"
types.setTypeParser(TIMESTAMP_OID, (value) => value && new Date(`${value}+00`));
export default class Database {
/**
* Gets the connection information through AWS Secrets Manager
*/
static getConnection = async () => {
const client = new AWS.SecretsManager({
region: '<region>',
});
if (process.env.databaseSecret == null) {
throw 'Database secret not defined';
}
const response = await client
.getSecretValue({ SecretId: process.env.databaseSecret })
.promise();
if (response.SecretString == undefined) {
throw 'Cannot find secret string';
}
return JSON.parse(response.SecretString);
};
static knexConnection = knex({
client: 'postgres',
connection: async () => {
const secret = await Database.getConnection();
return {
host: secret.host,
port: secret.port,
user: secret.username,
password: secret.password,
database: secret.dbname,
ssl: true,
};
},
});
}
Any guidance on how to solve this issue or even where to start looking would be greatly appreciated.
First of all, it is not a good idea to bypass ssl verification, and doing so can make you vulnerable to various exploits and skips a critical step in the TLS handshake.
What you can do is programmatically download the ca certificate chain bundle from Amazon and place it in the root directory of the lambda along side the handler.
wget https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem -P path/to/handler
Note: you can do this in your buildspec.yaml or in your script that packages the zip file that gets uploaded to aws
Then set the ssl configuration option to the contents of the pem file in your code postgres client configuration, like this:
let pgClient = new postgres.Client({
user: 'postgres',
host: 'rds-cluster.cluster-abc.us-west-2.rds.amazonaws.com',
database: 'mydatabase',
password: 'postgres',
port: 5432,
ssl: {
ca: fs.readFileSync(path.resolve('rds-combined-ca-bundle.pem'), "utf-8")
}
})
I know this is old, but just ran into this today. Running with node 10 and an older version of the pg library worked just fine. Updating to node 16 with pg version 8.x caused this error (simplified):
UNABLE_TO_GET_ISSUER_CERT_LOCALLY
In the past, you could indeed just set the ssl parameter to true or 'true' and it would work with the default AWS RDS certificate. Now, it seems we need to at least tell node/pg to ignore the cert verification (since it's self generated).
Using ssl: 'no-verify' works, enabling ssl and telling pg to ignore the verification of the cert chain.
source
UPDATE
For clarity, here's what the connection string would look like. With Knex, the same client info is passed to pg, so it should look similar to a pg client connection.
static knexConnection = knex({
client: 'postgres',
connection: async () => {
const secret = await Database.getConnection();
return {
host: secret.host,
port: secret.port,
user: secret.username,
password: secret.password,
database: secret.dbname,
ssl: 'no-verify',
};
}

Mongoose fails to reconnect after replicaset not found error

I'm running a small website that connects to MongoDB Atlas. It has a replicaset with 3 members. For the most part, everything works just fine, but every now an then Atlas' replicaset crashes (or something?) and Mongoose stops working from then on. It throws a single error - MongoError: no primary found in replicaset and that's it.
It doesn't fire mongoose.connection.on('error'), and no errors are reported after this point. It just fails to return any data.
It's sort of hard for me to debug this, as this is running in production, and I have no way of telling when replicaset will fail. Here's how I connect:
function connect() {
tries++;
mongoose.Promise = Promise;
const { uri, options } = config.mongoDb;
return mongoose.connect(uri, options);
}
let db = connect();
db
.connection
.on('error', (err) => {
Raven.captureException(err);
db.disconnect();
})
.on('disconnected', () => {
db = connect();
});
And my options look like this:
options: {
server: {
autoReconnect: true,
ssl: true,
poolSize: 10,
socket_option: {
keepAlive: true
}
}
}
Has anyone had similar problems before? Any idea what I'm doing wrong here, or how to actually catch the error, so I can properly reconnect?

Trouble Connecting to compose.io mongodb instance with mongoose

I am using these instructions:
https://help.compose.com/docs/connecting-to-mongodb#mongoose-node-and-compose
This is the connection string they gave me:
MONGO_URI=mongodb://*****:******#aws-us-west-2-portal.1.dblayer.com:15782,aws-us-west-2-portal.0.dblayer.com:15782/mydbname?ssl=true
This is the code I am trying:
certFileBuff = [fs.readFileSync(MONGO_CERT_PATH)]
options = {
mongos: true,
sslCA: certFileBuff,
sslValidate: false,
ssl: true
}
mongoose.connect(MONGO_URI, options)
At this point I get Authentication Failure errors:
if i remove ?ssl=true from the connection string I get the error:
no mongos proxy available
Does anyone have mongoose working with compose.io recently?
If you are not using sharding in mongo set mongos to false.
You can read about sharding in mongo here https://docs.mongodb.com/manual/core/sharded-cluster-query-router/
As it turns out. I had a password that didn't get along with the connection string. I changed the password to something shorter and it worked. fwiw the password that didn't work was: tapasleechframegoldrabbitsauceb88 (Obviously I'm not using anymore)
What I learned:
If you get a mongos proxy error its probably bad config. Make sure ssl=true is set on the connection string and ssl: true in your config.
If you get code: 18 authentication error then the password is either wrong or not formatted properly.
Here is my final working code:
const MONGO_URI = mongodb://****:*****#aws-us-west-2-portal.1.dblayer.com:15782,aws-us-west-2-portal.0.dblayer.com:15782/dbName?ssl=true
let ca, options = {}
if(MONGO_CERT_PATH) {
try {
ca = fs.readFileSync(encodeURI(MONGO_CERT_PATH))
options = {
mongos: {
ssl: true,
sslValidate: true,
sslCA: [ ca ]
}
}
} catch(err) {
logger.warn('mongo ssl cert missing')
}
}
let db = mongoose.connection
db.on('error', (e) => logger.error('connection error:', e))
db.once('open', () => {
logger.info('db connected')
})
mongoose.connect(MONGO_URI, options)

Resources