Using Connection URIs with Read Replication in Sequelize - node.js

If I have a connection URI, I can use that normally with Sequelize as such:
const sequelize = new Sequelize('postgres://user:pass#example.com:5432/dbname');
However, if I want to use Read and Write replication (https://sequelize.org/master/manual/read-replication.html), then there doesn't seem an option to use connection URI. Can I pass connection URI strings to read and write in the replication option as in:
const sequelize = new Sequelize(null, null, null, {
dialect: 'postgres',
replication: {
read: [
'postgres://user:pass#reader.example.com:5432/dbname',
'postgres://user:pass#anotherreader.example.com:5432/dbname'
],
write: 'postgres://user:pass#writer.example.com:5432/dbname'
}
})
EDIT:
I have already found a solution to the issue. and that is using an npm library like connection string to parse the connection string as shown below:
const write_uri = new ConnectionString(uri);
const sequelize = new Sequelize(null, null, null, {
dialect: 'postgres',
replication: {
read: [
'postgres://user:pass#reader.example.com:5432/dbname',
'postgres://user:pass#anotherreader.example.com:5432/dbname'
],
write: {
host: write_uri.hosts[0].name,
username: write_uri.user,
password: write_uri.password,
database: write_uri.path[0],
port: write_uri.hosts[0].port
}
}
});
But, that is not what I'm looking for.

As per sequelize source at master, you can't.
According to the sequelize source docs at sequelize.js, The Sequelize Constructor accepts options agrument, like this
constructor(database, username, password, options){
}
Where options.replication should be an object with two properties, read and write. Write should be an object (a single server for handling writes), and read an array of object (several servers to handle reads). Each read/write server can have the following properties: host, port, username, password, database.
you need to pass an array of objects to read:[] with connection values as props instead of passing strings.

You can pass config object to sequelize constructor even if you use uri connection.
Look the example in from the docs:
https://sequelize.org/master/class/lib/sequelize.js~Sequelize.html#instance-constructor-constructor
// with uri
const sequelize = new Sequelize('mysql://localhost:3306/database', {})
Look at the constructor overloading definition:
Sequelize(uri: string, options?: Sequelize.Options): Sequelize.Sequelize
Just pass the options you need.

Related

I can use two connections with different database using prisma?

I'm trying to use different databases with the prisma, one database would be for read-only and the other for write, I need to configure it in my project and use it inside the UseCase, how to do that?
I tried to configure two schemas but I couldn't call them inside the UseCase
Here's an example of how you can create two PrismaClientInstances, one with read access and one with write access
import { PrismaClient } from '#prisma/client'
const client1 = new PrismaClient({ datasources: { db: { url: 'postgres://localhost/db1' }} })
const client2 = new PrismaClient({ datasources: { db: { url: 'postgres://localhost/db2' }} })
You are essentially overriding the database url while instantiating a prisma client.
Here client1 could be read-only and client2 can be with write access.
You can read more about it here: #2443

AdonisJS 5, dynamically connect database

Hi used Adonis Js 5 with the new version. I have multiple database clients with the same database structure, can I create a new database connection without declaring the config at config/database.ts? it means I can create a connection on the fly.
Yes finally I can make it, with this:
Database.manager.patch(this.database.name, {
client: 'mysql',
connection: {
host: Env.get('DB_HOST'),
port: Env.get('DB_PORT'),
user: this.database.username,
password: this.database.password,
database: this.database.name,
},
debug: Env.get('DB_DEBUG', false),
})
Database.manager.connect(this.database.name)

NodeJS mysql2 - should I handle pool disconnections?

I use mysql2 module in my NodeJS project.
I understand the concept of database pooling in mysql2 module
(https://www.npmjs.com/package/mysql2).
Before using pool, I used regular connection with mysql2.createConnection() function,
but after some time I got 'PROTOCOL_CONNECTION_LOST' error.
My code is:
db.js:
const mysql = require('mysql2');
const pool = mysql.createPool({
host: 'sql server',
user: 'username',
database: 'database',
password: 'pass',
port: 3306,
connectionLimit: 10,
queueLimit: 0
});
module.exports = pool;
And how I using it:
const db = require('db');
db.query(...);
The query() and execute() functions on pool instance automatically call release() function of pool instance, and that is very good (because I don't need to write that command manually after any query).
But I need to understand: if I work like that (with pool, instead of without pool), there is a guarentee
'PROTOCOL_CONNECTION_LOST' will not be thrown? Should I handle that or the pooling mechanism does it automatically?
I ask because I saw on the internet some code that, for example, re-create the connection.
With pool, I need to re-create the connection sometime?
Thanks a lot!

Mongoose with ReplicaSet on Atlas

I have a replica set on MongoDB Atlas and this is my mongo shell connection string which connects perfectly:
$ mongo "mongodb://MY_SERVER-shard-00-00-clv3h.mongodb.net:27017,MY_SERVER-shard-00-01-clv3h.mongodb.net:27017,MY_SERVER-shard-00-02-clv3h.mongodb.net:27017/MY_DATABASE?replicaSet=MY_REPLICASET-NAME-shard-0" --ssl --username MY_USERNAME --password MY_PASSWORD --authenticationDatabase MY_ADMIN_DATABASE
How Can I convert it to use in mongoose? How Can I build my uri and options variable?
I tried the following without success:
// connection string using mongoose:
var uri = 'mongodb://MY_USER:MY_PASSWORD#' +
'MY_SERVER-shard-00-00-clv3h.mongodb.net:27017,' +
'MY_SERVER-shard-00-01-clv3h.mongodb.net:27017,' +
'MY_SERVER-shard-00-02-clv3h.mongodb.net:27017/MY_DATABASE';
var options = {
replset: {
ssl: true,
authSource: 'MY_ADMIN_DATABASE',
rs_name: 'MY_REPLICASET_NAME-shard-0'
}
};
mongoose.connect(uri, options);
var db = mongoose.connection;
I've tried including user: and pass: on options, removing MY_USER:MY_PASSWORD# from uri, change rs_name to replicaSet, every unsuccessful attempt. It seems that mongoose is not considering the authSource option.
Using the mongojs, it works fine with the following code:
// connection string using mongojs:
var uri = 'mongodb://MY_USER:MY_PASSWORD#' +
'MY_SERVER-shard-00-00-clv3h.mongodb.net:27017,' +
'MY_SERVER-shard-00-01-clv3h.mongodb.net:27017,' +
'MY_SERVER-shard-00-02-clv3h.mongodb.net:27017/MY_DATABASE';
var options = {
ssl: true,
authSource: 'MY_ADMIN_DATABASE',
replicaSet: 'MY_REPLICASET_NAME-shard-0'
};
var db = mongojs(uri,'', options);
But, I need to use mongoose because the ODM in my project.
How can I build my uri and options variable using mongoose?
ON MONGODB 3.4.x
I resolved this issue putting the 'options' value directly in 'uri' string, according to documentation (http://mongoosejs.com/docs/connections.html) on 'Replica Set Connections' section.
// connection string using mongoose:
var uri = 'mongodb://MY_USER:MY_PASSWORD#' +
'MY_SERVER-shard-00-00-clv3h.mongodb.net:27017,' +
'MY_SERVER-shard-00-01-clv3h.mongodb.net:27017,' +
'MY_SERVER-shard-00-02-clv3h.mongodb.net:27017/MY_DATABASE' +
'ssl=true&replicaSet=MY_REPLICASET_NAME-shard-0&authSource=MY_ADMIN_DATABASE';
mongoose.connect(uri);
var db = mongoose.connection;
Now, it is working fine!
NOTICE WITH MONGODB 3.6
On MongoDB Atlas using the version 3.6.x, the connection string changed to use a DNS server making the link shorter.
mongodb+srv://MY_USER:MY_PASSWORD#MY_SERVER.mongodb.net/MY_DATABASE
...if you use this connection string in your application, this will connect with success but it will be able to read and write only with atlas users with higher privilegies access (atlasAdmin, readWriteAnyDatabase...).
To you work with an specific user with privilege only to readWrite your database, you will need to keep the same connection string used in MongoDB 3.4 because the mongoose not recognized the DNS option (mongodb+srv).
P.S. all the new resources from MongoDB 3.6.x will continue working normally!
Add username and password to database connection
mongodb://[username:password#]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]]
Standard Connection String Format

my .env variables wont return for database login in node.js

Having a problem with .env variables in node.js, am using dotenv to handle the variables.
straight forward
require('dotenv').load()
and then checked console log to see if I can access then variables like so
console.log("database Name: "+process.env.DB_NAME);
console.log("database Username: "+process.env.DB_USER);
etc and all return fine.
the part of the code that calls the db is the following-
var Sequelize = require('sequelize');
module.exports = new Sequelize('dbname, 'username', 'password', {
host: 'hostname',
dialect: 'mssql',
dialectOptions: {
instanceName: 'namehere'
}
});
I replace the dbname with process.env.DB_NAME and all is fine, nothing falls over but the moment I try and use DB_USER in there then it fails to connect and shows user as ' ' in the log.
module.exports = new Sequelize(process.env.DB_NAME, process.env.USER_DB, 'password', { etc
the .env file itself is very straight forward and nothing out of the ordinary. structured as
NAME="value"
NAME2="value2"
etc. I though maybe once a value has been console logged it wont be available and turned off the console logs, but no luck. I also tried changing the name of the value encase that was reserved. please advise.
nevermind I was being silly. I presumed it only needed to be required once but not the case.

Resources