Sequelize with Singleton - node.js

I'm working with Sequelize in node.js and the idea is to use the Singleton pattern.
Reading about how node works with the modules caching and some singleton examples
My file in this moment is:
const DBManager = (function () {
// Instance stores a reference to the Singleton
let instance: any;
let db: string = null;
let user: string;
let password: string;
let host: string;
let sequelize: Sequelize.Sequelize;
function init(bdName: string) {
db = bdName;
user = process.env.MYSQL_DB_USERNAME || 'root';
password = process.env.MYSQL_DB_PASSWORD || 'root';
host = process.env.MYSQL_DB_HOST || 'localhost';
return {
open: () => {
sequelize = new Sequelize(db, user, password, {
host: host,
dialect: 'mysql',
pool: {
max: 5,
min: 0,
acquire: 30000,
idle: 10000
},
operatorsAliases: false,
logging: !process.env.HIDE_LOGS
});
},
testConnection: () => {
return sequelize.authenticate();
},
getManagerObject: () => {
return sequelize;
},
close: () => {
sequelize.close();
}
};
}
return {
// Get the Singleton instance if one exists
// or create one if it doesn't
getInstance: (bd?: string) => {
if (!instance) {
instance = init(bd);
}
return instance;
}
};
})();
export default DBManager;
So, as expected when i require this file anywhere in my project the references are the same and works as expected.
I'm not sure if this is the right way to implement the Singleton pattern, or if there is a defined and documented one, because the oficial documentation does not say anything about this.

There's usually no need for explicit singleton implementation. JS modules (CommonJS and ES modules in particular) are evaluated only once under normal circumstances, exported class instance is efficiently a singleton.
There's no need for IIFE as well because modules have their own scopes. Since init function isn't reused, there's possibly no need for it either. It can be simplified to:
export default {
open: ...
testConnection: ...
...
};
This abstraction isn't practical. There's already sequelize instance, creating wrapper methods for its own methods doesn't serve a good purpose.
Since a connection is usable after it's established, it makes sense to just export a promise of a connection, similarly to the one that is shown in this answer.
If the configuration (database name) is available in database manager module, it's preferable to just use it in-place:
// db.js
const dbName = someConfig.db;
const sequelize = new Sequelize(dbName, ...);
export default sequelize.authenticate().then(() => sequelize);
Which is used like:
import dbConnection from './db';
dbConnection.then(sequelize => { /* all code that depends on the connection */ });
If there may be several connections, or the configuration isn't available on import, factory function is exported instead:
// db.js
export default dbName => {
const sequelize = new Sequelize(dbName, ...);
sequelize.authenticate().then(() => sequelize);
}
Singleton instances are naturally handled with modules:
// foo-db.js
import getDbConnection from './db';
export default getFooDbName().then(dbName => getDbConnection(dbName));
And used like:
import dbConnection from './foo-db';
dbConnection.then(sequelize => { /* all code that depends on the connection */ });

Related

NodeJs & MongoDb , find data in a collection and insert it to another collection

I want to perform a database migration from an old model to another .
I have a database in MongoDB inside 2 collection the first one represents the old model and the other one present the model that the data would be transfer to it .
Thank you for sharing with me if there is a way to get the job done , perhaps a link or anything that could help please.
Note: example i get the data from large (instance of the old model) i must get it and transfer it to another instance width (step by step till having the second model).
import dotenv from 'dotenv';
dotenv.config();
async function main() {
const MONGO_URL = process.env.MONGO_URL || '';
const client = new MongoClient(MONGO_URL);
await listDatabases(client);
try {
await client.connect();
console.log('server is connected');
} finally {
await client.close();
}
}
main().catch(console.error);
async function listDatabases(client: { db: (arg0: string) => { (): any; new (): any; admin: { (): { (): any; new (): any; listDatabases: { (): any; new (): any } }; new (): any } } }) {
const databases = await client.db('park').admin().listDatabases();
console.log('Databases:');
databases.databases.forEach((db: { name: any }) => console.log(` - ${db.name}`));
}
async function ...{
}

How can I use two different databases in one single node app?

I have install Hbase client and PostgreSql client install but how to connect two databases in a single node application
import { error } from "console";
const hbase = require('hbase');
export class Db {
private conn = hbase();
private config = { host: '0.0.0.0', port: 8080 };
public client = new hbase.Client(this.config);
constructor() {
this.conn = new hbase.Client(this.config);
}
public conection() {
this.conn.table('messages').exists((error: string, succuss: string) => {
if (!succuss) {
this.createTable('messages', 'message_data');
}
});
}
private createTable(TblName: string, CF: string) {
this.conn.table(TblName).create(CF, function (error: string, success: string) {
console.log(success);
return success
});
}
}
I would suggest creating two different classes for Hbase and PostgreSql. ANd use them in your application whenever needed.
Another thing also use dependency injection in the constructor instead of defining configs in class. That way you can inject any DB configuration in an instance.
Here's code example
Create Class to manage HBaseDB connection
import { error } from "console";
const hbase = require('hbase');
export class HBaseDB {
//Inject this config object in your class constructor
//private config = { host: '0.0.0.0', port: 8080 };
//Here we have injected config object
constructor(config) {
this.conn = new hbase.Client(config);
}
public conection() {
this.conn.table('messages').exists((error: string, succuss: string) => {
if (!succuss) {
this.createTable('messages', 'message_data');
}
});
}
private createTable(TblName: string, CF: string) {
this.conn.table(TblName).create(CF, function (error: string, success: string) {
console.log(success);
return success
});
}
}
Create Class to manage PostgreSQL connection
const pg = require('pg');
export class PostgresqlDB {
constructor(config) {
//config contains database,username,password etc... configs
this.pool = new pg.Pool({ config })
this.conn = undefined
}
public async conection() {
//If connection is already connected, no need to connect again
//This will save time
if (this.conn === undefined)
this.conn = await this.pool.connect()
}
public async query(query) {
await this.conection()
const response = await this.conn.query(query)
return response
}
}
Now you can use them in code like
const pgDB = require('./PostgresqlDB')
const hbaseDB = require('./HBaseDB')
const hbaseDBConn = new hbaseDB({ host: '0.0.0.0', port: 8080 })
const pgDBConn = new pgDB({ database: 'test', user:'test',password:'test'})
Note: Above code is for understanding purposes only, You may need to add validations and correct some syntax for actual use

Nodejs Mongoose 'Operation `XXX.find()` buffering timed out after 10000ms'

index.ts is the entry point of this NodeJS program.
This is the code in index.ts:
import JobWorker from "./worker";
import { SwitchPlan } from "./jobs";
const worker = new JobWorker();
worker.addJob(SwitchPlan);
This is worker.ts:
import { CronJob } from "cron";
import mongoose from "mongoose";
import Config from "./config";
import logger from "./logger";
export default class JobWorker {
private jobs: CronJob[];
private config: {
NAME: string;
MONGO_URL: string;
};
constructor() {
this.config = Config;
this.connectDB();
this.jobs = [];
}
public async connectDB(): Promise<void> {
try {
await mongoose.connect(this.config.MONGO_URL,
{ useUnifiedTopology: true, useNewUrlParser: true, useCreateIndex: true },
);
logger.info("\nMONGODB has been connected\n");
} catch(err) {
logger.error("ERROR occurred while connecting to the database");
}
}
addJob(cronJob: CronJob) {
this.jobs.push(cronJob);
}
}
This is jobs.ts:
import moment from "moment";
import {
DatabaseOperations, Vehicle,
Plan1Doc, Plan1, VehicleDoc
} from "common-lib";
import logger from "../logger";
import { CronJob } from "cron";
const vehicleOps = new DatabaseOperations(Vehicle);
const SwitchPlan = new CronJob("25 * * * * *", async (): Promise<void> => {
const date: Date = moment(new Date()).startOf("date").toDate();
const expiringVehicles: VehicleDoc[] = vehicleOps.getAllDocuments(
{ "inspection.startTime": {
"$gte": date, "$lte": moment(date).startOf("date").add(1, "day").toDate()
}
},
{}, { pageNo: 0, limit: 0 }
).then((result: any) => {
logger.info("dsada");
}).catch((err: any) => {
logger.info("ssd");
});
});
SwitchPlan.start();
export { SwitchPlan };
I have omitted parts of code which are irrelevant to this problem. I ran this code through a debugger and there's no issue with the config. MonggoDB connected is getting printed at the start of the program. However the then block after getAllDocuments in jobs.ts is never reached and it always goes in the error block with the message, Operation vehicleinventories.find() buffering timed out after 10000ms. The getAllDocuments uses MongoDB's find() method and is working correctly because I am using this method in other projects where I have no such issues.
So far I have tried, deleting Mongoose from node_modules and reinstalling, tried connecting to MongoDB running on localhost, but the issue remains unsolved.
EDIT: DatabaseOperations class:
import { Model, Schema } from "mongoose";
class DatabaseOperations {
private dbModel: Model<any>;
constructor(dbModel: Model<any>) {
this.dbModel = dbModel;
}
getAllDocuments(
query: any,
projections: any,
options: { pageNo: number; limit: number },
sort?: any
): any {
const offset = options.limit * options.pageNo;
return this.dbModel
.find(query, projections)
.skip(offset)
.limit(options.limit)
.sort(sort ? sort : { createdAt: -1 })
.lean();
}
}
in your jobs.ts file you have the following line
SwitchToTier1Plan.start();
This line is called the moment you required the class file, hence before mongoose is connected, and all the models defined. Could this be the issue?
Another thing I noted is u are using mongoose.connect which may be wrong since mongoose.connect creates a global connection.
which means each new Worker you will be attempting to override the mongoose property with previous connection
Though i'm not sure what the implication is, but it could be because your .find could be using the old connection.
Since you are writing class, I would recommend using mongoose.createConnection which creates a new connection for each class initiation.

Loading files from all directories inside specific directory

Good time of the day,
I've been trying to add 'modularity' to my application by splitting the Vuex store into many different locations.
So far, i'm totally fine with loading 'local' modules (inside the store folder) with the following piece of code:
const localRequireContext = require.context('./modules', false, /.*\.js$/);
const localModules = localRequireContext.keys().map((file) => [file.replace(/(^.\/)|(\.js$)/g, ''), localRequireContext(file)]).reduce((localModules, [name, module]) => {
if (module.namespaced === undefined) {
module.namespaced = true;
}
return { ...localModules, [name]: module };
}, {});
const createStore = () => {
return new Vuex.Store({
modules: localModules
})
};
export default createStore;
However, what i'm trying to achieve seems to be rather impossible for me (i'm not new to the web app development, but actually never had a chance to play around with 'core' libraries of Node.js, Webpack, etc).
I have the following structure
root
|-assets
|-components
|-config
|-lang
|-layouts
|-libs
|-middleware
|-modules
|----company
|--------module
|-----------store
|--------------index.js (module index)
|-pages
|-plugins
|-store
|----index.js (main index)
So what i'm trying to achieve, is to get to the ~/modules folder, go inside each of company directory (namespace for module), open the module directory (the name of the module), navigate to the store folder and import index.js file, with roughly the following content:
import module from '../config/module';
export const namespace = [module.namespace, module.name].join('/');
export const state = () => ({
quotes: null,
team: null,
articles: null
});
export const getters = {
quotes: (state) => state.quotes,
team: (state) => state.team,
articles: (state) => state.articles
};
As i've already said, i'm not much of a 'guru' when it comes to these complicated (for me) things, so any help is really appreciated!
So far, i went the 'dumb road' and just tried to use the following:
const modulesRequireContext = require.context('../modules/**/**/store', false, /.*\.js$/);
But, no luck it is - Cannot find module 'undefined'
The final file (in my mind) should look something like this:
import Vue from 'vue';
import Vuex from 'vuex';
Vue.use(Vuex);
const localRequireContext = require.context('./modules', false, /.*\.js$/);
const localModules = localRequireContext.keys().map((file) => [file.replace(/(^.\/)|(\.js$)/g, ''), localRequireContext(file)]).reduce((localModules, [name, module]) => {
if (module.namespaced === undefined) {
module.namespaced = true;
}
return { ...localModules, [name]: module };
}, {});
const modulesRequireContext = require.context('CORRECT_WAY_OF_SEARCHING_IN_SUB_DIRECTORIES', false, /.*\.js$/);
const addedModules = modulesRequireContext.keys().map((file) => [file.replace(/(^.\/)|(\.js$)/g, ''), modulesRequireContext(file)]).reduce((addedModules, [name, module]) => {
return { ...addedModules, [module.namespace]: module };
}, {});
let modules = { ...localModules, ...addedModules };
const createStore = () => {
return new Vuex.Store({
modules: modules
})
};
export default createStore;

Nestjs resolve custom startup dependency

I have this factory which resolves redis:
import {RedisClient} from "redis";
export const RedisProvider = {
provide: 'RedisToken',
useFactory: async () => {
return new Promise((resolve, reject) => {
let redisClient = new RedisClient({
host: 'resolver_redis'
});
redisClient.on('ready', function () {
resolve(redisClient);
});
});
}
};
But in addition to that, i wanna check if a specific key exists in redis, and if its not i want to retrive it using microservices, but for that i need to inject the microservices "client", i wanna change it to someting like that:
import {RedisClient} from "redis";
export const RedisProvider = {
provide: 'RedisToken',
useFactory: async () => {
return new Promise((resolve, reject) => {
let redisClient = new RedisClient({
host: 'resolver_redis'
});
redisClient.on('ready', async function () {
let campaigns = await redisClient.get('campaigns');
if ( ! campaigns) {
// Note: "client" is not available in this scope
client.send('get:campaigns').subscribe(async function (campaigns) {
await redisClient.set('campaigns', campaigns);
resolve(redisClient);
});
}
else {
resolve(redisClient);
}
});
});
}
};
the only problem is that i dont have access to the "client", or do i?
it can also be anther provider if it makes more sense, but then i will also need priority for loading the providers, i am doing this because the application requires this data for startup stuff
I don't recommend adding more logic in providers factory, since it could be more controlled in the consumer of this provider ie: the services
in your services code you could do something like this
import { Component, Inject, OnModuleInit } from '#nestjs/common';
...
#Component()
export class MyService implements OnModuleInit {
constructor(#Inject(REDIS_TOKEN) private readonly redis: RedisClient) {}
onModuleInit() {
const campaigns = await this.redis.get('campaigns');
...
}
}
I assume that REDIS_TOKEN is constant saved on constants.ts file and equal to RedisToken.
this will be more controlled.
the code of checking for the campaigns key will be executed once your Module init.
for more information about Module Lifecycle checkout: Modules Lifecycle

Resources