Let's say I have a module that provides a connection to something.
export const CONN string = 'CONN';
#Global()
#Module({})
export class ConnModule implements OnApplicationShutdown {
constructor(#Inject(CONN) private readonly conn: Connection) {}
onApplicationShutdown() {
this.conn.close();
}
static forRoot(options): DynamicModule {
const connectionProvider = {
provide: CONN,
useFactory: async (): Promise<Connection> => {
const connection = new Connection(options);
await connection.establish();
return connection;
},
};
return {
module: ConnModule ,
providers: [connectionProvider],
exports: [connectionProvider],
};
}
}
This connection is used through the application:
export class AService {
constructor(
#Inject(CONN) protected readonly conn: Connection,
) {}
someMethod() {
this.conn.doSomething(); // this may get 401ed
}
export class BService {
constructor(
#Inject(CONN) protected readonly conn: Connection,
) {}
otherMethod() {
this.conn.doSomethingElse(); // this may get 401ed
}
This is a connection to 3rd party app and it may start returning 401 anytime needing us to reauthenticate / recreate a connection.
How can I create a new connection on error and replace the current provider for all the modules that are using it?
Something like:
OnConnectionUnauthorizedException ->
create a new Connection ->
replace the current stale connection object that CONN provides with a new one ->
retry the original action
Thank you!
This sounds concerning to me. As it is hard to communicate to the outside what the status of your application is. Also, what will happen to processing events/requests during the time of the connection re-establishing? You are going to handle a lot of accidental complexity. Consider health checks at least to expose your application's status.
Yet there is a solution that I can imagine: Instead of providing the connection, provide a service that holds the connection in a private field. Make a getConection method, possibly async to gap disconnected times, and return the connection. This way you don't have to replace the injected value.
I would furthermore really discourage replacing an injected value. As this is really hard to communicate to code that might be dependent on this value. ("e.g. hey here is a new instance).
export class ConnectionService {
connection: Conection | undefined;
// Pro Tip: make getConnection async and leverage e.g. Promise.race to implement
// a timeout so that if the connection is down this code e.g. delays 5 seconds and
// if the connection gets re-established, answers with the new connection or throws
// a "Couldnt reconnect" error
getConnection(){
return this.connection;
}
}
This solution should be your last resort and instead, you should challenge the source of the required reconnect.
Related
I am looking for an option to use nest as a back-end Gateway service -
The idea is to poll on DB changes ( and maybe later to move it to event driven ) - de facto no listener would be required here.
On change the Nest would update a 3rd pt API calls
What would be best practice here ?
Take a look here, I'm doing something similar to what you're after: https://github.com/nerdybeast/sith-api/blob/feature/redis-cache/src/modules/api/sobjects/trace-flag/TraceFlagPoller.ts
I created a class that "polls" a backend and emits an event when it detects a change in that backend. You could have other code that listens for this event which makes the call to your 3rd party api.
UPDATE:
As you stated, Nest does have a basic application context which skips the http service setup, here's how you can do that:
index.ts
import { NestFactory } from '#nestjs/core';
import { ApplicationModule } from './ApplicationModule';
import { DatabaseService } from './DatabaseService';
(async () => {
const app = await NestFactory.createApplicationContext(ApplicationModule);
const databaseService = app.get<DatabaseService>(DatabaseService);
await databaseService.poll();
})();
DatabaseService.ts
#Injectable()
export class DatabaseService {
private expectedResult: any;
public async poll() : Promise<void> {
const result = await getData();
if(result !== this.expectedResult) {
this.expectedResult = result;
await axios.post('https://some-url.com', result);
}
//Poll every 5 seconds or whatever
setTimeout(() => this.poll(), 5000);
}
}
This could be the solution if you had to poll the database instead of being able to subscribe to it. With this approach, when you start the app, it will poll forever, constantly updating your 3rd party api.
I would start the index.ts file with pm2 or forever so that you can have a graceful restart if your process crashes for some reason.
I would personally use typeORM subscribers like I have done many times for similar requirements. However I use an eventEmitter to not block the saving action. This is a snippet of what I usually do.
#Injectable()
export class EntityUpdate implements EntitySubscriberInterface {
constructor(
#InjectConnection() readonly connection: Connection,
#InjectEventManager() emitter: AppEvents<AbstractEntity>,
) {
connection.subscribers.push(this);
}
afterInsert(event: InsertEvent<AbstractEntity>): void {
this.emitter('entity', {
method: 'update',
entity,
});
}
}
Then I could listen to the event anywhere within my application and handle that status change of the entity
I have multiple #WebsocketGateways in my project, one implementing the OnGatewayConnection life cycle hook. It seems like the life cycle hook is called once for each Gateway, even though only one is implementing them. Is this the default behavior, a bug or am I doing something wrong?
CommonGateway
#WebSocketGateway()
export class CommonGateway implements OnGatewayConnection, OnGatewayDisconnect {
#WebSocketServer() server;
users: number = 0;
handleConnection() {
this.users++;
console.log('USER CONNECTED: ', this.users);
}
handleDisconnect() {
this.users--;
console.log('USER Disconnected: ', this.users);
}
}
DatesGateway
import {
WebSocketGateway,
SubscribeMessage,
WsResponse,
} from '#nestjs/websockets';
import { CommonService } from 'src/common/common.service';
#WebSocketGateway()
export class DatesGateway {
constructor(private readonly commonService: CommonService) {}
#SubscribeMessage('dates-now')
onNow(client): Promise<WsResponse<Date>> {
return Promise.resolve(this.commonService.now).then(now => ({
event: 'dates-now',
data: now,
}));
}
}
Console screenshot
A small repo demonstrating the issue can be found here
thanks
Update
This was fixed in 6.10.13.
Kamil's guidance to leave the #WebSocketGateway decorator empty now applies in the case where you have multiple WebSocket gateways (and wish to avoid handleConnection being called multiple times).
Pre v6.10.13
I just encountered this issue after creating several gateways. I'm using version 6.10.1 of #nestjs/platform-ws and #nestjs/websockets.
With the solution below, a single WebSocket server will be created and handleConnection will be called once (as desired). The handleDisconnect event always (correctly) fired once. It is important to note that:
A namespace must be included when defining the gateway.
When using the platform-ws adapter the namespaces can be either unique or identical.
When using the platform-socket.io adapter the namespaces must be identical.
A port must be included when defining the gateway. The ports should be identical but different from the HTTP server port to avoid EADDRINUSE and UnhandledPromiseRejectionWarning: Error: The HTTP/S server is already being used by another WebSocket server errors.
If you create many gateways you may see the MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 close listeners added. Use emitter.setMaxListeners() to increase limit warning. I still need to look into this.
app.gateway.ts
#WebSocketGateway(8080, {namespace: 'foo'})
export class AppGateway implements OnGatewayConnection, OnGatewayDisconnect {
constructor() {}
public handleConnection(): void {}
public handleDisconnect(): void {}
#SubscribeMessage('app:event')
public handleGet(): WsResponse<string> {
return {
event: 'app:event',
data: 'foobar'
};
}
}
user.gateway.ts
#WebSocketGateway(8080, {namespace: 'foo'})
export class UserGateway {
constructor() {}
#SubscribeMessage('user:event')
public handleGet(): WsResponse<string> {
return {
event: 'user:event',
data: 'foobar'
};
}
}
main.ts
import { NestFactory } from '#nestjs/core';
import { AppModule } from './app.module';
import { WsAdapter } from '#nestjs/platform-ws';
async function bootstrap() {
const app = await NestFactory.create(AppModule);
app.useWebSocketAdapter(new WsAdapter(app));
await app.listen(80);
}
bootstrap();
More insight into this would be welcome.
I found a solution.
Obviously it is a bug (?) in NestJS that it initiates multiple namespaces for all of your Gateways if you dont define any namespace at all. Then for each is a connection event.
The solution is to define the same namespace on each of your WebSocketGateways
#WebSocketGateway({namespace: 'yournamespace'})
And connect to the server in the client like so:
io.connect('http://localhost:4200/yournamespace');
This is because you have declared the gateway multiple times in modules instead of declaring it as an export in the main module and importing it in another. I also had a similar problem and found either removing an exporting or setting your module as a global decorator module helped. Please research the global decorator for the module before proceeding to use it: https://docs.nestjs.com/modules
so I'm using RabbitMQ for some Projects and i noticed that i ll use some duplicate code all the Time that's why i decided to make a Wrapper Class or Interface that have some function to use RabbitMQ direct without repeating the code all the time. i began to do this yesterday and i already had some Problems since i wanted to use OOP and Javascript can be complicated when using OOP (at least i think so)
I began with creating a class IRAbbitMQ with function init to initialize a connection and create a channel, i knew that i cant use nested classes so instead i wanted to use Factory functions, i tried to make the connection and channel a part of the class IRabbitMQ properties but i dont know why that gave me undefined when i create an instance of it
class IRabbitMQ {
constructor() {
this.init(rabbitMQServer); // rabbitMQServer for example 'localhost//5672'
}
// establish a Connection to RAbbitMQ Server
async init(host) {
try {
let connection = await amqplib.connect(host);
let channel = await connection.createChannel();
channel.prefetch(1);
console.log(' [x] Awaiting RPC requests');
this.connection = connection;
this.channel = channel;
}
catch(err) {
console.error(err);
}
}
// Close the Connection with RabbitMQ
closeConnection() {
this.connection.close();
}
log() {
console.log(this.connection);
}
EventPublisher() {
function init(IRabbit, publisherName) {
if(!IRabbit.connection) {
throw new Error('Create an Instance of IRabbitMQ to establish a Connection');
}
let ch = IRabbit.channel;
console.log(ch);
}
return {
init : init
}
}
}
var r = new IRabbitMQ();
r.log();
when i run the code the output is undefined, i dont know why since i m initializing the connection and channel properties in the init function and then called that function in the constructor so that should be initialized when i create an object of the Wrapper class. i wanted also to take some advices from you wether it is good to use classes or is there any other better way to create a Wrapper class or Interface for RabbitMQ to make it easy to use it and not have to duplicate Code.
Not really an answer, but I was able to successfully log the connection with this example code. I trimmed out other code to just focus on the .log() part that was logging a undefined.
Code is far from perfect, but works at least
const amqplib = require('amqplib');
class IRabbitMQ {
constructor() { }
async init(host) {
try {
const connection = await amqplib.connect(host);
const channel = await connection.createChannel();
channel.prefetch(1);
console.log(' [x] Awaiting RPC requests');
this.connection = connection;
this.channel = channel;
}catch(err) {
console.error(err);
}
}
log() {
console.log(this.connection);
}
}
async function createInstance(){
const instance = new IRabbitMQ();
try {
await instance.init('amqp://localhost');
}catch (e) {
throw new Error('OOPS!');
}
return instance;
}
async function runLogic() {
const r = await createInstance();
r.log();
}
runLogic().catch(console.log);
Just comment if you'd want me to give additional advice/tips, but this seems to work for me.
So no matter what I've read, even once I do it right, I can't seem to get the the hang of async and await. For example I have this in my startup.
startup.js
await CommandBus.GetInstance();
await Consumers.GetInstance();
Debugging jumps to the end of the get instance for CommandBus (starting up a channel for rabbitmq) and start Consumers.GetInstance() which fails since channel is null.
CommandBus.js
export default class CommandBus {
private static instance: CommandBus;
private channel: any;
private conn: Connection;
private constructor() {
this.init();
}
private async init() {
//Create connection to rabbitmq
console.log("Starting connection to rabbit.");
this.conn = await connect({
protocol: "amqp",
hostname: settings.RabbitIP,
port: settings.RabbitPort,
username: settings.RabbitUser,
password: settings.RabbitPwd,
vhost: "/"
});
console.log("connecting channel.");
this.channel = await this.conn.createChannel();
}
static async GetInstance(): Promise<CommandBus> {
if (!CommandBus.instance) {
CommandBus.instance = new CommandBus();
}
return CommandBus.instance;
}
public async AddConsumer(queue: Queues) {
await this.channel.assertQueue(queue);
this.channel.consume(queue, msg => {
this.Handle(msg, queue);
});
}
}
Consumers.js
export default class Consumers {
private cb: CommandBus;
private static instance: Consumers;
private constructor() {
this.init();
}
private async init() {
this.cb = await CommandBus.GetInstance();
await cb.AddConsumer(Queues.AuthResponseLogin);
}
static async GetInstance(): Promise<Consumers> {
if (!Consumers.instance) {
Consumers.instance = new Consumers();
}
return Consumers.instance;
}
}
Sorry I realize this is in Typescript, but I imagine that doesn't matter. The issue occurs specifically when calling cb.AddConsumer which can be found CommandBus.js. It tries to assert a queue against a channel that doesn't exist yet. What I don't understand, is looking at it. I feel like I've covered all the await areas, so that it should wait on channel creation. The CommandBus is always fetched as a singleton. I don't if this poses issues, but again it is one of those areas that I cover with awaits as well. Any help is great thanks everyone.
You can't really use asynchronous operations in a constructor. The problem is that the constructor needs to return your instance so it can't also return a promise that will tell the caller when it's done.
So, in your Consumers class, await new Consumers(); is not doing anything useful. new Consumers() returns a new instance of a Consumers object so when you await that it doesn't actually wait for anything. Remember that await does something useful with you await a promise. It doesn't have any special powers to await your constructor being done.
The usual way around this is to create a factory function (which can be a static in your design) that returns a promise that resolves to the new object.
Since you're also trying to make a singleton, you would cache the promise the first time you create it and always return the promise to the caller so the caller would always use .then() to get the finished instance. The first time they call it, they'd get a promise that was still pending, but later they'd get a promise that was already fulfilled. In either case, they just use .then() to get the instance.
I don't know TypeScript well enough to suggest to you the actual code for doing this, but hopefully you get the idea from the description. Turn GetInstance() into a factory function that returns a promise (that you cache) and have that promise resolve to your instance.
Something like this:
static async GetInstance(): Promise<Consumers> {
if (!Consumers.promise) {
let obj = new Consumers();
Consumers.promise = obj.init().then(() => obj);
}
return Consumers.promise;
}
Then, the caller would do:
Consumers.getInstance().then(consumer => {
// code here to use the singleton consumer object
}).catch(err => {
console.log("failed to get consumer object");
});
You will have to do the same thing in any class that has async operations involved in initializing the object (like CommandBus) too and each .init() call needs to call the base class super.init().then(...) so base class can do its thing to get properly initialized too and the promise your .init() is linked to the base class too. Or, if you're creating other objects that themselves have factory functions, then your .init() needs to call those factory functions and link their promises together so the .init() promise that is returned is linked to the other factory function promises too (so the promise your .init() returns will not resolve until all dependent objects are all done).
I am using typeorm with typescript in my node Js application. I am trying to figure out the way of using the single DB connection for all functions in the class. For example, I have two functions my class and want to use the global/single connection for all functions instead of creating a connection in every function as shown below:
export class SQLDBService implements IDatabaseService{
private readonly logger = getLogger("SQLDBService");
private connection:Connection;
getConversation(conversationId: string): ConversationEntity {
let conversationEntity = new ConversationEntity();
createConnection(/*...*/).then(async connection => {
let dbObj = await connection.getRepository(ConversationEntity).findOne({
conversationId: Equal(conversationId)
});
if(dbObj)
conversationEntity = dbObj;
});
return conversationEntity;
}
pushWrapUp(conversationId: string, wrapUp: string): void {
createConnection().then(async connection => {
let conversationEntity = await connection.getRepository(ConversationEntity).findOne({
conversationId: Equal(conversationId)
});
if(conversationEntity){
conversationEntity.wrapUp = wrapUp;
conversationEntity.endTime = new Date();
await connection.manager.save(conversationEntity);
}
});
}}
Can someone point me in the right direction?
The code above doesn't efficiently use async..await because promises aren't chained, this results in poor error handling and improper control flow.
As the documentation explains,
TypeORM's Connection does not setup a database connection as it might seem, instead it setups a connection pool. <...> Connection pool setup is established once connect method of the Connection is called. connect method is called automatically if you setup your connection using createConnection function. Disconnection (closing all connections in the pool) is made when close is called. Generally, you must create connection only once in your application bootstrap, and close it after you completely finished working with the database.
createConnection is supposed to be called only once on application initialization. Since it's asynchronous, initialization routine should wait for it before using TypeORM models.
As the documentation suggests, getConnection() can be used instead of createConnection. Since the purpose is to get a repository for default connection, getRepository can be used instead:
It's:
import {getRepository} from "typeorm";
...
async getConversation(conversationId: string): ConversationEntity {
let conversationEntity = new ConversationEntity();
let dbObj = getRepository(ConversationEntity).findOne({
conversationId: Equal(conversationId)
});
if(dbObj) conversationEntity = dbObj;
return conversationEntity;
}
You should use a global connection pool which will create, hold, and take care the used connections for you. I am not familiar with node.js, so I cannot give out a name of this kind 3rd party library. But there must be some, since the connection pool is a widely accepted design pattern.
this quite minimal refactoring should do the trick
export class SQLDBService implements IDatabaseService {
private readonly logger = getLogger("SQLDBService");
private connection:Connection;
init() {
this.connection = await createConnection(/*...*/)
}
getConversation(conversationId: string): ConversationEntity {
let conversationEntity = new ConversationEntity();
let dbObj = await this.connection.getRepository(ConversationEntity).findOne({
conversationId: Equal(conversationId)
});
if(dbObj)
conversationEntity = dbObj;
return conversationEntity;
}
pushWrapUp(conversationId: string, wrapUp: string): void {
let conversationEntity = await this.connection.getRepository(ConversationEntity).findOne({
conversationId: Equal(conversationId)
});
if(conversationEntity){
conversationEntity.wrapUp = wrapUp;
conversationEntity.endTime = new Date();
await this.connection.manager.save(conversationEntity);
}
}
}
const db = new SQLDBService()
try {
await db.init()
}
catch (error) {
console.error("db connection error")
console.error(error)
console.error("db connection error")
}