Async/await confusion using singeltons - node.js

So no matter what I've read, even once I do it right, I can't seem to get the the hang of async and await. For example I have this in my startup.
startup.js
await CommandBus.GetInstance();
await Consumers.GetInstance();
Debugging jumps to the end of the get instance for CommandBus (starting up a channel for rabbitmq) and start Consumers.GetInstance() which fails since channel is null.
CommandBus.js
export default class CommandBus {
private static instance: CommandBus;
private channel: any;
private conn: Connection;
private constructor() {
this.init();
}
private async init() {
//Create connection to rabbitmq
console.log("Starting connection to rabbit.");
this.conn = await connect({
protocol: "amqp",
hostname: settings.RabbitIP,
port: settings.RabbitPort,
username: settings.RabbitUser,
password: settings.RabbitPwd,
vhost: "/"
});
console.log("connecting channel.");
this.channel = await this.conn.createChannel();
}
static async GetInstance(): Promise<CommandBus> {
if (!CommandBus.instance) {
CommandBus.instance = new CommandBus();
}
return CommandBus.instance;
}
public async AddConsumer(queue: Queues) {
await this.channel.assertQueue(queue);
this.channel.consume(queue, msg => {
this.Handle(msg, queue);
});
}
}
Consumers.js
export default class Consumers {
private cb: CommandBus;
private static instance: Consumers;
private constructor() {
this.init();
}
private async init() {
this.cb = await CommandBus.GetInstance();
await cb.AddConsumer(Queues.AuthResponseLogin);
}
static async GetInstance(): Promise<Consumers> {
if (!Consumers.instance) {
Consumers.instance = new Consumers();
}
return Consumers.instance;
}
}
Sorry I realize this is in Typescript, but I imagine that doesn't matter. The issue occurs specifically when calling cb.AddConsumer which can be found CommandBus.js. It tries to assert a queue against a channel that doesn't exist yet. What I don't understand, is looking at it. I feel like I've covered all the await areas, so that it should wait on channel creation. The CommandBus is always fetched as a singleton. I don't if this poses issues, but again it is one of those areas that I cover with awaits as well. Any help is great thanks everyone.

You can't really use asynchronous operations in a constructor. The problem is that the constructor needs to return your instance so it can't also return a promise that will tell the caller when it's done.
So, in your Consumers class, await new Consumers(); is not doing anything useful. new Consumers() returns a new instance of a Consumers object so when you await that it doesn't actually wait for anything. Remember that await does something useful with you await a promise. It doesn't have any special powers to await your constructor being done.
The usual way around this is to create a factory function (which can be a static in your design) that returns a promise that resolves to the new object.
Since you're also trying to make a singleton, you would cache the promise the first time you create it and always return the promise to the caller so the caller would always use .then() to get the finished instance. The first time they call it, they'd get a promise that was still pending, but later they'd get a promise that was already fulfilled. In either case, they just use .then() to get the instance.
I don't know TypeScript well enough to suggest to you the actual code for doing this, but hopefully you get the idea from the description. Turn GetInstance() into a factory function that returns a promise (that you cache) and have that promise resolve to your instance.
Something like this:
static async GetInstance(): Promise<Consumers> {
if (!Consumers.promise) {
let obj = new Consumers();
Consumers.promise = obj.init().then(() => obj);
}
return Consumers.promise;
}
Then, the caller would do:
Consumers.getInstance().then(consumer => {
// code here to use the singleton consumer object
}).catch(err => {
console.log("failed to get consumer object");
});
You will have to do the same thing in any class that has async operations involved in initializing the object (like CommandBus) too and each .init() call needs to call the base class super.init().then(...) so base class can do its thing to get properly initialized too and the promise your .init() is linked to the base class too. Or, if you're creating other objects that themselves have factory functions, then your .init() needs to call those factory functions and link their promises together so the .init() promise that is returned is linked to the other factory function promises too (so the promise your .init() returns will not resolve until all dependent objects are all done).

Related

Mark an asynchronous service function as asynchronous later in the controller

I have learned that a function with an asynchronous call (e.g. query to a database) is marked as await and the whole function block as async. However, I can apparently define some asynchronous functions without await and call them later with await (in my controller) and for others I am forced immediately to use await in my service class (VSC editor).
I have a user service class with CRUD operations. I can define findOne(), create() and find() without await, even though they perform asynchronous operations. In the controller I use them with async-await and I don't get an error from VSC even if I forget it. However, I have to use my update() and remove() functions in my service class with await because VSC shows me an error and says that I am missing await. Why do the update() and remove() functions have to be immediately marked with await and the others three do not? The functions save(), findOne() and find() have the same Promise return value as my other two functions and access the same repository.
My code (service class):
#Injectable()
export class UsersService {
constructor(#InjectRepository(User) private repo: Repository<User>) {}
create(email: string, password: string) {
const user = this.repo.create({ email, password });
return this.repo.save(user);
}
findOne(id: number) {
return this.repo.findOne(id);
}
find(email: string) {
return this.repo.find({ email });
}
async update(id: number, attrs: Partial<User>) {
const user = await this.findOne(id);
if (!user) {
throw new NotFoundException('user not found');
}
Object.assign(user, attrs);
return this.repo.save(user);
}
async remove(id: number) {
const user = await this.findOne(id);
if (!user) {
throw new NotFoundException('user not found');
}
return this.repo.remove(user);
}
}
Where is the difference and should I then rather always mark all my CRUD operations in the service class immediately as async-await in order to be able to call them later in the controller without async-await?
PS: Sorry if my text is still written too confusing. Why do I have to write await this.findOne() in the function remove(), but I can use this function findOne() with this.repo.findOne(id) without await in the same class, although repo.findOne() is an asynchronous function?
You need to use await because you want the value resolved by the promise returned by this.findOne(id)
And this.repo.find() will return a promise as it's async, thus UsersService#findOne returns a Promise too. So:
return await this.repo.findOne(id) will behave the same as:
return this.repo.findOne(id)
Learn about async/await:
https://nodejs.dev/learn/modern-asynchronous-javascript-with-async-and-await
https://javascript.info/async-await
https://jakearchibald.com/2017/await-vs-return-vs-return-await

How to do chained node-redis transactions with singleton wrapper functions

I have a Typescript-backed Express.js project that uses a singleton Redis client.
The singleton includes wrapper functions to Redis commands needed for my application (e.g., SADD).
Singleton
Here is a snippet of my singleton Redis client service, which is relevant to my question:
/**
* Set up a singleton class instance to interface
* with the Redis database, along with helper async
* functions that provide functionality.
*/
var redis = require("redis");
// https://github.com/redis/node-redis/issues/1673
type RedisClientType = ReturnType<typeof redis.createClient>;
type RedisClientOptionsType = Parameters<typeof redis.createClient>[0];
export class Redis {
private static instance: Redis;
private static client: RedisClientType;
constructor() {
if (Redis.instance)
return Redis.instance;
Redis.instance = this;
Redis.client = null;
}
/* ... */
async initializeClient(options: RedisClientOptionsType) {
Redis.client = redis.createClient(options);
Redis.client.on('connect', function() {
Redis.instance.log('Client connected');
});
Redis.client.on('error', function(err: Error) {
Redis.instance.log(`Could not communicate with Redis client [${err}]`);
});
await Redis.client.connect();
}
async shutdownClient() {
await Redis.client.quit();
}
async multi() {
await Redis.client.multi();
}
async exec() {
await Redis.client.exec();
}
/* ... */
async sAdd(k: string, v: string) {
return await Redis.client.sAdd(k, v);
}
}
Individual calls to sAdd, sMembers, etc. work fine. So the client itself is initialized correctly, and it is able to process basic Redis calls.
What I would like to do is perform some chained transactions, e.g., from a rudimentary POST request using the singleton Redis client service, process some data from an uploaded file, and then add some key-value pairs (to start):
import { Redis } from '#/service/redis';
/* ... */
export const myPost = async (req: Request, res: Response) => {
const redis = new Redis(); // initialized client, as defined above
const k = 'my-key';
const v = 'my-value';
await redis
.multi()
.sAdd(k, v)
.exec();
}
Problem
The problem is that I get two errors with the chained await ... call.
First error
The first error is related to the await keyword, just before the multi/sAdd/exec call:
'await' expressions are only allowed within async functions and at the top levels of modules.ts(1308)
This await is within the async-ed post function. (I assume that I need to this handle the results of the underlying Promise chain.)
Second error
The second error is related to the sAdd wrapper:
Property 'sAdd' does not exist on type 'Promise<void>'.ts(2339)
I tried to add a return type to the call to multi:
async multi(): Promise<RedisClientType> {
await Redis.client.multi();
}
But this did not resolve the error with sAdd.
Question
What changes do I make to the service/singleton, which would allow calls to wrapper functions to be chained?

Export resolved promise result

Consider I have a.js with following class
class Connector {
constructor (url) {
this.url = url;
this.conneciton = null;
}
async connect() {
this.connection = await someThidPartyModule.connect(url);
return this;
}
}
// here I would like to do something like
// export default new Connector().connect();
Then use in b.js, c.js etc. connection from resovled connect method:
import Connector from 'a.js';
Connector.connection.callSomeMethod(); // here connection already exists after that promise resolved
As far as I aware it is not possible to do this, but maybe the some hack or workaround exists?
So after some tries found following:
export Promise is not a good idea, as on each import we also will get Promise, event if it is already resolved
Better to export class instance, call connect method and then allow to use it in all other files
Sadly anyway promise moving up to main endpoint file, where we have to make initialization async function and inside wait for promise to resolve
Also tried to export class with static create method, but that leave us with instance on the far end, which cannot be exported to other files.

what is the best method to multi request with callback or promise on node.js

I have a situation. I use Node.js to connect to a special hardware. let assume that I have two functions to access the hardware.
hardware.send('command');
hardware.on('responce', callback);
At first, I made a class to interface this to the application layer like this (I write simplified code over here for better understanding)
class AccessHardware {
constructor() {
}
updateData(callback) {
hardware.on('responce', callback);
hardware.send('command');
}
}
Now, the problem is that if there are multiple requests from the application layer to this access layer, they should not send multiple 'command' to the hardware. Instead, they should send one command and all of those callbacks can be served once the hardware answer the command.
So I update the code something like this:
class AccessHardware {
constructor() {
this.callbackList = [];
hardware.on('responce', (value) => {
while (this.callbackList.length > 0) {
this.callbackList.pop()(value);
}
});
}
updateData(callback) {
if (this.callbackList.length == 0) {
hardware.send('command');
}
this.callbackList.push(callback);
}
}
Of course, I prefer to use promise to handle the situation. so what is your suggestion to write this code with promise?
Next question, is this approach to make a 'list of callbacks' good?
I prefer to use promise to handle the situation. So what is your suggestion to write this code with promise?
You'd store a promise in your instance that will be shared between all method callers that want to share the same result:
class AccessHardware {
constructor(hardware) {
this.hardware = hardware;
this.responsePromise = null;
}
updateData() {
if (!this.responsePromise) {
this.responsePromise = new Promise(resolve => {
this.hardware.on('responce', resolve);
this.hardware.send('command');
});
this.responsePromise.finally(() => {
this.responsePromise = null; // clear cache as soon as command is done
});
}
return this.responsePromise;
}
}
Btw, if hardware is a global variable, there's no reason to use a class here.
Is the current solution to make a 'list of callbacks' good?
Yes, that's fine as well for a non-promise approach.

Is there a common pattern for node modules that need to async load large files before they are useful?

I don't like anything I've seen so far...
Module
import lineReader from 'line-reader'
let bigFile = './huge-file.csv'
export default class DooDad {
constructor() {
this.dictionary = new Map()
// Something like this seems like it'd be nice...
// await this.load()
}
load() {
return new Promise((resolve, reject) => {
lineReader.eachLine(bigFile, (line, last) => {
// this.dictionary = The contents of huge-file.csv
})
})
}
doStuff(foo) {
// Uses this.dictionary to do something interesting
// Problem: Unusable without first calling and waiting for this.load()
}
}
Usage
import DooDad from '../doodad'
let doodad = new DooDad()
// We have to first call and wait for load() before doodad is useable
doodad.load().then(x => {
doodad.doStuff()
})
Seems like you'd either want to...
1) Make the loading synchronous
2) Make a static create function on DooDad that returns a promise that resolves to a new instance of a DooDad
3) Make the constructor return a Promise (seems weird) Asynchronous constructor
4) Emit an event when its done loading
5) Leave it how it is
6) ????
doodad.load().then() makes perfect sense to me. You don't want constructors to be async so it makes sense to have .load() be where the async stuff is.
The other pattern I've seen is you export a factory-type function that returns a promise and when that promise is resolved, the resolved value is your fully formed object. This has the advantage of there is no access to the object until the async stuff is done and there is no temptation by calling code to try to use it before it is ready.
import makeDooDad from '../doodad'
makeDooDad().then(doodad => {
// you only get access to the object here after it's been fully
// initialized
doodad.doStuff();
});
And, the makeDooDad() factory function does something like this inside of your module:
function makeDooDad() {
let d = new DooDad();
// fully initialize the doodad object before resolving the promise
// and before returning the object
return d.load().then(() => {
// let the object itself by the resolved value
return d;
});
}
As for your other options:
Make the loading synchronous
This could be OK only if it is only done at server startup time. There is generally no real cost to doing some synchronous I/O at server startup time and often it makes things a lot simpler. For example, require() itself does synchronous I/O.
Make a static create function on DooDad that returns a promise that resolves to a new instance of a DooDad
That is essentially what I recommended above with the factory function. This is often times a good option.
Make the constructor return a Promise (seems weird) Asynchronous constructor
No. Don't really want to do that. Constructors should return objects, not promises. Use a factory function to return a promise.
Emit an event when its done loading
There are other pieces of code that do this such as creating a writeStream, emits an open event on the stream when the stream is actually open. But, in the days of promises, this isn't my favorite way of doing things for other types of objects that aren't already using lots of events.
Leave it how it is
It's OK as is, but I prefer the factory function that returns a promise.

Resources