I have a Typescript-backed Express.js project that uses a singleton Redis client.
The singleton includes wrapper functions to Redis commands needed for my application (e.g., SADD).
Singleton
Here is a snippet of my singleton Redis client service, which is relevant to my question:
/**
* Set up a singleton class instance to interface
* with the Redis database, along with helper async
* functions that provide functionality.
*/
var redis = require("redis");
// https://github.com/redis/node-redis/issues/1673
type RedisClientType = ReturnType<typeof redis.createClient>;
type RedisClientOptionsType = Parameters<typeof redis.createClient>[0];
export class Redis {
private static instance: Redis;
private static client: RedisClientType;
constructor() {
if (Redis.instance)
return Redis.instance;
Redis.instance = this;
Redis.client = null;
}
/* ... */
async initializeClient(options: RedisClientOptionsType) {
Redis.client = redis.createClient(options);
Redis.client.on('connect', function() {
Redis.instance.log('Client connected');
});
Redis.client.on('error', function(err: Error) {
Redis.instance.log(`Could not communicate with Redis client [${err}]`);
});
await Redis.client.connect();
}
async shutdownClient() {
await Redis.client.quit();
}
async multi() {
await Redis.client.multi();
}
async exec() {
await Redis.client.exec();
}
/* ... */
async sAdd(k: string, v: string) {
return await Redis.client.sAdd(k, v);
}
}
Individual calls to sAdd, sMembers, etc. work fine. So the client itself is initialized correctly, and it is able to process basic Redis calls.
What I would like to do is perform some chained transactions, e.g., from a rudimentary POST request using the singleton Redis client service, process some data from an uploaded file, and then add some key-value pairs (to start):
import { Redis } from '#/service/redis';
/* ... */
export const myPost = async (req: Request, res: Response) => {
const redis = new Redis(); // initialized client, as defined above
const k = 'my-key';
const v = 'my-value';
await redis
.multi()
.sAdd(k, v)
.exec();
}
Problem
The problem is that I get two errors with the chained await ... call.
First error
The first error is related to the await keyword, just before the multi/sAdd/exec call:
'await' expressions are only allowed within async functions and at the top levels of modules.ts(1308)
This await is within the async-ed post function. (I assume that I need to this handle the results of the underlying Promise chain.)
Second error
The second error is related to the sAdd wrapper:
Property 'sAdd' does not exist on type 'Promise<void>'.ts(2339)
I tried to add a return type to the call to multi:
async multi(): Promise<RedisClientType> {
await Redis.client.multi();
}
But this did not resolve the error with sAdd.
Question
What changes do I make to the service/singleton, which would allow calls to wrapper functions to be chained?
Related
I am building a Typescript package around amqplib's Promise API to make it simpler for me to send messages between two RabbitMQ queues.
This is a method inside a class of said package that is responsible for sending a message to RabbitMQ. It's not actually sending messages, because I can't see them from CloudAMQP.
class PostOffice {
connection: any;
constructor(connection: any) {
this.connection = connection;
}
//...
// TMQMessage is an object that can be trivially serialized to JSON
// The Writer class holds the channel because I'm having trouble
// making Typescript class member updates stick after a method returns.
async send(writer: Writer, message: TMQMessage) {
//...RabbitMQ code begins here
let channel = writer.stageConsume[0]; // This channel is created elsewhere
// Queue is created here and I can see it in CloudAMQP.
await channel.assertQueue(message.to + "/stage/1", {
durable: true,
}); // `message.to` is a string indicating destination, e.g. 'test/hello'
// According to docs, the content should be converted to Buffer
await channel.sendToQueue(message.to + "/stage/1", Buffer.from(JSON.stringify(message)));
channel.close()
}
}
This is a class I made that creates connections and channels, which is being used here:
import { Writer } from "./Writer";
export class ConnectionFactory {
url: string;
amqp: any;
constructor(url: string) {
this.url = url;
this.amqp = require('amqplib');
}
async connect(timeout: number = 2000) {
let connection = await this.amqp.connect(this.url, {
// timeout for a message acknowledge.
timeout, // Make the timeout 2s per now
})
return connection;
}
async createChannels(connection: any, writer: Writer) {
//... relevant code starts here
let stageChannels = [];
stageChannels.push(await connection.createChannel())
writer.stageConsume = stageChannels;
return writer;
}
}
}
My connection factory seems to be working properly, because I can see the connections from CloudAMQP's dashboard. Also I can see the Queues that have been created (asserted in my code) from the dashboard too. However, I am unable to get amqplib to send a message out to CloudAMQP.
Here's the (async) code I'm using to call my package:
let Cf = new ConnectionFactory(url)
let connection = await Cf.connect()
let writer = new Writer();
let message: TMQMessage = {
//... message content in JSON
}
writer = await Cf.createChannels(connection, writer)
let po = new PostOffice(connection)
await po.send(writer, message)
What seems to be wrong?
Probably want to await the close method too. But in general I would recommend amqp-client.js (that also have TypeScript definitions), https://github.com/cloudamqp/amqp-client.js/
so I'm using RabbitMQ for some Projects and i noticed that i ll use some duplicate code all the Time that's why i decided to make a Wrapper Class or Interface that have some function to use RabbitMQ direct without repeating the code all the time. i began to do this yesterday and i already had some Problems since i wanted to use OOP and Javascript can be complicated when using OOP (at least i think so)
I began with creating a class IRAbbitMQ with function init to initialize a connection and create a channel, i knew that i cant use nested classes so instead i wanted to use Factory functions, i tried to make the connection and channel a part of the class IRabbitMQ properties but i dont know why that gave me undefined when i create an instance of it
class IRabbitMQ {
constructor() {
this.init(rabbitMQServer); // rabbitMQServer for example 'localhost//5672'
}
// establish a Connection to RAbbitMQ Server
async init(host) {
try {
let connection = await amqplib.connect(host);
let channel = await connection.createChannel();
channel.prefetch(1);
console.log(' [x] Awaiting RPC requests');
this.connection = connection;
this.channel = channel;
}
catch(err) {
console.error(err);
}
}
// Close the Connection with RabbitMQ
closeConnection() {
this.connection.close();
}
log() {
console.log(this.connection);
}
EventPublisher() {
function init(IRabbit, publisherName) {
if(!IRabbit.connection) {
throw new Error('Create an Instance of IRabbitMQ to establish a Connection');
}
let ch = IRabbit.channel;
console.log(ch);
}
return {
init : init
}
}
}
var r = new IRabbitMQ();
r.log();
when i run the code the output is undefined, i dont know why since i m initializing the connection and channel properties in the init function and then called that function in the constructor so that should be initialized when i create an object of the Wrapper class. i wanted also to take some advices from you wether it is good to use classes or is there any other better way to create a Wrapper class or Interface for RabbitMQ to make it easy to use it and not have to duplicate Code.
Not really an answer, but I was able to successfully log the connection with this example code. I trimmed out other code to just focus on the .log() part that was logging a undefined.
Code is far from perfect, but works at least
const amqplib = require('amqplib');
class IRabbitMQ {
constructor() { }
async init(host) {
try {
const connection = await amqplib.connect(host);
const channel = await connection.createChannel();
channel.prefetch(1);
console.log(' [x] Awaiting RPC requests');
this.connection = connection;
this.channel = channel;
}catch(err) {
console.error(err);
}
}
log() {
console.log(this.connection);
}
}
async function createInstance(){
const instance = new IRabbitMQ();
try {
await instance.init('amqp://localhost');
}catch (e) {
throw new Error('OOPS!');
}
return instance;
}
async function runLogic() {
const r = await createInstance();
r.log();
}
runLogic().catch(console.log);
Just comment if you'd want me to give additional advice/tips, but this seems to work for me.
My problem is the following: I want to test a method that uploads a buch of data into an AWS S3 bucket. The problem is: I don't want to really upload data every time I am testing and I don't want to care about credentials sitting in the env. So I want to setup Sinon's fake-server module to simulate the upload and return the same results then S3 would. Sadly, it seems to be difficult to find a working example with code using async/await.
My test looks like this:
import {skip, test, suite} from "mocha-typescript";
import Chai from "chai";
import {S3Uploader} from "./s3-uploader.class";
import Sinon from "sinon";
#suite
class S3UploaderTest {
public server : Sinon.SinonFakeServer | undefined;
before() {
this.server = Sinon.fakeServer.create();
}
after() {
if (this.server != null) this.server.restore();
}
#test
async "should upload a file to s3 correctly"(){
let spy = Sinon.spy();
const uploader : S3Uploader = new S3Uploader();
const upload = await uploader.send("HalloWelt").toBucket("onetimeupload.test").toFolder("test/hw.txt").upload();
Chai.expect(upload).to.be.a("object");
}
}
Inside of the uploader.upload() method, I resolved a promise out of a callback. So how can I simulate the uploading-process?
Edit: Here is the code of the s3-uploader:
import AWS from "aws-sdk";
export class S3Uploader {
private s3 = new AWS.S3({ accessKeyId : process.env.ACCESS_KEY_ID, secretAccessKey : process.env.SECRET_ACCESS_KEY });
private params = {
Body: null || Object,
Bucket: "",
Key: ""
};
public send(stream : any) {
this.params.Body = stream;
return this;
}
public toBucket(bucket : string) {
this.params.Bucket = bucket;
return this;
}
public toFolder(path : string) {
this.params.Key = path;
return this;
}
public upload() {
return new Promise((resolve, reject) => {
if (process.env.ACCESS_KEY_ID == null || process.env.SECRET_ACCESS_KEY == null) {
return reject("ERR_NO_AWS_CREDENTIALS");
}
this.s3.upload(this.params, (error : any, data : any) => {
return error ? reject(error) : resolve(data);
});
});
}
}
Sinon fake servers are something you might use to develop a client that itself makes requests, instead of a wrapper around an existing client like AWS.S3, like you're doing. In this case, you're better off just stubbing the behavior of AWS.S3 instead of testing the actual requests it makes. That way you can avoid testing the implementation details of AWS.S3.
Since you're using TypeScript and you've made your s3 client private, you're going to need to make some changes to expose it to your tests. Otherwise, you won't be able to stub its methods without the TS compiler complaining about it. You also won't be able to write assertions using the params object, for similar reasons.
Since I don't use TS regularly, I'm not too familiar with it's common dependency injection techniques, but one thing you could do is add optional constructor arguments to your S3Uploader class that can overwrite the default s3 and arguments properties, like so:
constructor(s3, params) {
if (s3) this.s3 = s3;
if (params) this.params = params;
}
After which, you can create a stub instance and pass it to your test instance like this:
const s3 = sinon.createStubInstance(AWS.S3);
const params = { foo: 'bar' };
const uploader = new S3Uploader(s3, params);
Once you have the stub instance in place, you can write assertions to make sure the upload method was called the way you want it to be:
sinon.assert.calledOnce(s3.upload);
sinon.assert.calledWith(s3.upload, sinon.match.same(params), sinon.match.func);
You can also affect the behavior the upload method using the sinon stub api. For example, to make it fail like so:
s3.upload.callsArgWith(1, null);
Or make it succeed like so:
const data = { whatever: 'data', you: 'want' };
s3.upload.callsArgWith(1, null, data);
You'll probably want a completely separate test for each of these cases, using an instance before hook to avoid duplicating the common setup stuff. Testing for success will involve simply awaiting the promise and checking that its result is the data. Testing for failure will involve a try/catch that ensures the promise was rejected with the proper error.
Also, since you seem to be doing actual unit tests here, I'll recommend testing each S3Uploader method separately instead of calling them all in once big test. This drastically reduces the number of possible cases you need to cover, making your tests a lot more straightforward. Something like this:
#suite
class S3UploaderTest {
params: any; // Not sure the best way to type this.
s3: any; // Same. Sorry, not too experienced with TS.
uploader: S3Uploader | undefined;
before() {
this.params = {};
this.s3 = sinon.createStubInstance(AWS.S3);
this.uploader = new S3Uploader(this.s3, this.params);
}
#test
"send should set Body param and return instance"() {
const stream = "HalloWelt";
const result = this.uploader.send(stream);
Chai.expect(this.params.Body).to.equal(stream);
Chai.expect(result).to.equal(this.uploader);
}
#test
"toBucket should set Bucket param and return instance"() {
const bucket = "onetimeupload.test"
const result = this.uploader.toBucket(bucket);
Chai.expect(this.params.Bucket).to.equal(bucket);
Chai.expect(result).to.equal(this.uploader);
}
#test
"toFolder should set Key param and return instance"() {
const path = "onetimeupload.test"
const result = this.uploader.toFolder(path);
Chai.expect(this.params.Key).to.equal(path);
Chai.expect(result).to.equal(this.uploader);
}
#test
"upload should attempt upload to s3"() {
this.uploader.upload();
sinon.assert.calledOnce(this.s3.upload);
sinon.assert.calledWith(
this.s3.upload,
sinon.match.same(this.params),
sinon.match.func
);
}
#test
async "upload should resolve with response if successful"() {
const data = { foo: 'bar' };
s3.upload.callsArgWith(1, null, data);
const result = await this.uploader.upload();
Chai.expect(result).to.equal(data);
}
#test
async "upload should reject with error if not"() {
const error = new Error('Test Error');
s3.upload.callsArgWith(1, error, null);
try {
await this.uploader.upload();
throw new Error('Promise should have rejected.');
} catch(err) {
Chai.expect(err).to.equal(err);
}
}
}
If I were doing this with mocha proper, I'd group each method's tests into a nested describe block. I'm not sure if that's encouraged or even possible with mocha-typescript, but if so you might consider it.
So no matter what I've read, even once I do it right, I can't seem to get the the hang of async and await. For example I have this in my startup.
startup.js
await CommandBus.GetInstance();
await Consumers.GetInstance();
Debugging jumps to the end of the get instance for CommandBus (starting up a channel for rabbitmq) and start Consumers.GetInstance() which fails since channel is null.
CommandBus.js
export default class CommandBus {
private static instance: CommandBus;
private channel: any;
private conn: Connection;
private constructor() {
this.init();
}
private async init() {
//Create connection to rabbitmq
console.log("Starting connection to rabbit.");
this.conn = await connect({
protocol: "amqp",
hostname: settings.RabbitIP,
port: settings.RabbitPort,
username: settings.RabbitUser,
password: settings.RabbitPwd,
vhost: "/"
});
console.log("connecting channel.");
this.channel = await this.conn.createChannel();
}
static async GetInstance(): Promise<CommandBus> {
if (!CommandBus.instance) {
CommandBus.instance = new CommandBus();
}
return CommandBus.instance;
}
public async AddConsumer(queue: Queues) {
await this.channel.assertQueue(queue);
this.channel.consume(queue, msg => {
this.Handle(msg, queue);
});
}
}
Consumers.js
export default class Consumers {
private cb: CommandBus;
private static instance: Consumers;
private constructor() {
this.init();
}
private async init() {
this.cb = await CommandBus.GetInstance();
await cb.AddConsumer(Queues.AuthResponseLogin);
}
static async GetInstance(): Promise<Consumers> {
if (!Consumers.instance) {
Consumers.instance = new Consumers();
}
return Consumers.instance;
}
}
Sorry I realize this is in Typescript, but I imagine that doesn't matter. The issue occurs specifically when calling cb.AddConsumer which can be found CommandBus.js. It tries to assert a queue against a channel that doesn't exist yet. What I don't understand, is looking at it. I feel like I've covered all the await areas, so that it should wait on channel creation. The CommandBus is always fetched as a singleton. I don't if this poses issues, but again it is one of those areas that I cover with awaits as well. Any help is great thanks everyone.
You can't really use asynchronous operations in a constructor. The problem is that the constructor needs to return your instance so it can't also return a promise that will tell the caller when it's done.
So, in your Consumers class, await new Consumers(); is not doing anything useful. new Consumers() returns a new instance of a Consumers object so when you await that it doesn't actually wait for anything. Remember that await does something useful with you await a promise. It doesn't have any special powers to await your constructor being done.
The usual way around this is to create a factory function (which can be a static in your design) that returns a promise that resolves to the new object.
Since you're also trying to make a singleton, you would cache the promise the first time you create it and always return the promise to the caller so the caller would always use .then() to get the finished instance. The first time they call it, they'd get a promise that was still pending, but later they'd get a promise that was already fulfilled. In either case, they just use .then() to get the instance.
I don't know TypeScript well enough to suggest to you the actual code for doing this, but hopefully you get the idea from the description. Turn GetInstance() into a factory function that returns a promise (that you cache) and have that promise resolve to your instance.
Something like this:
static async GetInstance(): Promise<Consumers> {
if (!Consumers.promise) {
let obj = new Consumers();
Consumers.promise = obj.init().then(() => obj);
}
return Consumers.promise;
}
Then, the caller would do:
Consumers.getInstance().then(consumer => {
// code here to use the singleton consumer object
}).catch(err => {
console.log("failed to get consumer object");
});
You will have to do the same thing in any class that has async operations involved in initializing the object (like CommandBus) too and each .init() call needs to call the base class super.init().then(...) so base class can do its thing to get properly initialized too and the promise your .init() is linked to the base class too. Or, if you're creating other objects that themselves have factory functions, then your .init() needs to call those factory functions and link their promises together so the .init() promise that is returned is linked to the other factory function promises too (so the promise your .init() returns will not resolve until all dependent objects are all done).
I'm trying to develop a NodeJS app connecting to Firebase. I can connect successfully, but I'm unable to figure how to manage the scope in the then call.
I'm using NodeJS 6.9.2
My test implementation looks like this:
const EventEmitter = require('events');
const fb = require('firebase')
class FireGateway extends EventEmitter {
constructor() {
super();
if ( this.instance ) {
return this.instance;
}
// INIT
var fbConfig = {
apiKey: "xxxxx",
authDomain: "xxxxx.firebaseapp.com",
databaseURL: "https://xxxxx.firebaseio.com/"
};
fb.initializeApp(fbConfig)
this.instance = this;
this.testvar = "aaa";
}
login() {
fb.auth().signInWithEmailAndPassword ("email", "pwd")
.catch(function(error) {
// Handle Errors here.
}).then( function(onresolve, onreject) {
if (onresolve) {
console.log(this.testvar);
// "Cannot read property 'testvar' of undefined"
this.emit('loggedin');
// error as well
}
})
}
}
module.exports = FireGateway;
------
...
var FireGateway = require('./app/fireGateway');
this.fireGW = new FireGateway();
this.fireGW.login();
....
Any idea how can I manage it?
The callback passed to then is being called asynchronously from another context, so the this doesn't correspond to the instantiated object.
Using ES6 arrow functions you can keep your object context, since an arrow function does not create its own this context.
By the way, the syntax you are using in the then method is not correct, then accepts two callbacks with one argument each one. Check the syntax here.
The catch before the then is not necessary as well I think, it would make more sense to put it at the end.
It would be something like this:
login() {
fb.auth().signInWithEmailAndPassword("email", "pwd")
.then(
(onResolve) => {
console.log(this.testvar);
this.emit('loggedin');
},
(onReject) = > {
// error handling goes here
});
}
On the other hand, it seems login method is doing an asynchronous operation, so you might want to wait for it to finish in your code. I would make the login method return a Promise, so you can wait for it outside:
login() {
return fb.auth().signInWithEmailAndPassword("email", "pwd")
...
}