Ensure an async method is never executed in parallel - node.js

There's an async method in some external library, which is not thread-safe (corrupts internal state when being called concurrently) and is very unlikely to be ever fixed in the near future. Yet I want the rest of my codebase to freely use it in any possible parallel vs. sequential combinations (in other words, I want my methods that use that method to still be thread-safe).
So I need to wrap that problematic method with some Active Object implementation, so that its invocations are always aligned (executed one after another).
My take on this is to use a chain of Promises:
var workQueue: Promise<any> = Promise.resolve();
function executeSequentially<T>(action: () => Promise<T>): Promise<T> {
return new Promise((resolve, reject) => {
workQueue = workQueue.then(() => {
return action().then(resolve, reject);
});
});
}
And then instead of directly calling that method:
const myPromise = asyncThreadUnsafeMethod(123);
I'll be calling it like this:
const myPromise = executeSequentially(() => asyncThreadUnsafeMethod(123));
But am I missing something? Any potential problems with this approach? Any better ideas?

There are no problems with this approach, using a promise queue is a standard pattern that will solve your problem.
While your code does work fine, I would recommend to avoid the Promise constructor antipattern:
let workQueue: Promise<void> = Promise.resolve();
const ignore = _ => {};
function executeSequentially<T>(action: () => Promise<T>): Promise<T> {
const result = workQueue.then(action);
workQueue = result.then(ignore, ignore);
return result;
}
Also you say that you'd be "calling the method" like executeSequentially(() => asyncThreadUnsafeMethod(123));, but this does leave the potential for mistakes where you forget the executeSequentially wrapper and still call the unsafe method directly. Instead I'd recommend to introduce an indirection where you wrap the entire safe call in a function and then export only that, having your codebase only interact with that facade and never with the library itself. To help with the creation of such a facade, I'd curry the executeSequentially function:
const ignore = _ => {};
function makeSequential<T, A extends unknown[]>(fn: (...args: A) => Promise<T>) => (...args: A) => Promise<T> {
let workQueue: Promise<void> = Promise.resolve();
return (...args) => {
const result = workQueue.then(() => fn(...args));
workQueue = result.then(ignore, ignore);
return result;
};
}
export const asyncSafeMethod = makeSequential(asyncThreadUnsafeMethod);
or just directly implement it for that one case
const ignore = _ => {};
let workQueue: Promise<void> = Promise.resolve();
export function asyncSafeMethod(x: number) {
const result = workQueue.then(() => asyncThreadUnsafeMethod(x));
workQueue = result.then(ignore, ignore);
return result;
}
import { asyncSafeMethod } from …;
asyncSafeMethod(123);

Related

Argument of type '() => Promise<string>' is not assignable to parameter of type 'Promise<string>'. ts(2345)

I'm defining a function that requires an asynchronous function as a parameter:
async function handle(def: Promise<string>) {
// ...
const data = await def;
console.log(`data equals: ${data}`)
}
I can succesfully execute this by passing a promise.
handle(new Promise(async (res, rej) => {
const data = await Promise.resolve("some data")
if (data == "invalid")
return rej("data is invalid")
res(data)
}))
I need the inner-function to be async, since I need to perform awaits inside. However, I dislike the async in the promise and read online that its considered an anti-pattern.
I thought it was a better idea to get rid of the promise and use a basic async function:
handle(async () => {
const data = await Promise.resolve("some data")
if (data == "invalid")
throw "data is invalid"
return data
})
But the TS compiler raises the error:
Argument of type '() => Promise<string>' is not assignable to parameter of type 'Promise<string>'. ts(2345)
I thought that Promises and async functions are somewhat interchangable. I read that async functions always return a promise. Apperantly I am not interpreting this correctly, but I'm unsure what the error is telling me.
I hope the problem I am sketching is clear. It would be greatly appreciated if someone could clarify the error or give suggestions on how to implement this in the desired way. Big thanks!
your function signature seems to be incorrect.
you are defining the parameter to be a Promise<string>, while you're really willing it to be () => Promise<string>.
async function handle(def: () => Promise<string>) {
// ...
const data = await def();
console.log(`data equals: ${data}`)
}
In general, the time to use the Promise constructor in a situation like this is right when you call a callback-based function that you want to promisify. But here, it doesn't look like there's anything to promisify, so the Promise constructor doesn't really help.
If
I need the inner-function to be async, since I need to perform awaits inside.
You can have an immediately invoked async IIFE - yes, using an async callback for the Promise constructor wouldn't be safe, because it wouldn't handle errors.
handle((async () => {
const data = await Promise.resolve("some data")
if (data == "invalid")
throw "data is invalid";
return data
})())
The other option is to use .then - .then is interchangeable with await in many situations:
handle(
Promise.resolve("some data")
.then((data) => {
if (data == "invalid")
throw "data is invalid";
return data;
})
);

How to return object after completeon of .map function in a function of type promise in nodejs

I am using nodejs for server side code. I am using .map() inside a method of type Promise. Here the loop execution executes perfectly,but the problem is that, method returns a value before the .map() function is fully executed.
Below is the code,where in i am using ,map() function. Here i want to return created_ids array only after the .map() is completely executed,but what's happening is created_ids will be returned before insert query is called.
public createDomainUser(user:any):Promise<any> {
let created_ids:any = [];
var md5 = require('md5');
(user['password']) = md5(user['password']);
return Promise.all(user.Boards.map(boardobj => {
BasePgRepo.db.query(USER_QUERIES['insert'],vals).then(rows => {
created_ids.push(rows);
})
return created_ids;
}))
}
"return created_ids" should be called after the completion of map(). Am i going wrong somewhere?
I also tried to clean your code. Your mistake was missing return before BasePgRepo call.
import md5 from "md5";
class MyClass {
async createDomainUser(user: any): Promise<any> {
let created_ids: any = [];
user['password'] = md5(user['password']);
await Promise.all(user.Boards.map((boardobj: any) =>
BasePgRepo.db.query(USER_QUERIES['insert'], vals)
.then((rows: any) => {
created_ids.push(rows);
})));
return created_ids;
}
}

Creating a promise from code that uses await

I would like to write a block of code using await syntax, immediately execute it, and create a promise that waits for execution to finish. I've come up with the following way to do this.
let makePromise = async () => {
return foo && await bar();
}
let promise = makePromise();
However, I find this hard to read and understand. Creating a function and then calling it right away seems counter-intuitive and goes against good practice in other programming languages. Is there a more idiomatic way to write this piece of code?
Particularly, this code is used in the following context.
let promises = items.map((item) => {
let makePromise = async () => {
return foo(item) && await bar(item);
}
return makePromise();
});
Why not using an async function directly? The following code behaves exactly the same as your example, but keep in mind, this results in an array of promises and awaits none of them.
function foo(x) { return "foo " + x; }
async function bar(x) { return "bar " + x; }
const items = [1];
let promises = items.map(
async item => foo(item) && await bar(item)
);
Promise.all(promises).then(x => console.log(x));
Async functions actually return a Promise. This is valid:
const x = async () => {
console.log(123)
}
x().then(() => {
console.log(456)
})
So in your case:
let promise = bar()

How to mock curry function with Jest?

add.js
export default a => b => a+b;
module.js
import add from './add';
export default {
add1: n => add(1)(n),
};
test/module.js
import add from '../add';
import module from '../module';
jest.mock('../add', () => () => jest.fn());
module.add1(6);
expect(add.mock.calls).toHaveLength(1);
this can be called, but add wouldn't be a mock function, instead add() is a mock function, but the call params were not recorded correctly.
jest.mock('../add', () => () => jest.fn(a => b => a+b));
has also tried this, which doesn't seem to work correctly as well.
jest.mock('../add', jest.fn(a => b => a+b));
this would throw the inline function error
Is there a correct way to mock curry function at the moment?
Simple version should look like this
jest.mock('../add', () => (a) => jest.fn(b => a+b));
So you mock add module with a function that return that when gets called it returns the spy, problem you can't test anything on the spy.
So we need to refactor it so you have the add1 think as a spy in scope of the test
import add from '../add'
jest.mock('../add', () => jest.fn)
const addC = jest.fn()
add.mockImplemetation((a) => {
addC.mockImplementation((b => a+b)
return addC
})
I know this is a bit old but with the following answer I would have saved so much time...
import module from "./module";
const mockOperation = {
add: jest.fn()
};
jest.mock("./add", () => () => {
return mockOperation.add;
});
describe("add ", () => {
it("is called at least once", () => {
module.add1(6);
expect(mockOperation.add.mock.calls).toHaveLength(1);
});
});
A very importante thing: the constant function that you want to mock must start with the word mock. Otherwise it will throw an error.
const mockOperation = {
add: jest.fn()
};
With the mockOperation constant assigned to the add file, by reference is calling to it's add method which is a jest.fn()
I've had to do this a lot recently so I wrote a helper function to help me out.
curryMock = (baseFn, maxCurries) => {
var callIndex = 0
var curries = []
let returnFn
baseFn.mockImplementation((arg) => {
curries[callIndex] = 0
const fn = returnFn(callIndex)
callIndex++
return fn
})
returnFn = (index: number) => {
if (curries[index] < maxCurries) {
curries[index]++
return (arg: any) => {
baseFn.mock.calls[index].push(arg)
return returnFn(index)
}
}
}
}
You pass the function a jest.fn() and curryMock modifies it's implementation so that all the arguments return functions are called with will be added to the original jest.fn().mock.calls array. The second argument specifies how many times to return a function. maxCurries = 0 is for a regular function () => {},
maxCurries = 1is for a regular function() => => {}`, etc.
In your case:
import add from '../add';
import module from '../module';
jest.mock('../add');
curryMock(add)
module.add1(6);
expect(add).toHaveBeenCalledWith(1, 6);
//equivalent to
//expect(add).toHaveBeenNthCalledWith(1, 1, 6);
//if you others to test..
module.add7(4)
expect(add).toHaveBeenNthCalledWith(2, 7, 4)
curryMock is a quick utility function that I wrote for personal use. I will likely clean it up and extend it later, but I hope you get the idea: testing curried functions does not have to laborious.

Testing Promises with multiple thens using testdoublejs

I am using testdouble for stubbing calls within my node.js project. This particular function is wrapping a promise and has multiple then calls within the function itself.
function getUser (rethink, username) {
return new Promise((resolve, reject) => {
let r = database.connect();
r.then(conn => database.table(tablename).filter({username}))
.then(data => resolve(data))
.error(err => reject(err));
});
}
So I am wanting to determine if the resolve and reject are handled correctly based on error conditions. Assume there is some custom logic in there that I need to validate.
For my test
import getUser from './user';
import td from 'testdouble';
test(t => {
const db = td.object();
const connect = td.function();
td.when(connect('options')).thenResolve();
const result = getUser(db, 'testuser');
t.verify(result);
}
The issue is that the result of connect needs to be a promise, so I use then resolve with a value which needs to be another promise that resolves or rejects.
The line it is relating to is the result of database.connect() is not a promise.
TypeError: Cannot read property 'then' of undefined
Anyone have success with stubbing this type of call with Test Double?
So figured out the resolution. There are a few things to note in the solution and that we encountered. In short the resolution ended up being this...
td.when(database.connect()).thenResolve({then: (resolve) => resolve('ok')});
This resolves a thenable that is returned when test double sees database connect. Then subsequent calls can also be added.
There is also a part to note if you send in an object to database.connect() you have to be aware that it is doing === equality checking and you will need to have a reference to that object for it to correctly use td.when.
Test double provides stubs for unit testing. And in your case 'db' is the object we need to mock. Creating the mocking db through
td.object(Database) // Database is the class or constructor of your db
will be the right choice, but to simply mock those methods you need in this case, I wouldn't pick that way.
Here's the tested module, 'some.js':
function getUser (database, username) {
return new Promise((resolve, reject) => {
let r = database.connect();
r.then(conn => database.table('table').filter({username:username}))
.then(data => resolve(data))
.catch(err => reject(err));
});
}
module.exports = getUser;
And the test file, using mocha and chai.expect, which is could also be any other unit test module here:
let td = require('testdouble');
let expect = require('chai').expect;
const getUser = require('./some');
describe('some.js',()=>{
it('getUser',()=>{
const db = {};
const name = 'name';
db.connect = td.function();
db.table = td.function('table');
db.filter = td.function('filter');
td.when(db.connect()).thenResolve(db);
td.when(db.table('table')).thenReturn(db);
td.when(db.filter({username: name})).thenResolve('some user data');
return getUser(db, name)
.then(user=>{
expect(user).to.equal('some user data')
})
.catch(e=>assert(e))
})
})
So please let me know if any of these confuse you.

Resources