Operators invoked multiple times for merged Observable although only one source emits - node.js

I have a function in which I'm calling an instance of Manager's onSpecificData() to which I'm subscribing in order to update my application's state (I'm managing a state on the server-side as well).
The problem is that in the SomeManager's implementation of onSpecificData() I'm merging 3 different Observables using merge() operator, which for some reason triggers the invocation of all the underlying Observable's operators even though only 1 of the sources is the one that's emitting a value
SomeManager.ts
export class DerivedManager implements Manager {
private driver: SomeDriver;
constructor(...) {
this.driver = new SomeDriver(...);
}
public onSpecificData(): Observable<DataType> {
return merge(
this.driver.onSpecificData(Sources.Source1).map((value) => {
return {source1: value};
}),
this.driver.onSpecificData(Sources.Source2).map((value) => {
return {source2: value};
}),
this.driver.onSpecificData(Sources.Source3).map((value) => {
return {source3: value};
})
);
}
Manager.ts
export type DataType = Partial<{value1: number, value2: number, value3: number}>;
export interface Manager {
onSpecificData(): Observable<DataType>;
}
SomeDriver.ts
export const enum Sources {
Source1,
Source2,
Source3,
}
export class SomeDriver extends Driver {
private static specificDataId = 1337; // some number
private handler: Handler;
constructor(...) {
super(...);
this.handler = new Handler(this.connection, ...);
// ...
}
// ...
onSpecificData(source: Sources): Observable<number> {
return this.handler
.listenToData<SpecificDataType>(
SomeDriver.specificDataId,
(data) => data.source === source)
).map((data) => data.value);
}
}
Driver.ts
export abstract class Driver {
protected connection: Duplex;
constructor(...) {
// init connection, etc...
}
public abstract onSpecificData(source: number);
// some implementations and more abstract stuff...
}
Handler.ts
export class Handler {
private data$: Observable<Buffer>;
constructor(private connection: Duplex, ...) {
this.data$ = Observable.fromEvent<Buffer>(connection as any, 'data');
}
listenToData<T>(dataId: number, filter?: (data: T) => boolean) {
return this.data$
.map((data) => {
// decode and transform
})
.filter((decodedData) => !decodedData.error && decodedData.value.id)
.do((decodedData) => {
console.log(`Got ${decodedData.value.id}`);
})
.map((decodedData) => decodedData.value.value as T)
.filter(filter || () => true);
}
}
And finally, subscribe()-ing:
export default function(store: Store<State>, manager: Manager) {
// ...
manager.onSpecificData()
.subscribe((data) => {
// update state according to returned data
});
}
As you can see, there is only 1 underlying Observable (data$) but apparently the operator chain in listenToData<T>() is invoked 3 times for each value emitted by it. I already know this is because of SomeManager#onSpecificData()'s merge of those 3 Observables, but I don't know why this happens. I want it to be invoked once for each value.
Help will be much appreciated.

I solved this in a "hacky" way, in my opinion. I replaced data$ with a Subject, created an observable from stream's 'data' event, moving all the shared logic to that observable and emit a value from the subject, like so:
export class Handler {
private dataSrc = new Subject<DecodedData>();
constructor(private connection: Duplex, ...) {
Observable.fromEvent<Buffer>(connection as any, 'data')
.map((data) => {
// decode and transform
})
.filter((decodedData) => !decodedData.error)
.do((decodedData) => {
console.log(`Got ${decodedData.value.id}`);
})
.subscribe((decodedData) => {
this.dataSrc.next(decodedData);
});
}
listenToData<T>(dataId: number, filter?: (data: T) => boolean) {
return this.dataSrc
.filter((decodedData) => decodedData.value.id === dataId)
.map((decodedData) => decodedData.value.value as T)
.filter(filter || () => true);
}
}
Not exactly the solution I was looking for, but it works. If anyone has a better solution, which better suits the "Rx way" to do stuff, I'd love to hear it.

Related

NestJS with Prisma Transactions

I'm trying to use a Prisma transaction in a NestJS project and I can't figure out a clean way to accomplish the following:
Have a service that will call other services and have all of them bound to a transaction. Eg:
#Injectable()
export class OrdersService {
constructor(private prismaService: PrismaService, ...) {}
async someFn() {
return await this.prismaService.$transaction(async (prismaServiceBoundToTransaction): Promise<any> => {
await this.userService.update() // This will perform an update using prismaService internally
await this.otherService.delete() // Again, it'll use prismaService
}
}
}
In this case, both user and other services will use their own prisma service and won't be bound to the Transaction.
Is there a way to accomplish this without passing the prismaServiceBoundToTx to each method?
The main problem I had when finding a suitable solution was, that the prisma client in the lambda of an interactive transaction is not a fully fledged Client, but just Prisma.TransactionClient, which is missing $on, $connect, $disconnect, $use and the $transaction methods.
If prisma would provide a full Client at this place, all you could do to solve the problem was just doing transactions like this:
**THIS DOES NOT WORK BECAUSE prismaServiceBoundToTransaction IS JUST OF TYPE Prisma.TransactionClient!!!**
return await this.prismaService.$transaction(async (prismaServiceBoundToTransaction): Promise<any> => {
const userService = new UserService(prismaServiceBoundToTransaction)
const otherService = new OtherService(prismaServiceBoundToTransaction)
//Following calls will use prismaServiceBoundToTransaction internally
await userService.update()
await otherService.delete()
}
Of course above only works, if UserService and OtherService are stateless.
So for my solution I created a new Interface that will offer all methods of Prisma.TransactionClient, but also a custom method to create a transaction.
All of the services like your UserService will only retrieve this exact interface, so they can't call $transaction but only my interactiveTransaction method!
export interface PrismaClientWithCustomTransaction
extends Readonly<Prisma.TransactionClient> {
interactiveTransaction<F>(
fn: (prisma: Prisma.TransactionClient) => Promise<F>,
options?: {
maxWait?: number | undefined;
timeout?: number | undefined;
isolationLevel?: Prisma.TransactionIsolationLevel | undefined;
}
): Promise<F>;
}
We then create a concrete class TransactionalPrismaClient that implements the mentioned interface and delivers it, by retrieving a Prisma.TransactionClient in it's constructor and forwarding all of its methods. Additionally we also just implement the interactiveTransaction method by executing the lambda method with the Prisma.TransactionClient
export class TransactionalPrismaClient<
T extends Prisma.PrismaClientOptions = Prisma.PrismaClientOptions,
U = 'log' extends keyof T
? T['log'] extends Array<Prisma.LogLevel | Prisma.LogDefinition>
? Prisma.GetEvents<T['log']>
: never
: never,
GlobalReject extends
| Prisma.RejectOnNotFound
| Prisma.RejectPerOperation
| false
| undefined = 'rejectOnNotFound' extends keyof T
? T['rejectOnNotFound']
: false
> implements PrismaClientWithCustomTransaction
{
constructor(private readonly transactionalClient: Prisma.TransactionClient) {}
$executeRaw<T = unknown>(
query: TemplateStringsArray | Prisma.Sql,
...values: any[]
): PrismaPromise<number> {
return this.transactionalClient.$executeRaw(query, ...values);
}
$executeRawUnsafe<T = unknown>(
query: string,
...values: any[]
): PrismaPromise<number> {
return this.transactionalClient.$executeRawUnsafe(query, ...values);
}
$queryRaw<T = unknown>(
query: TemplateStringsArray | Prisma.Sql,
...values: any[]
): PrismaPromise<T> {
return this.transactionalClient.$queryRaw(query, ...values);
}
$queryRawUnsafe<T = unknown>(
query: string,
...values: any[]
): PrismaPromise<T> {
return this.transactionalClient.$queryRawUnsafe(query, ...values);
}
get otherEntity(): Prisma.OtherEntityDelegate<GlobalReject> {
return this.transactionalClient.otherEntity;
}
get userEntity(): Prisma.UserEntityDelegate<GlobalReject> {
return this.transactionalClient.userEntity;
}
async interactiveTransaction<F>(
fn: (prisma: Prisma.TransactionClient) => Promise<F>,
options?: {
maxWait?: number | undefined;
timeout?: number | undefined;
isolationLevel?: Prisma.TransactionIsolationLevel | undefined;
}
): Promise<F> {
return await fn(this.transactionalClient);
}
}
And in your PrismaService we also need to implement the interactiveTransaction method, so that it satifies our defined interface PrismaClientWithCustomTransaction.
#Injectable()
export class PrismaService
extends PrismaClient
implements OnModuleInit, PrismaClientWithCustomTransaction
{
private readonly logger = new ConsoleLogger(PrismaService.name);
async onModuleInit() {
this.logger.log('Trying to connect to db.');
await this.$connect();
}
async enableShutdownHooks(app: INestApplication) {
this.$on('beforeExit', async () => {
await app.close();
});
}
interactiveTransaction<R>(
fn: (prisma: Prisma.TransactionClient) => Promise<R>,
options?: {
maxWait?: number | undefined;
timeout?: number | undefined;
isolationLevel?: Prisma.TransactionIsolationLevel | undefined;
},
numRetries = 1
): Promise<R> {
let result: Promise<R> | null = null;
for (let i = 0; i < numRetries; i++) {
try {
result = this.$transaction(fn, options);
} catch (e) {
if (e instanceof Prisma.PrismaClientKnownRequestError) {
//TODO?
} else {
throw e;
}
}
if (result != null) {
return result;
}
}
throw new Error(
'No result in transaction after maximum number of retries.'
);
}
}
Because in our services we now expect the PrismaClientWithCustomTransaction interface, the auto injecting of NestJs wont work anymore and we have to provide PrismaService using a token:
providers: [
{
provide: 'PRISMA_SERVICE_TOKEN',
useClass: PrismaService,
},
],
exportt class UserService{
constructor(#Inject('PRISMA_SERVICE_TOKEN') private readonly prisma: PrismaClientWithCustomTransaction){}
}
Alright so now we can do the following:
#Injectable()
export class OrdersService {
constructor( #Inject('PRISMA_SERVICE_TOKEN')
private readonly prisma: PrismaClientWithCustomTransaction, ...) {}
async someFn() {
return await this.prisma.interactiveTransaction(
async (client) => {
//You can still use client directly, if you dont need nested transaction logic
return client.userEntity.create(...)
//Or create services for nested usage
const transactionalClient = new TransactionalPrismaClient(client);
const userService = new UserService(transactionalClient);
return userService.createUser(...);
});
},
{ isolationLevel: Prisma.TransactionIsolationLevel.RepeatableRead }
);
}
}
And if you need the $on, $connect, $disconnect, $use, you can of course still inject the original PrismaService with its regular interface.

Emit events between different tabs

I'm writing a web app where tab1 opens tab2 and needs to reload when tab2 is closed but nothing happens when I send the next() value (in debug I saw it get executed only once when the component is initialized). I assume it has something to do with the different browser tabs
the shared service that should allow communication between the two:
private tabClosedSource = new ReplaySubject<boolean>();
tabClosedEvent = this.tabClosedSource.asObservable();
toggleTabClosed() {
this.tabClosedSource.next(true);
}
on tab1 :
constructor(private service: ExampleService) {}
ngOnInit() {
this.service.tabClosedEvent.subscribe(
event => {
if (event) {
location.reload();
}
});
// had some issues with the path starting with '#' so I ended up replacing values
openTab2() {
window.open(window.location.href.replace('tab1', 'tab2'));
}
on tab2:
constructor(private service: ExampleService) {}
async onContinueClicked() {
api requests...
procces data...
this.service.toggleTabClosed();
window.close();
}
Your requirement is achievable with the help of Broadcast Channel API. Full credit goes to another blog, I've just updated it to suit your needs and by the way I learned some new things too.
First you need to create a service to communicate.
import { Injectable, NgZone } from '#angular/core';
import { MonoTypeOperatorFunction, Observable, Subject } from 'rxjs';
import { map } from 'rxjs/operators';
// helper method to notify zone
function runInZone<T>(zone: NgZone): MonoTypeOperatorFunction<T> {
return (source) => {
return new Observable(observer => {
const onNext = (value: T) => zone.run(() => observer.next(value));
const onError = (e:any) => zone.run(()=> observer.error(e));
const onComplete = () => zone.run(()=> observer.complete());
return source.subscribe(onNext, onError, onComplete);
});
};
}
#Injectable({
providedIn: 'root'
})
export class BroadcastHelperService {
private broadcastChannel: BroadcastChannel;
private onMessage = new Subject<any>();
constructor(
private ngZone: NgZone) {
this.broadcastChannel = new BroadcastChannel('testChannel');
this.broadcastChannel.onmessage = (message: any) => { this.onMessage.next(message) }
}
publish(message: string): void {
this.broadcastChannel.postMessage(message);
}
getMessage(): Observable<any> {
return this.onMessage.pipe(
runInZone(this.ngZone),
map((message: any) => message.data));
}
}
In tab1 component write code to reload page.
export class Tab1Component implements OnInit {
worker: any;
constructor(
public broadcastHelper: BroadcastHelperService
) { }
ngOnInit(): void {
this.broadcastHelper.getMessage().subscribe((message: any) => {
console.log(message);
if(message === 'close') {
window.location.reload();
}
})
}
}
In tab2 component, write code to broadcast message when the tab is closed.
export class Tab2Component {
constructor(
public broadcastHelper: BroadcastHelperService
) { }
#HostListener('window:beforeunload', ['$event'])
beforeUnloadHander(event: any) {
this.broadcastHelper.publish('close');
}
}

How to use AsyncLocalStorage for an Observable?

I'd like to use use AsyncLocalStorage in a NestJs Interceptor:
export interface CallHandler<T = any> {
handle(): Observable<T>;
}
export interface NestInterceptor<T = any, R = any> {
intercept(context: ExecutionContext, next: CallHandler<T>): Observable<R> | Promise<Observable<R>>;
}
The interceptor function gets a next CallHandler that returns an Observable.
I cannot use run in this case (the run callback will exit immediately before the callHandler.handle() observable has finished):
intercept(context: ExecutionContext, callHandler: CallHandler): Observable<any> | Promise<Observable<any>> {
const asyncLocalStorage = new AsyncLocalStorage();
const myStore = { some: 'data'};
return asyncLocalStorage.run(myStore, () => callHandler.handle());
}
See broken replit-example
The solution I came up with is this:
const localStorage = new AsyncLocalStorage();
export class MyInterceptor implements NestInterceptor {
intercept(context: ExecutionContext, callHandler: CallHandler): Observable<any> | Promise<Observable<any>> {
const resource = new AsyncResource('AsyncLocalStorage', { requireManualDestroy: true });
const myStore = { some: 'data' };
localStorage.enterWith(myStore);
return callHandler.handle().pipe(
finalize(() => resource.emitDestroy())
);
}
}
See working replit example
This seems to work fine, but I am not sure if this is really correct - and it looks messy and error-prone. So I wonder:
Is this correct at all?
Is there a better/cleaner way to handle this?
Below is the solution I came up with. My understanding of the problem is that you need the run function to receive a callback function that will fully encapsulate the execution of the handler, however, the intercept function is expected to return an observable that has not yet been triggered. This means that if you encapsulate the observable itself in the run callback function, it will not have been triggered yet.
My solution, below, is to return a new observable that, when triggered, will be responsible for triggering (i.e. subscribing to) the call handler itself. As a result, the promise we create in the run call can fully encapsulate the handle function and it's async callbacks.
Here is the general functionality in a stand-alone function so that you can see it all together:
intercept(context: ExecutionContext, next: CallHandler<any>): Observable<any> {
return new Observable((subscribe) => {
asyncStorage.run({}, () => new Promise(resolve => {
next.handle().subscribe(
result => {
subscribe.next(result);
subscribe.complete();
resolve();
},
error => {
subscribe.error(err);
resolve();
}
);
}));
});
}
Next, I took that concept and integrated it into my interceptor below.
export class RequestContextInterceptor implements NestInterceptor {
constructor(
private readonly requestContext: RequestContext,
private readonly localStorage: AsyncLocalStorage<RequestContextData>
) {}
intercept(context: ExecutionContext, next: CallHandler<any>): Observable<any> {
const contextData = this.requestContext.buildContextData(context);
return new Observable((subscribe) => {
void this.localStorage.run(contextData, () => this.runHandler(next, subscribe));
});
}
private runHandler(next: CallHandler<any>, subscribe: Subscriber<any>): Promise<void> {
return new Promise<void>((resolve) => {
next.handle().subscribe(
(result) => {
subscribe.next(result);
subscribe.complete();
resolve();
},
(err) => {
subscribe.error(err);
resolve();
}
);
});
}
}
It's worth noting that the Promise that is created during the run call does not have a rejection path. This is intentional. The error is passed on to the observable that is wrapping the promise. This means that the outer observable will still succeed or error depending upon what the inner observable does, however, the promise that wraps the inner observable will always resolve regardless.
Here is a solution for cls-hooks:
return new Observable(observer => {
namespace.runAndReturn(async () => {
namespace.set("some", "data")
next.handle()
.subscribe(
res => observer.next(res),
error => observer.error(error),
() => observer.complete()
)
})
})
here is our current solution to the problem:
we create an observable that will simply pass all emissions to the callHandler
the important part is, that we subscribe inside the localStorage.run method
const localStorage = new AsyncLocalStorage();
export class MyInterceptor implements NestInterceptor {
intercept(context: ExecutionContext, callHandler: CallHandler): Observable<any> | Promise<Observable<any>> {
const myStore = { some: 'data' };
return new Observable((subscriber) => {
const subscription = localStorage.run(myStore, () => {
/**
* - run the handler function in the run callback, so that myStore is set
* - subscribe to the handler and pass all emissions of the callHandler to our subscriber
*/
return callHandler.handle().subscribe(subscriber);
});
/**
* return an unsubscribe method
*/
return () => subscription.unsubscribe();
});
}
}

How to pass a child Interface to a parent class?

I have this:
LocationController.ts
import {GenericController} from './_genericController';
interface Response {
id : number,
code: string,
name: string,
type: string,
long: number,
lat: number
}
const fields = ['code','name','type','long','lat'];
class LocationController extends GenericController{
tableName:string = 'location';
fields:Array<any> = fields;
}
const locationController = new LocationController();
const get = async (req, res) => {
await locationController._get(req, res);
}
export {get};
GenericController.ts
interface Response {
id : number
}
export class GenericController{
tableName:string = '';
fields:Array<any> = [];
_get = async (req, res) => {
try{
const id = req.body['id'];
const send = async () => {
const resp : Array<Response> = await db(this.tableName).select(this.fields).where('id', id)
if (resp[0] === undefined) {
// some error handling
}
res.status(status.success).json(resp[0]);
}
await send();
}catch (error){
// some error handling
}
}
}
What I want to do is to pass the Response interface from LocationController to the GenericController parent, so that the response is typed accurately depending on how the child class has defined it. Clearly it doesn't work like this since the interface is defined outside of the class so the parent has no idea about the Response interface in the LocationController.ts file.
I've tried passing interface as an argument in the constructor, that doesn't work. So is there a way I can make this happen? I feel like I'm missing something really simple.
Typically, generics are used in a situation like this. Here's how I'd do it:
interface Response {
id: number;
}
// Note the generic parameter <R extends Response>
export class GenericController<R extends Response> {
tableName: string = "";
fields: Array<any> = [];
_get = async (req, res) => {
try {
const id = req.body["id"];
const send = async () => {
// The array is now properly typed. You don't know the exact type,
// but you do know the constraint - R is some type of `Response`
let resp: Array<R> = await db(this.tableName).select(this.fields).where("id", id);
if (resp[0] === undefined) {
// some error handling
}
res.status(status.success).json(resp[0]);
};
await send();
} catch (error) {
// some error handling
}
};
}
import { GenericController } from "./_genericController";
interface Response {
id: number;
code: string;
name: string;
type: string;
long: number;
lat: number;
}
const fields = ["code", "name", "type", "long", "lat"];
// Here we tell the GenericController exactly what type of Response it's going to get
class LocationController extends GenericController<Response> {
tableName: string = "location";
fields: Array<any> = fields;
}
const locationController = new LocationController();
const get = async (req, res) => {
await locationController._get(req, res);
};
export { get };
If this is not enough and you wish to somehow know the exact response type you're going to get, I believe the only way is a manual check. For example:
import { LocationResponse } from './locationController';
// ... stuff
// Manual runtime type check
if (this.tableName === 'location') {
// Manual cast
resp = resp as Array<LocationResponse>
}
// ...
You could also check the form of resp[0] (if (resp[0].hasOwnProperty('code')) { ... }) and cast accordingly. There are also nicer ways to write this, but the basic idea remains the same.
Generally, a properly written class should be unaware of any classes that inherit from it. Putting child-class-specific logic into your generic controller is a code smell. Though as always, it all depends on a particular situation.

How to access class metadata from method decorator

I'm having two decorators. A class decorator and a method decorator.
The class decorator defines metadata which I want to access in the method decorator.
ClassDecorator:
function ClassDecorator(topic?: string): ClassDecorator {
return (target) => {
Reflect.defineMetadata('topic', topic, target);
// I've also tried target.prototype instead of target
return target;
};
}
MethodDecorator:
interface methodDecoratorOptions {
cmd: string
}
function MethodDecorator(options: decoratorOptions) {
return function (target, propertyKey: string, descriptor: PropertyDescriptor) {
// HERE IS MY PROBLEM
console.log('metaData is: ', Reflect.getMetadata('topic', target));
}
}
And this is my Class definition:
#ClassDecorator('auth')
export class LoginClass {
#MethodDecorator({
cmd: 'login'
})
myMethod() {
console.log('METHOD CALLED');
}
}
THE PROBLEM:
The following line of the MethodDecorator returns metaData is: undefined. Why is it undefined?
console.log('metaData is: ', Reflect.getMetadata('topic', target));
THE QUESTION:
How can I access the metadata defined by the ClassDecorator from the MethodDecorator?
The problem is the order in which decorators get executed. Method decorators are executed first, class decorators are executed after. This makes sense if you think about it, the class decorators need the complete class to act upon, and creating the class involves creating the methods and calling their decorators first.
A simple workaround would be for the method decorator to register a callback that would then be called by the class decorator after the topic was set:
function ClassDecorator(topic?: string): ClassDecorator {
return (target) => {
Reflect.defineMetadata('topic', topic, target.prototype);
let topicFns: Array<() => void> = Reflect.getMetadata("topicCallbacks", target.prototype);
if (topicFns) {
topicFns.forEach(fn => fn());
}
return target;
};
}
interface methodDecoratorOptions {
cmd: string
}
function MethodDecorator(options: methodDecoratorOptions) {
return function (target: any, propertyKey: string, descriptor: PropertyDescriptor) {
let topicFns: Array<() => void> = Reflect.getMetadata("topicCallbacks", target);
if (!topicFns) {
Reflect.defineMetadata("topicCallbacks", topicFns = [], target);
}
topicFns.push(() => {
console.log('metaData is: ', Reflect.getMetadata('topic', target));
});
}
}
#ClassDecorator('auth')
class LoginClass {
#MethodDecorator({
cmd: 'login'
})
myMethod() {
console.log('METHOD CALLED');
}
}

Resources