ChainLink general API get request Example not working - get

I'm developing a project with using a smartcontract that have to make an API call.
I'm tring to follow the ChainLink guide at this link. I'd like to call the requestRiskData() from another function an than use the output (risk variable updated) to do some stuff. But since it takes like 30 seconds to update the risk value after the oracle call how i can wait to the risk variable to be updated?
Can someone send help?
Thx
I will attach the code here:
//SPDX-License-Identifier: UNLICENSED
pragma solidity ^0.8.0;
import "#chainlink/contracts/src/v0.8/ChainlinkClient.sol";
import "#chainlink/contracts/src/v0.8/ConfirmedOwner.sol";
contract APIConsumer is ChainlinkClient, ConfirmedOwner {
using Chainlink for Chainlink.Request;
uint256 public risk;
bytes32 private jobId;
uint256 private fee;
string public a= "0x6e1db9836521977ee93651027768f7e0d5722a33";
event RequestRisk(bytes32 indexed requestId, uint256 risk);
/**
* #notice Initialize the link token and target oracle
*
* Goerli Testnet details:
* Link Token: 0x326C977E6efc84E512bB9C30f76E30c160eD06FB
* Oracle: 0xCC79157eb46F5624204f47AB42b3906cAA40eaB7 (Chainlink DevRel)
* jobId: ca98366cc7314957b8c012c72f05aeeb
*
*/
constructor() ConfirmedOwner(msg.sender) {
setChainlinkToken(0x326C977E6efc84E512bB9C30f76E30c160eD06FB);
setChainlinkOracle(0xCC79157eb46F5624204f47AB42b3906cAA40eaB7);
jobId = "ca98366cc7314957b8c012c72f05aeeb";
fee = (1 * LINK_DIVISIBILITY) / 10; // 0,1 * 10**18 (Varies by network and job)
}
// Set the path to find the desired data in the API response, where the response format is:
// {
//"data":{
// "0x6e1db9836521977ee93651027768f7e0d5722a33":{
// "risk":{
// "score":....
// }
// }
// }
// }
function requestRiskData() public returns (bytes32 requestId) {
Chainlink.Request memory req = buildChainlinkRequest(jobId, address(this), this.fulfill.selector);
req.add("get","https://demo.anchainai.com/api/address_risk_score?proto=eth&address=0x6e1db9836521977ee93651027768f7e0d5722a33&apikey=demo_api_key");
req.add("path", "data,0x6e1db9836521977ee93651027768f7e0d5722a33,risk,score");
return sendChainlinkRequest(req, fee);
}
function fulfill(
bytes32 _requestId,
uint256 _risk
) public recordChainlinkFulfillment(_requestId) {
emit RequestRisk(_requestId, _risk);
risk = _risk;
}
function withdrawLink() public onlyOwner {
LinkTokenInterface link = LinkTokenInterface(chainlinkTokenAddress());
require(
link.transfer(msg.sender, link.balanceOf(address(this))),
"Unable to transfer"
);
}
function myStuff() public payable{
requestRiskData();
if(risk>2){
//DO MY STUFF
}
}
}
I tried to write the myStuff() function but i don't know how to handle this.

Can you please clarify what you mean by "I'd like to call the requestRiskData() from another function an than use the output (risk variable updated) to do some stuff"
When you say "use the output to do some stuff" do you mean outside the contract (front end) or from inside the smart contract?
If you want to use the value of _risk returned by the oracle, you should just add that logic (or call a helper function that implements that logic) after the value of _risk is set in your fulfill() callback.
so in your case, the following lines should be moved into the end of the fulfill() callback:
if(risk>2){
//DO MY STUFF
}
If you're using the value of _risk from the front end, then use ethersjs or web3js to listen for the RequestRisk event in your front end.
On a side note, you may want to consider renaming that event as its not in the request function but in the callback, so its really should be a RiskReceived or event of that sort of name.
hope this helps!

Related

Suggestion to Build a multiplayer Texas Holdem Poker game using NodeJS, Flutter

I have been building basic and complex mobile apps in Android and Flutter also have knowledge of NodeJS. I have already built a basic NodeJs multiplayer server for Texas Holdem Poker. Mulitple rooms and table logic are still remaining.
I want to develop a client mobile app in Flutter as I have deep knowledge of flutter.
I have been exploring the packages and convenient tools that are going to be used in Flutter but am still clueless.
This game development in Flutter is a new challenge for me so I would like to ask whether the technology stack for such a game is good or not?
Shall I consider switching from Flutter to something else?
I'd say go for it. I created a card game myself while learning Flutter - just needed a fun project to see it end to end.
Similarly to you, I used node.js in the backend, enabling me to play the game with my friends.
I ended up going with GraphQL for my back end services (using AWS AppSync service). Since GraphQL will give you push notification over websockets, it was ideal to send data between different clients playing the game. Node.js deployed on AWS Lambda with Dynamo DB for persistence worked without a problem.
I also ended up doing an AI algorithm (using Monte Carlo Tree Search) that enabled me to play the game against the AI. And as the final bit - I ported my back end node.js interface to dart, so I could play the game against the AI fully off-line.
I think I spent the most time figuring out how to do the card animation, especially since I wanted for the card moves to happen sequentially. AI would sometimes play the moves too fast, so there were multiple cards flying around at the same time. Once this was done - the rest of it was easy.
If you need more details, let me know. I'm happy to share bits of the code with you.
Edit: here are some code details. I'll try to edit out parts of code that you won't need, so probably some of it will fail to compile at first... And keep in mind - some things may be overcomplicated in my code - as I got the things up and running I would move on, so there is a lot to improve here...
First the back-end. As I mentioned, I used AWS: Dynamo DB to store the game data; a single lambda function to do all the work; and AppSync API: simply because it offers message push to the clients - you don't need to keep polling the back-end to see if there are any changes. And finally - I used Cognito user pool to authenticate users. In my app you can create your account, validate it through email etc.
I used AWS Amplify to setup a back-end: this is AWS framework that enables you to easily deploy your back-end even if you don't know that much about AWS. It is actually meant for app developers that don't need or want to learn about AWS in detail. Even if you know your way around AWS - it is still very useful, since it will automate a lot of security wiring for you, and will help you automate the deployment.
Currently, there is official amplify-flutter package; at the time I did this I used 3rd party package https://pub.dev/packages/amazon_cognito_identity_dart_2.
Now I'm assuming that you got your back-end setup: you deployed your node.js code in Lambda, and now you need your AppSync schema. Here's mine:
type Match #aws_iam #aws_cognito_user_pools {
matchId: Int
matchData: AWSJSON!
}
type PlayEvent #aws_iam #aws_cognito_user_pools {
matchId: Int!
success: Boolean!
playerId: Int!
eventType: String!
inputData: AWSJSON
eventData: AWSJSON
matchData: AWSJSON
}
input PlayEventInput {
matchId: Int!
eventType: String!
eventData: AWSJSON
playerId: Int
}
type Query {
getMatch(matchId: Int): Match #function(name: "tresetaFv2-${env}") #aws_iam #aws_cognito_user_pools
}
type Mutation {
playEvent(input: PlayEventInput!): PlayEvent #function(name: "tresetaFv2-${env}") #aws_iam #aws_cognito_user_pools
}
type Subscription {
onPlayEvent(matchId: Int, success: Boolean): PlayEvent #aws_subscribe(mutations: ["playEvent"])
}
As you can see, I'm using AWSJSON data type a lot - instead of coding my entire game schema in GraphQL, I'm simply passing JSON back and forth.
Few things to understand:
MatchData type holds your game state: what card each player holds, what are the cards in the deck, on the table, who's turn is it to play etc.
Query getMatch will be fired by the client app after selecting the match from the active matches list. When you join the game, this is how you fetch the game state.
Mutation playEvent is how you play your move: you pass PlayEventInput type - specifying matchID and playerID, event Type (in your case you can play card, fold, call...). The playEvent Mutation will return PlayEvent - telling you if the move was successful (was it legal at the time? Was your turn to play?), and it will return MatchData - the new game state after the move was played.
And finally - Subscription onPlayEvent. After each client joins the match (by match ID), it will subscribe to this subscription. As you can see from the subscription definition - it will subscribe to playEvent mutation: each time a player plays a move, it will notify all the others about the move - and pass on the result of the Mutation, so everyone will get the full game state to refresh their UI. A nice trick here is - you subscribe to mutations that have success=true, so you don't push any message for failed moves.
You will also see some annotations - this is how you tell Amplify to wire things for you:
#function(name: "tresetaFv2-${env}"): you tell it to call Lambda funciton called "tresetaFv2-${env}" to actuall do the work
#aws_iam #aws_cognito_user_pools - this is how you tell it that this API is scured by Cognito user pools.
So now we have the AWS back-end setup. So how do you actually run the query and subscribe to the events coming from the back-end?
First, I use this UserService in my code: https://github.com/furaiev/amazon-cognito-identity-dart-2/blob/main/example/lib/user_service.dart
This is my AppSync Service - this is just a generic way of calling AppSync:
import 'dart:convert';
import 'package:amazon_cognito_identity_dart_2/cognito.dart';
import 'package:flutter/foundation.dart';
import 'package:http/http.dart' as http;
import 'package:treseta_app/auth/auth_services.dart';
import 'package:web_socket_channel/web_socket_channel.dart';
import 'package:web_socket_channel/io.dart';
import 'package:web_socket_channel/web_socket_channel.dart';
class AppSyncImpl {
final String _endPoint;
String get _wsEndPoint => _endPoint.replaceAll('https', 'wss').replaceAll('appsync-api', 'appsync-realtime-api');
String get _host => _endPoint.replaceAll('https://', '').replaceAll('/graphql', '');
UserService _userService;
set userService(UserService userService) => this._userService = userService;
AppSyncImpl(this._endPoint, this._userService);
Future<http.Response> getHttpResponse(Map<String, String> query) async {
CognitoUserSession session = await _userService.getSession();
return http.post(
_endPoint,
headers: {
'Authorization': session.getAccessToken().getJwtToken(),
'Content-Type': 'application/json',
},
body: json.encode(query),
);
}
Future<Map<String, dynamic>> glQueryMutation(Map<String, String> query) async {
http.Response response;
try {
response = await getHttpResponse(query);
} catch (e) {
print(e);
}
return json.decode(response.body);
}
int wsTimeoutInterval;
void disconnectWebSocketChannel(WebSocketChannel wsChannel, String uniqueKey) {
wsChannel.sink.add(json.encode({'type': 'stop', 'id': uniqueKey}));
}
Future<WebSocketChannel> wsSubscription(
{#required Map<String, String> query,
#required String uniqueKey,
Function(Map<String, dynamic>) listener}) async {
var jwtToken = (await _userService.getSession()).getIdToken().getJwtToken();
var header = {'host': _host, 'Authorization': jwtToken};
var encodedHeader = Base64Codec().encode(Utf8Codec().encode(json.encode(header)));
// Note 'e30=' is '{}' in base64
var wssUrl = '$_wsEndPoint?header=$encodedHeader&payload=e30=';
var channel = IOWebSocketChannel.connect(wssUrl, protocols: ['graphql-ws']);
channel.sink.add(json.encode({"type": "connection_init"}));
channel.stream.listen((event) {
var e = json.decode(event);
switch (e['type']) {
case 'connection_ack':
wsTimeoutInterval = e['payload']['connectionTimeoutMs'];
var register = {
'id': uniqueKey,
'payload': {
'data': json.encode(query),
'extensions': {'authorization': header}
},
'type': 'start'
};
var payload = json.encode(register);
channel.sink.add(payload);
break;
case 'data':
listener(e);
break;
case 'ka':
print('Reminder: keep alive is not yet implemented!!!');
break;
case 'start_ack':
print('Ws Channel: Subscription started');
break;
default:
print('Unknown event $event');
}
}, onError: (error, StackTrace stackTrace) {
print('Ws Channel: $error');
}, onDone: () {
channel.sink.close();
print('Ws Channel: Done!');
});
return channel;
}
}
And this is the actual back-end service; notice how there is a function that corresponds to each query, mutation and subscription in GraphQL Schema (getMatchData, playEvent, subscribeWS):
import 'dart:async';
import 'dart:convert';
import 'package:flutter/foundation.dart';
import 'package:flutter/widgets.dart';
import 'package:treseta_app/appsync/app_sync.dart';
import 'package:treseta_app/auth/auth_services.dart';
import 'package:treseta_app/models/api_models.dart' as api;
import 'package:meta/meta.dart';
import 'package:treseta_app/models/game_model.dart';
import 'package:web_socket_channel/web_socket_channel.dart';
class AwsBackendService extends ChangeNotifier {
final subscriptionStreamCtrl = StreamController<api.PlayEvent>.broadcast();
Stream get subscriptionStream => subscriptionStreamCtrl.stream;
UserService userService;
String endPoint;
AppSyncImpl _appSyncImpl;
BuildContext context;
AwsBackendService({#required this.context, this.userService, this.endPoint})
: _appSyncImpl = AppSyncImpl(endPoint, userService);
#override
Future<api.Match> getMatchData(int matchId) async {
final query = {
'operationName': 'GetMatch',
'query': '''query GetMatch { getMatch(matchId:$matchId){matchData, playerId}}'''
};
User user = await userService.getCurrentUser();
String username = user.name;
var matchData = await _appSyncImpl.glQueryMutation(query);
var match = api.Match.fromJson(matchData['data']['getMatch']);
match.matchData.username = username;
return match;
}
Future<api.PlayEvent> playEvent(int matchId, int playerId, String eventType, api.PlayEventInputData eventData) async {
var encoded = json.encode(eventData.toJson()).replaceAll("\"", "\\\"");
final query = {
'operationName': 'PlayEvent',
'query':
'''mutation PlayEvent { playEvent(input:{matchId:$matchId, eventType:"$eventType", eventData:"$encoded"}){matchId, eventData, success, playerId, eventType, inputData, matchData}}''',
};
api.PlayEvent result =
await _appSyncImpl.glQueryMutation(query).then((value) => api.PlayEvent.fromJson(value['data']['playEvent']));
User user = await userService.getCurrentUser();
String username = user.name;
result.matchData.username = username;
return result;
}
WebSocketChannel _wsClient;
String _wsUniqeKey;
Future<void> subscribeWs(int matchId, int playerId, MatchData model) async {
final query = {
'operationName': 'OnPlayEvent',
'query':
'subscription OnPlayEvent { onPlayEvent(matchId:$matchId, success:true) {matchId, success,playerId,eventType,inputData,eventData,matchData}}',
};
_wsUniqeKey = UniqueKey().toString();
_wsClient = await _appSyncImpl.wsSubscription(
query: query,
uniqueKey: _wsUniqeKey,
listener: (Map<String, dynamic> message) {
var playEvent = api.PlayEvent.fromJson(message['payload']['data']['onPlayEvent']);
subscriptionStreamCtrl.sink.add(playEvent);
});
return;
}
void disconnect() {
try {
if (_wsClient != null) {
_appSyncImpl.disconnectWebSocketChannel(_wsClient, _wsUniqeKey);
}
} catch (e) {
print(e);
}
}
#override
void dispose() {
super.dispose();
subscriptionStreamCtrl.close();
disconnect();
}
}
You will see that each event we receive from AppSync subscription is just pumped into a Stream object.
And finally, my main ChangeNotifier Provider that holds the match data and notifies the UI is something like this - you will see how the incoming subscription events are processed, and UI is notified that there's a new event to be animated - a card being thrown, or new hand dealt etc.
import 'dart:async';
import 'package:flutter/widgets.dart';
import 'package:treseta_app/backend/backend_service.dart';
import 'package:treseta_app/models/api_models.dart';
import 'package:treseta_app/models/game_model.dart';
class Treseta extends ChangeNotifier {
MatchData model;
int playerId;
AwsBackendService _backendService;
StreamSubscription backendServiceSubscription;
AwsBackendService get backendService => _backendService;
set backendService(BackendService value) {
if (backendServiceSubscription != null) backendServiceSubscription.cancel();
_backendService = value;
// this is where we process the incoming subscription messages:
backendServiceSubscription = _backendService.subscriptionStream.listen((inEvent) {
PlayEvent playEvent = inEvent as PlayEvent;
playCard(
inData: playEvent,
playerId: playEvent.playerId,
card: playEvent.inputData.card);
});
}
int _matchId;
int get matchId => _matchId;
set matchId(int value) {
_matchId = value;
subscribeAndGetMatchData();
}
Future<void> disconnect() async => backendService.disconnect();
Future<MatchData> getMatchData() async {
var match = await backendService.getMatchData(matchId);
playerId = match.playerId;
model = match.matchData;
return model;
}
Future<void> subscribeAndGetMatchData() async {
if (matchId != null) {
disconnect();
await getMatchData();
await subscribe();
}
return;
}
Future<void> subscribe() async {
backendService.subscribeWs(matchId, playerId, this.model);
}
void playCard(
{GameCard card,
GlobalKey sourceKey,
GlobalKey targetKey,
int playerId = 0,
PlayEvent inData}) async {
// Animation notifier logic goes here...
}
#override
void dispose() {
disconnect();
backendServiceSubscription.cancel();
super.dispose();
}
}
There you go - I hope this helps.
There is another bit on how to animate the cards - the big challenge is to run your animation sequentially - even if the players are quick enough to play cards very quickly. If you want, I can post some details around it as well.
*** One more edit ****
I created a repo with a simple animation example. If I find time, I'll write some comments in Readme file...
https://github.com/andrija78/cards_demo

What are Backendless's rules for returning database objects from API Services

I set up a simple Backendless API Service and am running it through CodeRunner. As a test, I'm simply getting a record from the database and returning it. I've tried every combination of return type definition in the class annotations that I can think of, and I've assured that the correct record exists and is being returned to the service, but I've never successfully had the record returned using the console, or via a SDK invocation. In every case, the body returned to the invocation is null. My current test uses "Object" as the return type for the getSchedule call - are database objects not objects?
Here is the entire service:
'use strict';
const { DateTime } = require("luxon");
const util = require("util");
class Scheduling {
/**
*
* #param {String} day
* #returns {Object}
*/
getSchedule( day ) {
let t = DateTime.fromISO(day).toMillis();
let q = Backendless.DataQueryBuilder.create().setWhereClause(`day = ${t}`);
Backendless.Data.of("schedules").find(q)
.then(rec => {
console.log(util.inspect(rec,3))
if (rec.length === 1) {
return rec[0]
}
else {
return {error: 404, msg: 'not found'}
}
})
}
}
Backendless.ServerCode.addService( Scheduling )
The "inspect" call indicates I am retrieving the correct record. No errors, the return status of the invocation is always 200. Obviously, I'm missing something about API service return types, please point me in the correct direction.
The problem is the response for the find method is returned after the invocation of getSchedule is complete (because the API invocation is asynchronous).
How about declaring the getSchedule with async and then await for the API invocation?
'use strict';
const { DateTime } = require("luxon");
const util = require("util");
class Scheduling {
/**
*
* #param {String} day
* #returns {Object}
*/
async getSchedule( day ) {
let t = DateTime.fromISO(day).toMillis();
let q = Backendless.DataQueryBuilder.create().setWhereClause(`day = ${t}`);
var rec = await Backendless.Data.of("schedules").find(q);
console.log(util.inspect(rec,3))
if (rec.length === 1) {
return rec[0]
}
else {
return {error: 404, msg: 'not found'}
}
}
}
Backendless.ServerCode.addService( Scheduling )

How exactly does the mergeMap operator work and in which cases is it used?

Before coming here I have read the official documentation of Rxjs and some other pages but I am still not clear. What I understood is this:
It is used to "join" 2 observables and thus obtain a single observable as a result, I also saw that it is used to "flatten" an observable (I am also not very clear).
Now ... I have days trying to program a user registry using Angular and Node.js with Express and I found a little tutorial which I decided to use and it has this code:
import { Injectable, Injector } from '#angular/core';
import { HttpClient, HttpInterceptor, HttpRequest, HttpHandler, HttpEvent, HttpErrorResponse } from '#angular/common/http';
import { Observable, throwError } from 'rxjs';
import { catchError, retry, mergeMap } from 'rxjs/operators'
import { AuthenticationService } from './authentication.service';
#Injectable({
providedIn: 'root'
})
export class AppInterceptor implements HttpInterceptor {
constructor(private injector: Injector) { }
intercept(req: HttpRequest<any>, next: HttpHandler): Observable<HttpEvent<any>> {
let accessToken = "", refreshToken = ""
const tokens = JSON.parse(sessionStorage.getItem("tokens"))
if (tokens) {
accessToken = tokens.accessToken
refreshToken = tokens.refreshToken
}
let clonHttp: HttpRequest<any>
clonHttp = tokens ? req.clone({ headers: req.headers.append("Authorization", `Bearer ${accessToken}`) }) : req
let auth = this.injector.get(AuthenticationService);
return next.handle(clonHttp)
.pipe(
catchError((error: HttpErrorResponse) => {
if (error.error instanceof ErrorEvent) {
console.log("error event")
} else if (error.status == 401) {
return auth.getNewAccessToken(refreshToken)
.pipe(
retry(3),
mergeMap(
(response: any) => {
tokens.accessToken = response.accessToken
sessionStorage.setItem("tokens", JSON.stringify(tokens))
clonHttp = req.clone({ headers: req.headers.append("Authorization", `Bearer ${response.accessToken}`) })
return next.handle(clonHttp)
}
)
)
} else if (error.status == 409) {
return throwError("User not logged")
} else {
if (error.error && error.error.message) {
return throwError(error.error.message)
} else {
return throwError("Check your connection")
}
}
})
)
}
}
If you see, when you use the MergeMap operator they only pass you the answer (a single observable), or at least that's what I can see. What I'm trying to say is that I don't see that they are using it with 2 observables or to mix 2 observables, which is what I have read in their official documentation, in fact, in the examples they show they always use it with 2 observables.
Honestly it has been too difficult for me to understand this operator, if someone could help me understand it in a simple way, I would be extremely grateful, in addition to understanding its use in that code that I show earlier. Greetings in advance. Thank you!
mergeMap, like many other so-called higher order mapping operators, maintains one or multiple inner observables.
An inner observable is created with the outer value and the provided function. The outer value essentially is just the value received from its source. For example:
of(1, 2, 3).pipe(
mergeMap((outerValue, index) => /* ... return an observable ... */)
).subscribe(); // `outerValue`: 1, 2, 3 (separately)
When an outer value comes in, a new inner observable will be created. I think the best way to understand this is to have a look at the source code:
// `value` - the `outerValue`
protected _next(value: T): void {
if (this.active < this.concurrent) {
this._tryNext(value);
} else {
this.buffer.push(value);
}
}
protected _tryNext(value: T) {
let result: ObservableInput<R>;
const index = this.index++;
try {
// Create the inner observable based on the `outerValue` and the provided function (`this.project`)
// `mergeMap(project)`
result = this.project(value, index);
} catch (err) {
this.destination.error(err);
return;
}
this.active++;
// Subscribe to the inner observable
this._innerSub(result, value, index);
}
Please disregard for now concurrent and buffer, we'll have a look at them a bit later.
Now, what happens when an inner observable emits ? Before going any further, it's worth mentioning that, although it's obvious, an inner observable requires an inner subscriber. We can see this in the _innerSub method from above:
private _innerSub(ish: ObservableInput<R>, value: T, index: number): void {
const innerSubscriber = new InnerSubscriber(this, value, index);
const destination = this.destination as Subscription;
destination.add(innerSubscriber);
// This is where the subscription takes place
subscribeToResult<T, R>(this, ish, undefined, undefined, innerSubscriber);
}
When an inner observable emits, the notifyNext method will be called:
notifyNext(outerValue: T, innerValue: R,
outerIndex: number, innerIndex: number,
innerSub: InnerSubscriber<T, R>): void {
this.destination.next(innerValue);
}
Where destination points to the next subscriber in the chain. For example, it can be this:
of(1)
.pipe(
mergeMap(/* ... */)
)
.subscribe({} /* <- this is the `destination` for `mergeMap` */)
This will be explained in more detail in What about the next subscriber in the chain below.
So, what does it mean to to mix 2 observables ?
Let's see this example:
of(2, 3, 1)
.pipe(
mergeMap(outerValue => timer(outerValue).pipe(mapTo(outerValue)))
)
.subscribe(console.log)
/* 1 \n 2 \n 3 */
When 2 arrives, mergeMap will subscribe to an inner observable that will emit in 200ms. This is an asynchronous action, but notice that the outer values(2, 3, 1) arrive synchronously. Next, 3 arrives and will create an inner obs. that will emit in 300ms. Since the current script has not finished executing yet, the callback queue is not yet considered. Now 1 arrives, and will create an inner obs. that will emit in 100 ms.
mergeMap has now 3 inner observables and will pass along the inner value of whichever inner observable emits.
As expected, we get 1, 2, 3.
So that's what mergeMap does. Mixing observables can be thought of this way: if an outer value comes and an inner observable has already been created, then mergeMap simply says: "no problem, I'll just create a new inner obs. and subscribe to it".
What about concurrent and buffer
mergeMap can be given a second argument, concurrent which indicates how many inner observables should handle at the same time. These number of active inner observables is tracked with the active property.
As seen in _next method, if active >= concurrent, the outerValues will be added to a buffer, which is a queue(FIFO).
Then, when one active inner observable completes, mergeMap will take the oldest value from the value and will create an inner observable out of it, using the provided function:
// Called when an inner observable completes
notifyComplete(innerSub: Subscription): void {
const buffer = this.buffer;
this.remove(innerSub);
this.active--;
if (buffer.length > 0) {
this._next(buffer.shift()!); // Create a new inner obs. with the oldest buffered value
} else if (this.active === 0 && this.hasCompleted) {
this.destination.complete();
}
}
With this in mind, concatMap(project) is just mergeMap(project, 1).
So, if you have:
of(2, 3, 1)
.pipe(
mergeMap(outerValue => timer(outerValue * 100).pipe(mapTo(outerValue)), 1)
)
.subscribe(console.log)
this will be logged:
2 \n 3 \n 1.
What about the next subscriber in the chain
Operators are functions that return another function which accepts an observable as their only parameter and return another observable.
When a stream is being subscribed to, each observable returned by an operator will have its own subscriber.
All these subscribers can be seen as a linked list. For example:
// S{n} -> Subscriber `n`, where `n` depends on the order in which the subscribers are created
of(/* ... */)
.pipe(
operatorA(), // S{4}
operatorB(), // S{3}
operatorC(), // S{2}
).subscribe({ /* ... */ }) // S{1}; the observer is converted into a `Subscriber`
S{n} is the parent(destination) of S{n+1}, meaning that S{1} is the destination of S{2}, S{2} is the destination of S{3} and so forth.
StackBlitz
Unexpected results
Compare these:
of(2, 1, 0)
.pipe(
mergeMap(v => timer(v * 100).pipe(mapTo(v)))
).subscribe(console.log)
// 0 1 2
of(2, 1, 0)
.pipe(
mergeMap(v => timer(v).pipe(mapTo(v)))
).subscribe(console.log)
// 1 0 2
As per MDN:
The specified amount of time (or the delay) is not the guaranteed time to execution, but rather the minimum time to execution. The callbacks you pass to these functions cannot run until the stack on the main thread is empty.
As a consequence, code like setTimeout(fn, 0) will execute as soon as the stack is empty, not immediately. If you execute code like setTimeout(fn, 0) but then immediately after run a loop that counts from 1 to 10 billion, your callback will be executed after a few seconds.
This section by MDN should clarify things as well.
I'd say this is environment-specific, rather than RxJs-specific.
In the second snippet, the delays are consecutive so that's why you're getting unexpected results. If you increase the delays just a bit, like: timer(v * 2), you should get the expected behavior.
So merge map is mainly used to resolve multiple inner observables concurrently and when all inner observables are resolved outer observable will resolve. I hope this helps.
Imagine you have to to read a list of ids from some async source, being it a remote service, a DB, a file on your file system.
Imagine that you have to launch an async query for each id to get the details.
Imagine you have to collect all the details for each id and do something else.
You end up having an initial Obsersable emitting a list and then a bunch of of Observables generated by that list. This is were you would use mergeMap.
The code would look like this
mySourceObs = getIdListFromSomewhere();
myStream = mySourceObs.pipe(
// after you get the list of the ids from your service, you generate a new strem
// which emits all the values of the list via from operator
concatMap(listOfIds => from(listOfIds)),
// for each id you get the details
mergeMap(id => getDetails(id),
)
If you subscribe to myStream you get a stream of details data, one for every id of the original list. The code would be simply
myStream.subscribe(
detail => {
// do what you have to do with the details of an id
}
)
MORE ON THE CODE REFERENCED IN THE QUESTION
My understanding of piece of code using mergeMap is the following:
you fetch a new token with auth.getNewAccessToken
If something goes wrong you retry 3 times
When you receive a fresh token, you do some stuff and then you clone something with next.handle(clonHttp)
Key point is that both auth.getNewAccessToken and next.handle(clonHttp) are async calls returning an Observable.
In this case you want to make sure that FIRST you get the response from auth.getNewAccessToken and ONLY THEN you call next.handle(clonHttp).
In this case the best way to code such logic is using concatMap which ensures that the second Observable is concatenated to the successful completion of the first one.
mergeMap and switchMap can also work in this scenario since auth.getNewAccessToken emits only ONCE and then completes, but the right semantic is given by concatMap (which by the way is the same as mergeMap with concurrency set to 1, but this is another story).

Calling a method from another method in Node.js

I am trying to make some modifications to a Node.js app that I forked, what I am trying to do is call a function within another one.
I have attempted a crack at this by simply calling the method as follows but I'm not really that familiar with Node.js so I'm not sure I'm doing it right.
'use strict';
/*
* initalize the class
*/
function MessageBot(bot, user, cache) {
this.bot = bot;
this.user = user;
this.cache = cache;
}
/*
* perform commands
*/
MessageBot.prototype.librarySearch = function(searchText) {
var self = this;
// call SOS function - this is the bit that doesn't work
MessageBot.prototype.callSos(somenumbervar);
}
MessageBot.prototype.callSos = function(number) {
// do something else here
var self = this;
var commandList = self.cache.get('commandList');
}
Remember that this ostensibly inherits the prototype (directly or indirectly). Therefore, you can just do this.callSos(somenumbervar).
If you want to reach the method through the prototype, you have to tell it what this is. With your current code, this in callSos() will be MessageBot.prototype -- certainly not what you want. So you can also do MessageBot.prototype.callSos.call(this, somenumbervar).
Which approach to take depends on how dynamic you want your objects to be. For example, if you want consumers of MessageBot to be able to "override" callSos() by installing their own method, then you should take the first approach (this.callSos()) as it will look up callSos in the object's inheritance chain. This process will only reach the method you've installed on the prototype if the method hasn't been overridden. IMO this is the approach you should take unless you have a very good reason not to.
See this example, which demonstrates how the two approaches differ regarding overriding, while also showing that both work with regards to passing the correct this value (since both methods can retrieve the expected value from this.data):
function ClassA(data) {
this.data = data;
}
ClassA.prototype.foo = function () {
console.log("this.bar() returned: " + this.bar());
console.log("ClassA.prototype.bar.call(this) returned: " + ClassA.prototype.bar.call(this));
};
ClassA.prototype.bar = function () {
return 'in the ClassA prototype, our data is ' + this.data;
};
console.log('--obj1-- (no override)');
var obj1 = new ClassA(3.14);
obj1.foo();
console.log('--obj2-- (override)');
var obj2 = new ClassA(42);
obj2.bar = function () {
return 'in an overriding method, our data is ' + this.data;
};
obj2.foo();

Issue with Javascript module system I'm trying to make and cant figure out

I have an idea I really like for a module system I'm trying to create for one of my projects, the problem is I cant seem to figure it out. The idea is pretty simple, nothing complicated, and very straightforward but I guess I'm missing something as I have spent hours on end trying stuff.
Essentially all I'm wanting to do is have 4 variables, public, protected, package, and private inside a module along with 3 functions augment, extend, and submodule.
The idea is that each module is an anonymous function that saves a public interface to a window variable and have everything else unaccessible. That is, anything in the public variable and the 3 functions are the only public interface.
Ideally, with this setup, you can directly augment any module with the augment function, extend the module as a parent/child relationship with the extend function, and create a submodule on that module with the submodule function. The variables will be accessible to depending on the variable and method chosen for example package will be accessible to all submodules and extensions while protected will not be accessible from a submodule.
This should form a chain as well where each submodule down or each extension can access its parent data and have room to add its own. I should also note that the intention is not like a class where each instance has its own copy of all the variables, rather, theres just one variable pool in each module and their linked to one another depending on actions taken.
What I have below is what I've come up with so far but its not quite right and doesnt fit quite well as it leans to a more class like setup and really doesnt work anything like how I wanted it as described above but should give you an idea of what I'm talking about.
Any help is greatly appreciated, code ans jsfiddle link below
https://jsfiddle.net/0ghbxdpo/
function Module()
{
// Accessible to everything
this._public = {};
// Accessible to all extensions and submodules
this._package = {};
// Accessible only to extensions of the same module
this._protected = {};
// Accessible only to augmentations
this._private = {};
}
// Augments this module and returns the newly augmented module, using
// the return value isn't nesasary as the actual module would have successfully
// been augmented
// Access to all properties
Module.prototype.augment = function augment(augmentation)
{
return augmentation({
_public: this._public,
_package: this._package,
_protected: this._protected,
_private: this._private
});
}
// Create a new module with this one as its parent
// Access to all but this modules private variables
Module.prototype.extend = function extend(extension)
{
var obj = new Module();
obj._public._parent = this._public;
obj._package._parent = this._package;
obj._protected._parent = this._protected;
// Execute the extension module under the context of a new module
// with all but the private variables as its parents
return extension.call(obj);
}
// Create a new module and assign it to one of the 4 scope variables, public by default
// Access only to public and package modules
Module.prototype.submodule = function submodule(moduleName, module, scope)
{
var obj = new Module();
obj._public._parent = this._public;
obj._package._parent = this._package;
// Run the submodule code under the context of the new module
var tmp = module.call(obj);
// Then assign it to the correct scope variables
if(scope === "protected")
return this._protected[moduleName] = tmp;
else if(scope === "package")
return this._package[moduleName] = tmp;
else if(scope === "private")
return this._private[moduleName] = tmp;
else
return this._public[moduleName] = tmp;
}
// Some test to play around and test it out
// This isnt ideal as theres no way to actually setup anything with the initial module
// Also it reveals all the scopes to the public
window.CoolTest = new Module();
// This fails with an error as "this" points to the wrong object so the whole
// thing is created wrong and fails. You cant use call, apply, or bind from here as
// the public should not have access to anyting but the public variables
// The other scope variables are supposed to be stored internally and automatically
// used to create the submodule without ever exposing them
window.CoolTest.submodule("sub", function()
{
this._public.hello = "world";
});
I figured it out and the complete code is in this newly updated jsFiddle link and below it in text to the completed Module system.
https://jsfiddle.net/0ghbxdpo/1/
/*
* Module System
* This is not intended to be a class emulation nor will it work very well if
* treated like one.
*
* This was created out of desire for sharing properties and functions among
* different components in a program. One where permissions to all of it is
* very fine grained and strictly enforced.
*
* It was also created out of desire for lazy loading and lazy augmentation. In
* other words, I want to be able to split a large module into several files
* and have it just work. I also want each module to be augmentable and
* extendable still supporting the lazyloading approach and still respecting
* permissions
*
* You create a module by instiating this class, by doing so you will get an
* empty module. You may also provide a function as parameter to do an initial
* custom setup, the function will be passed an internal private object which
* contains the actual class so you can modify where fit. You have access to the
* entire class. A second parameter may be passed for special cirumstances where
* you want to access some variable inside the constructor function you provide.
*
* After the module is created it returns the public interface, this interface
* contains only the public variable contents and 3 methods that work on the
* module. There is intentionally no way to access the full module as it remains
* closed off tightly to the public.
*
* You may perform 3 actions on the module.
* "_augment" which re-gives you full control over the module and all of its
* properties.
*
* "_extend" which creates a new module that contins "_parent" links to the
* parents public, package, and protected variables. You can directly edit
* the parent object or you can add your own properties outside of the
* parent. Any changes you make to the parent will be reflected in the
* parent while any chnages you make outside of the parent will not be
* accessible by it
*
* "_submodule" which does what extend does except that instead of leaving out
* just the parents private variables, it also leaves out the parents
* protected variables. Not only does it do this, but it also ammends
* one of its scope variables, public by default, and attaches the modules
* public interface to it.
*/
var Module = (function ()
{
// initFunc is the otpional configuration function
// It is passed the internal module object and scopeRef if provided
// scopeRef is purely used for passing a custom parameter to the init function
function Module(initFunc, scopeRef)
{
// Declare 4 scope variables
// Augmentation can always modify these values
this._public = {}; // Accessible to anyone
this._package = {}; // Accessible to extensions and submodules
this._protected = {}; // Accessible only to extensions
this._private = {}; // Not accessible anywhere
// Scope related reasons
this._self = this;
// This is a bit of trickory to allow a public interface entirely closed
// off from the actual module, it pre-executes and saves the contents
// to be used later
// the parameter is the trick in how we access the real module
this._publicReturn = function publicReturn(self)
{
// We prepare the public interface by directly referencing the modules
// public variable
var ret = self._public;
// Then we ammend the interface and the public variable by adding in
// the 3 methods that can be used to work on the real internal module
ret._augment = self.augment.bind(self);
ret._extend = self.extend.bind(self);
ret._submodule = self.submodule.bind(self);
return ret;
}(this);
// Now that thats all out of the way, we process the given init function
// if provided
if (initFunc)
initFunc(this, scopeRef);
// And we return only the public interface
return this._publicReturn;
}
// This modifies the module by re-opening up the entire module for any
// kind of modification, works much the same way as the constructor
Module.prototype.augment = function (initFunc, scopeRef)
{
initFunc(this, scopeRef);
return this._publicReturn;
};
// This creates a brand new module with parent references to this
// modules public, package, and protected variable scopes
Module.prototype.extend = function(initFunc, scopeRef)
{
// Start with blank module
var obj = new Module();
// Augment in the parent references
obj._augment(function(_obj, _scopeRef)
{
_obj._public._parent = _scopeRef._public;
_obj._package._parent = _scopeRef._package;
_obj._protected._parent = _scopeRef._protected;
}, this);
// Allow the user to do augmentation if provided
if(initFunc)
obj._augment(initFunc, scopeRef);
// Return the given public interface
return obj;
};
// This creates a brand new module with parent references to this
// modules public and package variable scopes. It also ammends the newly
// made module to one of this modules variable scopes, by default public.
Module.prototype.submodule = function (moduleName, initFunc, scopeRef, scope)
{
// Start with blank module
var obj = new Module();
// Augment in the parent references
obj._augment(function(_obj, _scopeRef)
{
_obj._public._parent = _scopeRef._public;
_obj._package._parent = _scopeRef._package;
}, this);
// Allow the user to do augmentation if provided
if(initFunc)
obj._augment(initFunc, scopeRef);
// Then assign it to the correct scope variables
if (scope === "protected")
return this._protected[moduleName] = obj;
else if (scope === "package")
return this._package[moduleName] = obj;
else if (scope === "private")
return this._private[moduleName] = obj;
else
return this._public[moduleName] = obj;
};
return Module;
})();

Resources