Race condition when using withLatestFrom as part of my pipeable operator - node.js

I have the following code that suffers from a race condition. Sometimes I can see the results, sometimes I cannot:
const op1 = ()=>{
const filesObs = from(['a','b','c']).pipe(delay(200))
return (obs)=>{
return obs
.pipe(delay(100))
.pipe(withLatestFrom(filesObs))
}
}
from([1,2,3,4,5]).pipe(op1()).subscribe(console.log);
As it is I don't see anything printed. But If I increase the 2nd delay to 300 I see the expected values:
[ 1, 'c' ]
[ 2, 'c' ]
[ 3, 'c' ]
[ 4, 'c' ]
[ 5, 'c' ]
Is there a way to always see the result by using observeOn or subscribeOn somewhere on my code or should I follow another best practice?

First of all this is not an issue specific to withLatestFrom being inside an operator.
The code below that is not used inside an operator also does not print anything (faces the same issue):
const filesObs = from(['a','b','c']).pipe(delay(200));
from([1,2,3,4,5]).pipe(delay(100),withLatestFrom(filesObs)).subscribe(console.log);
According to the provided desired output in the question we need to get the last element of the letters stream and pair it with each value from the numbers stream. But what withLatestFrom() will get is the last emitted element at each point in time. To justify this, consider adding some delay between the emitted elements of the letters stream (1st line).
//adding a delay on each of the emitted letters.
const filesObs = from(['a','b','c']).pipe(concatMap((v)=>of(v).pipe(delay(50))))
from([1,2,3,4,5]).pipe(delay(100),withLatestFrom(filesObs)).subscribe(console.log);
[ 1, 'a' ]
[ 2, 'a' ]
[ 3, 'a' ]
[ 4, 'a' ]
[ 5, 'a' ]
As you can see the above is not the desired output.
Also, I am not sure if it is an rxjs bug, but withLatestFrom() will skip the value if there is nothing emitted yet on the observable arguement. See below how it skips the first number (because at the moment it is emits it, nothing has been emitted yet on filesObs).
const filesObs = from(['a','b','c']).pipe(concatMap((v)=>of(v).pipe(delay(50))))
//Now adding a delay on each of the emitted numbers.
from([1,2,3,4,5]).pipe(concatMap((v)=>of(v).pipe(delay(25)))).pipe(withLatestFrom(filesObs)).subscribe(console.log);
[ 2, 'a' ]
[ 3, 'a' ]
[ 4, 'b' ]
[ 5, 'b' ]
solution
One solution to the problem is to get the last() element of the letters stream and repeat() it. Then map each number with the first() element of filesObs, which will now always be the last one ('c'):
const filesObs = from(['a','b','c']).pipe(delay(200), last(), repeat())
from([1,2,3,4,5]).pipe(delay(100)).pipe(mergeMap(v=>filesObs.pipe(first(),map(v2=>[v2,v])))).subscribe(console.log);
And the same inside an operator:
const op1 = ()=>{
const filesObs = from(['a','b','c']).pipe(delay(200), last(), repeat())
return (obs)=>{
return obs
.pipe(delay(100))
.pipe(mergeMap(v=>filesObs.pipe(first(),map(v2=>[v2,v]))))
}
}
from([1,2,3,4,5]).pipe(op1()).subscribe(console.log);
Both of the above will output the below value, independent to the delay values:
[ 'c', 1 ]
[ 'c', 2 ]
[ 'c', 3 ]
[ 'c', 4 ]
[ 'c', 5 ]

Related

Groovy: Adding elements to a collection

I've the following collection
def a = [
b:[
[
c: "x",
d: ["y","z"]
]
],
b1:[
[
c: "x1",
d: ["y1","z1"]
]
]
]
I want to add a new element "w" to d:["y", "z"] to have this d:["y", "z", "w"]
I've tried a.put(d:"w"), a.add(d:"w") but getting exception
groovy.lang.MissingMethodException: No signature of method: java.util.LinkedHashMap.put() is applicable for argument types: (String) values: [w]
You have take into account all your nesting here. You have a map to
list to map. The main problem is the list now, since your example makes
it not clear, how many list items b could hold. So the solution for
your exact example is:
a.b[0].d << "w"

How to iterate a list up to certain element/index in terraform?

This is simple in other language by using for or do while statement. Being a novice, I still can't figure out to do it in Terraform.
The real case that I need to do is to build a connection string to mongoDB replica-set service. The replication_factor is 3, 5 or 7 which means I need to create a list of 2, 4, or 6 hostnames/addresses.
I come up with the following code sofar:
locals {
uri_list = [
"#${alicloud_mongodb_instance.tyk_mongodb-test.id}1.mongodb.${var.region}.rds.aliyuncs.com:${var.db_port}",
"#${alicloud_mongodb_instance.tyk_mongodb-test.id}2.mongodb.${var.region}.rds.aliyuncs.com:${var.db_port}",
"#${alicloud_mongodb_instance.tyk_mongodb-test.id}3.mongodb.${var.region}.rds.aliyuncs.com:${var.db_port}",
"#${alicloud_mongodb_instance.tyk_mongodb-test.id}4.mongodb.${var.region}.rds.aliyuncs.com:${var.db_port}",
"#${alicloud_mongodb_instance.tyk_mongodb-test.id}5.mongodb.${var.region}.rds.aliyuncs.com:${var.db_port}",
"#${alicloud_mongodb_instance.tyk_mongodb-test.id}6.mongodb.${var.region}.rds.aliyuncs.com:${var.db_port}"
]
uri_list_out = [
for uriname in local.uri_list :
lower(uriname)
if substr(uriname,length("${alicloud_mongodb_instance.tyk_mongodb-test.id}") + 1, 1) < var.mongos_config["${var.environment}"]["replication_factor"]
]
}
What I expect from
output "uri_list_out" {
value = local.uri_list_out
}
is the first two elements of uri_list but instead I got only [1,2] for replication_factor = 3. Seems like if instruction in for also modify the output ???
I appreciate any hints to solve this problem.
Hendro
I believe what you really need is the slice(list, startindex, endindex) function:
uri_list_out = [
for uri in slice(local.uri_list, 0, var.mongos_config[var.environment]["replication_factor"] - 1) :
replace(uri, "/^#/", "") # Remove the leading '#'
]
The docs for the slice function
> slice(["a", "b", "c", "d"], 1, 3)
[
"b",
"c",
]

conditional iteritems just processes no result/no error

i am trying to "loop" this dataframe and add a column based on the condition of another columns. i think i achieve my result with this logic but cant seem to find the right iterator or other method to create my new column. In this example I am using the iteritems but this code just spins for minutes and no result is given,i manually cancel the code. my dataframe has 400,000 columns. a screenshot of df is included.
the goal is to fill the instances where ['close'] == ['prev'] and replace the 0 value with most recent trade signal (either +, or -).
for index, col in df.T.iteritems():
if col['Close'] > col['prev']:
col['trade2'] = '+'
x = '+'
continue
elif col['Close'] < col['prev']:
col['trade2'] = '-'
x = '-'
continue
elif col['Close'] == col['prev']:
col['trade2'] = x
I created the test DataFrame as follows:
df = pd.DataFrame(data=[
[ 36.50, 36.53, '-' ],
[ 36.53, 36.50, '+' ],
[ 36.53, 36.53, '0' ],
[ 36.53, 36.53, '0' ],
[ 36.53, 36.53, '0' ],
[ 36.51, 36.53, '-' ],
[ 36.51, 36.51, '0' ],
[ 36.53, 36.51, '+' ],
[ 36.53, 36.53, '0' ],
[ 36.53, 36.53, '0' ],
[ 36.53, 36.53, '0' ]],
columns=['Close', 'prev', 'trade'],
index=range(5,16))
To compute trade2 value for the current row, we have
to define the following function:
def cmp(row):
global lastResult
if row.Close > row.prev:
lastResult = '+'
elif row.Close < row.prev:
lastResult = '-'
return lastResult
It uses a global variable lastResult, which will be initially set
to '0'.
And instead of your loop with iteritems, you should just apply this
function to each row and substitute the result to the new column,
starting from initial setting of lastResult (as mentioned above):
lastResult = '0'
df['trade2'] = df.apply(cmp, axis=1)
This is just the way how operations of such type should be performed.
And one remark concerning your code.
If you use iteritems, then:
within the loop you should add results to some list,
after the loop, this list should be substituted to a new column.
Another, quicker solution
If you already have trade column, you can execute:
df['trade2'] = df.trade.replace(to_replace='0')
This method relies on the fact that trade column is always a string
(either a minus, a plus or '0'), so to_replace argument will match (string) zero cases.
The next argument - value - has not been given, so it defaults to
None.
In such circumstances, the pad replace method is assumed, meaning
fill values forward.
This way, either '+' or '-' is "replicated" down the column, giving just
the same result.
For source DataFrame with 1100 rows (your DataFrame replicated 100 times)
I got the execution time over 100 times shorter than with the first
method.

Puppet in cycle added empty elements in array

$hash_arr_1 = { b => 2, c => 3, f => 1 }
$arr = ['a', 'c', 'd', 'f', 'e']
$hash_arr_2 = $arr.map |$param| {
if has_key($hash_arr_1, $param) {
{$param => $hash_arr_1[$param]}
}
}
notice($hash_arr_2)
Result: [{ , c => 3, , f => 1, ,}]
How to do that there are no empty elements in the array?
The problem here is that you are using the map lambda function when really you want to be using filter. Summary from linked documentation is as follows:
Applies a lambda to every value in a data structure and returns an array or hash containing any elements for which the lambda evaluates to true.
So the solution for you is:
$hash_arr_2 = $hash_arr_1.filter |$key, $value| { $key in $arr }
This will iterate through the keys of the hash $hash_arr_1, check if the key exists as a member of the array $arr with the provided conditional, and then return a hash with only the key value pairs that evaluated to true.

get common elements from a result which consists of more than 1 array in arangodb

I want to fetch common elements from multiple arrays. The no. of arrays resulted would keep changing depending upon the no. of tags in array a[].
As a first step, my query and result I get is as shown below:
let a=["Men","Women","Accessories"]
let c=(for i in a
Let d=Concat("Tags/",i)
return d)
for i in c
let m=(for y in outbound i TC
return y._key)
return m
and result I get is:
[
[
"C1",
"C5",
"C7",
"C3"
],
[
"C2",
"C5",
"C6",
"C4"
],
[
"C7",
"C5",
"C6"
]
]
From this result, I want only common element as a result i.e "C5" (here).
How can I get that?
This question has also been asked and answered on github.
The function INTERSECTION() returns the intersection of all specified arrays and APPLY() is used to pass a dynamic amount of nested arrays.
The query
let D = [["C1","C5","C7","C3"],["C2","C5","C6","C4"],["C7","C5","C6"]]
RETURN APPLY("INTERSECTION", D)
results in:
[
[
"C5"
]
]

Resources