I've tried to model the situation I have in a real code.
The use case is simple:
There is:
Future {blocking {
// in there it calls nested futures
}}
More detailed (global Execution context is used as parent one, but there is was attemp to use separete one for children):
object ExContexts {
val main = scala.concurrent.ExecutionContext.Implicits.global
val ecChildren = scala.concurrent.ExecutionContext.fromExecutor(Executors.newFixedThreadPool(1))
}
object NestedFutures2 extends App {
import scala.concurrent.ExecutionContext.Implicits.global
val cores = Runtime.getRuntime.availableProcessors
val works = parentWork(ExContexts.main) // main EC
val result1: Seq[Seq[Future[String]]] = Await.result(Future.sequence(works), Duration.Inf)
println("parents are done their work")
val result2: Seq[String] = Await.result(Future.sequence(result1.flatten), Duration.Inf)
// ---
def parentWork(ex:ExecutionContext): Seq[Future[Seq[Future[String]]]] = {
val works: Seq[Future[Seq[Future[String]]]] =
(1 to cores * 2) map { x =>
Future {
blocking { // will create new thread/place if needed
val parentName = Thread.currentThread.getName
println("parent: " + parentName + " started an action")
val playFutureOutcomes: Seq[Future[String]] = (1 to 10) map {stuffId =>
childPlay(parentName = parentName)(ExContexts.ecChildren)
}
Thread.sleep(1000)
println(s"[${timeStamp()}] parent: " + parentName + " has finished the action")
playFutureOutcomes
}
}
}
works
}
def childPlay(parentName:String)(ex:ExecutionContext):Future[String] = {
Future {
Thread.sleep(2000) // two seconds play session
val threadName = Thread.currentThread.getName
// log
println("child: " + threadName + " of " + parentName + " parent")
Thread.currentThread.getName
}
}
def timeStamp(pattern:String = "ss:mm : hh"): String = {
val fmt = DateTimeFormat.forPattern(pattern)
fmt.print(DateTime.now)
}
}
From the output (5 of 8 parrent futures are done) I see that children make use of parents threads...:
[01:31 : 05] parent: ForkJoinPool-1-worker-5 has finished the action
[01:31 : 05] parent: ForkJoinPool-1-worker-11 has finished the action
[01:31 : 05] parent: ForkJoinPool-1-worker-1 has finished the action
[01:31 : 05] parent: ForkJoinPool-1-worker-3 has finished the action
[01:31 : 05] parent: ForkJoinPool-1-worker-7 has finished the action
child: ForkJoinPool-1-worker-13 of ForkJoinPool-1-worker-1 parent
child: ForkJoinPool-1-worker-5 of ForkJoinPool-1-worker-5 parent
child: ForkJoinPool-1-worker-1 of ForkJoinPool-1-worker-1 parent
child: ForkJoinPool-1-worker-11 of ForkJoinPool-1-worker-11 parent
child: ForkJoinPool-1-worker-3 of ForkJoinPool-1-worker-3 parent
child: ForkJoinPool-1-worker-7 of ForkJoinPool-1-worker-7 parent
child: ForkJoinPool-1-worker-13 of ForkJoinPool-1-worker-11 parent
...
I expect parents one to done their work in ~1 second as it was if we don't have childrent. Also I would expect to see two connecton pools instead of one.
Q: what would be the suggestion to achive it?
UPDATE:
I guess it is all about Join-Pool based upon Implicit.glocal is built. All nested Furue will be joined to one pool?
// What (I think) I see is that if we have 'global' execution context as parent/main,
// then whatever we pass as execution context to the child Future still will use global context
// and still will be using ONE connection pool.
// And then therefore wll have performance problem if children do not use 'blocking' -
// parents will be stuck in cores-long connection pool
// So: Children will 'inherit' execution context, but not 'blocking'
// and as children connected/joined to the execution context the parents will not be able to do
// anything about it but wait for the free space in the pool (when child is done)
// new additional threads will not be creating for the next parent because (i guess)
// Global execution context will not see the reason of/for it.
// Like a snapshot of the moment of time:
// Connection Pool: [ch1, ch2, ch3, p1]
// It things:Ok there is one parent (p1) in the pool if one more parent
// future will come I create for it new thread it the pool.
// But because we have more joined children than parents in the list
// the moment when Global ex context can help parent will never come.
Should then I not to use that global context if I want parents to be independant on childrent and vice versa? (funny how many people in Internet are playing with global only.)
Say I know ma app has 20 000 clients and each of them makes 1-100 sec long operations (op1->op1.op1) all the time. Then I just could dedicate one executon context with 20 000 connecton pool (max). And as many execution context as needed for all sort of long-running operations (parent->child where child is not reportng to the parent). Say: printingExCon, logExCon, readStufExCon, writeStuffExCon
But if I know that one oparation should include anoter one and they maybe joined then (only) I may use that global Ex context?
Otherwise if I would use global ex context as may man context - all and everyone would join to it and I would end up having performance problem that I explained at te beginning of this Sunday post..
UPDATE:
I wasted good chunk of time trying to understand why fixed version (with 'ex' passed to childPlay's Future{}(ex) was producing an output that ignore sleeps and behaves weired on many other cases. The problem was fixed when instead of running/buildng in IntellyJIDEA 14 (by CTRL-SHIFT+F10) i build it wit sbt run..
Related
I have 2 components in react-admin the child one set the state of the parent one.
the child component has useEffect to trigger the following method in parent component
const [approved, setAproved] = useState([]);
useEffect(() => {
console.log(JSON.stringify(approved))
}, [approved])
const setapprovedamount = (id, approvedAmount) => {
if(approved.length!==0)
{
// if the child run 3 times
// this line runs 3 times
console.log("set Aproved with" +id+ " and amount : "+approvedAmount)
// but this line update only the last object
setAproved(
approved.map(item =>
(item.id == id)
? {...item, totalApproved : approvedAmount}
: item
)
)
}
else
{
setAproved(approved =>[...approved, {
id: id,
totalApproved: approvedAmount
}] )
}
}
useEffect(() => {
console.log(JSON.stringify(approved))
}, [approved])
So **for example** if I have 3 times load, the Console is:
file.js:168 set Aproved with44 and amount : 799.71
file.js:168 set Aproved with45 and amount : 845.98
file.js:168 set Aproved with46 and amount : 890.83
file.js:96
[{"id":44,"totalApproved":null},{"id":45,"totalApproved":null},{"id":46,"totalApproved":890.83}]
so the method runs 3 times as it is triggered but, it is setting only the last value rendered in the state array in the parent component
Because of how State and Lifecycle works.
If you wanted to fire multiple and chained state updates throughsetState, you need to let the component complete its cycle at each state update - first things first.
Therefore, the basic solution is to imply the use of chained setTimeout calls to delegate each state update, letting each function call complete a component, full life cycle.
setTimeout(function() {
setState(firstState)
setTimeout(function() {
setState(secondState)
});
});
TL;DR
I want to delay a publication, but can't figure out how to, er, combine the parts
In Brief
I have a Publisher
let generator = PassthroughSubject<Bool, Never>()
and want somehow to use the modifier
.delay(for: 2, scheduler: RunLoop.main)
so that when I call
generator.send(true)
the message is sent two seconds after the call to send()
Looking at the docs for Publishers.Delay made the type error more clear, but doesn't help me to find the right way to hook things up.
Code
import SwiftUI
import Combine
// Exists just to subscribe.
struct ContainedView : View {
private let publisher: AnyPublisher<Bool, Never>
init(_ publisher: AnyPublisher<Bool, Never> = Just(false).dropFirst().eraseToAnyPublisher()) {
self.publisher = publisher
}
var body: some View {
Rectangle().onReceive(publisher) { _ in print("Got it") }
}
}
struct ContentView: View {
let generator = PassthroughSubject<Bool, Never>()
// .delay(for: 2, scheduler: RunLoop.main)
// Putting it here doesn't work either.
var body: some View {
VStack {
Button("Tap") {
// Does not compile
self.generator.delay(for: 2, scheduler: RunLoop.main).send(true)
// Value of type 'Publishers.Delay<PassthroughSubject<Bool, Never>, RunLoop>' has no member 'send'
// https://developer.apple.com/documentation/combine/publishers/delay
// Does not compile
self.generator.send(true).delay(for: 2, scheduler: RunLoop.main)
// Value of tuple type '()' has no member 'delay'
// Just a broken-up version of the first try.
let delayed = self.generator.delay(for: 2, scheduler: RunLoop.main)
delayed.send(true)
// This, of course, builds and works.
self.generator.send(true)
print("Sent it")
}
ContainedView(generator.eraseToAnyPublisher())
.frame(width: 300, height: 200)
}
}
}
You can use the debounce property of a publisher to delay the publishing.
$yourProperty
.debounce(for: 0.8, scheduler: RunLoop.main)
.eraseToAnyPublisher()
.delay(for: 2, scheduler: RunLoop.main) is likely exactly what you need, but it'll be key to see how you're subscribing to fully understand the issue. Delay doesn't delay the sending of the value when using send() with a subject - that's a link for imperative code and sends the data whenever send is invoked, typically against some already existing subscription.
While you have a subscriber in the first bit of code, there isn't one with the subject to pin these together.
For example, if you updated:
Just(false).dropFirst().eraseToAnyPublisher()
to
Just(false).dropFirst().eraseToAnyPublisher().delay(for: 2, scheduler: RunLoop.main)
Then the print statement should trigger ~2 second after the the init() was invoked. Depending on what you're trying to accomplish here, using a closure trigger such as onAppear might make a lot more sense, having that call the subject.send(), which you can then delay as you like in the publisher chain that happens before whatever subscribes to it.
var cancellables: [AnyCancellable] = []
let generator = PassthroughSubject<Bool, Never>()
let generated = generator.delay(for: 2, scheduler: RunLoop.main).sink { value in
print(value.description + " " + Date().timeIntervalSinceReferenceDate.description)
}
print(Date().timeIntervalSinceReferenceDate.description)
generator.send(true)
generator.send(false)
output
641453284.840604
true 641453286.841731
false 641453286.847715
While answering a question I attempted to implement a setup where the main thread joins the efforts of the CommonPool to execute a number of independent tasks in parallel (this is how java.util.streams operates).
I create as many actors as there are CommonPool threads, plus a channel for the main thread. The actors use rendezvous channels:
val resultChannel = Channel<Double>(UNLIMITED)
val poolComputeChannels = (1..commonPool().parallelism).map {
actor<Task>(CommonPool) {
for (task in channel) {
task.execute().also { resultChannel.send(it) }
}
}
}
val mainComputeChannel = Channel<Task>()
val allComputeChannels = poolComputeChannels + mainComputeChannel
This allows me to distribute the load by using a select expression to find an idle actor for each task:
select {
allComputeChannels.forEach { chan ->
chan.onSend(task) {}
}
}
So I send all the tasks and close the channels:
launch(CommonPool) {
jobs.forEach { task ->
select {
allComputeChannels.forEach { chan ->
chan.onSend(task) {}
}
}
}
allComputeChannels.forEach { it.close() }
}
Now I have to write the code for the main thread. Here I decided to serve both the mainComputeChannel, executing the tasks submitted to the main thread, and the resultChannel, accumulating the individual results into the final sum:
return runBlocking {
var completedCount = 0
var sum = 0.0
while (completedCount < NUM_TASKS) {
select<Unit> {
mainComputeChannel.onReceive { task ->
task.execute().also { resultChannel.send(it) }
}
resultChannel.onReceive { result ->
sum += result
completedCount++
}
}
}
resultChannel.close()
sum
}
This gives rise to the situation where mainComputeChannel may be closed from a CommonPool thread, but the resultChannel still needs serving. If the channel is closed, onReceive will throw an exception and onReceiveOrNull will immediately select with null. Neither option is acceptable. I didn't find a way to avoid registering the mainComputeChannel if it's closed, either. If I use if (!mainComputeChannel.isClosedForReceive), it will not be atomic with the registration call.
This leads me to my question: what would be a good idiom to select over channels where some may get closed by another thread while others are still live?
The kotlinx.coroutines library is currently missing a primitive to make it convenient. The outstanding proposal is to add receiveOrClose function and onReceiveOrClosed clause for select that would make writing code like this possible.
However, you will still have to manually track the fact that your mainComputeChannel was closed and stop selecting on it when it was. So, using a proposed onReceiveOrClosed clause you'll write something like this:
// outside of loop
var mainComputeChannelClosed = false
// inside loop
select<Unit> {
if (!mainComputeChannelClosed) {
mainComputeChannel.onReceiveOrClosed {
if (it.isClosed) mainComputeChannelClosed = true
else { /* do something with it */ }
}
}
// more clauses
}
See https://github.com/Kotlin/kotlinx.coroutines/issues/330 for details.
There are no proposals on the table to further simplify this kind of pattern.
I would like to set up a number of threads operating concurrently on a channel, and every one of those threads should be also feeding the channel. One of the threads would decide when to stop. However, this is the closest I have come to doing that:
use Algorithm::Evolutionary::Simple;
my $length = 32;
my $supplier = Supplier.new;
my $supply = $supplier.Supply;
my $channel-one = $supply.Channel;
my $pairs-supply = $supply.batch( elems => 2 );
my $channel-two = $pairs-supply.Channel;
my $single = start {
react {
whenever $channel-one -> $item {
say "via Channel 1:", max-ones($item);
}
}
}
my $pairs = start {
react {
whenever $channel-two -> #pair {
my #new-chromosome = crossover( #pair[0], #pair[1] );
say "In Channel 2: ", #new-chromosome;
$supplier.emit( #new-chromosome[0]);
$supplier.emit( #new-chromosome[1]);
}
}
}
await (^10).map: -> $r {
start {
sleep $r/100.0;
$supplier.emit( random-chromosome($length) );
}
}
$supplier.done;
This stops after a number of emissions. And it's probably not running concurrently anyway. I am using channels instead of supplies and taps because these are not run concurrently, but asynchronously. I need supplies because I want to have a seudo-channel that takes the elements in pairs, as it's done above; I haven't seen the way of doing that with pure channels.
There is no difference above if I change the supply's emit to channel's send.
So several questions here
Are these react blocks run in different threads? If not, what would be the way of doing that?
Even if they are not, why does it stop even if $pairs is emitting to the channel all the time?
Could I have "batch" channels created automatically from single-item channels?
Update 1: if I eliminate $supplier.done from the end, it will just block. If I create a promise in whenever, one for each read, it just blocks and does nothing.
The answer is here, stripped down to the minimum necessary
my Channel $c .= new;
my Channel $c2 = $c.Supply.batch( elems => 2).Channel;
my Channel $output .= new;
my $count = 0;
$c.send(1) for ^2;
my $more-work = start react whenever $c2 -> #item {
if ( $count++ < 32 ) {
$c.send( #item[1]);
my $sum = sum #item;
$c.send( $sum );
$output.send( $sum );
} else {
$c.close;
}
}
await $more-work;
loop {
if my $item = $output.poll {
$item.say
} else {
$output.close;
}
if $output.closed { last };
}
A second channel that batches the first channel every two elements is used via the creation of a supply from a channel ($c.Supply), batching that supply in batches of two (batch( elems => 2)) and turning it back into a channel. A third channel is created for output.
In order to not exhaust the supply and hang the channel, every second element that is read from the first (and actually, only) channel is put back there. So the second channel that reads in twos is never hanged or waiting for new elements.
An output channel is created for every new element, and an external counter to finish the operation when it's needed; that output channel is read in a non-blocking way, and closed when there's nothing left to read in the last line.
To answer precisely to my original questions:
Yes, they are, only they are stealing elements from each other.
Because the two threads were reading from the same channel. The first one to stumble into an element, reads it.
Yes, by turning channels into supplies, batching them and turning them back into channels. Only bear in mind that they are not copies, they will be sharing the self same elements.
I have two futures. I want to execute them in order. For example:
val ec: ExecutionContextExecutor = ExecutionContext.Implicits.global
val first=Future.successful(...)
val second=Future.successful(...)
When first is completed then second should be executed. The problem is that second should return Future[Object] not Future[Unit] so
I can not use completed, andThen etc. functions
I can not block the process using await or Thread.sleep(...)
I can not use for loop since execution context is defined like this.
first.flatmap( _=> second) will not execute in order.
How can I do that?
As soon as you assign a Future to a val, that Future is scheduled and will execute as soon as possible. To prevent this you have two options:
Define the Future in a def
Define the Future where you want to use it.
Here's an example of #1:
def first: Future[Int] = Future { Thread.sleep(5000); 1 }
def second(i: Int): Future[Unit] = Future { println(i) }
first.flatMap(i => second(i))
And here's an example of #2:
for(
i <- Future { Thread.sleep(5000); 1 };
_ <- Future { println(i) }
) yield ()
Both examples will wait for 5 seconds and print then 1