In Scala, as explained in the PR that introduced it, parasitic allows to steal
execution time from other threads by having its
Runnables run on the Thread which calls execute and then yielding back control
to the caller after all its Runnables have been executed.
It appears to be a neat trick to avoid context switches when:
you are doing a trivial operation following on a Future coming from an actually long running operation, or
you are working with an API that doesn't allow you to specify an ExecutionContext for a Future but you but you would like to make sure the operation continues on that same thread, without introducing a different threadpool
The PR that originally introduced parasitic further explains that
When using parasitic with abstractions such as Future it will in many cases be non-deterministic
as to which Thread will be executing the logic, as it depends on when/if that Future is completed.
This concept is also repeated in the official Scala documentation in the paragraphs about Synchronous Execution Contexts:
One might be tempted to have an ExecutionContext that runs computations within the current thread:
val currentThreadExecutionContext = ExecutionContext.fromExecutor(
new Executor {
// Do not do this!
def execute(runnable: Runnable) { runnable.run() }
})
This should be avoided as it introduces non-determinism in the execution of your future.
Future {
doSomething
}(ExecutionContext.global).map {
doSomethingElse
}(currentThreadExecutionContext)
The doSomethingElse call might either execute in doSomething’s thread or in the main thread, and therefore be either asynchronous or synchronous. As explained here a callback should not be both.
I have a couple of doubts:
how is parasitic different from the synchronous execution context in the Scala documentation?
what is the source of non-determinism mentioned in both sources? From the comment in the PR that introduced parasitic it sounds like if doSomething completes very quickly it may return control to the main thread and you may end up actually not running doSomethingElse on a global thread but on the main one. That's what I could make of it, but I would like to confirm.
is there a reliable way to have a computation run on the same thread as its preceding task? I guess using a lazy implementation of the Future abstraction (e.g. IO in Cats) could make this more easy and reliable, but I'd like to understand if this is possible at all using the Future from the standard library.
parasitic has an upper bound on stack recursion, to try to mitigate the risk of StackOverflowErrors due to nested submissions, and can instead defer Runnables to a queue.
The source of non-determinisism is: If the Future is not yet completed: it will register to run on the completing thread. if the Future is completed it will run on the registering thread. Since those two situations can depend on timing, it is not deterministic which thread will execute the code.
How do you know A) which Thread that is, and B) whether it would ever be able to execute another task again?
I find that it is easier to think about Futures as read-handles for values that may or may not exist at a specific point in time. That nicely untangles the notion of Threads from the picture, and now it is rather about: When a value is available, I want to do X—and here is the ExecutionContext that will do X.
Related
I've been using various concurrency constructs for a while now without much consideration for how all the magic happens, which has recently made me increasingly uneasy.
In an attempt to remedy this ... feeling, I have been reading up on how async works under the hood. When I say async, in this case I'm referring to userland / greenthread / cooperative multitasking, although I assume some of the concepts will also apply to traditional OS managed threads insofar as a scheduler and workers are involved.
I see how a worker can suspend itself and let other workers execute, but at the lowest level in non-blocking library code, how does the scheduler know when a previously suspended worker's job is done and to wake up that worker?
For example if you fire up a worker in some sort of async block and perform an operation that would normally block (e.g. HTTP request, SQL query, other I/O), then even though your calling code is async, that operation (library code) better play nice with your async framework or you've effectively defeated the purpose of using it and blocked your scheduler from calling other waiting operations (the, What Color is Your Function problem) while it waits for your blocking call, which was executed inside your non-blocking calling code, to complete.
So now we've got async code calling other async library code, and now I'm asking myself the question all over again - how does the async library code know when to suspend and resume operation?
The idea of firing off a HTTP request, moving on, and returning later to check for results is weird to think about for me - not conceptually but from an implementation standpoint.
How do you perform a partial operation, e.g. sending TCP packets and then continuing with the rest of the program execution, only to come back later and check if results have been delivered. Delivered to what? A socket?
Now we're another layer deep and you are using socket selects to avoid creating threads and blocking, but, again...
how do those sockets start an operation, move on before completion, and then how does select know when data is available?
Are you continually checking some buffer to see if bytes have been delivered in an infinite loop and moving on if not?
Anyhow - I think you see where I'm going here....
I focused mainly on HTTP as a motivating example, but the same question applies for any normally blocking operations - how does it all work at the bottom?
Here are some of the resources I found helpful while researching the topic and which informed this question:
David Beazley's excellent video Build Your Own Async where he walks you through a simple implementation of a scheduler which fire callbacks and suspend execution by sleeping on a waiting queue. I found this video tremendously instructive, but it stops a bit short in that it shows you how using an async sleep frees up the scheduler to execute other workers, but doesn't really go into what would happen when you call code in those workers that itself must be non-blocking so it plays nice with the scheduler.
How does non-blocking IO work under the hood - This got me further along in my understanding, but still left with a few uncertainties.
As we all know, use of Future in Scala requires declaration of an execution context, which creates a thread pool that is used to run the code that appears in Future. My question is if use of Future.successful or Future.failed skips this request to the thread pool. The performance implications are important for correct use of Future.
To provide some background, I have been told by a coworker with years of Scala experience that direct use of Future() to wrap some block of code is discouraged because, to be meaningful, the code in the Future call has to have the potential to throw an exception, and this is undesirable. In our codebase, directly throwing an exception is considered un-functional style and also has performance implications, since throwing an exception is an operation that is relatively slow, with performance costs, and is therefore to be avoided. My question has two parts:
1) I want to know if there are other potential implications of avoiding direct calls to Future(): does use of Future.successful and Future.failed save computing resources by not scheduling work on a thread from the thread pool? By creating an already completed Future, do these two programming constructs avoid requesting a thread from the thread pool?
To be concrete, let us compare two blocks of code:
a)
val fut = Future(1+1)
and
b)
val fut = Future.successful(1+1)
Does b) save computing resources by not going to the thread pool?
2) In addition, I would also like to know if a thread is requested from the thread pool when mapping or flatmapping a Future. For example, I've noticed that any code which involves a flatmap on a Future requires an execution context to be defined. Does that mean that the flatmap operation itself is scheduled on a thread from the thread pool? Is it the case that every map or flatmap operation on a Future occurs on a thread from the thread pool? In that case, is requesting a thread from the thread pool inevitable as long as we are mapping or flatmapping a Future, even if it is already completed due to use of Future.successful and Future.failed?
To be concrete, let us look at an example:
val fut = Future.successful(1+1).map(x => x+1)
Does the map operation occur inside a thread requisitioned from the thread pool managed by the execution context?
This is just for me to better understand why use of Future.successful and Future.failed is considered better style than directly wrapping some code with Future(). To repeat what I said above, I already know that throwing exceptions is a performance bottleneck, but does use of Future.successful and Future.failed avoid requesting a thread from the thread pool completely and thereby reduce the performance overhead? Is this completely irrelevant since mapping or flatmapping a Future has to be scheduled on a thread from the thread pool anyway?
Future.successful and Future.failed do not require an execution context, therefore no thread is consumed by them. However calling map/flatMap/foreach on them does require an execution context for the callback passed in, therefore a thread is indeed consumed. More than performance considerations, calling Future.successful(v) rather than Future(v) is appropriate to semantically indicate that we already have the value v and we are just lifting it into a Future. Performance gain from Future.successful(1+1) as opposed to Future(1+1) is likely negligible in a real-world system where true bottleneck will be slow I/O such as remote API calls, database lookups, etc.
In Documentation, Dart is Single Threaded but to perform two operations at a time we use future objects which work same as thread.
Use Future objects (futures) to perform asynchronous operations.
If Dart is single threaded then why it allows to perform asynchronous operations.
Note: Asynchronous operations are parallel operations which are called threads
You mentioned that :
Asynchronous operations are parallel operations which are called threads
First of all, Asynchronous operations are not exactly parallel or even concurrent. Its just simply means that we do not want to block our flow of execution(Thread) or wait for the response until certain work is done. But the way we implement Asynchronous operations could decide either it is parallel or concurrent.
Parallellism vs Concurrency ?
Parallelism is actually doing lots of things simultaneously at the
same time. ex - You are walking and at the same time you're digesting
you food. Both tasks are completely running parallel and exactly at the
same time.
While
Concurrency is the illusion of Parallelism.Tasks seems to be Executed
parallel but they aren't. It like handing lots of things at a time but
only doing one task at a specific time. ex - You are walking and suddenly stop to tie your show lace. After tying your shoe lace you again start walking.
Now coming to Dart, Future Objects along with async and await keywords are used to perform asynchronous task. Here asynchronous doesn't means that tasks will be executed parallel or concurrent to each other. Instead in Dart even the asynchronous task is executed on the same thread which means that while we wait for another task to be completed, we will continue executing our synchronous code . Future Objects are used to represent the result of task which will be done at some time in future.
If you want to really execute your task concurrently then consider using Isolates(Which runs in separate thread and doesn't shares it memory with the main thread(or spawning thread).
Why? Because it is a necessity. Some operations, like http requests or timers, are asynchronous in nature.
There are isolates which allow you to execute code in a different process. The difference to threads in other programming languages is that isolates do not share memory with each other (which would lead to concurrency issues), they only communicate through messages.
To receive these messages (or wrapped in a Future, the result of it), Dart uses an event loop.
The Event Loop and Dart
Are Futures in Dart threads?
Dart is single threaded, but it can call native code(like c/c++) to perform asynchronous operations, which can introduce new thread.
In Flutter, Flutter engine is implement in c++, which provide the low-level implementation of Flutter’s core API, including asynchronous tasks like file and network I/O through new thread underneath.
Like Dart, JavaScript is also single threaded, I find this video very helpful to understand "Single Threaded" thing. what the heck is event loop
Here are a few notes:
Asynchronous doesn't mean multi-threaded. It means the code is not run at the same time. Usually asyncronous just means that it is scheduled to be run on the same thread (Isolate) after other tasks have finished.
Dart isn't actually single threaded. You can create another thread by creating another Isolate. However, within an Isolate the Dart code runs on a single thread and separate Isolates don't share memory. They can only communicate by messages.
A Future says that a value (or an error) will be returned at some point in the future. It doesn't say which thread the work is done on. Most futures are done on the current Isolate, but some futures (IO, for example) can be done on separate threads.
See this answer for links to more resources.
I have an article explaining this https://medium.com/#truongsinh/flutter-dart-async-concurrency-demystify-1cc739aaae57
In short, Flutter/Dart is not technically single-threaded, even though Dart code is executed in a single thread. Dart is a concurrent language with message passing pattern, that can take full advantage of modern multi-core architecture, without worrying about lock or mutex. Blocking in Dart can be either I/O-bound or CPU-bound, which should be solved, respectively, by Future and Dart’s Isolate/Flutter’s compute.
Here are the definitions by Wikipedia:
Asynchrony, in computer programming, refers to the occurrence of events independently of the main program flow and ways to deal with such events. These may be "outside" events such as the arrival of signals, or actions instigated by a program that take place concurrently with program execution, without the program blocking to wait for results.
And:
Concurrent computing is a form of computing in which several computations are executed during overlapping time periods—concurrently—instead of sequentially (one completing before the next starts).
In the context of single-threaded computation, do 'asynchronous', 'non-blocking', and 'concurrent' imply one another?
If not, could you give me a counter-example?
Note that I have excluded the word 'parallel' as it implies multiple threads.
Non-blocking operations are based on two approaches:
by simply returning without data (when no data is available - in such cases the caller has to "come back" by itself and "read" again)
by using callbacks. In that context "blocking" means that you wait for an operation to reach a certain state - whereas "non-blocking" means that you trigger the operation - and when that state is reached, you are notified.
Please note: both options do not imply concurrency or multiple threads on the client side. You absolutely can implement such a system using a single process ( think coroutines or node.js for example ).
In that sense: a non-blocking operation is always asynchronous - as you don't know when it will have results for you - or when it will call you back. Both concepts can be be implemented using concurrency, but there is absolute need for doing it that way.
Non-blocking and concurrent don't really apply to single threaded programs, due to the fact that they refer to ways of managing multiple threads. Non-blocking means that a program doesn't wait for all threads to finish before moving on, and concurrent computation can only happen if you have multiple threads doing the calculation. (Someone please correct me if I'm wrong.)
Asynchrony is the only term that applies to single threaded programming, in the form of human input, communication with other programs, etc. Because of this, no, they don't imply each other in the context of single threaded programs.
Looked at a few other questions but didn't quite find what I was looking for. Im using Scala but my questions is very high level and so is hopefully agnostic of any languages.
A regular scenario:
Thread A runs a function and there is some blocking work to be done (say a DB call).
The function has some non-blocking code (eg. Async block in Scala) to cause some sort of 'worker' Thread B (in a different pool) to pick up the I/O task.
The method in Thread A completes returning a Future which will eventually contain the result and Thread A is returned to its pool to quickly pick up another request to process.
Q1. Some thread somewhere usually has to wait?
My understanding of non-blocking architectures is that the common approach is to still have some Thread waiting/blocking on the I/O work somewhere - its just a case of having different pools which have access to different cores so that a small number of request processing threads can manage a large number of concurrent requests without ever waiting on a CPU core.
Is this a correct general understanding?
Q2. How the callback works ?
In the above scenario - Thread B that is doing the I/O work will run the callback function (provided by Thread A) if/when the I/O work has completed - which completes the Future with some Result.
Thread A is now off doing something else and has no association any more with the original request. How does the Result in the Future get sent back to the client socket? I understand that different languages have different implementations of such a mechanism but at a high level my current assumption is that (regardless of the language/framework) some framework/container objects must always be doing some sort of orchestration so that when a Future task is completed the Result gets sent back to the original socket handling the request.
I have spent hours trying to find articles which will explain this but every article seems to just deal with real low-level details. I know Im missing some details but i am having difficulty asking my question because Im not quite sure which parts Im missing :)
My understanding of non-blocking architectures is that the common approach is to still have some Thread waiting/blocking on the I/O work somewhere
If a thread is getting blocked somewhere, it is not really a non-blocking architecture. So no, that's not really a correct understanding of it. That doesn't mean that this is necessarily bad. Sometimes you just have to deal with blocking (using JDBC, for example). It would be better to push it off into a fixed thread pool designated for blocking, rather than allowing the entire application to suffer thread starvation.
Thread A is now off doing something else and has no association any more with the original request. How does the Result in the Future get sent back to the client socket?
Using Futures, it really depends on the ExecutionContext. When you create a Future, where the work is done depends on the ExecutionContext.
val f: Future[?] = ???
val g: Future[?] = ???
f and g are created immediately, and the work is submitted to a task queue in the ExecutionContext. We cannot guarantee which will actually execute or complete first in most cases. What you do with the values matters is well. Obviously if you use an Await to wait for the completion of the Futures, then we block the current thread. If we map them and do something with the values, then we again need another ExecutionContext to submit the task to. This gives us a chain of tasks that are asynchronously getting submitted and re-submitted to the executor for execution every time we manipulate the Future.
Eventually there needs to be some onComplete at the end of that chain to return the pass along that value to something, whether it's writing to stream, or something else. ie., it is probably out of the hands of the original thread.
Q1: No, at least not at the user code level. Hopefully your async I/O ultimately comes down to an async kernel API (e.g. select()). Which in turn will be using DMA to do the I/O and trigger an interrupt when it's done. So it's async at least down to the hardware level.
Q2: Thread B completes the Future. If you're using something like onComplete, then thread B will trigger that (probably by creating a new task and handing that task off to a thread pool to pick it up later) as part of the completing call. If a different thread has called Await to block on the Future, it will trigger that thread to resume. If nothing has accessed the Future yet, nothing in particular happens - the value sits there in the Future until something uses it. (See PromiseCompletingRunnable for the gritty details - it's surprisingly readable).