I have seen several comments to the effect that Executors are better than Threads, but if you have a number of Threads communicating via bounded buffers (as in Flow-Based Programming) why would you use Executors when you have to use Threads anyway (with newCachedThreadPool (?)). Also, I use methods like isAlive(), interrupt() - how do I get hold of the Thread handle?
Does anyone have sample code that I can plagiarize? ;-)
Executors are basically an abstraction over Threads. They make you isolate your potentially parallel logic in Runnable/Callable instances while liberating you from the duties of manually creating and starting a thread or managing a pool. You still need to handle dependencies as part of your application logic.
If you want to interact / are comfortable with Threads for your application logic, you may skip using Executors. Regarding getting hold of the thread, you can always execute Thread.currentThread() to get hold of the current thread from any executing context.
Related
Reading about ReaactiveX(like here), it states something like:
An advantage of this approach is that when you have a bunch of tasks that are not dependent on each
other, you can start them all at the same time rather than waiting for each one to finish before
starting the next one — that way, your entire bundle of tasks only takes as long to complete as the
longest task in the bundle.
Are not we all doing this already using multi threading programming? So how are two things different actually?
This is a broader topic about light-weight async tasks in general vs threads.
A big difference is in cost and speed. Threads are expensive, and OSs generally limit the number that you can create. Every thread has room for an entire full stack set aside in case it's needed (you get a StackOverflow if it wasn't enough). If you have more threads than processors, then switching tasks means saving off all the current thread info and loading the new thread into registers, etc.
ReactiveX libraries work with callbacks, so the only memory needed is the object with the callback data. Switching ReactiveX tasks is just a method call.
You can have many millions of ReactiveX tasks in progress at once, not so much with threads.
Most slow tasks (like file or network IO) actually do a lot of waiting. Why allocate an entire thread just to do nothing but wait?
With ReactiveX, the tasks are just simple objects that are waiting, just sitting in a queue.
Now, ReactiveX is built on top of threading. Those millions of tasks (just callback object in memory) when actually running are running on some thread. And ReactiveX, tasks aren't all really "running" at the same time (only a thread running on a core can actually do something). Most tasks do a lot of waiting so really those millions of ReactiveX tasks, are really just all "waiting" at the same time by hanging out in a queue.
Also, consider a scenario like Javascript, which is a single threaded environment. Multi-threading just isn't an option there. Even if you can create threads, avoiding concurrency or simplifying UI code that needs thread affinity can be nice when many tasks are all managed on a single thread.
Even in multiple thread scenarios, ReactiveX can be really helpful since it's API guarantees synchronous event callback for a particular stream, even if that steam is using many threads to generate the data.
I'm trying to make a thread pool for a game engine and I've been considering how my system should react to third party libraries spawning their own threads.
From what I've read, it is ideal to only have one thread for each CPU you have access to. So if my third party physics update spawns four threads, it would be ideal to turn off four threads from my thread pool while it is running, then turn them back on afterwards, that way multiple threads are never contending over one CPU.
My question is about the underlying mechanics behind functionality like conditional variables. Since spawning threads is expensive, having four threads wait on a conditional variable and then notifying them when the physics is done seems like a much better option than joining four threads and re-spawning them afterwards. But if they are waiting on a variable, are the threads truly "asleep" or are they still contending for CPU resources in the background?
Although you did not write what platform you are programming on, in most implementations threads that are waiting consume little to no CPU resources.
They do however use some memory (to save the stack, etc.), so you should avoid spawning an excessive number of threads and trying to reuse them as much as possible, since as you noted, spawning a new thread is an expensive operation on most platforms.
Even though you did not provide a lot of information, I'm guessing that in your scenario letting the threads wait is a much better option, as a small number of threads will not use a lot of resources and possibly having to spawn new threads frequently will affect performance badly on almost all platforms.
Any good third party library should give you the option of running it's work through your thread pool, to avoid that problem in the first place.
For example here's the documentation on how you can do that with PhysX - https://developer.nvidia.com/sites/default/files/akamai/physx/Docs/TaskManager.html
I overheard a coworker saying that a Task is basically a lightweight thread. Coming from a C++ background (where threads where the lightest weight processing unit), this seems counter-intuitive to me.
Aren't Tasks just as heavy as Threads?
You need to distinguish between a unit of work (Tasks) from the underlying process used to host/execute them. It isn't even necessary for Tasks to run on other threads. For example, Tasks can be executed in a single threaded application that periodically yields control to the task pool.
Even when Tasks are executed on separate threads, there is usually not a 1 to 1 relationship between Task and Thread. The threads are preallocated as part of a pool, and then tasks are scheduled to run on these threads as available. Creating a new task does not require the overhead of creating a thread, it only requires the cost of an enque in a task queue.
This makes tasks inherently more scalable. I can have millions of tasks throughout the lifetime of my application, but only ever actually use some constant number of threads.
Typically a "thread" implies mandatory concurrency. Starting up a thread requires allocating a stack and internal OS data structures for it. In contrast, a "task" often refers to a piece of work for which concurrency is optional, hence a parallel framework (such as OpenMP, Cilk Plus, TBB, PPL) can use the same thread to execute many tasks, by serializing the tasks, and converting optional parallelism to real parallelism only as necessary to keep the machine busy.
You are right - everything runs on a thread under the covers.
The reason people say that a Task is more lightweight than a Thread is that Microsoft put a lot of thought into having Tasks make efficient use of Threads, and the implementation is probably much lighter weight than what the average developer would come up with on their own using the Thread class.
EDIT
A more clear explanation is that a Task object is lighter weight than a Thread object, and while each Task is eventually run on a Thread, creating N Task objects concurrently leads to less than N concurrent Thread objects being used, for large N.
I don't quite understand the difference between threads and lightweight threads. From an API perspective both types of threads are identical so where exactly does the difference come in. Is it at the implementation level where a lightweight thread is managed by a higher level runtime than the OS thread scheduler or is it something else? Also, is there set of heuristics that people use to decide which type of thread to use in specific scenarios?
In what context, lightweight threads could represent threads which are implemented by a library, for example threads can be simulated in a library by switching between lightweight threads at an event handling layer, these lightweight threads are queued up and processed by a singe OS thread, the advantage of this is that since context switching is handled in the library switching can occur when the processing of data is complete and so the data does not need to be loaded back into the CPU's cache next time this lightweight thread becomes active.
Lightweight threads could also refer to co-operative threads (or fibers), these are threads where you have to explicitly yield to give other lightweight threads a chance, this has the same advantage in that the context switching can occur at a place you know you have finished processing some data and so you know it will not be need again.
Alternativly Lightweight threads could mean normal OS threads and the non-lightweight threads could mean processes, process have at least one thread within them and also have there own memory and other resources, they are more expensive than threads because you can not share data between thread easily and it can be a more expensive operation for the OS to create processes.
Can someone list some comparison points between Thread Spawning vs Thread Pooling, which one is better? Please consider the .NET framework as a reference implementation that supports both.
Thread pool threads are much cheaper than a regular Thread, they pool the system resources required for threads. But they have a number of limitations that may make them unfit:
You cannot abort a threadpool thread
There is no easy way to detect that a threadpool completed, no Thread.Join()
There is no easy way to marshal exceptions from a threadpool thread
You cannot display any kind of UI on a threadpool thread beyond a message box
A threadpool thread should not run longer than a few seconds
A threadpool thread should not block for a long time
The latter two constraints are a side-effect of the threadpool scheduler, it tries to limit the number of active threads to the number of cores your CPU has available. This can cause long delays if you schedule many long running threads that block often.
Many other threadpool implementations have similar constraints, give or take.
A "pool" contains a list of available "threads" ready to be used whereas "spawning" refers to actually creating a new thread.
The usefulness of "Thread Pooling" lies in "lower time-to-use": creation time overhead is avoided.
In terms of "which one is better": it depends. If the creation-time overhead is a problem use Thread-pooling. This is a common problem in environments where lots of "short-lived tasks" need to be performed.
As pointed out by other folks, there is a "management overhead" for Thread-Pooling: this is minimal if properly implemented. E.g. limiting the number of threads in the pool is trivial.
For some definition of "better", you generally want to go with a thread pool. Without knowing what your use case is, consider that with a thread pool, you have a fixed number of threads which can all be created at startup or can be created on demand (but the number of threads cannot exceed the size of the pool). If a task is submitted and no thread is available, it is put into a queue until there is a thread free to handle it.
If you are spawning threads in response to requests or some other kind of trigger, you run the risk of depleting all your resources as there is nothing to cap the amount of threads created.
Another benefit to thread pooling is reuse - the same threads are used over and over to handle different tasks, rather than having to create a new thread each time.
As pointed out by others, if you have a small number of tasks that will run for a long time, this would negate the benefits gained by avoiding frequent thread creation (since you would not need to create a ton of threads anyway).
My feeling is that you should start just by creating a thread as needed... If the performance of this is OK, then you're done. If at some point, you detect that you need lower latency around thread creation you can generally drop in a thread pool without breaking anything...
All depends on your scenario. Creating new threads is resource intensive and an expensive operation. Most very short asynchronous operations (less than a few seconds max) could make use of the thread pool.
For longer running operations that you want to run in the background, you'd typically create (spawn) your own thread. (Ab)using a platform/runtime built-in threadpool for long running operations could lead to nasty forms of deadlocks etc.
Thread pooling is usually considered better, because the threads are created up front, and used as required. Therefore, if you are using a lot of threads for relatively short tasks, it can be a lot faster. This is because they are saved for future use and are not destroyed and later re-created.
In contrast, if you only need 2-3 threads and they will only be created once, then this will be better. This is because you do not gain from caching existing threads for future use, and you are not creating extra threads which might not be used.
It depends on what you want to execute on the other thread.
For short task it is better to use a thread pool, for long task it may be better to spawn a new thread as it could starve the thread pool for other tasks.
The main difference is that a ThreadPool maintains a set of threads that are already spun-up and available for use, because starting a new thread can be expensive processor-wise.
Note however that even a ThreadPool needs to "spawn" threads... it usually depends on workload - if there is a lot of work to be done, a good threadpool will spin up new threads to handle the load based on configuration and system resources.
There is little extra time required for creating/spawning thread, where as thread poll already contains created threads which are ready to be used.
This answer is a good summary but just in case, here is the link to Wikipedia:
http://en.wikipedia.org/wiki/Thread_pool_pattern
For Multi threaded execution combined with getting return values from the execution, or an easy way to detect that a threadpool has completed, java Callables could be used.
See https://blogs.oracle.com/CoreJavaTechTips/entry/get_netbeans_6 for more info.
Assuming C# and Windows 7 and up...
When you create a thread using new Thread(), you create a managed thread that becomes backed by a native OS thread when you call Start – a one to one relationship. It is important to know only one thread runs on a CPU core at any given time.
An easier way is to call ThreadPool.QueueUserWorkItem (i.e. background thread), which in essence does the same thing, except those background threads aren’t forever tied to a single native thread. The .NET scheduler will simulate multitasking between managed threads on a single native thread. With say 4 cores, you’ll have 4 native threads each running multiple managed threads, determined by .NET. This offers lighter-weight multitasking since switching between managed threads happens within the .NET VM not in the kernel. There is some overhead associated with crossing from user mode to kernel mode, and the .NET scheduler minimizes such crossing.
It may be important to note that heavy multitasking might benefit from pure native OS threads in a well-designed multithreading framework. However, the performance benefits aren’t that much.
With using the ThreadPool, just make sure the minimum worker thread count is high enough or ThreadPool.QueueUserWorkItem will be slower than new Thread(). In a benchmark test looping 512 times calling new Thread() left ThreadPool.QueueUserWorkItem in the dust with default minimums. However, first setting the minimum worker thread count to 512, in this test, made new Thread() and ThreadPool.QueueUserWorkItem perform similarly.
A side effective of setting a high worker thread count is that new Task() (or Task.Factory.StartNew) also performed similarly as new Thread() and ThreadPool.QueueUserWorkItem.