When I call par on collections, it seems to create about 5-10 threads, which is fine for CPU bound tasks.
But sometimes I have tasks which are IO bound, in which case I'd like to have 500-1000 threads pulling from IO concurrently - doing 10-15 threads is very slow and I see my CPUs mostly sitting idle.
How can I achieve this?
You could wrap your blocking io operations in blocking block:
(0 to 1000).par.map{ i =>
blocking {
Thread.sleep(100)
Thread.activeCount()
}
}.max // yield 67 on my pc, while without blocking it's 10
But you should ask yourself a question if you should use parallel collections for IO operations. Their use case is to perform a CPU heavy task.
I would suggest you to consider using futures for IO calls.
You should also consider using a custom execution context for that task because the global execution context is a public singleton and you don't have control what code uses it and for which purpose. You could easily starve parallel computations created by external libraries if you used all threads from it.
// or just use scala.concurrent.ExecutionContext.Implicits.global if you don't care
implicit val blockingIoEc: ExecutionContextExecutor = ExecutionContext.fromExecutor(
Executors.newCachedThreadPool()
)
def fetchData(index: Int): Future[Int] = Future {
//if you use global ec, then it's required to mark computation as blocking to increase threads,
//if you use custom cached thread pool it should increase thread number even without it
blocking {
Thread.sleep(100)
Thread.activeCount()
}
}
val futures = (0 to 1000).map(fetchData)
Future.sequence(futures).onComplete {
case Success(data) => println(data.max) //prints about 1000 on my pc
}
Thread.sleep(1000)
EDIT
There is also a possibility to use custom ForkJoinPool using ForkJoinTaskSupport:
import java.util.concurrent.ForkJoinPool //scala.concurrent.forkjoin.ForkJoinPool is deprecated
import scala.util.Random
import scala.collection.parallel
val fjpool = new ForkJoinPool(2)
val customTaskSupport = new parallel.ForkJoinTaskSupport(fjpool)
val numbers = List(1,2,3,4,5).par
numbers.tasksupport = customTaskSupport //assign customTaskSupport
Let's say I have a stateful client smth like
trait Client {
case class S(connection: Connection, failures: Int) // Connection takes 10 seconds to create
def fetchData: State[S, Data] // Takes 10 seconds to fetch
}
I want to make it stateful because the creation of connection is expensive (so I want to cache it) + I have failures count to signal if there are too many failures in a row, then basically recreate connection.
What I've understood from State monad is that effectively it should be calculated on one thread, chaining immutable states sequentially on that thread. I can't afford it in this case, because I fetch operation takes huge amount of time, while all I need is quickly read the state, use the connection from there to start expensive async call. On the other hand, I can't make it State[S, Task[Data]], bacause I need to modify failures in S if the task fetchData fails. So I modified client to be
import fs2.Task
trait Client {
case class S(connection: Connection, failures: Int) // Connection takes 10 seconds to create
def fetchData: StateT[Task, S, Data] // Takes 10 seconds to fetch
}
The thing is I can come up with some Monoid that can add the states irrelevant if what was the initial state. E.g. If I have S1 -> (S2, Data), S1 -> (S3, Data), I could still add S2 and S3 to arrive at the final state regardless of what came before - S2 or S3.
Now I have a kind of Java server (Thrift) that use the handler. I haven't dig into details of how it working precisely yet, but let's assume that it listens for incoming connections and then spawns a thread on which it handles the data retrieval. So in my case I want to block the thread until Task is finished:
class Server(client: Client) {
def handle: Data {
run(client.fetchData)
}
private var state: S = Monoid[S].mempty
private def run(st: State[S, Data]): Data {
val resTask: Task[(S, Data)] = st.run(state)
val (newState, res) = Await.result(resTask.unsafeRunAsyncFuture)
state.synchonized({
state = Monoid[S].mappend(state, newState)
})
res
}
}
My questions are:
1) Am I doing it the right / best way here (my focus is efficiency / no bugs)
2) Do I really need State monad here, or it's better to use ordinary var and work with it?
3) In the examples of State monad I always saw that state is propagated to the top level of the app and handled there. But maybe it's better to handle it on Client level and expose stateless interface to the app?
4) Is this code thread safe, or I should make state some sort of synchronized var? I know that in Objective C this line state = Monoid[S].mappend(state, newState) could crash if in the middle of assignment the state was read from another thread.
I have a function that boils down to:
while(doWork)
{
config = generateConfigurationForTesting();
result = executeWork(config);
doWork = isDone(result);
}
How can I rewrite this for efficient asynchronous execution, assuming all functions are thread safe, independent of previous iterations, and probably require more iterations than the maximum number of allowable threads ?
The problem here is we don't know how many iterations are required in advance so we can't make a dispatch_group or use dispatch_apply.
This is my first attempt, but it looks a bit ugly to me because of arbitrarily chosen values and sleeping;
int thread_count = 0;
bool doWork = true;
int max_threads = 20; // arbitrarily chosen number
dispatch_queue_t queue =
dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
while(doWork)
{
if(thread_count < max_threads)
{
dispatch_async(queue, ^{ Config myconfig = generateConfigurationForTesting();
Result myresult = executeWork();
dispatch_async(queue, checkResult(myresult)); });
thread_count++;
}
else
usleep(100); // don't consume too much CPU
}
void checkResult(Result value)
{
if(value == good) doWork = false;
thread_count--;
}
Based on your description, it looks like generateConfigurationForTesting is some kind of randomization technique or otherwise a generator which can make a near-infinite number of configuration (hence your comment that you don't know ahead of time how many iterations you will need). With that as an assumption, you are basically stuck with the model that you've created, since your executor needs to be limited by some reasonable assumptions about the queue and you don't want to over-generate, as that would just extend the length of the run after you have succeeded in finding value ==good measurements.
I would suggest you consider using a queue (or OSAtomicIncrement* and OSAtomicDecrement*) to protect access to thread_count and doWork. As it stands, the thread_count increment and decrement will happen in two different queues (main_queue for the main thread and the default queue for the background task) and thus could simultaneously increment and decrement the thread count. This could lead to an undercount (which would cause more threads to be created than you expect) or an overcount (which would cause you to never complete your task).
Another option to making this look a little nicer would be to have checkResult add new elements into the queue if value!=good. This way, you load up the initial elements of the queue using dispatch_apply( 20, queue, ^{ ... }) and you don't need the thread_count at all. The first 20 will be added using dispatch_apply (or an amount that dispatch_apply feels is appropriate for your configuration) and then each time checkResult is called you can either set doWork=false or add another operation to queue.
dispatch_apply() works for this, just pass ncpu as the number of iterations (apply never uses more than ncpu worker threads) and keep each instance of your worker block running for as long as there is more work to do (i.e. loop back to generateConfigurationForTesting() unless !doWork).
How can I limit number of threads that are being executed at the same time?
Here is sample of my algorithm:
for(i = 0; i < 100000; i++) {
Thread.start {
// Do some work
}
}
I would like to make sure that once number of threads in my application hits 100, algorithm will pause/wait until number of threads in the app goes below 100.
Currently "some work" takes some time to do and I end up with few thousands of threads in my app. Eventually it runs out of threads and "some work" crashes. I would like to fix it by limiting number of pools that it can use at one time.
Please let me know how to solve my issue.
I believe you are looking for a ThreadPoolExecutor in the Java Concurrency API. The idea here is that you can define a maximum number of threads in a pool and then instead of starting new Threads with a Runnable, just let the ThreadPoolExecutor take care of managing the upper limit for Threads.
Start here: http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/ThreadPoolExecutor.html
import java.util.concurrent.*;
import java.util.*;
def queue = new ArrayBlockingQueue<Runnable>( 50000 )
def tPool = new ThreadPoolExecutor(5, 500, 20, TimeUnit.SECONDS, queue);
for(i = 0; i < 5000; i++) {
tPool.execute {
println "Blah"
}
}
Parameters for the ThreadBlockingQueue constructor: corePoolSize (5), this is the # of threads to create and to maintain if the system is idle, maxPoolSize (500) max number of threads to create, 3rd and 4th argument states that the pool should keep idle threads around for at least 20 seconds, and the queue argument is a blocking queue that stores queued tasks.
What you'll want to play around with is the queue sizes and also how to handle rejected tasks. If you need to execute 100k tasks, you'll either have to have a queue that can hold 100k tasks, or you'll have to have a strategy for handling a rejected tasks.
When writing multithreaded applications, one of the most common problems experienced is race conditions.
My questions to the community are:
What is the race condition?
How do you detect them?
How do you handle them?
Finally, how do you prevent them from occurring?
A race condition occurs when two or more threads can access shared data and they try to change it at the same time. Because the thread scheduling algorithm can swap between threads at any time, you don't know the order in which the threads will attempt to access the shared data. Therefore, the result of the change in data is dependent on the thread scheduling algorithm, i.e. both threads are "racing" to access/change the data.
Problems often occur when one thread does a "check-then-act" (e.g. "check" if the value is X, then "act" to do something that depends on the value being X) and another thread does something to the value in between the "check" and the "act". E.g:
if (x == 5) // The "Check"
{
y = x * 2; // The "Act"
// If another thread changed x in between "if (x == 5)" and "y = x * 2" above,
// y will not be equal to 10.
}
The point being, y could be 10, or it could be anything, depending on whether another thread changed x in between the check and act. You have no real way of knowing.
In order to prevent race conditions from occurring, you would typically put a lock around the shared data to ensure only one thread can access the data at a time. This would mean something like this:
// Obtain lock for x
if (x == 5)
{
y = x * 2; // Now, nothing can change x until the lock is released.
// Therefore y = 10
}
// release lock for x
A "race condition" exists when multithreaded (or otherwise parallel) code that would access a shared resource could do so in such a way as to cause unexpected results.
Take this example:
for ( int i = 0; i < 10000000; i++ )
{
x = x + 1;
}
If you had 5 threads executing this code at once, the value of x WOULD NOT end up being 50,000,000. It would in fact vary with each run.
This is because, in order for each thread to increment the value of x, they have to do the following: (simplified, obviously)
Retrieve the value of x
Add 1 to this value
Store this value to x
Any thread can be at any step in this process at any time, and they can step on each other when a shared resource is involved. The state of x can be changed by another thread during the time between x is being read and when it is written back.
Let's say a thread retrieves the value of x, but hasn't stored it yet. Another thread can also retrieve the same value of x (because no thread has changed it yet) and then they would both be storing the same value (x+1) back in x!
Example:
Thread 1: reads x, value is 7
Thread 1: add 1 to x, value is now 8
Thread 2: reads x, value is 7
Thread 1: stores 8 in x
Thread 2: adds 1 to x, value is now 8
Thread 2: stores 8 in x
Race conditions can be avoided by employing some sort of locking mechanism before the code that accesses the shared resource:
for ( int i = 0; i < 10000000; i++ )
{
//lock x
x = x + 1;
//unlock x
}
Here, the answer comes out as 50,000,000 every time.
For more on locking, search for: mutex, semaphore, critical section, shared resource.
What is a Race Condition?
You are planning to go to a movie at 5 pm. You inquire about the availability of the tickets at 4 pm. The representative says that they are available. You relax and reach the ticket window 5 minutes before the show. I'm sure you can guess what happens: it's a full house. The problem here was in the duration between the check and the action. You inquired at 4 and acted at 5. In the meantime, someone else grabbed the tickets. That's a race condition - specifically a "check-then-act" scenario of race conditions.
How do you detect them?
Religious code review, multi-threaded unit tests. There is no shortcut. There are few Eclipse plugin emerging on this, but nothing stable yet.
How do you handle and prevent them?
The best thing would be to create side-effect free and stateless functions, use immutables as much as possible. But that is not always possible. So using java.util.concurrent.atomic, concurrent data structures, proper synchronization, and actor based concurrency will help.
The best resource for concurrency is JCIP. You can also get some more details on above explanation here.
There is an important technical difference between race conditions and data races. Most answers seem to make the assumption that these terms are equivalent, but they are not.
A data race occurs when 2 instructions access the same memory location, at least one of these accesses is a write and there is no happens before ordering among these accesses. Now what constitutes a happens before ordering is subject to a lot of debate, but in general ulock-lock pairs on the same lock variable and wait-signal pairs on the same condition variable induce a happens-before order.
A race condition is a semantic error. It is a flaw that occurs in the timing or the ordering of events that leads to erroneous program behavior.
Many race conditions can be (and in fact are) caused by data races, but this is not necessary. As a matter of fact, data races and race conditions are neither the necessary, nor the sufficient condition for one another. This blog post also explains the difference very well, with a simple bank transaction example. Here is another simple example that explains the difference.
Now that we nailed down the terminology, let us try to answer the original question.
Given that race conditions are semantic bugs, there is no general way of detecting them. This is because there is no way of having an automated oracle that can distinguish correct vs. incorrect program behavior in the general case. Race detection is an undecidable problem.
On the other hand, data races have a precise definition that does not necessarily relate to correctness, and therefore one can detect them. There are many flavors of data race detectors (static/dynamic data race detection, lockset-based data race detection, happens-before based data race detection, hybrid data race detection). A state of the art dynamic data race detector is ThreadSanitizer which works very well in practice.
Handling data races in general requires some programming discipline to induce happens-before edges between accesses to shared data (either during development, or once they are detected using the above mentioned tools). this can be done through locks, condition variables, semaphores, etc. However, one can also employ different programming paradigms like message passing (instead of shared memory) that avoid data races by construction.
A sort-of-canonical definition is "when two threads access the same location in memory at the same time, and at least one of the accesses is a write." In the situation the "reader" thread may get the old value or the new value, depending on which thread "wins the race." This is not always a bug—in fact, some really hairy low-level algorithms do this on purpose—but it should generally be avoided. #Steve Gury give's a good example of when it might be a problem.
A race condition is a situation on concurrent programming where two concurrent threads or processes compete for a resource and the resulting final state depends on who gets the resource first.
A race condition is a kind of bug, that happens only with certain temporal conditions.
Example:
Imagine you have two threads, A and B.
In Thread A:
if( object.a != 0 )
object.avg = total / object.a
In Thread B:
object.a = 0
If thread A is preempted just after having check that object.a is not null, B will do a = 0, and when thread A will gain the processor, it will do a "divide by zero".
This bug only happen when thread A is preempted just after the if statement, it's very rare, but it can happen.
Many answers in this discussion explains what a race condition is. I try to provide an explaination why this term is called race condition in software industry.
Why is it called race condition?
Race condition is not only related with software but also related with hardware too. Actually the term was initially coined by the hardware industry.
According to wikipedia:
The term originates with the idea of two signals racing each other to
influence the output first.
Race condition in a logic circuit:
Software industry took this term without modification, which makes it a little bit difficult to understand.
You need to do some replacement to map it to the software world:
"two signals" ==> "two threads"/"two processes"
"influence the output" ==> "influence some shared state"
So race condition in software industry means "two threads"/"two processes" racing each other to "influence some shared state", and the final result of the shared state will depend on some subtle timing difference, which could be caused by some specific thread/process launching order, thread/process scheduling, etc.
Race conditions occur in multi-threaded applications or multi-process systems. A race condition, at its most basic, is anything that makes the assumption that two things not in the same thread or process will happen in a particular order, without taking steps to ensure that they do. This happens commonly when two threads are passing messages by setting and checking member variables of a class both can access. There's almost always a race condition when one thread calls sleep to give another thread time to finish a task (unless that sleep is in a loop, with some checking mechanism).
Tools for preventing race conditions are dependent on the language and OS, but some comon ones are mutexes, critical sections, and signals. Mutexes are good when you want to make sure you're the only one doing something. Signals are good when you want to make sure someone else has finished doing something. Minimizing shared resources can also help prevent unexpected behaviors
Detecting race conditions can be difficult, but there are a couple signs. Code which relies heavily on sleeps is prone to race conditions, so first check for calls to sleep in the affected code. Adding particularly long sleeps can also be used for debugging to try and force a particular order of events. This can be useful for reproducing the behavior, seeing if you can make it disappear by changing the timing of things, and for testing solutions put in place. The sleeps should be removed after debugging.
The signature sign that one has a race condition though, is if there's an issue that only occurs intermittently on some machines. Common bugs would be crashes and deadlocks. With logging, you should be able to find the affected area and work back from there.
Microsoft actually have published a really detailed article on this matter of race conditions and deadlocks. The most summarized abstract from it would be the title paragraph:
A race condition occurs when two threads access a shared variable at
the same time. The first thread reads the variable, and the second
thread reads the same value from the variable. Then the first thread
and second thread perform their operations on the value, and they race
to see which thread can write the value last to the shared variable.
The value of the thread that writes its value last is preserved,
because the thread is writing over the value that the previous thread
wrote.
What is a race condition?
The situation when the process is critically dependent on the sequence or timing of other events.
For example,
Processor A and processor B both needs identical resource for their execution.
How do you detect them?
There are tools to detect race condition automatically:
Lockset-Based Race Checker
Happens-Before Race Detection
Hybrid Race Detection
How do you handle them?
Race condition can be handled by Mutex or Semaphores. They act as a lock allows a process to acquire a resource based on certain requirements to prevent race condition.
How do you prevent them from occurring?
There are various ways to prevent race condition, such as Critical Section Avoidance.
No two processes simultaneously inside their critical regions. (Mutual Exclusion)
No assumptions are made about speeds or the number of CPUs.
No process running outside its critical region which blocks other processes.
No process has to wait forever to enter its critical region. (A waits for B resources, B waits for C resources, C waits for A resources)
You can prevent race condition, if you use "Atomic" classes. The reason is just the thread don't separate operation get and set, example is below:
AtomicInteger ai = new AtomicInteger(2);
ai.getAndAdd(5);
As a result, you will have 7 in link "ai".
Although you did two actions, but the both operation confirm the same thread and no one other thread will interfere to this, that means no race conditions!
I made a video that explains this.
Essentially it is when you have a state with is shared across multiple threads and before the first execution on a given state is completed, another execution starts and the new thread’s initial state for a given operation is wrong because the previous execution has not completed.
Because the initial state of the second execution is wrong, the resulting computation is also wrong. Because eventually the second execution will update the final state with the wrong result.
You can view it here.
https://youtu.be/RWRicNoWKOY
Here is the classical Bank Account Balance example which will help newbies to understand Threads in Java easily w.r.t. race conditions:
public class BankAccount {
/**
* #param args
*/
int accountNumber;
double accountBalance;
public synchronized boolean Deposit(double amount){
double newAccountBalance=0;
if(amount<=0){
return false;
}
else {
newAccountBalance = accountBalance+amount;
accountBalance=newAccountBalance;
return true;
}
}
public synchronized boolean Withdraw(double amount){
double newAccountBalance=0;
if(amount>accountBalance){
return false;
}
else{
newAccountBalance = accountBalance-amount;
accountBalance=newAccountBalance;
return true;
}
}
public static void main(String[] args) {
// TODO Auto-generated method stub
BankAccount b = new BankAccount();
b.accountBalance=2000;
System.out.println(b.Withdraw(3000));
}
Try this basic example for better understanding of race condition:
public class ThreadRaceCondition {
/**
* #param args
* #throws InterruptedException
*/
public static void main(String[] args) throws InterruptedException {
Account myAccount = new Account(22222222);
// Expected deposit: 250
for (int i = 0; i < 50; i++) {
Transaction t = new Transaction(myAccount,
Transaction.TransactionType.DEPOSIT, 5.00);
t.start();
}
// Expected withdrawal: 50
for (int i = 0; i < 50; i++) {
Transaction t = new Transaction(myAccount,
Transaction.TransactionType.WITHDRAW, 1.00);
t.start();
}
// Temporary sleep to ensure all threads are completed. Don't use in
// realworld :-)
Thread.sleep(1000);
// Expected account balance is 200
System.out.println("Final Account Balance: "
+ myAccount.getAccountBalance());
}
}
class Transaction extends Thread {
public static enum TransactionType {
DEPOSIT(1), WITHDRAW(2);
private int value;
private TransactionType(int value) {
this.value = value;
}
public int getValue() {
return value;
}
};
private TransactionType transactionType;
private Account account;
private double amount;
/*
* If transactionType == 1, deposit else if transactionType == 2 withdraw
*/
public Transaction(Account account, TransactionType transactionType,
double amount) {
this.transactionType = transactionType;
this.account = account;
this.amount = amount;
}
public void run() {
switch (this.transactionType) {
case DEPOSIT:
deposit();
printBalance();
break;
case WITHDRAW:
withdraw();
printBalance();
break;
default:
System.out.println("NOT A VALID TRANSACTION");
}
;
}
public void deposit() {
this.account.deposit(this.amount);
}
public void withdraw() {
this.account.withdraw(amount);
}
public void printBalance() {
System.out.println(Thread.currentThread().getName()
+ " : TransactionType: " + this.transactionType + ", Amount: "
+ this.amount);
System.out.println("Account Balance: "
+ this.account.getAccountBalance());
}
}
class Account {
private int accountNumber;
private double accountBalance;
public int getAccountNumber() {
return accountNumber;
}
public double getAccountBalance() {
return accountBalance;
}
public Account(int accountNumber) {
this.accountNumber = accountNumber;
}
// If this method is not synchronized, you will see race condition on
// Remove syncronized keyword to see race condition
public synchronized boolean deposit(double amount) {
if (amount < 0) {
return false;
} else {
accountBalance = accountBalance + amount;
return true;
}
}
// If this method is not synchronized, you will see race condition on
// Remove syncronized keyword to see race condition
public synchronized boolean withdraw(double amount) {
if (amount > accountBalance) {
return false;
} else {
accountBalance = accountBalance - amount;
return true;
}
}
}
You don't always want to discard a race condition. If you have a flag which can be read and written by multiple threads, and this flag is set to 'done' by one thread so that other thread stop processing when flag is set to 'done', you don't want that "race condition" to be eliminated. In fact, this one can be referred to as a benign race condition.
However, using a tool for detection of race condition, it will be spotted as a harmful race condition.
More details on race condition here, http://msdn.microsoft.com/en-us/magazine/cc546569.aspx.
Consider an operation which has to display the count as soon as the count gets incremented. ie., as soon as CounterThread increments the value DisplayThread needs to display the recently updated value.
int i = 0;
Output
CounterThread -> i = 1
DisplayThread -> i = 1
CounterThread -> i = 2
CounterThread -> i = 3
CounterThread -> i = 4
DisplayThread -> i = 4
Here CounterThread gets the lock frequently and updates the value before DisplayThread displays it. Here exists a Race condition. Race Condition can be solved by using Synchronzation
A race condition is an undesirable situation that occurs when two or more process can access and change the shared data at the same time.It occurred because there were conflicting accesses to a resource . Critical section problem may cause race condition. To solve critical condition among the process we have take out only one process at a time which execute the critical section.