Multithreading - Calculations between all pairs in a set - multithreading

Question

I have n elements (e.g. A, B, C and D) and need to do calculations between all of those.
Calculation 1 = A with B
Calculation 2 = A with C
Calculation 3 = A with D
Calculation 4 = B with C
Calculation 5 = B with D
Calculation 6 = C with D
In reality there are more than 1000 elements and I want to parallelise the process.
Note that I can't access an element from 2 threads simultaneously. This for example makes it impossible to do Calculation 1 and Calculation 2 at the same time because they both use the element A.
Edit: I could access an element from 2 threads but it makes everything very slow if i just split up the calculations and depend on locks for threadsafety.
Is there already an distribution algorithm for these kind of problems?
It seems like a lot of people must have had the same problem already but i couldn't find anything in the great internet. ;)
Single thread example code:
for (int i = 0; i < elementCount; i++)
{
for (int j = i + 1; j < elementCount; j++)
{
Calculate(element[i], element[j]);
}
}

Answer 1

You can apply round-robin tournament algorithm that allows to organize all possible pairs (N*(N-1) results).
All set elements (players) form two rows, column is pair at the
current round. First element is fixed, others are shifted in cyclic manner.
So you can run up to N/2 threads to get results for the first pairs set, then reorder indexes and continue
Excerpt from wiki:
The circle method is the standard algorithm to create a schedule for a round-robin tournament. All competitors are assigned to numbers, and then paired in the first round:
Round 1. (1 plays 14, 2 plays 13, ... )
1 2 3 4 5 6 7
14 13 12 11 10 9 8
then fix one of the contributors in the first or last column of the table (number one in this example) and rotate the others clockwise one position
Round 2. (1 plays 13, 14 plays 12, ... )
1 14 2 3 4 5 6
13 12 11 10 9 8 7
Round 3. (1 plays 12, 13 plays 11, ... )
1 13 14 2 3 4 5
12 11 10 9 8 7 6
until you end up almost back at the initial position
Round 13. (1 plays 2, 3 plays 14, ... )
1 3 4 5 6 7 8
2 14 13 12 11 10 9

Answer 2

It is simple enough to prove there is no way to distribute your calculations so that collisions never occur (that is, unless you manually order the computations and place round-boundaries, like #Mbo suggests), meaning that there is no distribution amongst multiple threads that will allow you to never lock.
Proof :
Given your requirement that any computation involving data object A should happen on a given thread T (only way to make sure you never lock on A).
Then it follows that thread T has to deal with at least one pair containing each other objects (B, C, D) of the input list.
It follows from the basic requirement that T is also to handle everything object-B related. And C. And D. So everything.
Therefore, only T can work.
QED. There is no possible parallelization that will never lock.
Way around #1 : map/reduce
That said... This is a typical case of divide and conquer. You are right that simple additions can require critical section locks, without the order of execution mattering. That is because your critical operation (addition) has a nice property, associativeness : A+(B+C) = (A+B)+C, on top of being commutative.
In other words, this operation is a candidate for a (parralel-friendly) reduce operation.
So the key here is probably :
Emit a stream of all interesting pairs
Map each pair to one or more partial results
Group each partial result by its master object (A, B, C)
Reduce each group by combining the partial results
A sample (pseudo) code
static class Data { int i = 0; }
static class Pair { Data d1; Data d2; }
static class PartialComputation { Data d; int sum; }
Data[] data = ...
Stream<Pair> allPairs = ... // Something like IntStream(0, data.length-1).flatMap(idx -> IntStream(idx+1 to data.length ).map(idx2 -> new Pair(data[idx], data[idx2])))
allPairs.flatMap(pair -> Stream.of(new ParticalComputation(pair.d1, pair.d1.i + pair.d2.i), new PartialComputation(pair.d2, pair.d2.i+pair.d1.i)) // Map everything, parallely, to partial results keyable by the original data object
allPairs.collect(Collectors.groupByParallel(
partialComp -> partialComp.d, // Regroup by the original data object
Collectors.reducing(0, (sum1, sum2) -> sum1.sum + sum2.sum)) // reduce by summing
))
Way around 2 : trust the implementations
Fact is, uncontended locks in java have gotten cheaper. On top of that, pure locking sometimes has better alternatives, like Atomic types in Java (e.g. AtomicLong if you are summing stuff), that use CAS instead of locking, which can be faster (google for it, I usually refer to the Java Concurrency In Practice book for hard numbers.)
The fact is, if you have 1000 to 10k different elements (which translates to at least millions of pairs) and, like, 8 CPUs, the contention (or probability that at least 2 of your 8 threads will be processing the same element) is pretty low. And I would rather measure it first-hand rather than saying upfront "I can not affor the locks", especially if the operation can be implemented using Atomic types.