How can I implement a multithreaded PostProcessor in ActivePivot? - activepivot

Question

I consider an ActivePivot instance to compute CVA (Credit Valuation Adjustment).
I have to apply a piece of logic on a large number of cells (20k for each counter-party), each being associated to a float array of size 10k. Even if ActivePivot is massively multithreaded, an ABasicPostProcessor will be applied in a mono-threaded way for each range location. How could I make it compute through my point location in a multi-threaded way?

Answer 1

I built the following class, which specialize ABasicPostProcessor (a Core class enabling fast implementation of per-point post-processor) by just adding the calls to doEvaluation in a multi-threaded way.
Given an ABasicPostProcessor specialisation, one simply has to extend AParallelBasicPostProcessor in order to gain parallel evaluation!
/**
* Specialization of ABasicPostProcessor which will call doEvaluation in a
* multithreaded way
*
* #author BLA
*/
public abstract class AParallelBasicPostProcessor<OutputType> extends ABasicPostProcessor<OutputType> {
private static final long serialVersionUID = -3453966549173516186L;
public AParallelBasicPostProcessor(String name, IActivePivot pivot) {
super(name, pivot);
}
#Override
public void evaluate(ILocation location, final IAggregatesRetriever retriever) throws QuartetException {
// Retrieve required aggregates
final ICellSet cellSet = retriever.retrieveAggregates(Collections.singleton(location), Arrays.asList(prefetchMeasures));
// Prepare a List
List<ALocatedRecursiveTask<OutputType>> tasks = new ArrayList<ALocatedRecursiveTask<OutputType>>();
// Create the procedure to hold the parallel sub-tasks
final ICellsProcedure subTasksGeneration = makeSubTasksGenerationProcedure(tasks);
cellSet.forEachLocation(subTasksGeneration, underlyingMeasures);
ForkJoinTask.invokeAll(tasks);
for (ALocatedRecursiveTask<OutputType> task : tasks) {
OutputType returnValue;
try {
returnValue = task.get();
} catch (InterruptedException e) {
throw new RuntimeException(e);
} catch (ExecutionException e) {
// re-throw the root cause of the ExecutionException
throw new RuntimeException(e.getCause());
}
// We can write only non-null aggregates
if (null != returnValue) {
writeInRetriever(retriever, task.getLocation(), returnValue);
}
}
}
protected void writeInRetriever(IAggregatesRetriever retriever, ILocation location, OutputType returnValue) {
retriever.write(location, returnValue);
}
protected ICellsProcedure makeSubTasksGenerationProcedure(List<ALocatedRecursiveTask<OutputType>> futures) {
return new SubTasksGenerationProcedure(futures);
}
/**
* {#link ICellsProcedure} registering a {#link ALocatedRecursiveTask} per
* point location
*/
protected class SubTasksGenerationProcedure implements ICellsProcedure {
protected List<ALocatedRecursiveTask<OutputType>> futures;
public SubTasksGenerationProcedure(List<ALocatedRecursiveTask<OutputType>> futures) {
this.futures = futures;
}
#Override
public boolean execute(final ILocation pointLocation, int rowId, Object[] measures) {
// clone the array of measures as it is internally used as a buffer
final Object[] clone = measures.clone();
futures.add(makeLocatedFuture(pointLocation, clone));
return true;
}
}
protected ALocatedRecursiveTask<OutputType> makeLocatedFuture(ILocation pointLocation, Object[] measures) {
return new LocatedRecursiveTask(pointLocation, measures);
}
/**
* A specialization of RecursiveTask by associating it to a
* {#link ILocation}
*
* #author BLA
*
*/
protected static abstract class ALocatedRecursiveTask<T> extends RecursiveTask<T> {
private static final long serialVersionUID = -6014943980790547011L;
public abstract ILocation getLocation();
}
/**
* Default implementation of {#link ALocatedRecursiveTask}
*
* #author BLA
*
*/
protected class LocatedRecursiveTask extends ALocatedRecursiveTask<OutputType> {
private static final long serialVersionUID = 676859831679236794L;
protected ILocation pointLocation;
protected Object[] measures;
public LocatedRecursiveTask(ILocation pointLocation, Object[] measures) {
this.pointLocation = pointLocation;
this.measures = measures;
if (pointLocation.isRange()) {
throw new RuntimeException(this.getClass() + " accepts only point location: " + pointLocation);
}
}
#Override
protected OutputType compute() {
try {
// The custom evaluation will be computed in parallel
return AParallelBasicPostProcessor.this.doEvaluation(pointLocation, measures);
} catch (QuartetException e) {
throw new RuntimeException(e);
}
}
#Override
public ILocation getLocation() {
return pointLocation;
}
}
}

Answer 2

The ActivePivot query engine is heavily multithreaded, the invocation of several post processors within a single query is done in parallel (unless one depends on the result of another of course). When the same post processor is executed several times over the locations involved in the query, that's also done in parallel. So before rolling up your sleeves, it is worth checking whether there isn't a more obvious bottleneck in your query plan.
Now the invocation of one post processor over one location is indeed an indivisible workload in the ActivePivot query engine. And in the case aggregates are not just numbers that sum in nanoseconds, but large or structured objects like vectors, there may be room for parallelism driven performance boost.
The ActivePivot query engine is built on top of a fork/join pool (http://docs.oracle.com/javase/tutorial/essential/concurrency/forkjoin.html). It means that your post processor code is always called from within the fork join pool, and that makes it possible to fork your own sub-tasks, then join them. That is considered an expert trick, don't try that without a fair understanding of how the fork join pool works.
Let's consider a post processor that for each evaluated location computes the maximum of several measures:
package com.quartetfs.pivot.sandbox.postprocessor.impl;
import com.quartetfs.biz.pivot.IActivePivot;
import com.quartetfs.biz.pivot.ILocation;
import com.quartetfs.biz.pivot.postprocessing.impl.ABasicPostProcessor;
import com.quartetfs.fwk.QuartetException;
import com.quartetfs.fwk.QuartetExtendedPluginValue;
/**
*
* Post processor that computes the MAX of several measures.
*
* #author Quartet FS
*
*/
#QuartetExtendedPluginValue(interfaceName = "com.quartetfs.biz.pivot.postprocessing.IPostProcessor", key = MaxPostProcessor.TYPE)
public class MaxPostProcessor extends ABasicPostProcessor<Double> {
/** serialVersionUID */
private static final long serialVersionUID = -8886545079342151420L;
/** Plugin type */
public static final String TYPE = "MAX";
public MaxPostProcessor(String name, IActivePivot pivot) {
super(name, pivot);
}
#Override
public String getType() { return TYPE; }
#Override
protected Double doEvaluation(ILocation location, Object[] measures) throws QuartetException {
double max = ((Number) measures[0]).doubleValue();
for(int i = 1; i < measures.length; i++) {
max = Math.max(max, ((Number) measures[i]).doubleValue());
}
return max;
}
}
In that post processor the leaf locations resulting from the evaluated range location will be computed one after the other. You can decide to create tasks instead, and execute those tasks in parallel through the fork join pool. I hope the following will get you started:
package com.quartetfs.pivot.sandbox.postprocessor.impl;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import jsr166y.ForkJoinTask;
import jsr166y.RecursiveTask;
import com.quartetfs.biz.pivot.IActivePivot;
import com.quartetfs.biz.pivot.ILocation;
import com.quartetfs.biz.pivot.cellset.ICellSet;
import com.quartetfs.biz.pivot.cellset.ICellsProcedure;
import com.quartetfs.biz.pivot.query.aggregates.IAggregatesRetriever;
import com.quartetfs.fwk.QuartetException;
import com.quartetfs.fwk.QuartetExtendedPluginValue;
/**
*
* Post processor that computes the MAX of several measures,
* evaluation of locations is performed in parallel.
*
* #author Quartet FS
*
*/
#QuartetExtendedPluginValue(interfaceName = "com.quartetfs.biz.pivot.postprocessing.IPostProcessor", key = ParallelMaxPostProcessor.TYPE)
public class ParallelMaxPostProcessor extends MaxPostProcessor {
/** serialVersionUID */
private static final long serialVersionUID = -8886545079342151420L;
/** Plugin type */
public static final String TYPE = "PMAX";
public ParallelMaxPostProcessor(String name, IActivePivot pivot) {
super(name, pivot);
}
#Override
public String getType() { return TYPE; }
#Override
public void evaluate(ILocation location, IAggregatesRetriever retriever)throws QuartetException {
try {
// Retrieve required aggregates
ICellSet cellSet = retriever.retrieveAggregates(Collections.singleton(location), Arrays.asList(prefetchMeasures));
// Evaluate the cell set to create tasks
ParallelEvaluationProcedure evalProcedure = new ParallelEvaluationProcedure();
cellSet.forEachLocation(evalProcedure);
// Execute the tasks in parallel and write results
evalProcedure.writeResults(retriever);
} catch(Exception e) {
throw new QuartetException("Evaluation of " + this + " on location " + location + " failed.", e);
}
}
/**
* Procedure evaluated on the cell set.
*/
protected class ParallelEvaluationProcedure implements ICellsProcedure {
/** List of tasks */
protected final List<MaxComputation> tasks = new ArrayList<ParallelMaxPostProcessor.MaxComputation>();
#Override
public boolean execute(ILocation location, int rowId, Object[] measures) {
Object[] numbers = measures.clone();
tasks.add(new MaxComputation(location, numbers));
return true; // continue
}
/** Once all the tasks are executed, write results */
public void writeResults(IAggregatesRetriever retriever) throws Exception {
// Invoke all the tasks in parallel
// using the fork join pool that runs the post processor.
ForkJoinTask.invokeAll(tasks);
for(MaxComputation task : tasks) {
retriever.write(task.location, task.get());
}
}
}
/**
* Max computation task. It illustrates our example well
* but in real-life this would be too little
* of a workload to deserve parallel execution.
*/
protected class MaxComputation extends RecursiveTask<Double> {
/** serialVersionUID */
private static final long serialVersionUID = -5843737025175189495L;
final ILocation location;
final Object[] numbers;
public MaxComputation(ILocation location, Object[] numbers) {
this.location = location;
this.numbers = numbers;
}
#Override
protected Double compute() {
try {
return doEvaluation(location, numbers);
} catch (QuartetException e) {
completeExceptionally(e);
return null;
}
}
}
}