This is a sample batch processing application. Initially I do create a thread pool for the batch processing and then execute those threads on the batching. Once the batch processing is started, it is keep looping and searching for entries in the queue for batching. That is a huge performance drain and the CPU usage goes to max.
The following block contains the sample code, I am currently working on.
BlockingQueue<BatchRequestEntry> batchingQueue;
ThreadPoolExecutor executorService executorService;
private boolean isRunning = false;
private ExecutorService getExecutorService() {
if (executorService == null) {
ThreadFactory threadFactory = new ThreadFactoryBuilder().setNameFormat("batch-processor-%d").build();
executorService = new ThreadPoolExecutor(batchProcessorConfiguration.getThreadCount(), batchProcessorConfiguration.getThreadCount(),
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<>(), threadFactory);
}
return executorService;
}
public synchronized void start() {
if (isRunning) {
return;
}
getExecutorService().submit(() -> {
while (isRunning) {
if (!batchingQueue.isEmpty()) {
List<BatchRequestEntry> entries = batchingQueue.batch(batchProcessorConfiguration.getBatchingMaxCount());
executorService.submit(() -> process(entries));
}
}
});
isRunning = true;
}
public SettableFuture<BatchResultEntry> append(BatchRequestEntry batchRequestEntry) {
if (!isRunning) {
start();
}
SettableFuture<BatchResultEntry> future = SettableFuture.create();
batchingQueue.append(batchRequestEntry);
futures.put(batchRequestEntry.getEntryId(), future);
return future;
}
What I want to provide as a solution is to capture the number of loops with empty batching queue. Then compare that value with a threshold value and sleep the threads in the thread pool. Once the new entries are appended to the batching queue, I want to interrupt the thread sleep and continue the loop for batching. I think that would solve the problem.
I want to figure how to do that with ThreadPoolExecutor and I would appreciate if there are better approaches to address this problem. Thank you!
Related
I have created a multithreaded Kafka consumer in which one thread is assigned to each of the partition (I have total 100 partitions). I have followed https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example link.
Below is the init method of my consumer.
consumer = kafka.consumer.Consumer.createJavaConsumerConnector(createConsumerConfig());
System.out.println("Kafka Consumer initialized.");
Map<String, Integer> topicCountMap = new HashMap<String, Integer>();
topicCountMap.put(topicName, 100);
Map<String, List<KafkaStream<byte[], byte[]>>> consumerMap = consumer.createMessageStreams(topicCountMap);
List<KafkaStream<byte[], byte[]>> streams = consumerMap.get(topicName);
executor = Executors.newFixedThreadPool(100);
In the above init method, I got the list of Kafka streams (total 100) which should be connected to each of the partition (Which is happening as expected).
Then I did submit each of the streams to a different thread using below snippet.
public Object call() {
for (final KafkaStream stream : streams) {
executor.execute(new StreamWiseConsumer(stream));
}
return true;
}
Below is the StreamWiseConsumer class.
public class StreamWiseConsumer extends Thread {
ConsumerIterator<byte[], byte[]> consumerIterator;
private KafkaStream m_stream;
public StreamWiseConsumer(ConsumerIterator<byte[], byte[]> consumerIterator) {
this.consumerIterator = consumerIterator;
}
public StreamWiseConsumer(KafkaStream kafkaStream) {
this.m_stream = kafkaStream;
}
#Override
public void run() {
ConsumerIterator<byte[], byte[]> consumerIterator = m_stream.iterator();
while(!Thread.currentThread().isInterrupted() && !interrupted) {
try {
if (consumerIterator.hasNext()) {
String reqId = UUID.randomUUID().toString();
System.out.println(reqId+ " : Event received by threadId : "+Thread.currentThread().getId());
MessageAndMetadata<byte[], byte[]> messageAndMetaData = consumerIterator.next();
byte[] keyBytes = messageAndMetaData.key();
String key = null;
if (keyBytes != null) {
key = new String(keyBytes);
}
byte[] eventBytes = messageAndMetaData.message();
if (eventBytes == null){
System.out.println("Topic: No event fetched for transaction Id:" + key);
continue;
}
String event = new String(eventBytes).trim();
// Some Processing code
System.out.println(reqId+" : Processing completed for threadId = "+Thread.currentThread().getId());
consumer.commitOffsets();
} catch (Exception ex) {
}
}
}
}
Ideally, it should start processing from all the 100 partitions in parallel. But it is picking some random number of events from one of the threads and processing it then some other thread starts processing from another partition. It seems like it's sequential processing but with different-different threads. I was expecting processing to happen from all the 100 threads. Am I missing something here?
PFB for the logs link.
https://drive.google.com/file/d/14b7gqPmwUrzUWewsdhnW8q01T_cQ30ES/view?usp=sharing
https://drive.google.com/file/d/1PO_IEsOJFQuerW0y-M9wRUB-1YJuewhF/view?usp=sharing
I doubt whether this is the right approach for vertically scaling kafka streams.
Kafka streams inherently supports multi thread consumption.
Increase the number of threads used for processing by using num.stream.threads configuration.
If you want 100 threads to process the 100 partitions, set num.stream.threads as 100.
I have a java thread which is running a path-finding algorithm in a constant while loop. Then, every so often I want to retrieve the most updated path from the thread. However, I am unsure how to do this, and think I might be doing it wrong.
My thread consists of the following code:
public class BotThread extends Thread {
Bot bot;
AStar pathFinder;
Player targetPlayer;
public List<boolean[]> plan;
public BotThread(Bot bot) {
this.bot = bot;
this.plan = new ArrayList<>();
pathFinder = new AStar(bot, bot.getLevelHandler());
}
public void run() {
while (true) {
System.out.println("THREAD RUNNING");
targetPlayer = bot.targetPlayer;
plan = pathFinder.optimise(targetPlayer);
}
}
public boolean[] getNextAction() {
return plan.remove(0);
}
}
I then create an object of BotThread, and call start(). Then when I call getNextAction() on the thread, I seem to receive a null pointer. Is this because I am not able to call another method on the thread whilst it is in the main loop? How should I do this properly?
This is because you are not giving enough time to thread to initialise plan Arraylist. You need to add sleeping time to the threads. Something like this while calling BotThread class from main:
int num_threads = 8;
BotThread myt[] = new BotThread[num_threads];
for (int i = 0; i < num_threads; ++i) {
myt[i] = new BotThread();
myt[i].start();
Thread.sleep(1000);
myt[i].getNextAction();
}
I got CyclicBarrier code from oracle page to understand it more. I modified it and now having one doubt.
Below code doesn't terminate but If I uncomment Thread.sleep condition, It works fine.
import java.util.Arrays;
import java.util.concurrent.BrokenBarrierException;
import java.util.concurrent.CyclicBarrier;
class Solver {
final int N;
final float[][] data;
boolean done = false;
final CyclicBarrier barrier;
class Worker implements Runnable {
int myRow;
Worker(int row) {
myRow = row;
}
public void run() {
while (!done) {
processRow(myRow);
try {
barrier.await();
} catch (InterruptedException ex) {
return;
} catch (BrokenBarrierException ex) {
return;
}
}
System.out.println("Run finish for " + Thread.currentThread().getName());
}
private void processRow(int row) {
float[] rowData = data[row];
for (int i = 0; i < rowData.length; i++) {
rowData[i] = 1;
}
/*try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}*/
done = true;
}
}
public Solver(float[][] matrix) {
data = matrix;
N = matrix.length;
barrier = new CyclicBarrier(N, new Runnable() {
public void run() {
for (int i = 0; i < data.length; i++) {
System.out.println("Data " + Arrays.toString(data[i]));
}
System.out.println("Completed:");
}
});
for (int i = 0; i < N; ++i)
new Thread(new Worker(i), "Thread "+ i).start();
}
}
public class CyclicBarrierTest {
public static void main(String[] args) {
float[][] matrix = new float[5][5];
Solver solver = new Solver(matrix);
}
}
Why Thread.sleep is required in above code?
I've not run your code but there may be a race condition, here is a scenario that reveals it:
you start the first thread, it runs during a certain amount of time sufficient for it to finish the processRow method call so it sets done to true and then waits on the barrier,
the other threads start but they see that all is "done" so they don't enter the loop and they'll never wait on the barrier, and end directly
the barrier will never be activated as only one of the N threads has reached it
deadlock
Why it is working with the sleep:
when one of the thread starts to sleep it lets the other threads work before marking the work as "done"
the other threads have enough time to work and can themselves reach the barrier
2 seconds is largely enough for 5 threads to end a processing that should not last longer than 10ms
But note that if your system is ovrerloaded it could too deadlock:
the first thread starts to sleep
the OS scheduler lets another application work during more than 2 seconds
the OS scheduler comes back to your application and the threads scheduler chooses the first thread again and lets it terminate, setting done to true
and here again the first scenario => deadlock too
And a possible solution (sorry not tested):
change your while loops for do/while loops:
do
{
processRow(myRow);
...
}
while (!done);
Here i attached my example that i used for performance test. Why there is so much diffrence between all this ? (This is sample console application)
class Program
{
internal class ThreadObj
{
public ManualResetEvent signalComplete { get; set; }
public int TaskItem { get; set; }
}
static void ThreadWork(object o)
{
ThreadObj obj = (ThreadObj)o;
System.Threading.Thread.Sleep(5000);
obj.signalComplete.Set();
}
static void Main(string[] args)
{
// Using new .net 4.0 Task
Stopwatch watch = new Stopwatch();
watch.Start();
System.Collections.Concurrent.ConcurrentBag<Task> tasks = new System.Collections.Concurrent.ConcurrentBag<Task>();
Parallel.For(0, 60, i =>
{
Task t = Task.Factory.StartNew(() =>
{
System.Threading.Thread.Sleep(5000);
}, TaskCreationOptions.PreferFairness);
tasks.Add(t);
});
Console.WriteLine("Waiting for task to finish");
Task.WaitAll(tasks.ToArray());
watch.Stop();
Console.WriteLine("Complete(Tasks) : Time " + watch.ElapsedMilliseconds.ToString());
// Using Thread
watch.Reset();
watch.Start();
System.Collections.Concurrent.ConcurrentBag<ManualResetEvent> tasksThreads = new System.Collections.Concurrent.ConcurrentBag<ManualResetEvent>();
Parallel.For(0, 60, i =>
{
ManualResetEvent signal = new ManualResetEvent(false);
tasksThreads.Add(signal);
Thread t = new Thread(new ParameterizedThreadStart(ThreadWork));
t.Start(new ThreadObj() { signalComplete = signal, TaskItem = i });
});
Console.WriteLine("Waiting for task to finish");
WaitHandle.WaitAll(tasksThreads.ToArray());
watch.Stop();
Console.WriteLine("Complete(Threads) : Time " + watch.ElapsedMilliseconds.ToString());
// Using ThreadPool
watch.Reset();
watch.Start();
System.Collections.Concurrent.ConcurrentBag<ManualResetEvent> tasksThreadPools = new System.Collections.Concurrent.ConcurrentBag<ManualResetEvent>();
Parallel.For(0, 60, i =>
{
ManualResetEvent signal = new ManualResetEvent(false);
tasksThreadPools.Add(signal);
ThreadObj obj = new ThreadObj() { signalComplete = signal, TaskItem = i };
ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadWork), obj);
});
Console.WriteLine("Waiting for task to finish");
WaitHandle.WaitAll(tasksThreadPools.ToArray());
watch.Stop();
Console.WriteLine("Complete(ThreadPool) : Time " + watch.ElapsedMilliseconds.ToString());
Console.ReadLine();
}
}
Please provide your suggetion on this.
Here is sample output that i got.
Waiting for task to finish
Complete(Tasks) : Time 28050
Waiting for task to finish
Complete(Threads) : Time 5435
Waiting for task to finish
Complete(ThreadPool) : Time 15032
You're test case is far from solid to begin with. When you perform actual computation work within the threadWork method, you'll find that the results are very different. TPL uses the threadpool internally so it's a matter of Threadpool vs Threads. The reason for the TPL to be so different compared to the Threadpool is likely in the nature of the Threadpool itself (will come back on that one later).
Look at the time it took for Threads to complete. Your test method only sleeps for 5 seconds, that's it. Now where did the other .43 second go to? Right, to the creation and destruction of the Thread itself and the associated overhead including context switching. The Threadpool has a queue of Threads that can be used to execute simultaneously. It's up to the Threadpool and it's configuration to create and destroy extra threads whenever it deems needed. When you schedule 60 items in the Threadpool , the Threadpool wont likely create 60 threads to handle all items simultaneously but rather use a sub amount of that and handle multiple items per thread. Since your test method is only sleeping, this explains the big difference between the time spent with threads and the time spent with the Threadpool.
Since TPL uses the Threadpool internally and before you're ThreadPool test ran, it is logical to assume that, at that stage: less threads were available in the Threadpool, but due to the run of the TPL, more threads were created for the Threadpool so in turn, when your Threadpool test ran, there were more threads available initially which explains the difference between TPL and the Threadpool.
Practically, you want to use the Threadpool as much as possible, especially for computational operations. When you need to synchronize with an external resource like downloading something from the web, i recommend not using a thread but one of the more advanced async options available in .NET for getting that particular resource.
My somewhat data-intensive wp7 app persists data as follows: I maintain a change journal reflecting all user activity, and every couple of seconds, a thread timer spins up a threadpool thread that flushes the change journal to a database inside a transaction. It looks something like this:
When the user exits, I stop the timer, flush the journal on the UI thread (takes no more than a second or two), and dismount the DB.
However, if the worker thread is active when the user exits, I can't figure out how to react gracefully. The system seems to kill the worker thread, so it never finishes its work and never gives up its lock on the database connection, and the ui thread then attempts to acquire the lock, and is immediately killed by the system. I tried setting a flag on the UI thread requesting the worker to abort, but I think the worker was interrupted before it read the flag. Everything works fine except for this 1 in 100 scenario where some user changes end up not being saved to the db, and I can't seem to get around this.
Very simplified code below:
private Timer _SweepTimer = new Timer(SweepCallback, null, 5000, 5000);
private volatile bool _BailOut = false;
private void SweepCallback(object state) {
lock (db) {
db.startTransaction();
foreach(var entry in changeJournal){
//CRUD entry as appropriate
if(_BailOut){
db.rollbackTransaction();
return;
}
}
db.endTransaction();
changeJournal.Clear();
}
}
private void RespondToSystemExit(){
_BailOut = true; //Set flag for worker to exit
lock(db){ //In theory, should acquire the lock after the bg thread bails out
SweepCallback(null);//Flush to db on the UI thread
db.dismount();//App is now ready to close
}
}
Well, just to close this question, I ended up using a manualresetevent instead of the locking, which is to the best of my understanding a misuse of the manualresetevent, risky and hacky, but its better than nothing.
I still don't know why my original code wasn't working.
EDIT: For posterity, I'm reposting the code to reproduce this from the MS forums:
//This is a functioning console app showing the code working as it should. Press "w" and then "i" to start and then interrupt the worker
using System;
using System.Threading;
namespace deadlocktest {
class Program {
static void Main(string[] args) {
var tester = new ThreadTest();
string input = "";
while (!input.Equals("x")) {
input = Console.ReadLine();
switch (input) {
case "w":
tester.StartWorker();
break;
case "i":
tester.Interrupt();
break;
default:
return;
}
}
}
}
class ThreadTest{
private Object lockObj = new Object();
private volatile bool WorkerCancel = false;
public void StartWorker(){
ThreadPool.QueueUserWorkItem((obj) => {
if (Monitor.TryEnter(lockObj)) {
try {
Log("Worker acquired the lock");
for (int x = 0; x < 10; x++) {
Thread.Sleep(1200);
Log("Worker: tick" + x.ToString());
if (WorkerCancel) {
Log("Worker received exit signal, exiting");
WorkerCancel = false;
break;
}
}
} finally {
Monitor.Exit(lockObj);
Log("Worker released the lock");
}
} else {
Log("Worker failed to acquire lock");
}
});
}
public void Interrupt() {
Log("UI thread - Setting interrupt flag");
WorkerCancel = true;
if (Monitor.TryEnter(lockObj, 5000)) {
try {
Log("UI thread - successfully acquired lock from worker");
} finally {
Monitor.Exit(lockObj);
Log("UI thread - Released the lock");
}
} else {
Log("UI thread - failed to acquire the lock from the worker");
}
}
private void Log(string Data) {
Console.WriteLine(string.Format("{0} - {1}", DateTime.Now.ToString("mm:ss:ffff"), Data));
}
}
}
Here is nearly identical code that fails for WP7, just make a page with two buttons and hook them
using System;
using System.Diagnostics;
using System.Threading;
using System.Windows;
using Microsoft.Phone.Controls;
namespace WorkerThreadDemo {
public partial class MainPage : PhoneApplicationPage {
public MainPage() {
InitializeComponent();
}
private Object lockObj = new Object();
private volatile bool WorkerCancel = false;
private void buttonStartWorker_Click(object sender, RoutedEventArgs e) {
ThreadPool.QueueUserWorkItem((obj) => {
if (Monitor.TryEnter(lockObj)) {
try {
Log("Worker acquired the lock");
for (int x = 0; x < 10; x++) {
Thread.Sleep(1200);
Log("Worker: tick" + x.ToString());
if (WorkerCancel) {
Log("Worker received exit signal, exiting");
WorkerCancel = false;
break;
}
}
} finally {
Monitor.Exit(lockObj);
Log("Worker released the lock");
}
} else {
Log("Worker failed to acquire lock");
}
});
}
private void Log(string Data) {
Debug.WriteLine(string.Format("{0} - {1}", DateTime.Now.ToString("mm:ss:ffff"), Data));
}
private void buttonInterrupt_Click(object sender, RoutedEventArgs e) {
Log("UI thread - Setting interrupt flag");
WorkerCancel = true;
//Thread.Sleep(3000); UNCOMMENT ME AND THIS WILL START TO WORK!
if (Monitor.TryEnter(lockObj, 5000)) {
try {
Log("UI thread - successfully acquired lock from worker");
} finally {
Monitor.Exit(lockObj);
Log("UI thread - Released the lock");
}
} else {
Log("UI thread - failed to acquire the lock from the worker");
}
}
}
}
Your approach should work when you operate from the Application_Deactivated or Application_Closing event. MSDN says:
There is a time limit for the Deactivated event to complete. The
device may terminate the application if it takes longer than 10
seconds to save the transient state.
So if you say it just takes just a few seconds this should be fine. Unless the docs don't tell the whole story. Or your worker thread takes longer to exit than you think.
As Heinrich Ulbricht already said you have <=10 sec to finish your stuff, but you should block MainThread to get them.
It means that even if you have BG thread with much work to do, but your UI thread just does nothing in OnClosingEvent/OnDeactivatingEvent - you will not get your 10 seconds.
Our application actually does eternal wait on UI thread in closing event to allow BG thread send some data thru sockets.