I have to spawn multiple threads in loop for configured no. of sites and then each thread has multiple child threads. After finishing all threads I need to do certain task to save data into database. How I come to know whether all task has finished.
my current code:
List<SocialDataConfig> configList = SetConfiguration();
foreach (SocialDataConfig config in configList)
{
new Thread(delegate()
{
FetchSocialData(config);
}).Start();
}
// save fetched data into database and log fetched information in database
private void FetchSocialData(SocialDataConfig config)
{
for (int i = 0; i < config.ThreadCount; i++)
{
Thread thread = new Thread(delegate()
{
FetchData(54764876, config);
});
thread.Start();
}
}
Instead of creating your own threads you can use the new Task library in C#.
You can then use something like this:
int n = 10;
// Construct started tasks
Task<int>[] tasks = new Task<int>[n];
for (int i = 0; i < n; i++)
{
tasks[i] = Task<int>.Factory.StartNew(action, i);
}
// Wait for all the tasks to finish.
Task.WaitAll(tasks);
Here is the MSDN documentation on WaitAll with a complete example.
Related
I have a java thread which is running a path-finding algorithm in a constant while loop. Then, every so often I want to retrieve the most updated path from the thread. However, I am unsure how to do this, and think I might be doing it wrong.
My thread consists of the following code:
public class BotThread extends Thread {
Bot bot;
AStar pathFinder;
Player targetPlayer;
public List<boolean[]> plan;
public BotThread(Bot bot) {
this.bot = bot;
this.plan = new ArrayList<>();
pathFinder = new AStar(bot, bot.getLevelHandler());
}
public void run() {
while (true) {
System.out.println("THREAD RUNNING");
targetPlayer = bot.targetPlayer;
plan = pathFinder.optimise(targetPlayer);
}
}
public boolean[] getNextAction() {
return plan.remove(0);
}
}
I then create an object of BotThread, and call start(). Then when I call getNextAction() on the thread, I seem to receive a null pointer. Is this because I am not able to call another method on the thread whilst it is in the main loop? How should I do this properly?
This is because you are not giving enough time to thread to initialise plan Arraylist. You need to add sleeping time to the threads. Something like this while calling BotThread class from main:
int num_threads = 8;
BotThread myt[] = new BotThread[num_threads];
for (int i = 0; i < num_threads; ++i) {
myt[i] = new BotThread();
myt[i].start();
Thread.sleep(1000);
myt[i].getNextAction();
}
I am trying to implement a thread-safe solution to keep a count of successful tasks that have been completed, which will ultimately get bound to label displayed on the UI. However, when I use the AtomicInteger below it locks up my UI when the tasks start running, however, if I remove all AtomicInteger refs everything works fine. Is there a non-blocking, thread-safe way which this can be accomplished?
public void handleSomeButtonClick(){
if(!dataModel.getSomeList().isEmpty()) {
boolean unlimited = false;
int count = 0;
AtomicInteger successCount = new AtomicInteger(0);
if(countSelector.getValue().equalsIgnoreCase("Unlimited"))
unlimited = true;
else
count = Integer.parseInt(countSelector.getValue());
while(unlimited || successCount.get() < count) {
Task task = getSomeTask();
taskExecutor.submit(task);
task.setOnSucceeded(event -> {
if (task.getValue())
log.info("Successfully Completed Task | Total Count: " + successCount.incrementAndGet());
else
log.error("Failed task");
});
}
}
}
Your loop waits for a certain number of tasks to be completed. It may even be an infinite loop.
This is not a good idea:
You block the calling thread which seems to be the JavaFX application thread.
You don't have any control of how many tasks are submitted. count could be 3, but since you only schedule the tasks in the loop, 1000 or more tasks could be created&scheduled before the first one completes.
Furthermore if you use onSucceeded/onFailed, you don't need to use AtomicInteger or any similar kind of synchronisation, since those handlers all run on the JavaFX application thread.
Your code could be rewritten like this:
private int successCount;
private void scheduleTask(final boolean unlimited) {
Task task = getSomeTask();
task.setOnSucceeded(event -> {
// cannot get a Boolean from a raw task, so I assume the task is successfull iff no exception happens
successCount++;
log.info("Successfully Completed Task | Total Count: " + successCount);
if (unlimited) {
// submit new task, if the number of tasks is unlimited
scheduleTask(true);
}
});
// submit new task on failure
task.setOnFailed(evt -> scheduleTask(unlimited));
taskExecutor.submit(task);
}
public void handleSomeButtonClick() {
if(!dataModel.getSomeList().isEmpty()) {
successCount = 0;
final boolean unlimited;
final int count;
if(countSelector.getValue().equalsIgnoreCase("Unlimited")) {
unlimited = true;
count = 4; // set limit of number of tasks submitted to the executor at the same time
} else {
count = Integer.parseInt(countSelector.getValue());
unlimited = false;
}
for (int i = 0; i < count; i++) {
scheduleTask(unlimited);
}
}
}
Note: This code runs the risk of handleButtonClick being clicked multiple times before the previous tasks have been completed. You should either prevent scheduling new tasks before the old ones are completed or use some reference type containing an int instead for the count, create this object in handleSomeButtonClick and pass this object to scheduleTask.
Your UI lock up means you do the counting(successCount.get() < count) in your FX application thread. I cannot understand why you keep submit the task in the while loop,
which one do you want to do? (1) start X(e.g. 10) task and count how many task is success. or (2) just keep starting new task and see the count go up.
if(2) then run the whole while loop in a background thread, update the UI in a Platform->runlater().
if(1) use the Future / CompletableFuture, or more powerful version Future in 3rd party package like vavr.
Your problem is future.get() block and wait for result.
This will be simple if you use Vavr library.
Because it can attach a code to its future which run automatically when success or fail.
So you don't have to wait.
Here is a example which using Vavr's future.
CheckedFunction0<String> thisIsATask = () -> {
if ( /*do something*/ ){
throw new Exception("Hey");
}
return "ABC";
};
List<Future<String>> futureList = new ArrayList<>();
for (int x = 0; x < 10; x++) {
futureList.add(Future.of(getExecutorService(), thisIsATask));
}
futureList.forEach((task) -> {
// This will run if success
task.onSuccess(s -> s.equals("ABC") ? Platform.runLater(()->UpdateCounter()) : wtf());
// Your get the exception if it is fail;
task.onFailure(e -> e.printStackTrace());
// task.onComplete() will run on any case when complete
});
This is not blocking, the code at onSucess onFailure or onComplete will run when the task is finish or an exception is catch.
Note: Future.of will use the executorService you pass in to run each task at new thread, the code you provide at onSuccess will continue to run at that thread once the task is done so if you calling javafx remember the Platform.runLater()
Also if you want to run something when all task is finish, then
// the code at onComplete will run when tasks all done
Future<Seq<String>> all = Future.sequence(futureList);
all.onComplete((i) -> this.btnXYZ.setDisable(false));
I've written a Node package that performs some intensive data processing. I'm able to watch progress via console.log, but I'd like to provide the end user with some way of monitoring it in an event-driven way. Basically, it would be like returning a promise, except instead of one then event it would have an event for each "step", and then finally resolve with the data.
Unfortunately, I don't know enough about Node streams (which I'm guessing is the thing I need) to do this. Can I get a few pointers? How can I create a stream which is updated every time there is, say, 1% more progress, and then finally gives the computed data?
EDIT: As an example, consider this loop:
for(let i = 0; i < N; i++) {
intensiveFunction();
console.log(`${i} of ${N} completed`);
}
What I want to do instead is
for(let i = 0; i < N; i++) {
intensiveFunction();
// send user a signal that i/N of the task has been completed
}
You don't need to use streams, You can use an EventEmitter, and emit the current progress, or any event you may want.
my-package.js
const EventEmitter = require('events');
// Create a class that extends from EventEmitter
// And now you can emit events when something happens, e.g., progress update
class MyPackage extends EventEmitter {
async intensiveFunction() {
// Something
}
async process() {
for(let i = 0; i < N; i++) {
await this.intensiveFunction();
this.emit('step', i, N); // Or send progress in % or whatever you want
}
}
}
module.exports = MyPackage;
index.js
const MyPackage = require('my-package');
const package = new MyPackage();
package.on('step', (step, total) => console.log(`${step}/${total}`));
package.process();
You can either provide a full events API, or mix it with promises. Meaning process can resolve once is done, or you can emit an end event, or do both.
async process() {
for(let i = 0; i < N; i++) {
await this.intensiveFunction();
this.emit('step', i, N); // Or send progress in % or whatever you want
}
// Emit the end event
this.emit('end', result);
// And resolve the promise
return result; // Whatever result is
}
for (int i = 0; i < 100,000; i++)
{
threadEvent.Invoke(i, new EventArgs());// tell processbar value
}
threadEvent += new EventHandler(method_threadEvent);
void method_threadEvent(object sender, EventArgs e)
{
int nowValue = Convert.ToInt32(sender);
nowValueDelegate now = new nowValueDelegate(setNow);
this.Invoke(now, nowValue);
}
private void setNow(int nowValue)
{
this.progressBar1.Value = nowValue;
}
private delegate void nowValueDelegate(int nowValue);
in the loop i do nothing, but it also waste alot of time !
why threadEvent.Invoke spend so much time ?
Invoking is an expensive operation, because it has to cross thread boundaries.
It's best to reduce the amount of invokes, by for instance only updating the progress bar for each percentage of work you do, rather than for each iteration of the loop. That way, only 100 updates need to be processed, rather than one for each iteration.
First thing you need to do is to calculate or estimate the current progress.
For a typical loop
for (int i = 0; i < someValue; ++i)
{
... // Work here
}
A good estimate of progress is (i / someValue) * 100, which gives the percentage of the loop that has been completed. To update the progress to the UI thread only when the next percentage has been reached you could do something in the line of:
int percentCompleted = 0;
threadEvent.Invoke(percentCompleted, new EventArgs()); // Initial progressbar value
for (int i = 0; i < someValue; ++i)
{
int newlyCompleted = (i / someValue) * 100;
if (newlyCompleted > percentCompleted)
threadEvent.Invoke(percentCompleted, new EventArgs());
percentCompleted = newlyCompleted;
... // Work here
}
Now finally, you could use BeginInvoke instead of Invoke to make sure the worker thread doesn't wait for the threadEvent to complete (PostMessage behaviour). This works well here because there is no return value from threadEvent that you need.
I have a program which has a Ui with which users choose the way to display and do small configurations. It also has a background procedure, which continuously reads data from the network and update the data to display.
Now I put them in one process:
background procedure:
STATE MainWindow::Rcv()
{
DeviceMAP::iterator dev;
for(dev= dev_map.begin(); dev!= dev_map.end(); dev++)
{
dev->second.rcvData();//receive data from the network, the time can be ignored.
BitLog* log = new BitLog();
dev->second.parseData(log);
LogItem* logItem = new LogItem();
logItem->time = QString::fromLocal8Bit(log->rcvTime.c_str());
logItem->name = QString::fromLocal8Bit(log->basicInfo.getName().c_str());
logItem->PIN = QString::fromLocal8Bit(log->basicInfo.getPIN().c_str()).toShort();
delete log;
add_logItem(logItem);
}
return SUCCESS;
}
add_logItem:
void MainWindow::add_logItem(LogItem* logItem)
{
writeToFile(logItem);
Device* r = getDevbyPIN(QString::number(logItem->PIN));
if(r == NULL)return;
devInfo_inside_widget::States state = logItem->state;
bool bool_list[portsNum_X];
for(int i =0; i < portsNum_X; i++)
{
bool_list[i] = 0;
}
for(int i = 0; i < portsNum; i++)
{
bool_list[i] = (logItem->BITS[i/8] >> (7 - i%8)) & 0x1;
}
r->refresh(state, logItem->time, bool_list);//update data inside...state, time , BITS...
IconLabel* icl = getIConLabelByDev(r);//update data
icl->refresh(state);
logDisplayQueue.enqueue(logItem);//write queue here
int size = logDisplayQueue.size();
if(size > 100)
{
logDisplayQueue.dequeue();//write queue here
}
}
The section above has not dealt with any ui operations yet, but when user push a radio button in the ui, the program has to filter the data in the queue and display it in the table widget:
ui operations:
void MainWindow::filter_log_display(bool bol)
{
row_selectable = false;
ui->tableWidget->setRowCount(0);//delete table items all
row_selectable = true;
int size_1 = logDisplayQueue.size() - 1;
ui->tableWidget->verticalScrollBar()->setSliderPosition(0);
if(size_1+1 < 100)
{
ui->tableWidget->setRowCount(size_1 + 1);
}
else
{
ui->tableWidget->setRowCount(100);//display 100 rows at most
}
if(bol)//filter from logDisplayQueue and display unworking-state-log rows
{
int index = 0;
for(int queue_i = size_1; queue_i >= 0; queue_i--)
{
LogItem* logItem = (LogItem*)logDisplayQueue.at(queue_i); // read queue here
if(logItem->state == STATE_WORK || logItem->state == STATE_UN)continue;
QString BITS_str = bits2Hexs(logItem->BITS);
ui->tableWidget->setItem(index, 0, new QTableWidgetItem(logItem->time));//time
ui->tableWidget->setItem(index, 1, new QTableWidgetItem(logItem->name));//name
ui->tableWidget->setItem(index, 2, new QTableWidgetItem(BITS_str));//BITS
if(queue_i == oldRowItemNo)ui->tableWidget->selectRow(index);
index++;
}
ui->tableWidget->setRowCount(index);
}
else//display all rows
{
for(int queue_i = size_1, index = 0; queue_i >= 0; queue_i--, index++)
{
LogItem* logItem = (LogItem*)logDisplayQueue.at(queue_i); //read queue here
QString BITS_str = bits2Hexs(logItem->BITS);//
finish = clock();
ui->tableWidget->setItem(index, 0, new QTableWidgetItem(logItem->time));//time
ui->tableWidget->setItem(index, 1, new QTableWidgetItem(logItem->name));//name
ui->tableWidget->setItem(index, 2, new QTableWidgetItem(BITS_str));//BITS
if(queue_i == oldRowItemNo)ui->tableWidget->selectRow(index);
}
}
}
So the queue is quite samll and the background procedure is quite frequent(nearly 500 times per sec). That is, the queue will be written 500 times in 1 sec, but displayed time from time by the user.
I want to split the functions into two threads and run them together, one rev and update data, one display.
If i do not use any lock or mutex, the user may get the wrong data, but if i force the write-data procedure enter critical section and leave critical section everytime, it will be a heavy overload. :)
Should I use CRITICAL_SECTION or something else, any suggestions related?(my words could be verbose for you :) , i only hope for some hints :)
I'd put "Recv" function in another QObject derived class, put it under other QThread not main gui thread and connect "logItemAdded(LogItem* item)" signal to main window's "addLogItem(LogItem* item)" slot.
for just quick and dirty hint my conceptual code follows.
#include <QObject>
class Logger : public QObject
{
Q_OBJECT
public:
Logger(QObject* parent=0);
virtual ~Logger();
signals:
void logItemAdded(LogItem* logItem);
public slots:
protected:
void Rcv()
{
// ...
// was "add_logItem(logItem)"
emit logItemAdded(logItem);
}
};
MainWindow::MainWindow(...)
{
Logger logger = new Logger;
// setup your logger
QThread* thread = new QThread;
logger->moveToThread(thread);
connect(thread, SIGNAL(finished()), thread, SLOT(deleteLater()));
thread->start();
}
Hope this helps.
Good luck.