There is main function:
int main(int argc, char *argv[]) {
QCoreApplication a(argc, argv);
Worker w;
QObject::connect(&w, SIGNAL(done()), &a, SLOT(quit()), Qt::QueuedConnection);
w.start();
int ret = a.exec();
w.quit();
w.wait();
return ret;
}
And there is Worker's definition:
class Worker : public QThread
{
Q_OBJECT
public:
Worker(QObject *parent=0);
protected:
void run();
protected slots:
void process_request();
private:
int ttl;
Messenger* messenger;
}
Worker::Worker(QObject * parent)
:QThread(parent),
ttl(5),
messenger(new Messenger(this))
{
moveToThread(this);
connect(messenger, SIGNAL(new_message()), SLOT(process_request()), Qt::QueuedConnection);
}
void Worker::finish(){
quit();
messenger->disconnectFromNetwork();
}
void Worker::run(){
messenger->connectToNetwork("somewhere");
exec();
emit done();
}
void Worker::process_request(){
net_message msg;
messenger->recv(msg);
// PROCESSING
messenger->send(msg);
BOOST_LOG_SEV(file_log, severity::notification) << "TTL = " << ttl;
if (--ttl == 0) {
finish();
}
}
Well, sorry for the long exposition. The idea was that the Messenger lives in the main thread, and pokes the Worker when it has a new message for it, and the Worker lives only a certain amount of messages, after which it stops and shuts down entire application.
But there is a problem: the log file has lines TTL = -1, and TTL = -2, and such. It shouldn't, and the only reason I can think about is that quit() doesn't quite ends the event loop: it allows pending events to be processed before returning from exec(). Is it so? If "no", then what may cause such a behaviour?
First of all "You're doing that wrong".
Secondly documentation doesn't say what is the state of queue in event loop after exit was called. It is possible that exec() returns after event queue is empty to make sure that all asynchronous cleanups are done (in case if this it top most event loop, in this case it is).
edit:
I've checked source code. Apparently QEventLoop::processEvents is call between each check that event loop should exit. So it looks like exec() returns only when queue is empty.
Related
Along with the main thread, i have one more thread that receives data to write them in a file.
std::queue<std::vector<int>> dataQueue;
std::mutex mutex;
void setData(const std::vector<int>& data) {
std::lock_guard<std::mutex> lock(mutex);
dataQueue.push(data);
}
void write(const std::string& fileName) {
std::unique_ptr<std::ostream> ofs = std::unique_ptr<std::ostream>(new zstr::ofstream(fileName));
while (store) {
std::lock_guard<std::mutex> lock(mutex);
while (!dataQueue.empty()) {
std::vector<int>& data= dataQueue.front();
ofs->write(reinterpret_cast<char*>(data.data()), sizeof(data[0])*data.size());
dataQueue.pop();
}
}
}
}
setData is used by the main thread and write is actually the writing thread. I use std::lock_quard to avoid memory conflict but when locking on the writing thread, it slows down the main thread as it has to wait for the Queue to be unlocked. But i guess i can avoid this as the threads never act on the same element of the queue at the same time.
So i would like to do it lock-free but i don't really understand how i should implement it. I mean, how can i do it without locking anything ? moreover, if the writing thread is faster than the main thread, the queue might be empty most of the time, so it should somehow waits for new data instead of looping infinitly to check for non empty queue.
EDIT: I changed simple std::lock_guard by std::cond_variable so that it could wait when the queue is empty. But the main thread can still be blocked as , when cvQeue.wait(.) is resolved, it reacquire the lock. moreover, what if the main thread does cvQueue.notify_one() but the writing thread is not waiting ?
std::queue<std::vector<int>> dataQueue;
std::mutex mutex;
std::condition_variable cvQueue;
void setData(const std::vector<int>& data) {
std::unique_lock<std::mutex> lock(mutex);
dataQueue.push(data);
cvQueue.notify_one();
}
void write(const std::string& fileName) {
std::unique_ptr<std::ostream> ofs = std::unique_ptr<std::ostream>(new zstr::ofstream(fileName));
while (store) {
std::lock_guard<std::mutex> lock(mutex);
while (!dataQueue.empty()) {
std::unique_lock<std::mutex> lock(mutex);
cvQueue.wait(lock);
ofs->write(reinterpret_cast<char*>(data.data()), sizeof(data[0])*data.size());
dataQueue.pop();
}
}
}
}
If you only have two threads, than you could use a lock-free single-producer-single-consumer (SPSC) queue.
A bounded version can be found here: https://github.com/rigtor/SPSCQueue
Dmitry Vyukov presented an unbounded version here: http://www.1024cores.net/home/lock-free-algorithms/queues/unbounded-spsc-queue (You should note though, that this code should be adapted to use atomics.)
Regarding a blocking pop operation - this is something that lock-free data structures do not provide since such an operation is obviously not lock-free. However, it should be relatively straight forward to adapt the linked implementations in such a way, that a push operation notifies a condition variable if the queue was empty before the push.
i guess i have something that met my needs. I did a LockFreeQueue that uses std::atomic. I can thus manage the state of the head/tail of the queue atomically.
template<typename T>
class LockFreeQueue {
public:
void push(const T& newElement) {
fifo.push(newElement);
tail.fetch_add(1);
cvQueue.notify_one();
}
void pop() {
size_t oldTail = tail.load();
size_t oldHead = head.load();
if (oldTail == oldHead) {
return;
}
fifo.pop();
head.store(++oldHead);
}
bool isEmpty() {
return head.load() == tail.load();
}
T& getFront() {
return fifo.front();
}
void waitForNewElements() {
if (tail.load() == head.load()) {
std::mutex m;
std::unique_lock<std::mutex> lock(m);
cvQueue.wait_for(lock, std::chrono::milliseconds(TIMEOUT_VALUE));
}
}
private:
std::queue<T> fifo;
std::atomic<size_t> head = { 0 };
std::atomic<size_t> tail = { 0 };
std::condition_variable cvQueue;
};
LockFreeQueue<std::vector<int>> dataQueue;
std::atomic<bool> store(true);
void setData(const std::vector<int>& data) {
dataQueue.push(data);
// do other things
}
void write(const std::string& fileName) {
std::unique_ptr<std::ostream> ofs = std::unique_ptr<std::ostream>(new zstr::ofstream(fileName));
while (store.load()) {
dataQueue.waitForNewElements();
while (!dataQueue.isEmpty()) {
std::vector<int>& data= dataQueue.getFront();
ofs->write(reinterpret_cast<char*>(data.data()), sizeof(data[0])*data.size());
dataQueue.pop();
}
}
}
}
I still have one lock in waitForNewElements but it is not locking the whole process as it is waiting for things to do. But the big improvement is that the producer can push while the consumer pop. It is only forbidden when LockFreQueue::tail and LockFreeQueue::head are the same. Meaning that the queue is empty and it enters the waiting state.
The thing that i'm not very satisfied at is cvQueue.wait_for(lock, TIMEOUT_VALUE). I wanted to do a simple cvQueue.wait(lock), but the problem is that when it comes to end the thread, I do store.store(false) in the main thread. So if the writing thread is waiting it will never end without a timeout. So, I set a big enough timeout so that most of the time the condition_variable is resolved by the lock, and when the thread ends it is resolved by the timeout.
If you feel that something must be wrong or must be improved, feel free to comment.
I have a timer that will create a new thread and wait for the timer to expire before calling the notify function. It works correctly during the first execution, but when the timer is started a second time, an exception is thrown trying to create the new thread. The debug output shows that the previous thread has exited before attempting to create the new thread.
Timer.hpp:
class TestTimer
{
private:
std::atomic<bool> active;
int timer_duration;
std::thread thread;
std::mutex mtx;
std::condition_variable cv;
void timer_func();
public:
TestTimer() : active(false) {};
~TestTimer() {
Stop();
}
TestTimer(const TestTimer&) = delete; /* Remove the copy constructor */
TestTimer(TestTimer&&) = delete; /* Remove the move constructor */
TestTimer& operator=(const TestTimer&) & = delete; /* Remove the copy assignment operator */
TestTimer& operator=(TestTimer&&) & = delete; /* Remove the move assignment operator */
bool IsActive();
void StartOnce(int TimerDurationInMS);
void Stop();
virtual void Notify() = 0;
};
Timer.cpp:
void TestTimer::timer_func()
{
auto expire_time = std::chrono::steady_clock::now() + std::chrono::milliseconds(timer_duration);
std::unique_lock<std::mutex> lock{ mtx };
while (active.load())
{
if (cv.wait_until(lock, expire_time) == std::cv_status::timeout)
{
lock.unlock();
Notify();
Stop();
lock.lock();
}
}
}
bool TestTimer::IsActive()
{
return active.load();
}
void TestTimer::StartOnce(int TimerDurationInMS)
{
if (!active.load())
{
if (thread.joinable())
{
thread.join();
}
timer_duration = TimerDurationInMS;
active.store(true);
thread = std::thread(&TestTimer::timer_func, this);
}
else
{
Stop();
StartOnce(TimerDurationInMS);
}
}
void TestTimer::Stop()
{
if (active.load())
{
std::lock_guard<std::mutex> _{ mtx };
active.store(false);
cv.notify_one();
}
}
The error is being thrown from my code block here:
thread = std::thread(&TestTimer::timer_func, this);
during the second execution.
Specifically, the error is being thrown from the move_thread function: _Thr = _Other._Thr;
thread& _Move_thread(thread& _Other)
{ // move from _Other
if (joinable())
_XSTD terminate();
_Thr = _Other._Thr;
_Thr_set_null(_Other._Thr);
return (*this);
}
_Thrd_t _Thr;
};
And this is the exception: Unhandled exception at 0x76ED550B (ucrtbase.dll) in Sandbox.exe: Fatal program exit requested.
Stack trace:
thread::move_thread(std::thread &_Other)
thread::operator=(std::thread &&_Other)
TestTimer::StartOnce(int TimerDurationInMS)
If it's just a test
Make sure the thread handler is empty or joined when calling the destructor.
Make everything that can be accessed from multiple threads thread safe (specifically, reading the active flag). Simply making it an std::atomic_flag should do.
It does seem like you are killing a thread handle pointing to a live thread, but hard to say without seeing the whole application.
If not a test
...then generally, when need a single timer, recurreing or not, you can just go away with scheduling an alarm() signal into itself. You remain perfectly single threaded and don't even need to link with the pthread library. Example here.
And when expecting to need more timers and stay up for a bit it is worth to drop an instance of boost::asio::io_service (or asio::io_service if you need a boost-free header-only version) into your application which has mature production-ready timers support. Example here.
You create the TestTimer and run it the first time via TestTimer::StartOnce, where you create a thread (at the line, which later throws the exception). When the thread finishes, it sets active = false; in timer_func.
Then you call TestTimer::StartOnce a second time. As active == false, Stop() is not called on the current thread, and you proceed to creating a new thread in thread = std::thread(&TestTimer::timer_func, this);.
And then comes the big but:
You have not joined the first thread before creating the second one. And that's why it throws an exception.
I am working on a c++ (11) project and on the main thread, I need to check the value of two variables. The value of the two variables will be set by other threads through two different callbacks. I am using two condition variables to notify changes of those two variables. Because in c++, locks are needed for condition variables, I am not sure if I should use the same mutex for the two condition variables or I should use two mutex's to minimize exclusive execution. Somehow, I feel one mutex should be sufficient because on one thread(the main thread in this case) the code will be executed sequentially anyway. The code on the main thread that checks (wait for) the value of the two variables wont be interleaved anyway. Let me know if you need me to write code to illustrate the problem. I can prepare that. Thanks.
Update, add code:
#include <mutex>
class SomeEventObserver {
public:
virtual void handleEventA() = 0;
virtual void handleEventB() = 0;
};
class Client : public SomeEventObserver {
public:
Client() {
m_shouldQuit = false;
m_hasEventAHappened = false;
m_hasEventBHappened = false;
}
// will be callbed by some other thread (for exampe, thread 10)
virtual void handleEventA() override {
{
std::lock_guard<std::mutex> lock(m_mutexForA);
m_hasEventAHappened = true;
}
m_condVarEventForA.notify_all();
}
// will be called by some other thread (for exampe, thread 11)
virtual void handleEventB() override {
{
std::lock_guard<std::mutex> lock(m_mutexForB);
m_hasEventBHappened = true;
}
m_condVarEventForB.notify_all();
}
// here waitForA and waitForB are in the main thread, they are executed sequentially
// so I am wondering if I can use just one mutex to simplify the code
void run() {
waitForA();
waitForB();
}
void doShutDown() {
m_shouldQuit = true;
}
private:
void waitForA() {
std::unique_lock<std::mutex> lock(m_mutexForA);
m_condVarEventForA.wait(lock, [this]{ return m_hasEventAHappened; });
}
void waitForB() {
std::unique_lock<std::mutex> lock(m_mutexForB);
m_condVarEventForB.wait(lock, [this]{ return m_hasEventBHappened; });
}
// I am wondering if I can use just one mutex
std::condition_variable m_condVarEventForA;
std::condition_variable m_condVarEventForB;
std::mutex m_mutexForA;
std::mutex m_mutexForB;
bool m_hasEventAHappened;
bool m_hasEventBHappened;
};
int main(int argc, char* argv[]) {
Client client;
client.run();
}
I have an object MainWorker ran as a separate thread thanks to moveToThread method.
MainWorker has a member SubWorker which is also ran as a separate thread. Both threads are working in infinite loops.
The idea is, MainWorker and SubWorker both perform some separate computations. Whenever SubWorker is done computing, it should notify MainWorker with the result.
Therefore I intuitively made first connection between signal emitted by SubWorker and a slot of MainWorker, but it wasn't working, so I made two more connections to rule out some potential problems:
connect(subWorker, &SubWorker::stuffDid, this, &MainWorker::reportStuff)); //1
connect(subWorker, &SubWorker::stuffDid, subWorker, &SubWorker::reportStuff); //2
connect(this, &MainWorker::stuffDid, this, &MainWorker::reportStuffSelf); //3
It seems, that what is not working is exactly what I need - cross thread communication, because connection 2 and 3 works as expected. My question is: How do I make connection 1 work?
Edit: Apparently, after Karsten's explanation, it is clear that infinite loop blocks the EventLoop. So the new question is, how can I send messages (signals, whatever) from an infinite loop thread to its parent thread?
I am new to Qt, there is a high chance that I got it completely wrong. Here goes the minimal (not)working example:
MainWorker.h
class MainWorker : public QObject
{
Q_OBJECT
public:
MainWorker() : run(false) {}
void doStuff()
{
subWorker = new SubWorker;
subWorkerThread = new QThread;
subWorker->moveToThread(subWorkerThread);
connect(subWorkerThread, &QThread::started, subWorker, &SubWorker::doStuff);
if(!connect(subWorker, &SubWorker::stuffDid, this, &MainWorker::reportStuff)) qDebug() << "connect failed";
connect(subWorker, &SubWorker::stuffDid, subWorker, &SubWorker::reportStuff);
connect(this, &MainWorker::stuffDid, this, &MainWorker::reportStuffSelf);
subWorkerThread->start();
run = true;
while(run)
{
QThread::currentThread()->msleep(200);
emit stuffDid();
}
}
private:
bool run;
QThread* subWorkerThread;
SubWorker* subWorker;
signals:
void stuffDid();
public slots:
void reportStuff()
{
qDebug() << "MainWorker: SubWorker did stuff";
}
void reportStuffSelf()
{
qDebug() << "MainWorker: MainWorker did stuff (EventLoop is not blocked)";
}
};
SubWorker.h
class SubWorker : public QObject
{
Q_OBJECT
public:
SubWorker() : run(false) {}
void doStuff()
{
run = true;
while(run)
{
qDebug() << "SubWorker: Doing stuff...";
QThread::currentThread()->msleep(1000);
emit stuffDid();
}
}
private:
bool run;
public slots:
void reportStuff()
{
qDebug() << "SubWorker: SubWorker did stuff";
}
signals:
void stuffDid();
};
main.cpp
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
MainWorker *mainWorker = new MainWorker;
QThread *mainWorkerThread = new QThread;
mainWorker->moveToThread(mainWorkerThread);
QObject::connect(mainWorkerThread, &QThread::started, mainWorker, &MainWorker::doStuff);
mainWorkerThread->start();
return a.exec();
}
Is there a way to start two (or more) C++11 threads and join() the first one that is finished?
An example scenario:
#include <iostream>
#include <thread>
using namespace std;
void prepare_item1() {std::cout << "Preparing 1" << std::endl;}
void consume_item1() {std::cout << "Consuming 1" << std::endl;}
void prepare_item2() {std::cout << "Preparing 2" << std::endl;}
void consume_item2() {std::cout << "Consuming 2" << std::endl;}
int main()
{
std::thread t1(prepare_item1);
std::thread t2(prepare_item2);
t1.join();
consume_item1();
t2.join();
consume_item2();
return 0;
}
I would have liked to do something like that instead:
int main()
{
std::thread t1(prepare_item1);
std::thread t2(prepare_item2);
finished_id=join_any(t1,t2)
if (finished_id==1)
{
consume_item1();
...
}
else if (finished_id==2)
{
consume_item2();
...
}
return 0;
}
Also, I would like the solution to be blocking, similar to the t.join() function.
Note: The real reason I need this is that I have two different blocking functions from which I receive commands, and whenever any of them is ready I would like to process the first command that arrives and continue to the next one when it is done. (sequential processing of commands from two parallel sources)
Thanks!
Here is a thread-safe multi-producer multi-consumer queue:
template<class T>
struct safe_queue {
std::deque<T> data;
std::atomic<bool> abort_flag = false;
std::mutex guard;
std::condition_variable signal;
template<class...Args>
void send( Args&&...args ) {
{
std::unique_lock<std::mutex> l(guard);
data.emplace_back(std::forward<Args>(args)...);
}
signal.notify_one();
}
void abort() {
abort_flag = true; // 1a
{ std::unique_lock<std::mutex>{guard}; }
signal.notify_all(); // 1b
}
std::experimental::optional<T> get() {
std::unique_lock<std::mutex> l(guard);
signal.wait( l, [this]()->bool{ // 2b
return !data.empty() || abort_flag.load(); // 2c
});
if (abort_flag.load()) return {};
T retval = std::move(data.front());
data.pop_front();
return retval;
}
};
have the threads shove data into the queue, and the main thread do a .get() on it.
If abort() is called, all waiting threads are woken up with an "empty" value from .get().
It uses std::experimental::optional, but you can replace that with something else (throw on abort? Whatever).
Code modified slightly from this other answer. Note that I think the other answer has some errors in it, which I corrected above, and attempts to solve a different problem.
The message you send could be the id of the thread that is ready to be waited upon, for example, or the work it has completed, or whatever.