As a part of a project I'm writing a logger function. This logger function sends an e-mail when the program wants to log something. Since it has happened that the SMTP server was non responsive, I've decided to do the sending of the mails in a separate thread.
This thread reads messages from an std::deque which is filled by the logging function.
The thread is setup as follows:
while (!boost::this_thread::interruption_requested())
{
EmailItem emailItem;
{
boost::unique_lock<boost::mutex> lock(mMutex);
while (mEmailBuffer.empty())
mCond.wait(lock);
bufferOverflow = mBufferOverflow;
mBufferOverflow = false;
nrOfItems = mEmailBuffer.size();
if (nrOfItems > 0)
{
emailItem = mEmailBuffer.front();
mEmailBuffer.pop_front();
}
}
if (nrOfItems > 0)
{
bool sent = false;
while(!sent)
{
try
{
..... Do something with the message .....
{
boost::this_thread::disable_interruption di;
boost::lock_guard<boost::mutex> lock(mLoggerMutex);
mLogFile << emailItem.mMessage << std::endl;
}
sent = true;
}
catch (const std::exception &e)
{
// Unable to send mail, an exception occurred. Retry sending it after some time
sent = false;
boost::this_thread::sleep(boost::posix_time::seconds(LOG_WAITBEFORE_RETRY));
}
}
}
}
The function log() adds a new message to the deque (mEmailBuffer) as follows:
{
boost::lock_guard<boost::mutex> lock(mMutex);
mEmailBuffer.push_back(e);
mCond.notify_one();
}
When the main program exits, the destructor of the logger object is called. This is where it goes wrong, the application crashes with an error:
/usr/include/boost/thread/pthread/mutex.hpp:45: boost::mutex::~mutex(): Assertion `!pthread_mutex_destroy(&m)' failed.
The destructor merely calls an interrupt on the thread and then joins it:
mQueueThread.interrupt();
mQueueThread.join();
In the main program, I use multiple different classes which make use of boost threading and mutex as well, could this cause this behaviour? Not calling the destructor of the logger object results in no errors, as does using the logger object and not doing anything else.
My guess is I am doing something very wrong, or there is a bug in the threading library when using multiple threads divided over several classes.
Does anyone have a idea what the reason for this error might be?
EDIT:
I did as #Andy T proposed and stripped the code as much as possible. I removed nearly everything in the function which is ran in a different thread. The thread now looks like:
void Vi::Logger::ThreadedQueue()
{
bool bufferOverflow = false;
time_t last_overflow = 0;
unsigned int nrOfItems = 0;
while (!boost::this_thread::interruption_requested())
{
EmailItem emailItem;
// Check for new log entries
{
boost::unique_lock<boost::mutex> lock(mMutex);
while (mEmailBuffer.empty())
mCond.wait(lock);
}
}
}
The problem still persists. Backtracking of the problem however showed me something different from the initial code:
#0 0x00007ffff53e9ba5 in raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1 0x00007ffff53ed6b0 in abort () at abort.c:92
#2 0x00007ffff53e2a71 in __assert_fail (assertion=0x7ffff7bb6407 "!pthread_mutex_lock(&m)", file=<value optimized out>, line=50, function=0x7ffff7bb7130 "void boost::mutex::lock()") at assert.c:81
#3 0x00007ffff7b930f3 in boost::mutex::lock (this=0x7fffe2c1b0b8) at /usr/include/boost/thread/pthread/mutex.hpp:50
#4 0x00007ffff7b9596c in boost::unique_lock<boost::mutex>::lock (this=0x7fffe48b3b40) at /usr/include/boost/thread/locks.hpp:349
#5 0x00007ffff7b958db in boost::unique_lock<boost::mutex>::unique_lock (this=0x7fffe48b3b40, m_=...) at /usr/include/boost/thread/locks.hpp:227
#6 0x00007ffff6ac2bb7 in Vi::Logger::ThreadedQueue (this=0x7fffe2c1ade0) at /data/repos_ViNotion/stdcomp/Logging/trunk/src/Logger.cpp:198
#7 0x00007ffff6acf2b2 in boost::_mfi::mf0<void, Vi::Logger>::operator() (this=0x7fffe2c1d890, p=0x7fffe2c1ade0) at /usr/include/boost/bind/mem_fn_template.hpp:49
#8 0x00007ffff6acf222 in boost::_bi::list1<boost::_bi::value<Vi::Logger*> >::operator()<boost::_mfi::mf0<void, Vi::Logger>, boost::_bi::list0> (this=0x7fffe2c1d8a0, f=..., a=...) at /usr/include/boost/bind/bind.hpp:253
#9 0x00007ffff6acf1bd in boost::_bi::bind_t<void, boost::_mfi::mf0<void, Vi::Logger>, boost::_bi::list1<boost::_bi::value<Vi::Logger*> > >::operator() (this=0x7fffe2c1d890) at /usr/include/boost/bind/bind_template.hpp:20
#10 0x00007ffff6aceff2 in boost::detail::thread_data<boost::_bi::bind_t<void, boost::_mfi::mf0<void, Vi::Logger>, boost::_bi::list1<boost::_bi::value<Vi::Logger*> > > >::run (this=0x7fffe2c1d760)
at /usr/include/boost/thread/detail/thread.hpp:56
#11 0x00007ffff2cc5230 in thread_proxy () from /usr/lib/libboost_thread.so.1.42.0
#12 0x00007ffff4d87971 in start_thread (arg=<value optimized out>) at pthread_create.c:304
#13 0x00007ffff549c92d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#14 0x0000000000000000 in ?? ()
Might it be possible that mMutex is not unlocked in the combination of using an unique_lock() and then interrupting the thread?
do you join your thread before exiting? as tyz suggested, your thread can still keep it locked when mutex is destroyed.
[EDIT]
you didn't provide complete example that can be compiled and run, it's hard to help w/o it.
check this simple example that should be similar to your one:
#include <boost/thread.hpp>
#include <boost/bind.hpp>
#include <queue>
class Test
{
public:
Test()
{
thread = boost::thread(boost::bind(&Test::thread_func, this));
}
~Test()
{
thread.interrupt();
thread.join();
}
void run()
{
for (size_t i = 0; i != 10000; ++i) {
boost::lock_guard<boost::mutex> lock(mutex);
queue.push(i);
condition_var.notify_one();
}
}
private:
void thread_func()
{
while (!boost::this_thread::interruption_requested())
{
{
boost::unique_lock<boost::mutex> lock(mutex);
while (queue.empty())
condition_var.wait(lock);
queue.pop();
}
}
}
private:
boost::thread thread;
boost::mutex mutex;
boost::condition_variable condition_var;
std::queue<int> queue;
};
int main()
{
Test test;
test.run();
return 0;
}
compare with your case
You should unlock mutex before you delete it.
Related
I'm reading C++ concurrency in action.
It introduces how to implement interrupting thread using std::condition_variable_any.
I try to understand the code more than a week, but I couldn't.
Below is the code and explanation in the book.
#include <condition_variable>
#include <future>
#include <iostream>
#include <thread>
class thread_interrupted : public std::exception {};
class interrupt_flag {
std::atomic<bool> flag;
std::condition_variable* thread_cond;
std::condition_variable_any* thread_cond_any;
std::mutex set_clear_mutex;
public:
interrupt_flag() : thread_cond(0), thread_cond_any(0) {}
void set() {
flag.store(true, std::memory_order_relaxed);
std::lock_guard<std::mutex> lk(set_clear_mutex);
if (thread_cond) {
thread_cond->notify_all();
} else if (thread_cond_any) {
thread_cond_any->notify_all();
}
}
bool is_set() const { return flag.load(std::memory_order_relaxed); }
template <typename Lockable>
void wait(std::condition_variable_any& cv, Lockable& lk);
};
thread_local static interrupt_flag this_thread_interrupt_flag;
void interruption_point() {
if (this_thread_interrupt_flag.is_set()) {
throw thread_interrupted();
}
}
template <typename Lockable>
void interrupt_flag::wait(std::condition_variable_any& cv, Lockable& lk) {
struct custom_lock {
interrupt_flag* self;
// (1) What is this lk for? Why is lk should be already locked when it is used in costume_lock constructor?
Lockable& lk;
custom_lock(interrupt_flag* self_, std::condition_variable_any& cond,
Lockable& lk_)
: self(self_), lk(lk_) {
self->set_clear_mutex.lock();
self->thread_cond_any = &cond;
}
void unlock() {
lk.unlock();
self->set_clear_mutex.unlock();
}
void lock() { std::lock(self->set_clear_mutex, lk); }
~custom_lock() {
self->thread_cond_any = 0;
self->set_clear_mutex.unlock();
}
};
custom_lock cl(this, cv, lk);
interruption_point();
cv.wait(cl);
interruption_point();
}
class interruptible_thread {
std::thread internal_thread;
interrupt_flag* flag;
public:
template <typename FunctionType>
interruptible_thread(FunctionType f) {
std::promise<interrupt_flag*> p;
internal_thread = std::thread([f, &p] {
p.set_value(&this_thread_interrupt_flag);
f();
});
flag = p.get_future().get();
}
void interrupt() {
if (flag) {
flag->set();
}
};
void join() { internal_thread.join(); };
void detach();
bool joinable() const;
};
template <typename Lockable>
void interruptible_wait(std::condition_variable_any& cv, Lockable& lk) {
this_thread_interrupt_flag.wait(cv, lk);
}
void foo() {
// (2) This is my implementation of how to use interruptible wait. Is it correct?
std::condition_variable_any cv;
std::mutex m;
std::unique_lock<std::mutex> lk(m);
try {
interruptible_wait(cv, lk);
} catch (...) {
std::cout << "interrupted" << std::endl;
}
}
int main() {
std::cout << "Hello" << std::endl;
interruptible_thread th(foo);
th.interrupt();
th.join();
}
Your custom lock type acquires the lock on the internal
set_clear_mutex when it’s constructed 1, and then sets the
thread_cond_any pointer to refer to the std:: condition_variable_any
passed in to the constructor 2.
The Lockable reference is stored for later; this must already be
locked. You can now check for an interruption without worrying about
races. If the interrupt flag is set at this point, it was set before
you acquired the lock on set_clear_mutex. When the condition variable
calls your unlock() function inside wait(), you unlock the Lockable
object and the internal set_clear_mutex 3.
This allows threads that are trying to interrupt you to acquire the
lock on set_clear_mutex and check the thread_cond_any pointer once
you’re inside the wait() call but not before. This is exactly what you
were after (but couldn’t manage) with std::condition_variable.
Once wait() has finished waiting (either because it was notified or
because of a spurious wake), it will call your lock() function, which
again acquires the lock on the internal set_clear_mutex and the lock
on the Lockable object 4. You can now check again for interruptions
that happened during the wait() call before clearing the
thread_cond_any pointer in your custom_lock destructor 5, where you
also unlock the set_clear_mutex.
First, I couldn't understand what is the purpose of Lockabel& lk in mark (1) and why it is already locked in constructor of custom_lock. (It could be locked in the very custom_lock constructor. )
Second there is no example in this book of how to use interruptible wait, so foo() {} in mark (2) is my guess implementation of how to use it. Is it correct way of using it ?
You need a mutex-like object (lk in your foo function) to call the interruptiple waiting just as you would need it for the plain std::condition_variable::wait function.
What's problematic (I also read the book and I have doubts about this example) is that the flag member points to a memory location inside the other thread which could finish right before calling flag->set(). In this specific example the thread only exists after we set the flag so that is okay, but otherwise this approach is limited in my opinion (correct me if I am wrong).
Along with the main thread, i have one more thread that receives data to write them in a file.
std::queue<std::vector<int>> dataQueue;
std::mutex mutex;
void setData(const std::vector<int>& data) {
std::lock_guard<std::mutex> lock(mutex);
dataQueue.push(data);
}
void write(const std::string& fileName) {
std::unique_ptr<std::ostream> ofs = std::unique_ptr<std::ostream>(new zstr::ofstream(fileName));
while (store) {
std::lock_guard<std::mutex> lock(mutex);
while (!dataQueue.empty()) {
std::vector<int>& data= dataQueue.front();
ofs->write(reinterpret_cast<char*>(data.data()), sizeof(data[0])*data.size());
dataQueue.pop();
}
}
}
}
setData is used by the main thread and write is actually the writing thread. I use std::lock_quard to avoid memory conflict but when locking on the writing thread, it slows down the main thread as it has to wait for the Queue to be unlocked. But i guess i can avoid this as the threads never act on the same element of the queue at the same time.
So i would like to do it lock-free but i don't really understand how i should implement it. I mean, how can i do it without locking anything ? moreover, if the writing thread is faster than the main thread, the queue might be empty most of the time, so it should somehow waits for new data instead of looping infinitly to check for non empty queue.
EDIT: I changed simple std::lock_guard by std::cond_variable so that it could wait when the queue is empty. But the main thread can still be blocked as , when cvQeue.wait(.) is resolved, it reacquire the lock. moreover, what if the main thread does cvQueue.notify_one() but the writing thread is not waiting ?
std::queue<std::vector<int>> dataQueue;
std::mutex mutex;
std::condition_variable cvQueue;
void setData(const std::vector<int>& data) {
std::unique_lock<std::mutex> lock(mutex);
dataQueue.push(data);
cvQueue.notify_one();
}
void write(const std::string& fileName) {
std::unique_ptr<std::ostream> ofs = std::unique_ptr<std::ostream>(new zstr::ofstream(fileName));
while (store) {
std::lock_guard<std::mutex> lock(mutex);
while (!dataQueue.empty()) {
std::unique_lock<std::mutex> lock(mutex);
cvQueue.wait(lock);
ofs->write(reinterpret_cast<char*>(data.data()), sizeof(data[0])*data.size());
dataQueue.pop();
}
}
}
}
If you only have two threads, than you could use a lock-free single-producer-single-consumer (SPSC) queue.
A bounded version can be found here: https://github.com/rigtor/SPSCQueue
Dmitry Vyukov presented an unbounded version here: http://www.1024cores.net/home/lock-free-algorithms/queues/unbounded-spsc-queue (You should note though, that this code should be adapted to use atomics.)
Regarding a blocking pop operation - this is something that lock-free data structures do not provide since such an operation is obviously not lock-free. However, it should be relatively straight forward to adapt the linked implementations in such a way, that a push operation notifies a condition variable if the queue was empty before the push.
i guess i have something that met my needs. I did a LockFreeQueue that uses std::atomic. I can thus manage the state of the head/tail of the queue atomically.
template<typename T>
class LockFreeQueue {
public:
void push(const T& newElement) {
fifo.push(newElement);
tail.fetch_add(1);
cvQueue.notify_one();
}
void pop() {
size_t oldTail = tail.load();
size_t oldHead = head.load();
if (oldTail == oldHead) {
return;
}
fifo.pop();
head.store(++oldHead);
}
bool isEmpty() {
return head.load() == tail.load();
}
T& getFront() {
return fifo.front();
}
void waitForNewElements() {
if (tail.load() == head.load()) {
std::mutex m;
std::unique_lock<std::mutex> lock(m);
cvQueue.wait_for(lock, std::chrono::milliseconds(TIMEOUT_VALUE));
}
}
private:
std::queue<T> fifo;
std::atomic<size_t> head = { 0 };
std::atomic<size_t> tail = { 0 };
std::condition_variable cvQueue;
};
LockFreeQueue<std::vector<int>> dataQueue;
std::atomic<bool> store(true);
void setData(const std::vector<int>& data) {
dataQueue.push(data);
// do other things
}
void write(const std::string& fileName) {
std::unique_ptr<std::ostream> ofs = std::unique_ptr<std::ostream>(new zstr::ofstream(fileName));
while (store.load()) {
dataQueue.waitForNewElements();
while (!dataQueue.isEmpty()) {
std::vector<int>& data= dataQueue.getFront();
ofs->write(reinterpret_cast<char*>(data.data()), sizeof(data[0])*data.size());
dataQueue.pop();
}
}
}
}
I still have one lock in waitForNewElements but it is not locking the whole process as it is waiting for things to do. But the big improvement is that the producer can push while the consumer pop. It is only forbidden when LockFreQueue::tail and LockFreeQueue::head are the same. Meaning that the queue is empty and it enters the waiting state.
The thing that i'm not very satisfied at is cvQueue.wait_for(lock, TIMEOUT_VALUE). I wanted to do a simple cvQueue.wait(lock), but the problem is that when it comes to end the thread, I do store.store(false) in the main thread. So if the writing thread is waiting it will never end without a timeout. So, I set a big enough timeout so that most of the time the condition_variable is resolved by the lock, and when the thread ends it is resolved by the timeout.
If you feel that something must be wrong or must be improved, feel free to comment.
I have a timer that will create a new thread and wait for the timer to expire before calling the notify function. It works correctly during the first execution, but when the timer is started a second time, an exception is thrown trying to create the new thread. The debug output shows that the previous thread has exited before attempting to create the new thread.
Timer.hpp:
class TestTimer
{
private:
std::atomic<bool> active;
int timer_duration;
std::thread thread;
std::mutex mtx;
std::condition_variable cv;
void timer_func();
public:
TestTimer() : active(false) {};
~TestTimer() {
Stop();
}
TestTimer(const TestTimer&) = delete; /* Remove the copy constructor */
TestTimer(TestTimer&&) = delete; /* Remove the move constructor */
TestTimer& operator=(const TestTimer&) & = delete; /* Remove the copy assignment operator */
TestTimer& operator=(TestTimer&&) & = delete; /* Remove the move assignment operator */
bool IsActive();
void StartOnce(int TimerDurationInMS);
void Stop();
virtual void Notify() = 0;
};
Timer.cpp:
void TestTimer::timer_func()
{
auto expire_time = std::chrono::steady_clock::now() + std::chrono::milliseconds(timer_duration);
std::unique_lock<std::mutex> lock{ mtx };
while (active.load())
{
if (cv.wait_until(lock, expire_time) == std::cv_status::timeout)
{
lock.unlock();
Notify();
Stop();
lock.lock();
}
}
}
bool TestTimer::IsActive()
{
return active.load();
}
void TestTimer::StartOnce(int TimerDurationInMS)
{
if (!active.load())
{
if (thread.joinable())
{
thread.join();
}
timer_duration = TimerDurationInMS;
active.store(true);
thread = std::thread(&TestTimer::timer_func, this);
}
else
{
Stop();
StartOnce(TimerDurationInMS);
}
}
void TestTimer::Stop()
{
if (active.load())
{
std::lock_guard<std::mutex> _{ mtx };
active.store(false);
cv.notify_one();
}
}
The error is being thrown from my code block here:
thread = std::thread(&TestTimer::timer_func, this);
during the second execution.
Specifically, the error is being thrown from the move_thread function: _Thr = _Other._Thr;
thread& _Move_thread(thread& _Other)
{ // move from _Other
if (joinable())
_XSTD terminate();
_Thr = _Other._Thr;
_Thr_set_null(_Other._Thr);
return (*this);
}
_Thrd_t _Thr;
};
And this is the exception: Unhandled exception at 0x76ED550B (ucrtbase.dll) in Sandbox.exe: Fatal program exit requested.
Stack trace:
thread::move_thread(std::thread &_Other)
thread::operator=(std::thread &&_Other)
TestTimer::StartOnce(int TimerDurationInMS)
If it's just a test
Make sure the thread handler is empty or joined when calling the destructor.
Make everything that can be accessed from multiple threads thread safe (specifically, reading the active flag). Simply making it an std::atomic_flag should do.
It does seem like you are killing a thread handle pointing to a live thread, but hard to say without seeing the whole application.
If not a test
...then generally, when need a single timer, recurreing or not, you can just go away with scheduling an alarm() signal into itself. You remain perfectly single threaded and don't even need to link with the pthread library. Example here.
And when expecting to need more timers and stay up for a bit it is worth to drop an instance of boost::asio::io_service (or asio::io_service if you need a boost-free header-only version) into your application which has mature production-ready timers support. Example here.
You create the TestTimer and run it the first time via TestTimer::StartOnce, where you create a thread (at the line, which later throws the exception). When the thread finishes, it sets active = false; in timer_func.
Then you call TestTimer::StartOnce a second time. As active == false, Stop() is not called on the current thread, and you proceed to creating a new thread in thread = std::thread(&TestTimer::timer_func, this);.
And then comes the big but:
You have not joined the first thread before creating the second one. And that's why it throws an exception.
I'm trying to implement an multi-thread job, a producer and a consumer, and basically what I want to do is, when consumer finishes the data, it notifies the producer so that producer provides new data.
The tricky part is, in my current impl, producer and consumer both notifies each other and waits for each other, I don't know how to implement this part correctly.
For example, see the code below,
mutex m;
condition_variable cv;
vector<int> Q; // this is the queue the consumer will consume
vector<int> Q_buf; // this is a buffer Q into which producer will fill new data directly
// consumer
void consume() {
while (1) {
if (Q.size() == 0) { // when consumer finishes data
unique_lock<mutex> lk(m);
// how to notify producer to fill up the Q?
...
cv.wait(lk);
}
// for-loop to process the elems in Q
...
}
}
// producer
void produce() {
while (1) {
// for-loop to fill up Q_buf
...
// once Q_buf is fully filled, wait until consumer asks to give it a full Q
unique_lock<mutex> lk(m);
cv.wait(lk);
Q.swap(Q_buf); // replace the empty Q with the full Q_buf
cv.notify_one();
}
}
I'm not sure this the above code using mutex and condition_variable is the right way to implement my idea,
please give me some advice!
The code incorrectly assumes that vector<int>::size() and vector<int>::swap() are atomic. They are not.
Also, spurious wakeups must be handled by a while loop (or another cv::wait overload).
Fixes:
mutex m;
condition_variable cv;
vector<int> Q;
// consumer
void consume() {
while(1) {
// Get the new elements.
vector<int> new_elements;
{
unique_lock<mutex> lk(m);
while(Q.empty())
cv.wait(lk);
new_elements.swap(Q);
}
// for-loop to process the elems in new_elements
}
}
// producer
void produce() {
while(1) {
vector<int> new_elements;
// for-loop to fill up new_elements
// publish new_elements
{
unique_lock<mutex> lk(m);
Q.insert(Q.end(), new_elements.begin(), new_elements.end());
cv.notify_one();
}
}
}
Maybe that is close to what you want to achive. I used 2 conditional variables to notify producers and consumers between each other and introduced variable denoting which turn is now:
#include <ctime>
#include <condition_variable>
#include <iostream>
#include <mutex>
#include <queue>
#include <thread>
template<typename T>
class ReaderWriter {
private:
std::vector<std::thread> readers;
std::vector<std::thread> writers;
std::condition_variable readerCv, writerCv;
std::queue<T> data;
std::mutex readerMutex, writerMutex;
size_t noReaders, noWriters;
enum class Turn { WRITER_TURN, READER_TURN };
Turn turn;
void reader() {
while (1) {
{
std::unique_lock<std::mutex> lk(readerMutex);
while (turn != Turn::READER_TURN) {
readerCv.wait(lk);
}
std::cout << "Thread : " << std::this_thread::get_id() << " consumed " << data.front() << std::endl;
data.pop();
if (data.empty()) {
turn = Turn::WRITER_TURN;
writerCv.notify_one();
}
}
}
}
void writer() {
while (1) {
{
std::unique_lock<std::mutex> lk(writerMutex);
while (turn != Turn::WRITER_TURN) {
writerCv.wait(lk);
}
srand(time(NULL));
int random_number = std::rand();
data.push(random_number);
std::cout << "Thread : " << std::this_thread::get_id() << " produced " << random_number << std::endl;
turn = Turn::READER_TURN;
}
readerCv.notify_one();
}
}
public:
ReaderWriter(size_t noReadersArg, size_t noWritersArg) : noReaders(noReadersArg), noWriters(noWritersArg), turn(ReaderWriter::Turn::WRITER_TURN) {
}
void run() {
int noReadersArg = noReaders, noWritersArg = noWriters;
while (noReadersArg--) {
readers.emplace_back(&ReaderWriter::reader, this);
}
while (noWritersArg--) {
writers.emplace_back(&ReaderWriter::writer, this);
}
}
~ReaderWriter() {
for (auto& r : readers) {
r.join();
}
for (auto& w : writers) {
w.join();
}
}
};
int main() {
ReaderWriter<int> rw(5, 5);
rw.run();
}
Here's a code snippet. Since the worker treads are already synchronized, requirement of two buffers is ruled out. So a simple queue is used to simulate the scenario:
#include "conio.h"
#include <iostream>
#include <thread>
#include <mutex>
#include <queue>
#include <atomic>
#include <condition_variable>
using namespace std;
enum state_t{ READ = 0, WRITE = 1 };
mutex mu;
condition_variable cv;
atomic<bool> running;
queue<int> buffer;
atomic<state_t> state;
void generate_test_data()
{
const int times = 5;
static int data = 0;
for (int i = 0; i < times; i++) {
data = (data++) % 100;
buffer.push(data);
}
}
void ProducerThread() {
while (running) {
unique_lock<mutex> lock(mu);
cv.wait(lock, []() { return !running || state == WRITE; });
if (!running) return;
generate_test_data(); //producing here
lock.unlock();
//notify consumer to start consuming
state = READ;
cv.notify_one();
}
}
void ConsumerThread() {
while (running) {
unique_lock<mutex> lock(mu);
cv.wait(lock, []() { return !running || state == READ; });
if (!running) return;
while (!buffer.empty()) {
auto data = buffer.front(); //consuming here
buffer.pop();
cout << data << " \n";
}
//notify producer to start producing
if (buffer.empty()) {
state = WRITE;
cv.notify_one();
}
}
}
int main(){
running = true;
thread producer = thread([]() { ProducerThread(); });
thread consumer = thread([]() { ConsumerThread(); });
//simulating gui thread
while (!getch()){
}
running = false;
producer.join();
consumer.join();
}
Not a complete answer, though I think two condition variables could be helpful, one named buffer_empty that the producer thread will wait on, and another named buffer_filled that the consumer thread will wait on. Number of mutexes, how to loop, and so on I cannot comment on, since I'm not sure about the details myself.
Accesses to shared variables should only be done while holding the
mutex that protects it
condition_variable::wait should check a condition.
The condition should be a shared variable protected by the mutex that you pass to condition_variable::wait.
The way to check the condition is to wrap the call to wait in a while loop or use the 2-argument overload of wait (which is
equivalent to the while-loop version)
Note: These rules aren't strictly necessary if you truly understand what the hardware is doing. However, these problems get complicated quickly when with simple data structures, and it will be easier to prove that your algorithm is working correctly if you follow them.
Your Q and Q_buf are shared variables. Due to Rule 1, I would prefer to have them as local variables declared in the function that uses them (consume() and produce(), respectively). There will be 1 shared buffer that will be protected by a mutex. The producer will add to its local buffer. When that buffer is full, it acquires the mutex and pushes the local buffer to the shared buffer. It then waits for the consumer to accept this buffer before producing more data.
The consumer waits for this shared buffer to "arrive", then it acquires the mutex and replaces its empty local buffer with the shared buffer. Then it signals to the producer that the buffer has been accepted so it knows to start producing again.
Semantically, I don't see a reason to use swap over move, since in every case one of the containers is empty anyway. Maybe you want to use swap because you know something about the underlying memory. You can use whichever you want and it will be fast and work the same (at least algorithmically).
This problem can be done with 1 condition variable, but it may be a little easier to think about if you use 2.
Here's what I came up with. Tested on Visual Studio 2017 (15.6.7) and GCC 5.4.0. I don't need to be credited or anything (it's such a simple piece), but legally I have to say that I offer no warranties whatsoever.
#include <thread>
#include <vector>
#include <mutex>
#include <condition_variable>
#include <chrono>
std::vector<int> g_deliveryBuffer;
bool g_quit = false;
std::mutex g_mutex; // protects g_deliveryBuffer and g_quit
std::condition_variable g_producerDeliver;
std::condition_variable g_consumerAccepted;
// consumer
void consume()
{
// local buffer
std::vector<int> consumerBuffer;
while (true)
{
if (consumerBuffer.empty())
{
std::unique_lock<std::mutex> lock(g_mutex);
while (g_deliveryBuffer.empty() && !g_quit) // if we beat the producer, wait for them to push to the deliverybuffer
g_producerDeliver.wait(lock);
if (g_quit)
break;
consumerBuffer = std::move(g_deliveryBuffer); // get the buffer
}
g_consumerAccepted.notify_one(); // notify the producer that the buffer has been accepted
// for-loop to process the elems in Q
// ...
consumerBuffer.clear();
// ...
}
}
// producer
void produce()
{
std::vector<int> producerBuffer;
while (true)
{
// for-loop to fill up Q_buf
// ...
producerBuffer = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 };
// ...
// once Q_buf is fully filled, wait until consumer asks to give it a full Q
{ // scope is for lock
std::unique_lock<std::mutex> lock(g_mutex);
g_deliveryBuffer = std::move(producerBuffer); // ok to push to deliverybuffer. it is guaranteed to be empty
g_producerDeliver.notify_one();
while (!g_deliveryBuffer.empty() && !g_quit)
g_consumerAccepted.wait(lock); // wait for consumer to signal for more data
if (g_quit)
break;
// We will never reach this point if the buffer is not empty.
}
}
}
int main()
{
// spawn threads
std::thread consumerThread(consume);
std::thread producerThread(produce);
// for for 5 seconds
std::this_thread::sleep_for(std::chrono::seconds(5));
// signal that it's time to quit
{
std::lock_guard<std::mutex> lock(g_mutex);
g_quit = true;
}
// one of the threads may be sleeping
g_consumerAccepted.notify_one();
g_producerDeliver.notify_one();
consumerThread.join();
producerThread.join();
return 0;
}
I have a reusable class that starts up an infinite thread. this thread can only be killed by calling a stop function that sets a kill switch variable. When looking around, there is quite a bit of argument over volatile vs atomic variables.
The following is my code:
program.cpp
int main()
{
ThreadClass threadClass;
threadClass.Start();
Sleep(1000);
threadClass.Stop();
Sleep(50);
threaClass.Stop();
}
ThreadClass.h
#pragma once
#include <atomic>
#include <thread>
class::ThreadClass
{
public:
ThreadClass(void);
~ThreadClass(void);
void Start();
void Stop();
private:
void myThread();
std::atomic<bool> runThread;
std::thread theThread;
};
ThreadClass.cpp
#include "ThreadClass.h"
ThreadClass::ThreadClass(void)
{
runThread = false;
}
ThreadClass::~ThreadClass(void)
{
}
void ThreadClass::Start()
{
runThread = true;
the_thread = std::thread(&mythread, this);
}
void ThreadClass::Stop()
{
if(runThread)
{
runThread = false;
if (the_thread.joinable())
{
the_thread.join();
}
}
}
void ThreadClass::mythread()
{
while(runThread)
{
//dostuff
Sleep(100); //or chrono
}
}
The code that i am representing here mirrors an issue that our legacy code had in place. We call the stop function 2 times, which will try to join the thread 2 times. This results in an invalid handle exception. I have coded the Stop() function in order to work around that issue, but my question is why would the the join fail the second time if the thread has completed and joined? Is there a better way programmatically to assume that the thread is valid before trying to join?