Consumer thread is not synchronized until producer finishes its execution - multithreading

Question

I'm learning C++ (14) and currently I'm working on the concurrency part.
I've written a small producer/consumer example to learn and understand condition_variables.
The idea is for one thread to fill up a vector with numbers and then the other to print it to console:
#include <iostream>
#include <thread>
#include <condition_variable>
#include <vector>
#include <atomic>
std::atomic<bool> done{false};
std::mutex ready_mutex;
std::condition_variable ready;
std::vector<int> numbers;
void produce() {
auto idx = 0;
while( true ) {
++idx;
std::lock_guard<std::mutex> lk(ready_mutex);
for(int i = 0; i < 5; ++i) {
numbers.push_back(i);
}
if( 5 == idx) {
done = true;
break;
}
ready.notify_one();
}
ready.notify_one();
}
void consume() {
while( true ) {
std::unique_lock<std::mutex> lk(ready_mutex);
ready.wait(lk, [](){ return !v.empty(); });
std::cout << "(" << numbers.size() << ")" << std::endl;
for(auto x: numbers) {
std::cout << x << ", ";
}
std::cout << std::endl;
numbers.clear();
std::cout << "(" << numbers.size() << ")" << std::endl;
std::cout.flush();
if(done) break;
}
}
int main() {
std::thread t(produce);
std::thread t2(consume);
t.join();
t2.join();
return 0;
}
The problem is that the program is not showing me the expected output.
What I'm expecting is:
(5) 0, 1, 2, 3, 4, (0)
(5) 0, 1, 2, 3, 4, (0)
(5) 0, 1, 2, 3, 4, (0)
(5) 0, 1, 2, 3, 4, (0)
(5) 0, 1, 2, 3, 4, (0)
But what I get is:
(25)
0, 1, 2, 3, 4, 0, 1, 2, 3, 4, 0, 1, 2, 3, 4, 0, 1, 2, 3, 4, 0, 1, 2, 3, 4,
(0)
For what I see, the producer thread runs all iterations in one go and then notifies the consumer thread, yet after a few hours I still don't understand why the producer is not waking up the consumer until the end.
Also, I have noticed that this problem persists if instead of using a global vector, I pass it as a reference to the threads:
void producerer(std::vector<int>& v);
void consumer(std::vector<int>& v);
std::vector<int> numbers;
std::thread t(producer, std::ref(numbers));
std::thread t2(consumerer, std::ref(numbers));
OS: Debian 9
Compilers: g++ 6.3.0, clang++ 3.8.1-24
Compiling flags: -Werror -Wextra -Wall -std=c++14 -O0 -g3 -pthread

Answer 1

The issue here is that when the producer releases the lock, you're expecting the consumer to acquire the lock next. However, because of the while loop in the producer, it's possible that the producer re-acquires the lock before the consumer can. If the producer only "produced" once and then yielded to the consumer, there wouldn't be an issue but since you want a ping-pong effect, you need to ensure that the producer waits its turn in the same way the consumer is waiting its turn.
One way to solve this is with another boolean. I'll call it canProduce and set up the producer in the same way you set up the consumer. I'll also repeat the entire process 15 times in main, to minimize the chance of a fluke false positive, should the producer still not be correctly waiting its turn.
#include <iostream>
#include <thread>
#include <condition_variable>
#include <vector>
#include <atomic>
std::atomic<bool> done{false};
std::atomic<bool> canProduce{true};
std::mutex ready_mutex;
std::condition_variable ready;
std::vector<int> numbers;
void produce() {
auto idx = 0;
while( true ) {
++idx;
std::unique_lock<std::mutex> lk(ready_mutex);
ready.wait(lk, [](){ return canProduce == true; });
for(int i = 0; i < 5; ++i) {
numbers.push_back(i);
}
canProduce = false;
// Manually unlocking thread to ensure that when the consumer thread
// wakes up, this thread is unlocked and the consumer thread can acquire
// the lock, otherwise it'll go back to sleep.
lk.unlock();
ready.notify_one();
if( 5 == idx) {
done = true;
break;
}
}
}
void consume() {
while( true ) {
std::unique_lock<std::mutex> lk(ready_mutex);
ready.wait(lk, [](){ return !numbers.empty(); });
std::cout << "(" << numbers.size() << ") ";
for(auto x: numbers) {
std::cout << x << ", ";
}
numbers.clear();
std::cout << "(" << numbers.size() << ")" << std::endl;
std::cout.flush();
if (done) {
break;
} else {
canProduce = true;
lk.unlock();
ready.notify_one();
}
}
}
int main() {
for (int i = 0; i < 15; i++) {
std::cout << "Attempt " << i << std::endl;
std::thread t(produce);
std::thread t2(consume);
t.join();
t2.join();
done = false;
canProduce = true;
}
return 0;
}
Output
Attempt 1
(5) 0, 1, 2, 3, 4, (0)
(5) 0, 1, 2, 3, 4, (0)
(5) 0, 1, 2, 3, 4, (0)
(5) 0, 1, 2, 3, 4, (0)
(5) 0, 1, 2, 3, 4, (0)
Attempt 2
(5) 0, 1, 2, 3, 4, (0)
(5) 0, 1, 2, 3, 4, (0)
(5) 0, 1, 2, 3, 4, (0)
(5) 0, 1, 2, 3, 4, (0)
(5) 0, 1, 2, 3, 4, (0)
...