Different mutex lock behaviour when using a condition_variable - multithreading

I use a mutex in two different situations:
- first example: I use the mutex with unique_lock to make sure that threads don't access the same resource simultaneously
- second example: I expand my first example to use a condition_variable, so that all threads wait until this additional thread notifies them.
Here is my first example
#include <thread>
#include <mutex>
#include <condition_variable>
#include <iostream>
using namespace std;
mutex Mutex;
condition_variable cv;
bool ready = false;
void print(const char* ThreadName,int WaitTime)
{
cout << ThreadName << " : Waiting to get lock!" << endl;
unique_lock<mutex> lock(Mutex);
cout << ThreadName << " : Got the lock" << endl;
this_thread::sleep_for(chrono::milliseconds(WaitTime));
while (!ready)
{
cv.wait(lock);
}
cout<< ThreadName << " : thread is finishing now...." << endl;
}
void execute(const char* ThreadName)
{
this_thread::sleep_for(chrono::milliseconds(2000));
cout<< ThreadName << "Thready is ready to be executed!" << endl;
ready = true;
cv.notify_all();
}
int main()
{
thread t1(print, "Print1",200);
thread t2(print, "Print2",1000);
thread t3(print, "Print3",500);
thread t4(print, "Print4",10);
thread te(execute, "Execute");
t1.join();
t2.join();
t3.join();
t4.join();
te.join();
return 0;
}
The result of this is :
Print1Print3 : Waiting to get lock!Print2 : Waiting to get lock!
Print2 : Got the lock
Print4 : Waiting to get lock!
: Waiting to get lock!
Print2 : thread is finishing now....
Print3 : Got the lock
Print3 : thread is finishing now....
Print4 : Got the lock
Print4 : thread is finishing now....
Print1 : Got the lock
Print1 : thread is finishing now....
We can see that the first thread that gets a hold of the mutex, can do his thing, and only once its finished, the next thread can get beyond the unique_lock lock(Mutex); statement
Now I expand this example to use a condition_variable
#include <thread>
#include <mutex>
#include <condition_variable>
#include <iostream>
using namespace std;
mutex Mutex;
condition_variable cv;
bool ready = false;
void print(const char* ThreadName,int WaitTime)
{
cout << ThreadName << " : Waiting to get lock!" << endl;
unique_lock<mutex> lock(Mutex);
cout << ThreadName << " : Got the lock" << endl;
this_thread::sleep_for(chrono::milliseconds(WaitTime));
while (!ready)
{
cv.wait(lock);
}
cout<< ThreadName << " : thread is finishing now...." << endl;
}
void execute(const char* ThreadName)
{
this_thread::sleep_for(chrono::milliseconds(2000));
cout<< ThreadName << "Thready is ready to be executed!" << endl;
ready = true;
cv.notify_all();
}
int main()
{
thread t1(print, "Print1",200);
thread t2(print, "Print2",1000);
thread t3(print, "Print3",500);
thread t4(print, "Print4",10);
thread te(execute, "Execute");
t1.join();
t2.join();
t3.join();
t4.join();
te.join();
return 0;
}
Output from this one is
Print1Print3: Waiting to get lock!
: Waiting to get lock!
Print2 : Waiting to get lock!
Print4 : Waiting to get lock!
Print3 : Got the lock
Print1 : Got the lock
Print4 : Got the lock
Print2 : Got the lock
ExecuteThready is ready to be executed!
Print2 : thread is finishing now....
Print4 : thread is finishing now....
Print1 : thread is finishing now....
Print3 : thread is finishing now....
What I don't understand is how all 4 threads can get a lock on the mutex, while there is nowhere a link between the condition_variable and the mutex?

...when there is nowhere a link between the condition_variable and the mutex?
The link is here:
cv.wait(lock);
The wait function does three things before it returns:
it unlocks the given lock,
it waits for some other thread to call cv.notify_all(), and then
it re-locks the lock.
Of course, if some other thread was awakened first, then it may have to wait to re-lock the lock after it has awakened from the notification.

Related

Interruptible sleep in std::thread

I have a simple C++11 thread program like below.
Code:
#include <iostream>
#include <thread>
#include <chrono>
#include <atomic>
int main(int argc, char *argv[]) {
std::cout << "My program starts" << std::endl;
std::atomic<bool> exit_thread(false);
std::thread my_thread = std::thread([&exit_thread]{
do {
std::cout << "Thread is doing something..." << std::endl;
std::this_thread::sleep_for(std::chrono::seconds(5));
} while (!exit_thread);
});
std::this_thread::sleep_for(std::chrono::seconds(12));
exit_thread = true;
std::cout << "Might have to wait to exit thread" << std::endl;
my_thread.join();
return 0;
}
As you can see above, there is a loop which has a sleep_for which makes the thread sleep for 5 seconds and then it wakes and loops again provided that exit_thread is set to false. Main thread waits for 12 seconds and prepares to exit firstly by setting exit_thread to true and then does a join on the thread. All good until now.
Problem:
Above is okay and works for objective. But there is a "potential problem". If the thread has just now started to sleep then it would take it 4 seconds more before it gets out of sleep to discover that it now needs to exit. This delays the exit process and destruction.
Question:
How to can I make the thread sleep in an interruptible way? So that I can interrupt the sleep and make the thread exit right away instead by cancelling out of sleep instead of waiting for the potential 4 or 3 or 2 seconds.
I think that the solution to this might be achievable using a std::condition_variable? Probably? I am looking for a piece of code to show how.
Note that my code runs on both clang and gcc.
We should be waiting on a condition variable or semaphore instead of sleeping. Here's the minimal change to do that:
#include <atomic>
#include <chrono>
#include <condition_variable>
#include <iostream>
#include <mutex>
#include <thread>
int main()
{
std::cout << "My program starts" << std::endl;
std::atomic<bool> exit_thread(false);
std::condition_variable cv;
std::mutex m;
std::thread my_thread = std::thread([&exit_thread,&cv,&m]{
do {
std::cout << "Thread is doing something..." << std::endl;
{
std::unique_lock<std::mutex> lock(m);
cv.wait_for(lock, std::chrono::seconds(5));
}
} while (!exit_thread);
});
std::this_thread::sleep_for(std::chrono::seconds(12));
{
std::lock_guard<std::mutex> guard(m);
exit_thread = true;
}
cv.notify_all();
std::cout << "Thread stops immediately" << std::endl;
my_thread.join();
}
Apparently, we do need the mutex:
Even if the shared variable is atomic, it must be modified under the
mutex in order to correctly publish the modification to the waiting
thread.

problem with understanding example of code from Boost.Fiber library

I am currently trying (in order to learn) to understand example of code from the Boost.Fiber library : https://www.boost.org/doc/libs/1_71_0/libs/fiber/examples/work_sharing.cpp
// Copyright Nat Goodspeed + Oliver Kowalke 2015.
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// http://www.boost.org/LICENSE_1_0.txt)
#include <chrono>
#include <condition_variable>
#include <cstddef>
#include <deque>
#include <iomanip>
#include <iostream>
#include <mutex>
#include <sstream>
#include <string>
#include <thread>
#include <boost/assert.hpp>
#include <boost/fiber/all.hpp>
#include <boost/fiber/detail/thread_barrier.hpp>
static std::size_t fiber_count{ 0 };
static std::mutex mtx_count{};
static boost::fibers::condition_variable_any cnd_count{};
typedef std::unique_lock< std::mutex > lock_type;
/*****************************************************************************
* example fiber function
*****************************************************************************/
//[fiber_fn_ws
void whatevah( char me) {
try {
std::thread::id my_thread = std::this_thread::get_id(); /**< get ID of initial thread >*/
{
std::ostringstream buffer;
buffer << "fiber " << me << " started on thread " << my_thread << '\n';
std::cout << buffer.str() << std::flush;
}
for ( unsigned i = 0; i < 10; ++i) { /**< loop ten times >*/
boost::this_fiber::yield(); /**< yield to other fibers >*/
std::thread::id new_thread = std::this_thread::get_id(); /**< get ID of current thread >*/
if ( new_thread != my_thread) { /**< test if fiber was migrated to another thread >*/
my_thread = new_thread;
std::ostringstream buffer;
buffer << "fiber " << me << " switched to thread " << my_thread << '\n';
std::cout << buffer.str() << std::flush;
}
}
} catch ( ... ) {
}
lock_type lk( mtx_count);
if ( 0 == --fiber_count) { /**< Decrement fiber counter for each completed fiber. >*/
lk.unlock();
cnd_count.notify_all(); /**< Notify all fibers waiting on `cnd_count`. >*/
}
}
//]
/*****************************************************************************
* example thread function
*****************************************************************************/
//[thread_fn_ws
void thread( boost::fibers::detail::thread_barrier * b) {
std::ostringstream buffer;
buffer << "thread started " << std::this_thread::get_id() << std::endl;
std::cout << buffer.str() << std::flush;
boost::fibers::use_scheduling_algorithm< boost::fibers::algo::shared_work >(); /**<
Install the scheduling algorithm `boost::fibers::algo::shared_work` in order to
join the work sharing.
>*/
b->wait(); /**< sync with other threads: allow them to start processing >*/
lock_type lk( mtx_count);
cnd_count.wait( lk, [](){ return 0 == fiber_count; } ); /**<
Suspend main fiber and resume worker fibers in the meanwhile.
Main fiber gets resumed (e.g returns from `condition_variable_any::wait()`)
if all worker fibers are complete.
>*/
BOOST_ASSERT( 0 == fiber_count);
}
//]
/*****************************************************************************
* main()
*****************************************************************************/
int main( int argc, char *argv[]) {
std::cout << "main thread started " << std::this_thread::get_id() << std::endl;
//[main_ws
boost::fibers::use_scheduling_algorithm< boost::fibers::algo::shared_work >(); /*<
Install the scheduling algorithm `boost::fibers::algo::shared_work` in the main thread
too, so each new fiber gets launched into the shared pool.
>*/
for ( char c : std::string("abcdefghijklmnopqrstuvwxyz")) { /*<
Launch a number of worker fibers; each worker fiber picks up a character
that is passed as parameter to fiber-function `whatevah`.
Each worker fiber gets detached.
>*/
boost::fibers::fiber([c](){ whatevah( c); }).detach();
++fiber_count; /*< Increment fiber counter for each new fiber. >*/
}
boost::fibers::detail::thread_barrier b( 4);
std::thread threads[] = { /*<
Launch a couple of threads that join the work sharing.
>*/
std::thread( thread, & b),
std::thread( thread, & b),
std::thread( thread, & b)
};
b.wait(); /*< sync with other threads: allow them to start processing >*/
{
lock_type/*< `lock_type` is typedef'ed as __unique_lock__< [#http://en.cppreference.com/w/cpp/thread/mutex `std::mutex`] > >*/ lk( mtx_count);
cnd_count.wait( lk, [](){ return 0 == fiber_count; } ); /*<
Suspend main fiber and resume worker fibers in the meanwhile.
Main fiber gets resumed (e.g returns from `condition_variable_any::wait()`)
if all worker fibers are complete.
>*/
} /*<
Releasing lock of mtx_count is required before joining the threads, otherwise
the other threads would be blocked inside condition_variable::wait() and
would never return (deadlock).
>*/
BOOST_ASSERT( 0 == fiber_count);
for ( std::thread & t : threads) { /*< wait for threads to terminate >*/
t.join();
}
//]
std::cout << "done." << std::endl;
return EXIT_SUCCESS;
}
I have a problem with to understand how the fibers can continue their execution, in the different thread when all the thread are waiting for the notify_all(), because when all the thread are blocking because of the wait function : cnd_count.wait( lk, [](){ return 0 == fiber_count; } ).
So if there are all blocked by the wait function, how is it possible that the fibers continue to execute, I thought that the fibers were executed by the thread with the help of their own scheduling manager. I have read that the threads are just execution unit that the fibers use in order to run their own callable or function in their. So why all the fibers continue to run when all the threads are blocking ?
See: https://www.boost.org/doc/libs/1_71_0/libs/fiber/doc/html/fiber/scheduling.html
Whenever a thread is suspended the scheduler run the next ready fiber, so whatevah got executed when cnd_count.wait is called.
You can try removing the line boost::fibers::use_scheduling_algorithm< boost::fibers::algo::shared_work >(); on thread_fn_ws, and you will find all the fiber are ran in the main thread, because scheduler is not installed and thus fiber won't be ran.

Deadlock after notify_one() (notifying thread locks mutex)

I have following problem. I have some multiple threads that do some work and one main thread that wakes them up when work is available. So far, I have managed to write some code using conditional variables and mutexes and most of the time this works okay, but from time to time, notifying thread will lock the mutex right after call notify_one(), thus blocking the notified thread and deadlocking.
I have written minimal code to illustrate this situation.
#include <iostream>
#include <thread>
#include <condition_variable>
std::mutex lock;
std::condition_variable cv;
void foo() {
std::cout << "Thread: Entering doWork()" << std::endl;
std::unique_lock<std::mutex> l(lock);
std::cout << "Thread: Acquired lock, going to wait." << std::endl;
cv.wait(l , []{return true;});
std::cout << "Thread: Done waiting, exit." << std::endl;
}
int main(void) {
std::unique_lock<std::mutex> l(lock);
std::cout << "MAIN: Creating thread." << std::endl;
std::thread t(foo);
std::cout << "MAIN: Unlocking mutex." << std::endl;
l.unlock();
std::cout << "MAIN: Notifying thread." << std::endl;
cv.notify_one();
//std::this_thread::sleep_for(std::chrono::seconds(1));
l.lock();
std::cout << "MAIN: Acquired lock." << std::endl;
std::cout << "MAIN: Joining thread." << std::endl;
t.join();
return 0;
}
In ideal situation, the output should be
MAIN: Creating thread.
MAIN: Unlocking mutex.
Thread: Entering doWork()
Thread: Acquired lock, going to wait.
MAIN: Notifying thread.
Thread: Done waiting, exit.
MAIN: Acquired lock.
MAIN: Joining thread.
but more often than not it is
MAIN: Creating thread.
MAIN: Unlocking mutex.
MAIN: Notifying thread.
MAIN: Acquired lock.
MAIN: Joining thread.
Thread: Entering doWork()
Is there any better way to eliminate chance of deadlock except of adding sleep into notifying thread (which I don't want to do)? Thank you in advance.
It's "condition variable" not "conditional variable", and the reason it's called that is that you use it to wait on some condition.
You aren't doing that, you just wait on a lambda that always returns true, and that's the cause of your problem. That and the fact your main thread is holding the lock all the time (why?!)
Sometimes the main thread runs the unlock, the notify_one, and the lock quickly, before the foo thread has even started. That means the foo thread misses the notification, then tries to acquire the lock, but can't because the main thread has it.
A condition variable is not like a semaphore, the notify_one` call does not set a state that can be detected later. If the condition variable isn't waiting when the notify_one call happens then it misses it, and it is gone forever. If you miss the notification and then sleep you will never wake up.
The solution is not to add arbitrary sleeps, that doesn't solve anything (ever!)
The correct solution is to have a condition that is tested, and to stop holding the lock when you're not updating any shared data. In the example below the condition being tested is "is the boolean ready true?" and the foo thread will wait until that condition is true. The main thread sets the variable, making the condition true, and then notifies the other thread that it should re-check the condition.
#include <iostream>
#include <thread>
#include <condition_variable>
std::mutex lock;
std::condition_variable cv;
bool ready = false;
void foo() {
std::cout << "Thread: Entering doWork()" << std::endl;
std::unique_lock<std::mutex> l(lock);
std::cout << "Thread: Acquired lock, going to wait." << std::endl;
cv.wait(l , []{return ready;});
std::cout << "Thread: Done waiting, exit." << std::endl;
}
int main(void) {
std::cout << "MAIN: Creating thread." << std::endl;
std::thread t(foo);
{
std::cout << "MAIN: Locking mutex." << std::endl;
std::unique_lock<std::mutex> l(lock);
ready = true;
}
std::cout << "MAIN: Notifying thread." << std::endl;
cv.notify_one();
std::cout << "MAIN: Joining thread." << std::endl;
t.join();
}

pthread_exit in signal handler causes segmentation fault

The program below sets SIG_ALRM handler for the whole process, creates a thread, sends SIG_ALRM signal to new created thread.
In SIG_ALRM handler pthread_exit is called.
The result - segmentation fault.
If you sleep before sending signal - OK.
It looks like new thread has not been started at the moment of pthread_exit.
I tried to locate segmentation fault with gdb but couldn't reproduce the crash with gdb.
What causes segmentation fault?
Thanks!
#include <signal.h>
#include <pthread.h>
#include <iostream>
#include <cassert>
using namespace std;
void* threadFunc(void* arg) {
cout << "thread: started. sleeping..: " << pthread_self() << endl;
sleep(10);
cout << "thread: exit" << endl;
return NULL;
}
void alrm_handler(int signo) {
cout << "alrm_handler: " << pthread_self() << endl;
pthread_exit(NULL); //if comment - no segmentation fault
}
int main() {
cout << "main: " << pthread_self() << endl;
struct sigaction act;
act.sa_handler = alrm_handler;
act.sa_flags = 0;
sigemptyset(&act.sa_mask);
sigaction(SIGALRM, &act, NULL);
pthread_t t;
int rc = pthread_create(&t, NULL, threadFunc, NULL);
assert(rc == 0);
// usleep(1000); //if Uncomment - no segmentation fault
rc = pthread_kill(t, SIGALRM);
assert(rc == 0);
pthread_join(t, NULL);
cout << "main: exit" << endl;
return 0;
}
The output:
main: 140130531731232
alrm_handler: 140130504095488
Segmentation fault
pthread_exit is not async-signal-safe. You cannot call it from signal handlers unless you can be sure the signal handler is not interrupting an async-signal-unsafe function. In particular, the time between calling pthread_create and the entry to your new thread's start function must be considered async-signal-unsafe - this is never explicitly spelled out in the standard, but you can think of the new thread as still being "in pthread_create" (which is async-signal-unsafe) if you like.
Give change for thread initialization process to be completed. so just uncomment the below line is the right approach.
usleep(1000);

Simple Detached pthread does not cancel! (cout blocks and interleaves even if mutexed)

I have a hard problem here, which I can not solve and do not find the right answer on the net:
I have created a detached thread with a clean up routing, the problem is that on my Imac and Ubuntu 9.1 (Dual Core). I am not able to correctly cancel the detached thread in the fallowing code:
#include <iostream>
#include <pthread.h>
#include <sched.h>
#include <signal.h>
#include <time.h>
pthread_mutex_t mutex_t;
using namespace std;
static void cleanup(void *arg){
pthread_mutex_lock(&mutex_t);
cout << " doing clean up"<<endl;
pthread_mutex_unlock(&mutex_t);
}
static void *thread(void *aArgument)
{
pthread_setcancelstate(PTHREAD_CANCEL_ENABLE,NULL);
pthread_setcanceltype(PTHREAD_CANCEL_DEFERRED,NULL);
pthread_cleanup_push(&cleanup,NULL);
int n=0;
while(1){
pthread_testcancel();
sched_yield();
n++;
pthread_mutex_lock(&mutex_t);
cout << " Thread 2: "<< n<<endl; // IF I remove this endl; --> IT WORKS!!??
pthread_mutex_unlock(&mutex_t);
}
pthread_cleanup_pop(0);
return NULL;
}
int main()
{
pthread_t thread_id;
pthread_attr_t attr;
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr,PTHREAD_CREATE_DETACHED);
int error;
if (pthread_mutex_init(&mutex_t,NULL) != 0) return 1;
if (pthread_create(&thread_id, &attr, &(thread) , NULL) != 0) return 1;
pthread_mutex_lock(&mutex_t);
cout << "waiting 1s for thread...\n" <<endl;
pthread_mutex_unlock(&mutex_t);
int n =0;
while(n<1E3){
pthread_testcancel();
sched_yield();
n++;
pthread_mutex_lock(&mutex_t);
cout << " Thread 1: "<< n<<endl;
pthread_mutex_unlock(&mutex_t);
}
pthread_mutex_lock(&mutex_t);
cout << "canceling thread...\n" <<endl;
pthread_mutex_unlock(&mutex_t);
if (pthread_cancel(thread_id) == 0)
{
//This doesn't wait for the thread to exit
pthread_mutex_lock(&mutex_t);
cout << "detaching thread...\n"<<endl;
pthread_mutex_unlock(&mutex_t);
pthread_detach(thread_id);
while (pthread_kill(thread_id,0)==0)
{
sched_yield();
}
pthread_mutex_lock(&mutex_t);
cout << "thread is canceled";
pthread_mutex_unlock(&mutex_t);
}
pthread_mutex_lock(&mutex_t);
cout << "exit"<<endl;
pthread_mutex_unlock(&mutex_t);
return 0;
}
When I replace the Cout with printf() i workes to the end "exit" , but with the cout (even locked) the executable hangs after outputting "detaching thread...
It would be very cool to know from a Pro, what the problem here is?.
Why does this not work even when cout is locked by a mutex!?
THE PROBELM lies in that COUT has a implicit cancelation point!
We need to code like this:
pthread_setcancelstate(PTHREAD_CANCEL_ENABLE,NULL);
pthread_testcancel();
pthread_setcancelstate(PTHREAD_CANCEL_DISABLE,NULL);
and make the thread at the beginning :
pthread_setcancelstate(PTHREAD_CANCEL_DISABLE,NULL);
That ensures that only pthread_cancel() has a cancelation point...
Try commenting out the line pthread_detach(thread_id); and run it. You are creating the thread as detached with your pthread_attr_t.
Either that, or try passing NULL instead of &attr in the pthread_create (so that the thread is not created detached) and run it.
I would guess that if the timing is right, the (already detached) thread is gone by the time the main thread attempts the pthread_detach, and you are going off into Never Never Land in pthread_detach.
Edit:
If cout has an implicit cancelation point as Gabriel points out, then most likely what happens is that the thread cancels while holding the mutex (it never makes it to pthreads_unlock_mutex after the cout), and so anybody else waiting on the mutex will be blocked forever.
If the only resource you need to worry about is the mutex, you could keep track of whether or not your thread has it locked and then unlock it in the cleanup, assuming that cleanup runs in the same thread.
Take a look here, page 157 on: PThreads Primer.

Resources