Dekker Mutual-Exclusion - multithreading

I'm studying the operating system and the program that Dekker wrote for his third attempt to Mutual Exclusion
I wrote my code in C ++ in Visual Studio, the code is below, but I wonder how these two threads are still in the critical area at the same time?
The output of the program is below
#include<iostream>
#include<conio.h>
#include<thread>
using namespace std;
bool flag0 = false;
bool flag1 = false;
void p1()
{
flag0 = true;
while (flag1);
for (int i = 1; i <= 10; i++)
cout << "p1 : " << i << endl;
flag0 = false;
}
void p2()
{
flag1 = true;
while (flag0);
for (int i = -1; i >= -10; i--)
cout << "p2 : " << i << endl;
flag1 = false;
}
int main()
{
thread t1(p1);
thread t2(p2);
t1.join();
t2.join();
_getch();
return 0;
}
Output:
p1 : p2 : -11
p2 : p1 : 2
p1 : 3
p1 : 4
p1 : 5
p1 : 6
-2
p2 : -3
p2 : -4
p2 : -5
p2 : -6
p2 : -7
p2 : -8
p2 : -9
p2 : p1 : -107
p1 : 8
p1 : 9
p1 : 10

" I wonder how these two threads are still in the critical area at the same time"
You've got two threads accessing the same variables, without any form of synchronization. In C++, that is a form of Undefined Behavior. Undefined Behavior means that anything can happen. "Two threads in the same critical area at the same time" is not even remotely surprising.

The problem is that your flags are ordinary bool variables, which results in a data race and therefore UB. That basically means that all bets are off about how your program behaves! For example, the compiler could hoist the load in the while-loop, effectively transforming the loop into an infinite loop. But infinite loops without side effects are also UB, so the compiler is well in its right to remove the loop entirely!
But even if the compiler does not perform these optimizations, the code still does not guarantee mutual exclusion, because the flag operations are not sequentially consistent. Essentially what can happen is that both threads store true in their respective flag, but it is not guaranteed that this updated value is visible to the other thread. So it can happen that the subsequent load operations in the while-loop still return false for both threads.
To get the desired behavior, the operations on the flags need to be sequentially consistent, which means that all such operations have a single total order. To achieve that you have to define your flags as std::atomic<bool>. All operations on atomics are sequentially consistent by default (unless specified otherwise).
However, since this is the third attempt by Dekker and not the final (correct) version, it does provide mutual exclusion (under a sequentially consistent memory model), but is prone to deadlocks!
For more details you should familiarize yourself with the C++ memory model. I can recommend the paper Memory Models for C/C++ Programmers which I have co-authored.

#include<iostream>
#include<conio.h>
#include<thread>
using namespace std;
bool turnop1 = false;
void p1()
{
for (int i = 1; i <= 10; i++)
{
while (turnop1 == false);
cout << "p1 : " << i << endl;
turnop1 = false;
}
}
void p2()
{
for (int t = -1; t >= -10; t--)
{
while (turnop1 == true);
cout << "p2 : " << t << endl;
turnop1 = true;
}
}
int main()
{
thread t1(p1);
thread t2(p2);
t2.join();
t1.join();
_getch();
return 0;
}
Modified your code and worked for me. Thanks! I know this is not completely right but this Dekker's idea/example is enough for a simple explanation.

Related

boost::thread not updating global variable

I am using a wrapper function in an external software to start a new thread, which updates a global variable, but yet this seems invisible to the main thread. I cant call join(), not to block the main thread and crash the software. boost::async, boost::thread and boost::packaged_task all behave the same way.
uint32 *Dval;
bool hosttask1()
{
while(*Dval<10)
{
++*Dval;
PlugIn::gResultOut << " within thread global value: " << *Dval << std::endl;
Sleep(500);
}
return false;
}
void SU_HostThread1(uint32 *value)
{
Dval = value;
*Dval = 2;
PlugIn::gResultOut << " before thread: " << *value << " before thread global: " << *Dval << std::endl;
auto myFuture = boost::async(boost::launch::async,&hosttask1);
//boost::thread thread21 = boost::thread(&hosttask1);
//boost::packaged_task<bool> pt(&hosttask1);
//boost::thread thread21 = boost::thread(boost::move(pt));
}
When I call the function:
number a=0
su_hostthread1(a)
sleep(2) //seconds
result(" function returned "+a+" \n")
OUTPUT:
before thread value: 2 before thread global value: 2
within thread global value: 3
within thread global value: 4
within thread global value: 5
within thread global value: 6
function returned 2
within thread global value: 7
within thread global value: 8
within thread global value: 9
within thread global value: 10
Any ideas?
Thanks in advance!
If you share data between threads, you must syncronize access to that data. The two possible ways are a mutex protecting said data and atomic operations. The simple reason is that caches and read/write reordering (both by CPU and compiler) exist. This is a complex topic though and it's nothing that can be explained in an answer here, but there are a few good books out there and also a bunch of code that gets it right.
The following code correctly reproduces what I intend to do. Mainly, the thread updates a global variable which the main thread correctly observes.
#include "stdafx.h"
#include <iostream>
#include <boost/thread.hpp>
#include <boost/chrono.hpp>
unsigned long *dataR;
bool hosttask1()
{
bool done = false;
std::cout << "In thread global value: " << *dataR << "\n"; //*value11 << *dataL <<
unsigned long cc = 0;
boost::mutex m;
while (!done)
{
m.lock();
*dataR = cc;
m.unlock();
cc++;
std::cout << "In thread loop global value: "<< *dataR << "\n";
if (cc==5) done = true;
}
return done;
}
void SU_HostThread1(unsigned long *value)
{
dataR = value;
std::cout << "Before thread value: " << *value << " Before thread global value: " << *dataR << "\n"; //*value11 << *dataL <<
auto myFuture = boost::async(boost::launch::async, &hosttask1);
return;
}
int main()
{
unsigned long value =1;
unsigned long *value11;
value11 = &value;
SU_HostThread1(value11);
boost::this_thread::sleep(boost::posix_time::seconds(1));
std::cout << "done with end value: " << *value11 << "\n";
return 0;
}
output:
Before thread value: 1 Before thread global value: 1
In thread global value: 1
In thread loop global value: 0
In thread loop global value: 1
In thread loop global value: 2
In thread loop global value: 3
In thread loop global value: 4
done with end value: 4
Yet when I copy this exactly to the SDK of the external software, the main thread does not update global value. Any ideas how this is so?
Thanks
output in external software:
before thread value: 1 before thread global value: 1
In thread global value: 1
In thread loop global value: 0
In thread loop global value: 1
In thread loop global value: 2
In thread loop global value: 3
In thread loop global value: 4
done with end value: 1
Likely this is because the compiler doesn't generally think about multithreading when optimising your code. If has seen you code checks a value repeatedly, and it knows that in single threading that value cannot change, so it just omitted the check.
If you declare the variable as volatile, then it will probably generate less efficient code that checks more often.
Of course you have to also understand that when a value is written, there are circumstances when it may not all be written in one go, so if you are unlucky enough to read it back when it is half-written, then you get back a garbage value. The fix for that is to declare it as std::atomic (which is automatically considered volatile by the optimiser), and then even more complex code will be emitted to ensure that the write and the read cannot intersect (or different processor primitives might be used for small objects such as integers)
most variables are not shared between threads, and when they are it is up to the programmer to identify those and balance optimisation against the thread synchronisation needs during design.

read variable value in main from thread in c++

I need to have a thread which executes a function in while loop(say increments an int value). In the main I need to have a while loop which executes some function(say a for loop which counts from 0 to 5) and then reads the current value of a variable in the thread. The thread must keep running its own while loop irrespective of whats going on in main. However the value of the thread variable must not change while main reads the variable.
I guess this problem can be solved using atomic. However this is a toy problem in which the variable in the thread is an int. In my actual problem the thread variable if of type Eigen::quaternionf or float[4]. So I need to ensure that the entire Eigen::quaternionf or float[4] is held constant when it is read from main.
The cout in the thread is only for debugging. If the code runs with thread safety, it can be removed. I read from another post that using cout in a thread safe manner may need to write a new wrapper around cout with a mutex. I want to avoid it.
My main concern is reading the variable in correct order in main.
My code fails(today is my first day with multithreading) and is as below along with observed output(selected parts). the code fails to keep the order of the output using cout(garbled output). Also I am not sure that the thread variable is correctly read by the main.
#include <thread>
#include <mutex>
#include <iostream>
int i = 0;
void safe_increment(std::mutex& i_mutex)
{
while(1)
{
std::lock_guard<std::mutex> lock(i_mutex);
++i;
std::cout << "thread: "<< std::this_thread::get_id() << ", i=" << i << '\n';
}
}
int main()
{
std::mutex i_mutex;
std::thread t1(safe_increment, std::ref(i_mutex));
while(1)
{
for(int k =0; k < 5; k++)
{
std::cout << "main: k =" << k << '\n';
}
std::lock_guard<std::mutex> lock(i_mutex);
std::cout << "main: i=" << i << '\n';
}
}
The output(selected parts) I get is
thread: 139711042705152, i=223893
thread: 139711042705152, i=223894
thread: 139711042705152, i=223895
main: i=223895
main: k =0
thread: main: k =1139711042705152
main: k =2
main: k =3
, i=main: k =4
223896
thread: 139711042705152, i=223897
thread: 139711042705152, i=223898
thread: 139711042705152, i=224801
thread: 139711042705152, i=224802
main: i=224802
main: k =0
main: k =1
thread: main: k =2
main: k =3
main: k =4
139711042705152, i=224803
thread: 139711042705152, i=224804
thread: 139711042705152, i=224805
i is properly synchronized with the mutex. well done! obviously this runs until you force it to stop, so when you do find a better way to end execution, be sure to join on your thread.
to fix the garbling, you need to synchronize on std::cout:
int main()
{
std::mutex i_mutex;
std::thread t1(safe_increment, std::ref(i_mutex));
while(1)
{
std::lock_guard<std::mutex> lock(i_mutex);//moved up here to sync on std::cout << k
for(int k =0; k < 5; k++)
{
std::cout << "main: k =" << k << '\n';
}
std::cout << "main: i=" << i << '\n';
if (i > 100) break;
}
t1.join(); //thread will continue and main will wait until it is done
//your thread needs to have some way out of its while(1) as well.
}
the thread can maybe be this:
void safe_increment(std::mutex& i_mutex)
{
while(1)
{
std::lock_guard<std::mutex> lock(i_mutex);
++i;
std::cout << "thread: "<< std::this_thread::get_id() << ", i=" << i << '\n';
if (i > 111) break;
}
}

std::async performance on Windows and Solaris 10

I'm running a simple threaded test program on both a Windows machine (compiled using MSVS2015) and a server running Solaris 10 (compiled using GCC 4.9.3). On Windows I'm getting significant performance increases from increasing the threads from 1 to the amount of cores available; however, the very same code does not see any performance gains at all on Solaris 10.
The Windows machine has 4 cores (8 logical) and the Unix machine has 8 cores (16 logical).
What could be the cause for this? I'm compiling with -pthread, and it is creating threads since it prints all the "S"es before the first "F". I don't have root access on the Solaris machine, and from what I can see there's no installed tool which I can use to view a process' affinity.
Example code:
#include <iostream>
#include <vector>
#include <future>
#include <random>
#include <chrono>
std::default_random_engine gen(std::chrono::system_clock::now().time_since_epoch().count());
std::normal_distribution<double> randn(0.0, 1.0);
double generate_randn(uint64_t iterations)
{
// Print "S" when a thread starts
std::cout << "S";
std::cout.flush();
double rvalue = 0;
for (int i = 0; i < iterations; i++)
{
rvalue += randn(gen);
}
// Print "F" when a thread finishes
std::cout << "F";
std::cout.flush();
return rvalue/iterations;
}
int main(int argc, char *argv[])
{
if (argc < 2)
return 0;
uint64_t count = 100000000;
uint32_t threads = std::atoi(argv[1]);
double total = 0;
std::vector<std::future<double>> futures;
std::chrono::high_resolution_clock::time_point t1;
std::chrono::high_resolution_clock::time_point t2;
// Start timing
t1 = std::chrono::high_resolution_clock::now();
for (int i = 0; i < threads; i++)
{
// Start async tasks
futures.push_back(std::async(std::launch::async, generate_randn, count/threads));
}
for (auto &future : futures)
{
// Wait for tasks to finish
future.wait();
total += future.get();
}
// End timing
t2 = std::chrono::high_resolution_clock::now();
// Take the average of the threads' results
total /= threads;
std::cout << std::endl;
std::cout << total << std::endl;
std::cout << "Finished in " << std::chrono::duration_cast<std::chrono::milliseconds>(t2 - t1).count() << " ms" << std::endl;
}
As a general rule, classes defined by the C++ standard library do not have any internal locking. Modifying an instance of a standard library class from more than one thread, or reading it from one thread while writing it from another, is undefined behavior, unless "objects of that type are explicitly specified as being sharable without data races". (N3337, sections 17.6.4.10 and 17.6.5.9.) The RNG classes are not "explicitly specified as being sharable without data races". (cout is an example of a stdlib object that is "sharable with data races" — as long as you haven't done ios::sync_with_stdio(false).)
As such, your program is incorrect because it accesses a global RNG object from more than one thread simultaneously; every time you request another random number, the internal state of the generator is modified. On Solaris, this seems to result in serialization of accesses, whereas on Windows it is probably instead causing you not to get properly "random" numbers.
The cure is to create separate RNGs for each thread. Then each thread will operate independently, and they will neither slow each other down nor step on each other's toes. This is a special case of a very general principle: multithreading always works better the less shared data there is.
There's an additional wrinkle to worry about: each thread will call system_clock::now at very nearly the same time, so you may end up with some of the per-thread RNGs seeded with the same value. It would be better to seed them all from a random_device object. random_device requests random numbers from the operating system, and does not need to be seeded; but it can be very slow. The random_device should be created and used inside main, and seeds passed to each worker function, because a global random_device accessed from multiple threads (as in the previous edition of this answer) is just as undefined as a global default_random_engine.
All told, your program should look something like this:
#include <iostream>
#include <vector>
#include <future>
#include <random>
#include <chrono>
static double generate_randn(uint64_t iterations, unsigned int seed)
{
// Print "S" when a thread starts
std::cout << "S";
std::cout.flush();
std::default_random_engine gen(seed);
std::normal_distribution<double> randn(0.0, 1.0);
double rvalue = 0;
for (int i = 0; i < iterations; i++)
{
rvalue += randn(gen);
}
// Print "F" when a thread finishes
std::cout << "F";
std::cout.flush();
return rvalue/iterations;
}
int main(int argc, char *argv[])
{
if (argc < 2)
return 0;
uint64_t count = 100000000;
uint32_t threads = std::atoi(argv[1]);
double total = 0;
std::vector<std::future<double>> futures;
std::chrono::high_resolution_clock::time_point t1;
std::chrono::high_resolution_clock::time_point t2;
std::random_device make_seed;
// Start timing
t1 = std::chrono::high_resolution_clock::now();
for (int i = 0; i < threads; i++)
{
// Start async tasks
futures.push_back(std::async(std::launch::async,
generate_randn,
count/threads,
make_seed()));
}
for (auto &future : futures)
{
// Wait for tasks to finish
future.wait();
total += future.get();
}
// End timing
t2 = std::chrono::high_resolution_clock::now();
// Take the average of the threads' results
total /= threads;
std::cout << '\n' << total
<< "\nFinished in "
<< std::chrono::duration_cast<
std::chrono::milliseconds>(t2 - t1).count()
<< " ms\n";
}
(This isn't really an answer, but it won't fit into a comment, especially with the command formatting an links.)
You can profile your executable on Solaris using Solaris Studio's collect utility. On Solaris, that will be able to show you where your threads are contending.
collect -d /tmp -p high -s all app [app args]
Then view the results using the analyzer utility:
analyzer /tmp/test.1.er &
Replace /tmp/test.1.er with the path to the output generated by a collect profile run.
If your threads are contending over some resource(s) as #zwol posted in his answer, you will see it.
Oracle marketing brief for the toolset can be found here: http://www.oracle.com/technetwork/server-storage/solarisstudio/documentation/o11-151-perf-analyzer-brief-1405338.pdf
You can also try compiling your code with Solaris Studio for more data.

What's the correct way of waiting for detached threads to finish?

Look at this sample code:
void OutputElement(int e, int delay)
{
this_thread::sleep_for(chrono::milliseconds(100 * delay));
cout << e << '\n';
}
void SleepSort(int v[], uint n)
{
for (uint i = 0 ; i < n ; ++i)
{
thread t(OutputElement, v[i], v[i]);
t.detach();
}
}
It starts n new threads and each one sleeps for some time before outputting a value and finishing. What's the correct/best/recommended way of waiting for all threads to finish in this case? I know how to work around this but I want to know what's the recommended multithreading tool/design that I should use in this situation (e.g. condition_variable, mutex etc...)?
And now for the slightly dissenting answer. And I do mean slightly because I mostly agree with the other answer and the comments that say "don't detach, instead join."
First imagine that there is no join(). And that you have to communicate among your threads with a mutex and condition_variable. This really isn't that hard nor complicated. And it allows an arbitrarily rich communication, which can be anything you want, as long as it is only communicated while the mutex is locked.
Now a very common idiom for such communication would simply be a state that says "I'm done". Child threads would set it, and the parent thread would wait on the condition_variable until the child said "I'm done." This idiom would in fact be so common as to deserve a convenience function that encapsulated the mutex, condition_variable and state.
join() is precisely this convenience function.
But imho one has to be careful. When one says: "Never detach, always join," that could be interpreted as: Never make your thread communication more complicated than "I'm done."
For a more complex interaction between parent thread and child thread, consider the case where a parent thread launches several child threads to go out and independently search for the solution to a problem. When the problem is first found by any thread, that gets communicated to the parent, and the parent can then take that solution, and tell all the other threads that they don't need to search any more.
For example:
#include <chrono>
#include <iostream>
#include <iterator>
#include <random>
#include <thread>
#include <vector>
void OneSearch(int id, std::shared_ptr<std::mutex> mut,
std::shared_ptr<std::condition_variable> cv,
int& state, int& solution)
{
std::random_device seed;
// std::mt19937_64 eng{seed()};
std::mt19937_64 eng{static_cast<unsigned>(id)};
std::uniform_int_distribution<> dist(0, 100000000);
int test = 0;
while (true)
{
for (int i = 0; i < 100000000; ++i)
{
++test;
if (dist(eng) == 999)
{
std::unique_lock<std::mutex> lk(*mut);
if (state == -1)
{
state = id;
solution = test;
cv->notify_one();
}
return;
}
}
std::unique_lock<std::mutex> lk(*mut);
if (state != -1)
return;
}
}
auto findSolution(int n)
{
std::vector<std::thread> threads;
auto mut = std::make_shared<std::mutex>();
auto cv = std::make_shared<std::condition_variable>();
int state = -1;
int solution = -1;
std::unique_lock<std::mutex> lk(*mut);
for (uint i = 0 ; i < n ; ++i)
threads.push_back(std::thread(OneSearch, i, mut, cv,
std::ref(state), std::ref(solution)));
while (state == -1)
cv->wait(lk);
lk.unlock();
for (auto& t : threads)
t.join();
return std::make_pair(state, solution);
}
int
main()
{
auto p = findSolution(5);
std::cout << '{' << p.first << ", " << p.second << "}\n";
}
Above I've created a "dummy problem" where a thread searches for how many times it needs to query a URNG until it comes up with the number 999. The parent thread puts 5 child threads to work on it. The child threads work for awhile, and then every once in a while, look up and see if any other thread has found the solution yet. If so, they quit, else they keep working. The main thread waits until solution is found, and then joins with all the child threads.
For me, using the bash time facility, this outputs:
$ time a.out
{3, 30235588}
real 0m4.884s
user 0m16.792s
sys 0m0.017s
But what if instead of joining with all the threads, it detached those threads that had not yet found a solution. This might look like:
for (unsigned i = 0; i < n; ++i)
{
if (i == state)
threads[i].join();
else
threads[i].detach();
}
(in place of the t.join() loop from above). For me this now runs in 1.8 seconds, instead of the 4.9 seconds above. I.e. the child threads are not checking with each other that often, and so main just detaches the working threads and lets the OS bring them down. This is safe for this example because the child threads own everything they are touching. Nothing gets destructed out from under them.
One final iteration can be realized by noticing that even the thread that finds the solution doesn't need to be joined with. All of the threads could be detached. The code is actually much simpler:
auto findSolution(int n)
{
auto mut = std::make_shared<std::mutex>();
auto cv = std::make_shared<std::condition_variable>();
int state = -1;
int solution = -1;
std::unique_lock<std::mutex> lk(*mut);
for (uint i = 0 ; i < n ; ++i)
std::thread(OneSearch, i, mut, cv,
std::ref(state), std::ref(solution)).detach();
while (state == -1)
cv->wait(lk);
return std::make_pair(state, solution);
}
And the performance remains at about 1.8 seconds.
There is still (sort of) an effective join with the solution-finding thread here. But it is accomplished with the condition_variable::wait instead of with join.
thread::join() is a convenience function for the very common idiom that your parent/child thread communication protocol is simply "I'm done." Prefer thread::join() in this common case as it is easier to read, and easier to write.
However don't unnecessarily constrain yourself to such a simple parent/child communication protocol. And don't be afraid to build your own richer protocol when the task at hand needs it. And in this case, thread::detach() will often make more sense. thread::detach() doesn't necessarily imply a fire-and-forget thread. It can simply mean that your communication protocol is more complex than "I'm done."
Don't detach, but instead join:
std::vector<std::thread> ts;
for (unsigned int i = 0; i != n; ++i)
ts.emplace_back(OutputElement, v[i], v[i]);
for (auto & t : threads)
t.join();

Interpreting time command output on a multi threaded program

I have a multi threaded program and I am profiling time taken starting before all pthread_create's and after all pthread_join's.
Now I find that this time, lets call it X, which is shown below in "Done in xms" is actually user + sys time of time output. In my app the number argument to a.out controls how many threads to spawn. ./a.out 1 spawn 1 pthread and ./a.out 2 spawns 2 threads where each thread does the same amount of work.
I was expecting X to be the real time instead of user + sys time. Can someone please tell me why this is not so? Then this really means my app is indeed running parallel without any locking between threads.
[jithin#whatsoeverclever tests]$ time ./a.out 1
Done in 320ms
real 0m0.347s
user 0m0.300s
sys 0m0.046s
[jithin#whatsoeverclever tests]$ time ./a.out 2
Done in 450ms
real 0m0.266s
user 0m0.383s
sys 0m0.087s
[jithin#whatsoeverclever tests]$ time ./a.out 3
Done in 630ms
real 0m0.310s
user 0m0.532s
sys 0m0.105s
Code
int main(int argc, char **argv) {
//Read the words
getWords();
//Set number of words to use
int maxWords = words.size();
if(argc > 1) {
int numWords = atoi(argv[1]);
if(numWords > 0 && numWords < maxWords) maxWords = numWords;
}
//Init model
model = new Model(MODEL_PATH);
pthread_t *threads = new pthread_t[maxWords];
pthread_attr_t attr;
void *status;
// Initialize and set thread joinable
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
int rc;
clock_t startTime = clock();
for(unsigned i=0; i<maxWords; i++) {
//create thread
rc = pthread_create(&threads[i], NULL, processWord, (void *)&words[i] );
if (rc){
cout << "Error:unable to create thread: " << i << "," << rc << endl;
exit(-1);
}
}
// free attribute and wait for the other threads
pthread_attr_destroy(&attr);
for(unsigned i=0; i<maxWords; i++) {
rc = pthread_join(threads[i], &status);
if (rc){
cout << "Error:unable to join thread: " << i << "," << rc << endl;
exit(-1);
}
}
clock_t endTime = clock();
float diff = (((float)endTime - (float)startTime) / 1000000.0F ) * 1000;
cout<<"Done in "<< diff << "ms\n";
delete[] threads;
delete model;
}
The clock function is specifically documented to return the processor time used by a process. If you want to measure wall time elapsed, it's not the right function.

Resources