Here is a simplified version of my situation:
void AppendToVector(std::vector<int>* my_vector) {
for (int i = 0; i < 100; i++) {
my_vector->push_back(i);
}
}
void CreateVectors(const int num_threads) {
std::vector<std::vector<int>* > my_vector_of_pointers(10);
ThreadPool pool(num_threads);
for (for int i = 0; i < 10; i++) {
my_vector_of_pointers[i] = new std::vector<int>();
pool.AddTask(AppendToVector,
&my_vector_of_pointers[i]);
}
}
My question is whether I need to put a mutex lock in AppendToVector when running this with multiple threads? My intuition tells me I do not have to because there is no possibility of two threads accessing the same data, but I am not fully confident.
Every thread is appending different std::vector (inside AppendToVector function) so there is no need for locking in your case. In case you change your code in the way more than one thread access same vector then you will need lock. Don't be confused because std::vectors you are passing to AppendToVector are them-selfs elements of main std::list, it matters only that here threads are manipulating with completely different (not shared) memory
Related
I am attempting to design a system in which I'll have multiple threads. All these threads share a common variable (which is atomic) at a time and when each thread starts it increments that variable and before that thread ends it decrements that variable. I would like a thread to block if the current value of that variable is greater than a specific limit value. A simple psuedocode is this:
std::atomic<int> var;
int limit = 3; //limit value
void doWork() //This method is run as multiple thread simultaneously
{
if(var.load() < limit)
{
//block until this condition is met
}
//increment
var++;
//Do some work
.....
//Decrement before leaving
var--
}
Now two approaches come to my mind. The first approach is to sleep while var.load < limit but this looks sub-optimal. The second approach is to use conditional_variable. I am using that approach below but I believe I have two issues here
The first thread will block on cv.wait(lk, []{return var.load() <
limit});
Multiple threads wont be doing work only one will do work
while others will be blocked.
I would appreciate it if someone can suggest on how I can remove these defects from the code below or if there is a better approach ?
std::condition_variable cv;
std::mutex cv_m;
std::atomic<int> var;
int limit = 3; //limit value
void doWork()
{
std::unique_lock<std::mutex> lk(cv_m);
std::cerr << "Waiting... \n";
cv.wait(lk, []{
return var.load() < limit
});
//Increment var
var++;
//Do some Work
...
//Decrement Work
var--
cv.notify_all();
}
Can somebody explain to me why this code is bad:
int data;
void* worker(void* arg __attribute__((unused))) {
pthread_mutex_t m;
pthread_mutex_init(&m, NULL);
for (int i = 0; i < N; i++) {
pthread_mutex_lock(&m);
data++;
pthread_mutex_unlock(&m);
}
pthread_mutex_destroy(&m);
return NULL;
}
And this is ok:
int data;
pthread_mutex_t m;
void* worker(void* arg __attribute__((unused))) {
for (int i = 0; i < N; i++) {
pthread_mutex_lock(&m);
data++;
pthread_mutex_unlock(&m);
}
return NULL;
}
// ...
pthread_mutex_init(&m, NULL);
// ...
pthread_mutex_destroy(&m);
// ..
Do i always need to declare mutex variables globally?
The problem with local mutex is that it's only a locally accessible version of the mutex ... therefore when a thread locks the mutex in order to share some globally accessible data, the data itself is not protected, since every other thread will have it's own local mutex that can be locked and unlocked. It defeats the whole point of mutual exclusion.
I suggest also to think about exception safety. In this particular example you are just doing data++ in the middle of mutex lock/unlock. So what if you are putting another statement before pthread_mutex_unlock(&m); in the future which can throw a exception. Read about RAII.
Could HippoMocks be used within concurrent testcases just like this:
synchronized startup phase
create mock
register expectations etc.
parallel testing phase
call methods on the mock
synchronized teardown phase
verify the mock
I did not find an explicit statement regarding this issue. Here and there it is mentioned, that mocking non-virtual methods would destroy the possibility for thread-safety (HippoMocks: is it possible to mock non-virtual methods?) or that thread-safety could be added quite easily (unfortunately without actually revealing how). GoogleMock answers this question very clearly (https://github.com/google/googletest/blob/master/googlemock/docs/CookBook.md#using-google-mock-and-threads), such an information would be helpful here, too.
No, HippoMocks is not designed to be thread safe.
However, if you follow some simple rules you should be able to use mocks in a multithreaded environment:
Do the setup sequentially in one thread and use one single MockRepository.
Using different mocks in different threads should be safe.
Using one mock in different threads is safe, when you use only OnCall() setups. Combine it with OnCall().Do() and you should be able do a lot of testing this way.
Don't use ExpectCall - it is not safe.
UPDATE: Okay, I did it. I wrote a small test for multithreading
class IMulti
{
public:
virtual void A() =0;
virtual int B(int a) = 0;
};
const int THREAD_ITERATIONS = 1000;
static DWORD WINAPI run_thread(LPVOID args)
{
IMulti* im = static_cast<IMulti*>(args);
for (int i=0; i<THREAD_ITERATIONS; i++)
{
im->A();
int result = im->B(22);
std::cout << "task says: " << i <<" result:" << result <<"\n";
}
std:: cout << "finished";
return 0;
}
TEST(check_HippoMocksCanMultiThreadedConcurrentReadingViaOnCall)
{
MockRepository mocks;
IMulti* im = mocks.Mock<IMulti>();
mocks.OnCall(im, IMulti::A);
mocks.OnCall(im, IMulti::B).Return(4711);
HANDLE handles[2];
handles[0] = CreateThread(NULL, 0, &run_thread, im, 0, NULL);
handles[1] = CreateThread(NULL, 0, &run_thread, im, 0, NULL);
WaitForMultipleObjects(2, handles, TRUE, INFINITE);
}
The result is, that it works fine.
Now I made it a little bit harder and replaced the second OnCall by the following:
for (int i = 0; i< THREAD_ITERATIONS*2; i++)
{
mocks.ExpectCall(im, IMulti::B).Return(i);
}
Here you will get crashes randomly (just play around with the THREAD_ITERATIONS counter). The reason is, that the matched expectations are somehow counted in the Mockrepository.
Doing the setup concurently crashes, as expected.
I have a class ChunkManager that has a few (supposed to be) asynchronous methods. These methods handle tasks in my game engine such as loading the map blocks (similar to Minecraft) on a different thread so as not to completely halt the main thread (they are lengthy operations)
Here is one of those methods:
void ChunkManager::asyncRenderChunks(){
boost::thread loadingThread(&ChunkManager::renderChunks,this);
}
Where renderChunks looks like:
void ChunkManager::renderChunks(){
activeChunksMutex->lock();
for(int z=0; z < CHUNK_MAX; z=z+1)
{
for(int y=0; y < CHUNK_MAX; y=y+1)
{
for(int x=0; x < CHUNK_MAX; x=x+1)
{
activeChunks[x][y][z]->Render(scnMgr);
}
}
}
activeChunksMutex->unlock();
}
This should work, right? However it crashes when this runs. I have a feeling it has to do with what I do with the thread after it's created, because if I put
loadingThread.join();
in the aforementioned method, it works fine, but the main thread is halted because obviously its just waiting for the new thread to finish, effectively bringing me back to square one.
Any advice?
Sorry if this is a retarded question, I am new to the concept of threads.
Thanks.
Update (4/9/2013):
I found this gem: http://threadpool.sourceforge.net/
..and solved my problem!
If you can join the thread, it must be joinable.
As it says in the documentation:
When the boost::thread object that represents a thread of execution is destroyed the program terminates if the thread is joinable.
You created a local thread object and immediately let it go out of scope: it is destroyed when ChunkManager::asyncRenderChunks returns.
Either:
make it a detached (non-joinable) thread
void ChunkManager::asyncRenderChunks() {
boost::thread loadingThread(&ChunkManager::renderChunks,this);
loadingThread.detach();
}
or create the thread object elsewhere and keep it alive
class ChunkManager {
boost::thread renderingThread;
bool renderChunkWork; // work to do flag
Chunk activeChunks[CHUNK_MAX][CHUNK_MAX][CHUNK_MAX];
boost::mutex activeChunksMutex;
boost::condition_variable activeChunksCV;
bool shutdown; // shutdown flag
void renderChunks() {
for(int z=0; z < CHUNK_MAX; ++z)
for(int y=0; y < CHUNK_MAX; ++y)
for(int x=0; x < CHUNK_MAX; ++x)
activeChunks[x][y][z]->Render(scnMgr);
}
void renderChunkThread() {
boost::unique_lock<boost::mutex> guard(activeChunksMutex);
while (true) {
while (!(renderChunkWork || shutdown))
activeChunksCV.wait(guard);
if (shutdown)
break;
renderChunks();
doRenderChunks = false;
}
}
public:
ChunkManager()
: loadingThread(&ChunkManager::renderChunkThread, this),
renderChunkWork(false), shutdown(false)
{}
~ChunkManager() {
{ // tell the rendering thread to quit
boost::unique_lock<boost::mutex> guard(activeChunksMutex);
renderChunkShutdown = true;
activeChunksCV.notify_one();
}
renderingThread.join()
}
void asyncRenderChunks() {
boost::unique_lock<boost::mutex> guard(activeChunksMutex);
if (!renderChunkWork) {
renderChunkWork = true;
activeChunksCV.notify_one();
}
}
};
NB. In general, creating threads on-the-fly is less good than creating your threads up-front, and just waking them when there's something to do. It avoids figuring out how to handle a second call to asyncRenderChunks before the last one is complete (start a second thread? block?), and moves the latency associated with thread creation.
Note on object lifetime
It's important to realise that in this code:
void ChunkManager::asyncRenderChunks() {
SomeType myObject;
}
the instance myObject will be created and then immediately destroyed.
It crashes, because in the current version of Boost.Thread, you have to either join() a thread or detach() it - otherwise ~thread would terminate the program. (In earlier versions ~thread used to call detach() automatically.)
So if you don't want to join the thread - just detach it:
boost::thread loadingThread(&ChunkManager::renderChunks,this);
loadingThread.detach();
I have a newbie question about Boost::Thread and Mutex.
I would like to start many parallel instances of the following Worker, and all of them write to the same std::vector:
struct Worker {
std::vector<double>* vec;
Worker(std::vector<double>* v) : vec(v) {}
void operator() {
// do some long computation and then add results to *vec, e.g.
for(std::size_t i = 0; i < vec->size(); ++i) {
(*vec)[i] += some_value;
}
}
};
I understand that the Worker has to lock vec before it write to it and unlock it when it's done (because all Workers write to the same vector). But how do I express that?
You need a boost::mutex to protect the vector, and you can use a boost::mutex::scoped_lock that'll lock the mutex in its constructor, and unlock it in the destructor
Keep in mind you need to use that same mutex everywhere where you access that instance of vec , be it reads or writes.
To get you going, you could do something like:
struct Worker {
boost::mutex &vec_mutex;
Worker(std::vector<double>* v,boost::mutex &v_mutex) : vec(v),vec_mutex(v_mutex) {}
void operator() {
// do some long computation and then add results to *vec, e.g.
boost::mutex::scoped_lock lock(vec_mutex);
for(std::size_t i = 0; i < vec->size(); ++i) {
(*vec)[i] += some_value;
}
}
};
For more advanced stuff, you should encapsulate the vector and mutex further, or sooner or later you'll forget that these needs to be connected and you'll access vec somewhere without holding the lock leading to very hard to debug problems. For problems such as this example, I'd rather have the workers use their own individual vectors , and combine the result in the a controlling thread when the workers are done.
OT but useful info I hope - since you are updating this vector a lot (or just for best practice) consider iterators to iterate over the vector elements. Making the worker code faster by not requiring use of vector<double>::operator[] will reduce the aggregate wait time of your worker threads.
for(std::vector<double>::iterator iter = vec->begin(); iter != vec->end(); ++iter) {
*iter += some_value;
}