I have a multithreaded app that has to read some data often, and occasionally that data is updated. I have problems with writing by using unique_lock and problems with reading by using upgrade_lock
There is examples of my problems:
void unlock(){
test.stream = 0;
test.mtx.unlock();
}
void lock_mtx(int i){
boost::unique_lock<boost::shared_mutex> lock(test.mtx);
test.stream = i;
boost::this_thread::sleep_for(boost::chrono::milliseconds(10000));
unlock();
boost::this_thread::sleep_for(boost::chrono::milliseconds(1000));
}
When I destruct lock , mutex is already unlocked by this thread, and sometimes it is locked by another thread, but destructor make it free again. After destruction of lock (in the first thread) third thread take mutex and I have two writers at the one moment
void lock_mtx(int i){
boost::upgrade_lock<boost::shared_mutex> lock(test.mtx);
read_from_locked();
boost::this_thread::sleep_for(boost::chrono::milliseconds(5000));
boost::upgrade_to_unique_lock<boost::shared_mutex> uniqueLock(lock);
write_to_locked();
boost::this_thread::sleep_for(boost::chrono::milliseconds(10000));
}
The second problem, is that when some thread takes upgrade_lock, other threads can't read shared objects
Both of problems occur in MS VisualStudio 2013 and Windows8 x64
void lock_mtx(int i)
{
{
boost::unique_lock<boost::shared_mutex> lock(test.mtx);
test.stream = i;
boost::this_thread::sleep_for(boost::chrono::milliseconds(10000));
}
boost::this_thread::sleep_for(boost::chrono::milliseconds(1000));
}
The purpose of unique_lock and lock_guard is to automatically unlock the mutex when it goes out of scope. This pattern is known as RAII.
Rule of thumb: Never manually call lock()/unlock() on your (Basic|Shared)Lockable objects. It's an antipattern because
It's extremely hard to get right (think of exception safety)
It usually indicates a code smell (locks being held across different method calls). If you even need this, consider making the RAII lock guard (lock_guard or unique_lock) a member of the containing class, or return the unique_lock so the caller has the option to explicitly adopt the lock, or to just let it be automatically released by the guard.
Related
How can I detect when a thread was ended (in a platform independent way)?
I have to store copies of objects for every thread and I want to know when I can dispose or redistribute it.
It's possibly via RAII and local_thread mechanism. We create a class that do usefull work in destructor.
class ThreadEndNotifer
{
public:
~ThreadEndNotifer()
{
// Do usefull work
useFullWork();
}
}
Next, we create local_thread variable. It can be global or class feild(thread_local class field is implicit static).
class Foo
{
private:
// Remember about initialization like static feild
thread_local ThreadEndNotifer mNotifer;
}
So, the useFullWork will be called every time when any thread are ending.
I like to create one global variable and init it only if needed, in this way I avoid overhead.
With multiple threads, in order to detect any one of them was ended, you need:
A shared condition_variable for notification, a shared mutex to lock, and shared variables for conditions of all threads.
void call_me_at_the_end_of_a_thread(int index_of_thread){
std::unique_lock<std::mutex> l(the_global_mutex);
array_of_bools[index_of_thread] = true;
num_of_dead_threads++; // global integer only for the convenience of checking before wait
std::notify_all_at_thread_exit(the_global_condition_variable, std::move(l));
}
You may use an array of bool or vector<bool> for checking which threads was ended. You may prefer notify_all to notify_all_at_thread_exit if you don't care about the timing of notifying after current thread has finished "completely".
void call_me_to_detect_thread_was_ended(void){
static int copy_of_num_of_dead_threads;
std::unique_lock<std::mutex> l(the_global_mutex);
while(num_of_dead_threads==copy_of_num_of_dead_threads)
the_global_condition_variable.wait(l);
std::cout<<num_of_dead_threads - copy_of_num_of_dead_threads<<" threads finished.\n";
copy_of_num_of_dead_threads=num_of_dead_threads;
}
num_of_dead_threads is only for simplicity. Check the array_of_bools to find out which threads have already finished.
I have this pretty straightforward piece of code that very rarely throws "System.ApplicationException : Object synchronization method was called from an unsynchronized block of code." when ReleaseMutex() is called.
I logically analyzed the flow of the method and just cannot understand how/why this could happen.
To my understanding, the ownership of mutex is guaranteed in this case:
readonly string mutexKey;
public Logger(string dbServer, string dbName)
{
this.mutexKey = ServiceManagerHelper.GetServiceName(dbServer, dbName);
}
private void Log(LogType type, string message, Exception ex)
{
using (var mutex = new Mutex(false, mutexKey))
{
bool acquiredMutex;
try
{
acquiredMutex = mutex.WaitOne(TimeSpan.FromSeconds(5));
}
catch (AbandonedMutexException)
{
acquiredMutex = true;
}
if (acquiredMutex)
{
try
{
// some application code here
}
finally
{
mutex.ReleaseMutex();
}
}
}
}
catch (AbandonedMutexException)
{
acquiredMutex = true;
}
This is a very serious bug in your code. Catching an AbandonedMutexException is never correct, it is a very serious mishap. Another thread acquired the mutex but terminated without calling ReleaseMutex(). You've irrecoverably lost synchronization and the mutex is no longer usable.
You were sort of lucky by making a mistake and assuming that you acquired the mutex anyway. You didn't. The ReleaseMutex() call will now bomb with the exception you quoted.
You cannot recover from this mishap, other than by terminating the program (the wise choice) or by disabling logging completely so the mutex will never be used again. Make the wise choice by removing the catch clause. Discovering the true source of the problem, that thread that crashed and didn't call ReleaseMutex(), is out of context for this question, there are no hints. You've been ignoring this problem, papered it over by catching AME, you can't ignore it.
In my case, i see the same behavior like Nathan Schubkegel. I use await's, and Thread.CurrentThread.ManagedThreadId gives another value for the "same" thread. I mean, thread was started with ManagedThreadId == 10, and Mutex was owned with this thread id, but later ReleaseMutex() causes ApplicationException with message: "Object synchronization method was called from an unsynchronized block of code", and i see that ManagedThreadId == 11 at this time :) . It seems, await sometimes changes thread id when returns. It seems, that is the reason. Mutex thinks that another thread wants to release it. It's sad, that Mutex documentation does not make ATTENTION on this moment.
So, you CAN NOT use asynchronous operator await between Mutex acquire and release. It's because C# compiler replaces plain operator await by asynchronous callback, and this callback can be made by ANOTHER thread. Usually, it's the same thread, but sometimes it's another thread (from thread pool).
Mutex checks thread. Only thread that acquired Mutex may release it. If you need synchronization without this checking, use Semaphore. SemaphoreSlim has asynchronous method WaitAsync() - it's cool.
This exception is raised when you call ReleaseMutex() from a thread that does not own the mutex. Search // some application code here for code that releases the mutex.
Also reconsider whether you're actually calling ReleaseMutex() from the same thread where you called WaitOne(). Example: I arrived at this post because I was using async/await and my code resumed on a different thread and tried to release a mutex the thread didn't own.
I have a worker thread, which sends some data over TCP, taking that data from several other threads. I need to fill some data, having sort of a mutex over it, and then call another thread's method, which would then unlock the mutex when finished while caller thread would continue its own job.
I've first implemented this using Qt as follows:
Data globalData;
QMutex mutex;
void requestSend() // several such functions in other threads
{
mutex.lock(); // we want to change the data
globalData=fillData();
invokeMethod(workerClass,"work",Qt::QueuedConnection);
}
void work() // a slot in a class instanced in worker thread
{
sendData(globalData);
mutex.unlock(); // data is now available to be changed
}
This seems reasonable and even works, but then I found this in the QMutex documentation:
void QMutex::unlock ()
Unlocks the mutex. Attempting to unlock a mutex in a different thread to the one that locked
it results in an error. Unlocking a mutex that is not locked results in undefined behavior.
I have two questions:
What's the reason of such restriction to unlock in a different thread? (and why don't I see the error the doc says about?)
What should I use instead of QMutex to achieve what I'm trying to? Would QWaitCondition be an adequate replacement?
The purpose of the mutex is to ensure that only one thread can access the data at any one time. Therefore, it doesn't really make sense to lock in one thread and unlock the same mutex in another.
If you're finding it works, you're probably just lucky at the moment, but doesn't mean it won't cause you issues if the timing of threads changes.
I'm not quite sure exactly what you're trying to do, but it appears that you have various threads that can write to the globalData and as soon as you write to it, you want another thread to send the data before more data writes to the globalData.
What I suggest is to create a mutex around the writing of the data and just call a signal to send the data to the thread that will send the data. Being on different threads, the data will be copied anyway: -
void requestSend() // several such functions in other threads
{
QMutexLocker locker(&mutex);
globalData=fillData();
emit SendData(globalData); // send signal to the thread which will send the data
}
Note that QMutexLocker is used to ensure the lock is released, even if an exception should occur.
Don't be too concerned about the copying of data in signals and slots; Qt is very efficient, and will only create a "copy on write", due to implicit sharing, if you use its container objects. Even if it has to make the copy for passing the data between the threads, you shouldn't really worry about it, unless you can see a performance issue.
Finally, note that implicit sharing and multithreading can work happily together, as you can read here.
When a thread has acquired the lock and execute the following code, Could the thread will unlock the lock it has acquired just with the return statement? some code like this.
static pthread_mutex_t mutex;
int foo()
{
pthread_mutex_lock(mutex);
.........
execute some code here and some errors happen
return -1;// but without pthread_mutex_unlock
pthread_mutex_unlock(mutext)
return 0;
}
Some errors happens before pthread_mutex_unlock statement and the thread returns to the callee. Will the thread give back the mutext lock for other threads without executing pthread_mutex_unlock?
No, the lock is not automatically released. This is why, in C++ code, it is common to use Resource Aquisition is Initialization (RAII), which takes advantage of construction/destruction to ensure that each call to the lock function has a corresponding call to unlock. If you are writing pure C code, though, you will need to make sure that you unlock the mutex, even in error situations, before returning.
Note that you can make your coding a little bit easier by doing the following:
static inline int some_function_critical_section_unsynchronized(void) {
// ...
}
int some_function(void) {
int status = 0;
pthread_mutex_lock(mutex);
status = some_function_critical_section_unsynchronized();
pthread_mutex_unlock(mutex);
return status;
}
In other words, if you can separate the logic into smaller functions, you may be able to tease out the locking code from your logic. Of course, sometimes this is not possible (like when coding in this fashion would make the critical section too large, and for performance, the less readable form is required).
If you can use C++, I would strongly suggest using boost::thread and boost::scoped_lock to ensure that the acquired mutex is automatically freed when its usage has gone out of scope.
No, it will not automatically unlock the mutex. You must explicitly call pthread_mutex_unlock() in the error path, if the mutex has been locked by the function.
A mutex is a programming concept that is frequently used to solve multi-threading problems. My question to the community:
What is a mutex and how do you use it?
When I am having a big heated discussion at work, I use a rubber chicken which I keep in my desk for just such occasions. The person holding the chicken is the only person who is allowed to talk. If you don't hold the chicken you cannot speak. You can only indicate that you want the chicken and wait until you get it before you speak. Once you have finished speaking, you can hand the chicken back to the moderator who will hand it to the next person to speak. This ensures that people do not speak over each other, and also have their own space to talk.
Replace Chicken with Mutex and person with thread and you basically have the concept of a mutex.
Of course, there is no such thing as a rubber mutex. Only rubber chicken. My cats once had a rubber mouse, but they ate it.
Of course, before you use the rubber chicken, you need to ask yourself whether you actually need 5 people in one room and would it not just be easier with one person in the room on their own doing all the work. Actually, this is just extending the analogy, but you get the idea.
A Mutex is a Mutually exclusive flag. It acts as a gate keeper to a section of code allowing one thread in and blocking access to all others. This ensures that the code being controlled will only be hit by a single thread at a time. Just be sure to release the mutex when you are done. :)
Mutual Exclusion. Here's the Wikipedia entry on it.
The point of a mutex is to synchronize two threads. When you have two threads attempting to access a single resource, the general pattern is to have the first block of code attempting access to set the mutex before entering the code. When the second code block attempts access, it sees that the mutex is set and waits until the first block of code is complete (and unsets the mutex), then continues.
Specific details of how this is accomplished obviously varies greatly by programming language.
When you have a multi-threaded application, the different threads sometimes share a common resource, such as a variable or similar. This shared source often cannot be accessed at the same time, so a construct is needed to ensure that only one thread is using that resource at a time.
The concept is called "mutual exclusion" (short Mutex), and is a way to ensure that only one thread is allowed inside that area, using that resource etc.
How to use them is language specific, but is often (if not always) based on a operating system mutex.
Some languages doesn't need this construct, due to the paradigm, for example functional programming (Haskell, ML are good examples).
What is a Mutex?
The mutex (In fact, the term mutex is short for mutual exclusion) also known as spinlock is the simplest synchronization tool that is used to protect critical regions and thus prevent race conditions. That is a thread must acquire a lock before entering into a critical section (In critical section multi threads share a common variable, updating a table, writing a file and so on), it releases the lock when it leaves critical section.
What is a Race Condition?
A race condition occurs when two or more threads can access shared data and they try to change it at the same time. Because the thread scheduling algorithm can swap between threads at any time, you don't know the order in which the threads will attempt to access the shared data. Therefore, the result of the change in data is dependent on the thread scheduling algorithm, i.e. both threads are "racing" to access/change the data.
Real life example:
When I am having a big heated discussion at work, I use a rubber
chicken which I keep in my desk for just such occasions. The person
holding the chicken is the only person who is allowed to talk. If you
don't hold the chicken you cannot speak. You can only indicate that
you want the chicken and wait until you get it before you speak. Once
you have finished speaking, you can hand the chicken back to the
moderator who will hand it to the next person to speak. This ensures
that people do not speak over each other, and also have their own
space to talk.
Replace Chicken with Mutex and person with thread and you basically have the concept of a mutex.
#Xetius
Usage in C#:
This example shows how a local Mutex object is used to synchronize access to a protected resource. Because each calling thread is blocked until it acquires ownership of the mutex, it must call the ReleaseMutex method to release ownership of the thread.
using System;
using System.Threading;
class Example
{
// Create a new Mutex. The creating thread does not own the mutex.
private static Mutex mut = new Mutex();
private const int numIterations = 1;
private const int numThreads = 3;
static void Main()
{
// Create the threads that will use the protected resource.
for(int i = 0; i < numThreads; i++)
{
Thread newThread = new Thread(new ThreadStart(ThreadProc));
newThread.Name = String.Format("Thread{0}", i + 1);
newThread.Start();
}
// The main thread exits, but the application continues to
// run until all foreground threads have exited.
}
private static void ThreadProc()
{
for(int i = 0; i < numIterations; i++)
{
UseResource();
}
}
// This method represents a resource that must be synchronized
// so that only one thread at a time can enter.
private static void UseResource()
{
// Wait until it is safe to enter.
Console.WriteLine("{0} is requesting the mutex",
Thread.CurrentThread.Name);
mut.WaitOne();
Console.WriteLine("{0} has entered the protected area",
Thread.CurrentThread.Name);
// Place code to access non-reentrant resources here.
// Simulate some work.
Thread.Sleep(500);
Console.WriteLine("{0} is leaving the protected area",
Thread.CurrentThread.Name);
// Release the Mutex.
mut.ReleaseMutex();
Console.WriteLine("{0} has released the mutex",
Thread.CurrentThread.Name);
}
}
// The example displays output like the following:
// Thread1 is requesting the mutex
// Thread2 is requesting the mutex
// Thread1 has entered the protected area
// Thread3 is requesting the mutex
// Thread1 is leaving the protected area
// Thread1 has released the mutex
// Thread3 has entered the protected area
// Thread3 is leaving the protected area
// Thread3 has released the mutex
// Thread2 has entered the protected area
// Thread2 is leaving the protected area
// Thread2 has released the mutex
MSDN Reference Mutex
There are some great answers here, here is another great analogy for explaining what mutex is:
Consider single toilet with a key. When someone enters, they take the key and the toilet is occupied. If someone else needs to use the toilet, they need to wait in a queue. When the person in the toilet is done, they pass the key to the next person in queue. Make sense, right?
Convert the toilet in the story to a shared resource, and the key to a mutex. Taking the key to the toilet (acquire a lock) permits you to use it. If there is no key (the lock is locked) you have to wait. When the key is returned by the person (release the lock) you're free to acquire it now.
In C#, the common mutex used is the Monitor. The type is 'System.Threading.Monitor'. It may also be used implicitly via the 'lock(Object)' statement. One example of its use is when constructing a Singleton class.
private static readonly Object instanceLock = new Object();
private static MySingleton instance;
public static MySingleton Instance
{
lock(instanceLock)
{
if(instance == null)
{
instance = new MySingleton();
}
return instance;
}
}
The lock statement using the private lock object creates a critical section. Requiring each thread to wait until the previous is finished. The first thread will enter the section and initialize the instance. The second thread will wait, get into the section, and get the initialized instance.
Any sort of synchronization of a static member may use the lock statement similarly.
To understand MUTEX at first you need to know what is "race condition" and then only you will understand why MUTEX is needed. Suppose you have a multi-threading program and you have two threads. Now, you have one job in the job queue. The first thread will check the job queue and after finding the job it will start executing it. The second thread will also check the job queue and find that there is one job in the queue. So, it will also assign the same job pointer. So, now what happens, both the threads are executing the same job. This will cause a segmentation fault. This is the example of a race condition.
The solution to this problem is MUTEX. MUTEX is a kind of lock which locks one thread at a time. If another thread wants to lock it, the thread simply gets blocked.
The MUTEX topic in this pdf file link is really worth reading.
Mutexes are useful in situations where you need to enforce exclusive access to a resource accross multiple processes, where a regular lock won't help since it only works accross threads.
Mutex: Mutex stands for Mutual Exclusion. It means only one process/thread can enter into critical section at a given time. In concurrent programming multiple threads/process updating the shared resource (any variable, shared memory etc.) may lead to some unexpected result. ( As the result depends upon the which thread/process gets the first access).
In order to avoid such an unexpected result we need some synchronization mechanism, which ensures that only one thread/process gets access to such a resource at a time.
pthread library provides support for Mutex.
typedef union
{
struct __pthread_mutex_s
{
***int __lock;***
unsigned int __count;
int __owner;
#ifdef __x86_64__
unsigned int __nusers;
#endif
int __kind;
#ifdef __x86_64__
short __spins;
short __elision;
__pthread_list_t __list;
# define __PTHREAD_MUTEX_HAVE_PREV 1
# define __PTHREAD_SPINS 0, 0
#else
unsigned int __nusers;
__extension__ union
{
struct
{
short __espins;
short __elision;
# define __spins __elision_data.__espins
# define __elision __elision_data.__elision
# define __PTHREAD_SPINS { 0, 0 }
} __elision_data;
__pthread_slist_t __list;
};
#endif
This is the structure for mutex data type i.e pthread_mutex_t.
When mutex is locked, __lock set to 1. When it is unlocked __lock set to 0.
This ensure that no two processes/threads can access the critical section at same time.