I'm writing a multi-threaded application using Core Data.
I'm under the impression that - [NSManagedObjectContext lock] does not work as a standard lock. As a proof of concept here is a sample built after the Xcode 3 "Command-line tool" template, "Core Data" flavor:
int main (int argc, const char * argv[])
{
objc_startCollectorThread();
NSManagedObjectContext *context = managedObjectContext();
[context lock];
[context lock];
NSLog(#"hello world! (context=%#)", context);
[context unlock];
[context unlock];
return 0;
}
That should result in a deadlock, instead when running under Mac OS X 10.6.8 or 10.7.4, I've got this log:
2012-07-18 16:53:40.206 test[20004:a0b] hello world! (context=<NSManagedObjectContext: 0x20000df40>)
Can anybody tell me what is happening?
(if I use instances of NSLock instead of the context, the deadlock happens as expected)
here is an excerpt from Apple Documentation about NSManagedObjectContext lock
lock
Attempts to acquire a lock on the receiver.
- (void)lock
Discussion
This method blocks a thread’s execution until the lock can be acquired. An application protects a critical section of code by requiring a thread to acquire a lock before executing the code. Once the critical section is past, the thread relinquishes the lock by invoking unlock.
Sending this message to a managed object context helps the framework to understand the scope of a transaction in a multi-threaded environment. It is preferable to use the NSManagedObjectContext’s implementation of NSLocking instead using of a separate mutex object.
If you lock (or successfully tryLock) a managed object context, the thread in which the lock call is made must have a retain until it invokes unlock. If you do not properly retain a context in a multi-threaded environment, this will result in deadlock.
Availability
Available in Mac OS X v10.4 and later.
It appears that for an unknown reason, subsequent NSManagedObject locks are ignored when happening in a same thread.
Here is another sample code working as expected (no log at execution):
#implementation NSManagedObjectContext (Test)
- (void)testLock:(id)sender
{
[self lock];
NSLog(#"hello world! (context=%#)", self);
[self unlock];
}
#end
int main (int argc, const char * argv[])
{
objc_startCollectorThread();
NSManagedObjectContext *context = managedObjectContext();
[context lock];
[NSThread detachNewThreadSelector:#selector(testLock:) toTarget:context withObject:nil];
sleep(2);
}
Note:
- (BOOL)[NSManagedObjectContext tryLock] also return YESfor any subsequent calls.
Related
The main question is: How we can wait for a thread in Linux kernel to complete? I have seen a few post concerned about proper way of handling threads in Linux kernel but i'm not sure how we can wait for a single thread in the main thread to be completed (suppose we need the thread[3] be done then proceed):
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/sched.h>
#include <linux/kthread.h>
#include <linux/slab.h>
void *func(void *arg) {
// doing something
return NULL;
}
int init_module(void) {
struct task_struct* thread[5];
int i;
for(i=0; i<5; i++) {
thread[i] = kthread_run(func, (void*) arg, "Creating thread");
}
return 0;
}
void cleanup_module(void) {
printk("cleaning up!\n");
}
AFAIK there is no equivalent of pthread_join() in kernel. Also, I feel like your pattern (of starting bunch of threads and waiting only for one of them) is not really common in kernel. That being said, there kernel does have few synchronization mechanism that may be used to accomplish your goal.
Note that those mechanisms will not guarantee that the thread finished, they will only let main thread know that they finished doing the work they were supposed to do. It may still take some time to really stop this tread and free all resources.
Semaphores
You can create a locked semaphore, then call down in your main thread. This will put it to sleep. Then you will up this semaphore inside of your thread just before exiting. Something like:
struct semaphore sem;
int func(void *arg) {
struct semaphore *sem = (struct semaphore*)arg; // you could use global instead
// do something
up(sem);
return 0;
}
int init_module(void) {
// some initialization
init_MUTEX_LOCKED(&sem);
kthread_run(&func, (void*) &sem, "Creating thread");
down(&sem); // this will block until thread runs up()
}
This should work but is not the most optimal solution. I mention this as it's a known pattern that is also used in userspace. Semaphores in kernel are designed for cases where it's mostly available and this case has high contention. So a similar mechanism optimized for this case was created.
Completions
You can declare completions using:
struct completion comp;
init_completion(&comp);
or:
DECLARE_COMPLETION(comp);
Then you can use wait_for_completion(&comp); instead of down() to wait in main thread and complete(&comp); instead of up() in your thread.
Here's the full example:
DECLARE_COMPLETION(comp);
struct my_data {
int id;
struct completion *comp;
};
int func(void *arg) {
struct my_data *data = (struct my_data*)arg;
// doing something
if (data->id == 3)
complete(data->comp);
return 0;
}
int init_module(void) {
struct my_data *data[] = kmalloc(sizeof(struct my_data)*N, GFP_KERNEL);
// some initialization
for (int i=0; i<N; i++) {
data[i]->comp = ∁
data[i]->id = i;
kthread_run(func, (void*) data[i], "my_thread%d", i);
}
wait_for_completion(&comp); // this will block until some thread runs complete()
}
Multiple threads
I don't really see why you would start 5 identical threads and only want to wait for 3rd one but of course you could send different data to each thread, with a field describing it's id, and then call up or complete only if this id equals 3. That's shown in the completion example. There are other ways to do this, this is just one of them.
Word of caution
Go read some more about those mechanisms before using any of them. There are some important details I did not write about here. Also those examples are simplified and not tested, they are here just to show the overall idea.
kthread_stop() is a kernel's way for wait thread to end.
Aside from waiting, kthread_stop() also sets should_stop flag for waited thread and wake up it, if needed. It is usefull for threads which repeat some actions infinitely.
As for single-shot tasks, it is usually simpler to use works for them, instead of kthreads.
EDIT:
Note: kthread_stop() can be called only when kthread(task_struct) structure is not freed.
Either thread function should return only after it found kthread_should_stop() return true, or get_task_struct() should be called before start thread (and put_task_struct() should be called after kthread_stop()).
Using C++11 STL with VS2013 to implementing a asynchronous print class.
Failing to get thread.join() returns with no deadlocking.
I am trying to debug and finally find this issue may caused by global/local class variable declaration. Here is the details and I dont know why it happened?
#include <iostream>
#include <string>
#include <chrono>
#include <mutex>
#include <thread>
#include <condition_variable>
#include "tbb/concurrent_queue.h"
using namespace std;
class logger
{
public:
~logger()
{
fin();
}
void init()
{
m_quit = false;
m_thd = thread(bind(&logger::printer, this));
//thread printer(bind(&logger::printer, this));
//m_thd.swap(printer);
}
void fin()
{
//not needed
//unique_lock<mutex> locker(m_mtx);
if (m_thd.joinable())
{
m_quit = true;
write("fin");
//locker.unlock();
m_thd.join();
}
}
void write(const char *msg)
{
m_queue.push(msg);
m_cond.notify_one();
}
void printer()
{
string msgstr;
unique_lock<mutex> locker(m_mtx);
while (1)
{
if (m_queue.try_pop(msgstr))
cout << msgstr << endl;
else if (m_quit)
break;
else
m_cond.wait(locker);
}
cout << "printer quit" <<endl;
}
bool m_quit;
mutex m_mtx;
condition_variable m_cond;
thread m_thd;
tbb::concurrent_queue<string> m_queue;
};
For more convenience I placed thread.join into class's destructor in order to ensure the m_thread can be quit normally.
I test the whole class and something wrong occured.
m_thd.join() never return when class logger declared as a global var
like this:
logger lgg;
void main()
{
lgg.init();
for (int i = 0; i < 100; ++i)
{
char s[8];
sprintf_s(s, 8, "%d", i);
lgg.write(s);
}
//if first call lgg.fin() here, m_thd can be joined normally
//lgg.fin();
system("pause");
//dead&blocked here and I observed that printer() finished successfully
}
If class logger declared as a local variable, it seems everything works well.
void main()
{
logger lgg;
lgg.init();
for (int i = 0; i < 100; ++i)
{
char s[8];
sprintf_s(s, 8, "%d", i);
lgg.write(s);
}
system("pause");
}
update 2015/02/27
I tried to delete std::cout in printer(), but program still blocked at same place, seems it is not the std::cout problem?
Deleting supernumerary lock in fin()
Globals and statics are constructed and destructed just prior or post to DllMain getting called respectively for DLL_PROCESS_ATTACH and DLL_PROCESS_DETACH. The problem with this is that it occurs inside the loader lock. Which is the most dangerous place on the planet to be if dealing with kernel objects as it may cause deadlock, or the application to randomly crash. As such you should never use thread primitives as statics on windows EVER. Thus dealing with threading in a destructor of a global object is basically doing the exact things we're warned not to do in DllMain.
To quote Raymond Chen
The building is being demolished. Don't bother sweeping the floor and emptying the trash cans and erasing the whiteboards. And don't line up at the exit to the building so everybody can move their in/out magnet to out. All you're doing is making the demolition team wait for you to finish these pointless housecleaning tasks.
and again:
If your DllMain function creates a thread and then waits for the thread to do something (e.g., waits for the thread to signal an event that says that it has finished initializing, then you've created a deadlock. The DLL_PROCESS_ATTACH notification handler inside DllMain is waiting for the new thread to run, but the new thread can't run until the DllMain function returns so that it can send a new DLL_THREAD_ATTACH notification.
This deadlock is much more commonly seen in DLL_PROCESS_DETACH, where a DLL wants to shut down its worker threads and wait for them to clean up before it unloads itself. You can't wait for a thread inside DLL_PROCESS_DETACH because that thread needs to send out the DLL_THREAD_DETACH notifications before it exits, which it can't do until your DLL_PROCESS_DETACH handler returns.
This also occurs even when using an EXE because the visual C++ runtime cheats and registers its constructors and destructors with the C runtime to be run when the runtime is loaded or unloaded, thus ending up with the same issue:
The answer is that the C runtime library hires a lackey. The hired lackey is the C runtime library DLL (for example, MSVCR80.DLL). The C runtime startup code in the EXE registers all the destructors with the C runtime library DLL, and when the C runtime library DLL gets its DLL_PROCESS_DETACH, it calls all the destructors requested by the EXE.
I'm wondering how you're using m_mtx. The normal pattern is that both thread lock it and both threads unlock it. But fin() fails to lock it.
Similarly unexpected is m_cond.wait(m_mtx). This would release the mutex, except that it isn't locked in the first place!
Finally, as m_mtx isn't locked, I don't see how m_quit = true should become visible in m_thd.
One problem you have is that std::condition_variable::notify_one is called while the same std::mutex that the waiting thread is holding, is held (happens when logger::write is called by logger::fin).
This causes the notified thread to immediately block again, and hence the printer thread will block possibly indefinitely upon destruction (or until spurious wakeup).
You should never notify while holding the same mutex as the waiting thread(s).
Quote from en.cppreference.com:
The notifying thread does not need to hold the lock on the same mutex as the one held by the waiting thread(s); in fact doing so is a pessimization, since the notified thread would immediately block again, waiting for the notifying thread to release the lock.
I'm trying to use a C++11 std::condition_variable, but when I try to lock the unique_lock associated with it from a second thread I get an exception "Resource deadlock avoided". The thread that created it can lock and unlock it, but not the second thread, even though I'm pretty sure the unique_lock shouldn't be locked already at the point the second thread tries to lock it.
FWIW I'm using gcc 4.8.1 in Linux with -std=gnu++11.
I've written a wrapper class around the condition_variable, unique_lock and mutex, so nothing else in my code has direct access to them. Note the use of std::defer_lock, I already fell in to that trap :-).
class Cond {
private:
std::condition_variable cCond;
std::mutex cMutex;
std::unique_lock<std::mutex> cULock;
public:
Cond() : cULock(cMutex, std::defer_lock)
{}
void wait()
{
std::ostringstream id;
id << std::this_thread::get_id();
H_LOG_D("Cond %p waiting in thread %s", this, id.str().c_str());
cCond.wait(cULock);
H_LOG_D("Cond %p woke up in thread %s", this, id.str().c_str());
}
// Returns false on timeout
bool waitTimeout(unsigned int ms)
{
std::ostringstream id;
id << std::this_thread::get_id();
H_LOG_D("Cond %p waiting (timed) in thread %s", this, id.str().c_str());
bool result = cCond.wait_for(cULock, std::chrono::milliseconds(ms))
== std::cv_status::no_timeout;
H_LOG_D("Cond %p woke up in thread %s", this, id.str().c_str());
return result;
}
void notify()
{
cCond.notify_one();
}
void notifyAll()
{
cCond.notify_all();
}
void lock()
{
std::ostringstream id;
id << std::this_thread::get_id();
H_LOG_D("Locking Cond %p in thread %s", this, id.str().c_str());
cULock.lock();
}
void release()
{
std::ostringstream id;
id << std::this_thread::get_id();
H_LOG_D("Releasing Cond %p in thread %s", this, id.str().c_str());
cULock.unlock();
}
};
My main thread creates a RenderContext, which has a thread associated with it. From the main thread's point of view, it uses the Cond to signal the rendering thread to perform an action and can also wait on the COnd for the rendering thread to complete that action. The rendering thread waits on the Cond for the main thread to send rendering requests, and uses the same Cond to tell the main thread it's completed an action if necessary. The error I'm getting occurs when the rendering thread tries to lock the Cond to check/wait for render requests, at which point it shouldn't be locked at all (because the main thread is waiting on it), let alone by the same thread. Here's the output:
DEBUG: Created window
DEBUG: OpenGL 3.0 Mesa 9.1.4, GLSL 1.30
DEBUG: setScreen locking from thread 140564696819520
DEBUG: Locking Cond 0x13ec1e0 in thread 140564696819520
DEBUG: Releasing Cond 0x13ec1e0 in thread 140564696819520
DEBUG: Entering GLFW main loop
DEBUG: requestRender locking from thread 140564696819520
DEBUG: Locking Cond 0x13ec1e0 in thread 140564696819520
DEBUG: requestRender waiting
DEBUG: Cond 0x13ec1e0 waiting in thread 140564696819520
DEBUG: Running thread 'RenderThread' with id 140564575180544
DEBUG: render thread::run locking from thread 140564575180544
DEBUG: Locking Cond 0x13ec1e0 in thread 140564575180544
terminate called after throwing an instance of 'std::system_error'
what(): Resource deadlock avoided
To be honest I don't really understand what a unique_lock is for and why condition_variable needs one instead of using a mutex directly, so that's probably the cause of the problem. I can't find a good explanation of it online.
Foreword: An important thing to understand with condition variables is that they can be subject to random, spurious wake ups. In other words, a CV can exit from wait() without anyone having called notify_*() first. Unfortunately there is no way to distinguish such a spurious wake up from a legitimate one, so the only solution is to have an additional resource (at the very least a boolean) so that you can tell whether the wake up condition is actually met.
This additional resource should be guarded by a mutex too, usually the very same you use as a companion for the CV.
The typical usage of a CV/mutex pair is as follows:
std::mutex mutex;
std::condition_variable cv;
Resource resource;
void produce() {
// note how the lock only protects the resource, not the notify() call
// in practice this makes little difference, you just get to release the
// lock a bit earlier which slightly improves concurrency
{
std::lock_guard<std::mutex> lock(mutex); // use the lightweight lock_guard
make_ready(resource);
}
// the point is: notify_*() don't require a locked mutex
cv.notify_one(); // or notify_all()
}
void consume() {
std::unique_lock<std::mutex> lock(mutex);
while (!is_ready(resource))
cv.wait(lock);
// note how the lock still protects the resource, in order to exclude other threads
use(resource);
}
Compared to your code, notice how several threads can call produce()/consume() simultaneously without worrying about a shared unique_lock: the only shared things are mutex/cv/resource and each thread gets its own unique_lock that forces the thread to wait its turn if the mutex is already locked by something else.
As you can see, the resource can't really be separated from the CV/mutex pair, which is why I said in a comment that your wrapper class wasn't really fitting IMHO, since it indeed tries to separate them.
The usual approach is not to make a wrapper for the CV/mutex pair as you tried to, but for the whole CV/mutex/resource trio. Eg. a thread-safe message queue where the consumer threads will wait on the CV until the queue has messages ready to be consumed.
If you really want to wrap just the CV/mutex pair, you should get rid of your lock()/release() methods which are unsafe (from a RAII point of view) and replace them with a single lock() method returning a unique_ptr:
std::unique_ptr<std::mutex> lock() {
return std::unique_ptr<std::mutex>(cMutex);
}
This way you can use your Cond wrapper class in rather the same way as what I showed above:
Cond cond;
Resource resource;
void produce() {
{
auto lock = cond.lock();
make_ready(resource);
}
cond.notify(); // or notifyAll()
}
void consume() {
auto lock = cond.lock();
while (!is_ready(resource))
cond.wait(lock);
use(resource);
}
But honestly I'm not sure it's worth the trouble: what if you want to use a recursive_mutex instead of a plain mutex? Well, you'd have to make a template out of your class so that you can choose the mutex type (or write a second class altogether, yay for code duplication). And anyway you don't gain much since you still have to write pretty much the same code in order to manage the resource. A wrapper class only for the CV/mutex pair is too thin a wrapper to be really useful IMHO. But as usual, YMMV.
I have been battling this for a long time now. My understanding of Autorelease is that when it doesnt need it any longer it will release it.
I was getting one of those evil EXC_BAD_ACCESS without any details.
It would just crash on:
int main(int argc, char *argv[])
{
#autoreleasepool {
return UIApplicationMain(argc, argv, nil, NSStringFromClass([Logix_AppDelegate class]));
}
}
With Zombies enabled I found that I was writing to CoreData in a loop
here is my initialization of the NSManagedObjectContext
NSManagedObjectContext *context = [[(MyAppDelegate *)[[UIApplication sharedApplication] delegate] managedObjectContext] autorelease];
Any thoughts here.... there is a lot of code, so I didnt want to paste volumes of stuff here.
autorelease adds an object to the current autorelease pool. That object will receive a release message (and potentially be deallocated) when the current autorelease pool is destroyed.
Sending autorelease to an object makes sense only if you own that object. In your case, the managedObjectContext method has not "new", "alloc" or "copy" in its name, so you don't own the returned object and must not call autorelease on it.
So you should replace that line by
NSManagedObjectContext *context = [(MyAppDelegate *)[[UIApplication sharedApplication] delegate] managedObjectContext];
See also Memory Management Policy in the "Advanced Memory Management Programming Guide":
You own any object you create
You create an object using a method
whose name begins with “alloc”, “new”, “copy”, or “mutableCopy” (for
example, alloc, newObject, or mutableCopy). ...
When you no longer need it, you must relinquish ownership of an object you own
You relinquish ownership of an object by sending it a release
message or an autorelease message. ...
You must not relinquish ownership of an object you do not own
I have a multi-threaded iOS App with a background thread that synchronizes data with the Cloud. I have read several questions regarding merging changes from a background thread to the main thread.
However, what I need to do is the opposite. I somehow need to make sure that a newly spawned background thread picks up data that was newly created within the moc of the main thread. Right before spawning the thread. For some reason, some data added to the main thread's poc even after a save does not show up in the moc of the new thread. I am definitely missing something while spawining the thread and its moc.
btw: I have a moc per thread and a shared persistent store coordinator.
Here is the code that sets up the moc per thread:
-(NSManagedObjectContext *) storageContext
NSThread *currentThread = [NSThread currentThread];
NSManagedObjectContext *context = [currentThread.threadDictionary objectForKey:#"context"];
if(context != nil)
{
return context;
}
context = [[[NSManagedObjectContext alloc] init] autorelease];
[context setPersistentStoreCoordinator: self.coordinator];
[currentThread.threadDictionary setObject:context forKey:#"context"];
return context;
And here is the code that spawns the thread
-(void)startBackgroundSync
AutoSync *target = [AutoSync withInit];
NSInvocationOperation *operation = [[NSInvocationOperation alloc] initWithTarget:target selector:#selector(sync) object:nil];
operation = [operation autorelease];
//Add the operation to the queue
[queue addOperation:operation];
A nudge in the right direction will be greatly appreciated
Thanks
Sohil