I'm having a conflict when saving a bunch of NSManagedObjects via an outside thread. For starters, I can tell you the following:
I'm using a separate MOC for each thread.
The MOCs share the same persistent store coordinator.
It's likely that an outside thread is modifying one or many of the records that I'm saving.
OK, so with that out of the way, here's what I'm doing.
In my outside thread, I'm doing some computation and updating a single value in a bunch of managed objects. I do this by looking up the object in the persistent store by my primary key, modifying the single decimal property, and then calling save on the bunch all at once.
In the meantime, I believe the main thread is doing some updating of its own.
When my outside thread does its big save on its managed object context, I get an exception thrown stating a large number of conflicts. All of the conflicts seem to be centered around a single relationship on each record. Though the managed object in the persistent store and my outside thread share the same ObjectID for this relationship, they don't share the same pointer. Based on what I see, that's the only thing that's different between the objects in my NSMergeConflict debug output.
It makes sense to me why the two objects have relationships with different pointers -- they're in different threads. However, as I understand it from Apple's documentation, the only thing cached when an object is first retrieved from the persistent store are the global IDs. So, one would think that when I run save on the outside thread MOC, it compares the ObjectIDs, sees they're the same, and lets it all through.
So, can anyone tell me why I'm getting a conflict?
Per the documentation in the Concurrency with Core Data chapter of The Core Data Programming Guide, the recommended configuration is for the contexts to share the same persistent store coordinator, not just the same persistent store.
Also, the section Track Changes in Other Threads Using Notifications of the same chapter states if you're tracking updates with the NSManagedObjectContextDidSaveNotification then you send -mergeChangesFromContextDidSaveNotification to the main thread's context so it can merge the changes. But if you're tracking with NSManagedObjectContextDidChangeNotification then the external thread should send the object IDs of the modified objects to the main thread which will then send -refreshObject:mergeChanges: to its context for each modified object.
And really, you should know if the main thread is also performing updates through its controller, and propagate its changes in like manner but in the opposite direction.
You need to have all your contexts listening for NSManagedObjectContextDidSaveNotification from any context that makes changes. Otherwise, only the front context will be aware of changes made on the background threads but the background context won't be aware of changes on the front thread.
So, if you have three threads and three context each of which makes changes, all three context must register for notifications from the other two.
Unfortunately, it seems as though this bug was actually being caused by something else -- I was calling the operation causing the error more than once at the same time when I shouldn't have been. Although this doesn't answer the initial question as to why pointers matter in conflicts, updating my code to prevent this situation has resolved my issue.
Related
Let's have a worker thread which is accessed from a wide variety of objects. This worker object has some public slots, so anyone who connects its signals to the worker's slots can use emit to trigger the worker thread's useful tasks.
This worker thread needs to be almost global, in the sense that several different classes use it, some of them are deep in the hierarchy (child of a child of a child of the main application).
I guess there are two major ways of doing this:
All the methods of the child classes pass their messages upwards the hierarchy via their return values, and let the main (e.g. the GUI) object handle all the emitting.
All those classes which require the services of the worker thread have a pointer to the Worker object (which is a member of the main class), and they all connect() to it in their constructors. Every such class then does the emitting by itself. Basically, dependency injection.
Option 2. seems much more clean and flexible to me, I'm only worried that it will create a huge number of connections. For example, if I have an array of an object which needs the thread, I will have a separate connection for each element of the array.
Is there an "official" way of doing this, as the creators of Qt intended it?
There is no magic silver bullet for this. You'll need to consider many factors, such as:
Why do those objects emit the data in the first place? Is it because they need to do something, that is, emission is a “command”? Then maybe they could call some sort of service to do the job without even worrying about whether it's going to happen in another thread or not. Or is it because they inform about an event? In such case they probably should just emit signals but not connect them. Its up to the using code to decide what to do with events.
How many objects are we talking about? Some performance tests are needed. Maybe it's not even an issue.
If there is an array of objects, what purpose does it serve? Perhaps instead of using a plain array some sort of “container” class is needed? Then the container could handle the emission and connection and objects could just do something like container()->handle(data). Then you'd only have one connection per container.
I need to be able to to grab objects from Core Data and keep them in a mutable array in memory in order to avoid constant fetching and slow UI/UX. The problem is that I grab the objects on other threads. I also do writing to these objects at times on other threads. Because of this I can't just save the NSManagedObjects in an array and just call something like myManagedObjectContext.performBlock or myObject.managedObjectContext.PerformBlock since you are not supposed to pass MOCs between threads.
I was thinking of using a custom object to throw the data I need from the CD objects into. This feels a little stupid since I already made a Model/NSManagedObject class for the entities and since the custom object would be mutable it still would not be thread safe. This means I would have to do something like a serial queue for object manipulation on multiple threads? So for example any time I want to read/write/delete an object I have to throw it into my object serialQueue.
This all seems really nasty so I am wondering are there any common design patterns for this problem or something similar? Is there a better way of doing this?
I doubt you need custom objects between Core Data and your UI. There is a better answer:
Your UI should read from the managed objects that are associated with the main thread (which it sounds like you are doing).
When you make changes on another thread those changes will update the objects that are on your main thread. That is what Core Data is designed to do.
You just need to listen to those changes and have your UI react to them.
There are several ways to do this:
NSFetchedResultsController. Kind of like your mutable array but has a delegate it will notify when objects change. Highly recommended
Listen for KVO changes on the property that you are displaying in your UI. Whenever the property changes you get a KVO notification and can react to it. More code but also more narrowly focused.
Listen for NSManagedObjectContextDidSaveNotification events via the NSNotification center and react to the notification. The objects that are being changed will be in the userInfo of the notification.
Of the three, using a NSFetchedResultsController is usually the right answer. When that in place you just change what you need to change on other threads, save the context and you are done. The UI will update itself.
One pattern is to pass along only the object ids, which are NSString objects, immutable and thus thread safe, and query on the main thread after those ids. This way every NSManagedObject will belong to the appropriate thread.
Alternatively, you can use mergeChangesFromContextDidSaveNotification which will update the objects from the main thread with the changes made on the secondary thread. You'd still need fetching for new objects, though.
The "caveat" is that you need to save the secondary context in order to get your hands on a notification like this. Also any newly created, but not saved objects from the main thread will be lost after applying the merge - however this might not pose problems if your main thread only consumes CoreData objects.
My application has NSOperation subclasses that fetch and operate on managed objects. My application also periodically purges rows from the database, which can result in the following race condition:
An background operation fetches a bunch of objects (from a thread-specific context). It will iterate over these objects and do something with their properties.
A bunch of rows are deleted in the main managed object context.
The background operation accesses a property on an object that was deleted from the main context. This results in an 'NSObjectInaccessibleException', reason: 'CoreData could not fulfill a fault'
Ideally, the objects that are fetched by the NSOperation can be operated on even if one is deleted in the main context. The best way I can think to achieve this is either to:
Call [request setReturnsObjectsAsFaults:NO] to ensure that Core Data won't try to fulfill a fault for an object that no longer exists in the main context. The problem here is I may need to access the object's relationships, which (to my understanding) will still be faulted.
Iterate through the managed objects up front and copy the properties I will need into separate non-managed objects. The problem here is that (I think) I will need to synchronize/lock this part, in case an object is deleted in the main context before I can finish copying.
Am I missing something obvious? It doesn't seem like what I'm trying to accomplish is too out of the ordinary. Thanks for your help.
You said each thread has its own context. That's good. However, they also need to stay synchronized with changes to each other (how depends on their hierarchy).
Are the all assigned to the same persistent store coordinator, or do they have parent/child relationships?
Siblings should monitor NSManagedObjectContextObjectsDidChangeNotification from other siblings. Parents will automatically get notified when a child context saves.
I ended up mitigating this by perform both fetches and deletes on the same queue.
Great question, I can only provide a partial answer and would really like to know this too. Unfortunately your own solution is more of a workaround but not really an answer. What if the background operation is very long and you can't resort to running it on the main thread?
One thing I can say is that you don't have to call [request setReturnsObjectsAsFaults:NO] since the fetch request will load the data into the row cache and will not go back to the database when a fault fires for one of the fetched objects (see Apples documentation for NSFetchRequest). This doesn't help with relationships though.
I've tried the following:
On NSManagedObjectContextWillSave notification, wait for the current background task to finish and prevent new tasks from starting with something like
-(void)contextWillSave:(NSNotification *)notification {
dispatch_sync(self.backgroundQueue, ^{
self.suspendBackgroundOperation = YES;
});
}
Unset suspendBackgroundOperation on NSManagedObjectContextDidSave notification
However the dispatch_sync call introduces possible dead locks so this doesn't really work either (see my related question). Plus it would still block the main thread until a potentially lengthy background operation finishes.
I have a Silverlight app where I've implemented the M-V-VM pattern so my actual UI elements (Views) are separated from the data (Models). Anyways, at one point after the user has gone and done some selections and possible other input, I'd like to asyncronously go though the model and scan it and compile a list of optiions that the user has changed (different from the default), and eventually update that on the UI as a summary, but that would be a final step.
My question is that if I use a background worker to do this, up until I actually want to do the UI updates, I just want to read current values in one of my models, I don't have to synchronize access to the model right? I'm not modifying data just reading current values...
There are Lists (ObservableCollections), so I will have to call methods of those collections like "_ABCCollection.GetSelectedItems()" but again I'm just reading, I'm not making changes. Since they are not primitives, will I have to synchronize access to them for just reads, or does that not matter?
I assume I'll have to sychronize my final step as it will cause PropertyChanged events to fire and eventually the Views will request the new data through the bindings...
Thanks in advance for any and all advice.
You are correct. You can read from your Model objects and ObservableCollections on a worker thread without having a cross-thread violation. Getting or setting the value of a property on a UI element (more specifically, an object that derives from DispatcherObject) must be done on the UI thread (more specifically, the thread on which the DispatcherObject subclass instance was created). For more info about this, see here.
Here's yet another question on Core Data and multithreading:
I'm writing an application on the iPhone that retrieves XML data from the internet, parses it in a background thread (using NSXMLparser) and saves the data in Core Data using its own NSManagedObjectContext. I have a class - let's call it DataRetriever - that does this for me.
There are different UIViewControllers that then retrieve the data to display it in their respective UITableViews, of course this happens on the main thread using NSFetchedResultsControllers and a single managed object context that is used for reading.
I've read the answer to this question, which tells me that I need to register for NSManagedObjectDidSaveNotifications on the background thread (this will be done by the DataRetriever class I suppose) and then call the mergeChangesFromContextDidSaveNotification method on the reading context from that class on the main thread. This, I think, is totally thread-unsafe. I might have interpreted this the wrong way, though.
I've also read this part of Apple's documentation on the subject (Track Changes in Other Threads Using Notifications), and it tells me to simply register for NSManagedObjectDidSaveNotifications coming from the reading context in the view controller on the main thread and then it would have to call mergeChangesFromContextDidSaveNotification to update its reading context.
I went with Apple's recommendations: I now have my view controllers register themselves to NSManagedObjectDidSaveNotifications on the main thread using the reading managed object context as the source of the notifications. Doing this on the writing context probably isn't thread safe, and Apple's documentation isn't very specific on this.
Result: No crashes, but I am not receiving any notifications either.
Side note: I've read in Apple's documentation that notifications don't automatically propagate to other threads and I might even be listening for notifications from the wrong context, but why is Apple telling me to do it this way, then?
Any help is greatly appreciated.
-- EDIT --
Just to be clear, I'm registering for notifications coming from a particular NSManagedObjectContext, Apple's documentation specifically states (here) that some system frameworks may use an instance of Core Data themselves, so I could be receiving notifications from contexts that don't concern me if I don't specify a source. The documentation I referred to earlier on doesn't say anything about this, though. Any comments on this design choice are welcome.
The UI runs on the main thread so you want any intensive processing that might bog the UI down done on another thread. You have the context in the main thread listen for notifications because the main thread context is usually the only one that needs to update itself because of changes by other context in other threads.
All this is thread safe because data won't be deleted from the persistent store as long as one or more context is still using it. So, if context A has an object with the data while context B deletes another object representing the same data, the object in context A remains alive until context A calls for a merge.
Basically, each context operates in its own little world until you call merge. The race conditions that normally bedevil thread based data operations don't occur with Core Data.