I have a background process receiving and applying changes to Core Data entities from a back end server using Restkit which works really well. All the entities have a version number property which updates when the back end accepts changes and publishes a new version. If an entity the user is viewing is changed I need to update the view with the latest version information.
Using KVO to observe version number for the current entity and refreshing the view when it changes works really well as long as version number is the last property.
That is, the 'column order' matters, and property updates are atomic. If the version number is the last property then when the observer is invoked all changes to all entity properties will have been applied.
If version number is not the last property defined, then when the observer is invoked the updated values of the properties after version will not been applied.
The solution is to change the database and ensure that version number is always last. This works however I cannot find anything in the documentation to suggest that the sequence of property changes is guaranteed.
I assume the only way to get a water-tight non-atomic notification is to register for managed object context change notifications and then process those notifications looking for changes to objects of interest. My concern with this is that it is not fined grained and there will be a lot of unnecessary processing to find relatively few things of interest.
Is this correct or is there a way to ensure an non-atomic view of an object when using KVO?
If you wanted to use KVO you would need to layer some change management on top, such as when the managed object is saved you check the version number and change another (non-persistent) attribute that is being observed. You can be sure that everything has been updated in a logical set when the object is saved.
Generally the context save notification is the approved approach. So long as you aren't making thousands of changes or making few large saves to the context it shouldn't be an issue. You can also look at using predicates to filter the changes and / or a fetched results controller (which does the observation for you).
Related
I built a homebrew data entity repository with a factory that defines retention policy by type (e.g. absolute or sliding expiration). The policy also specifies the cache type as httpcontext request, session, or application. A MemoryCache is maintained by a caching proxy in all 3 cache types. Anyhow, I have a data entity service tied to the repository which does the load and save for our primary data entity. The idea is you use the entity repository and don't need to care if the entity is cached or retrieved from it's data source (db in this case).
An obvious assumption would be that you would need to synchronise the load/save events as you would need to save the cached entity before loading the entity from it's data source.
So I was investigating a data integrity issue in production today... :)
Today I read there can be a good long gap between the entity being removed from the MemoryCache and the CacheItemRemovedCallback event firing (default 20 seconds). The simple lock I had around the load and save data ops was insufficient. Furthermore the CacheItemRemovedCallback was in it's own context outside of HttpContext making things interesting. It meant I needed to make the callback function static as I was potentially assigning a disposed instance to the event.
So once I realised there was was the possibility of a gap whereby my data entity no longer existed in cache but might not have been saved to it's data source might explain the 3 corrupt orders out of 5000. While filling out a long form it would be easy to perform work beyond the policy's 20 minute sliding expiration on the primary data entity. That means if they happen to submit at the same moment of expiration an interesting race condition between the load (via request context) and save (via cache expired callback) emerges.
With a simple lock it was the roll of the dice, would save or load win? Clearly we need a save before the next load from the data source (db). Ideally when an item expires from the cache it is atomically written to it's data source. with the entity gone from the cache but the expired callback not yet fired a load operation can slip in. In this case the entity will not be found in the cache so will default to load from the data source. However, as the save operation may not have commenced resulting in data integrity corruption and will likely clobber your now saved cached data.
To accomplish synchronisation I need a named signalling lock so I settled on EventWaitHandle. A named lock is created per user which is < 5000. This allows the Load to wait on a signal from the expired event which Saves the entity (whose thread exists in its own context outside HttpContext). So in the save it is easy to grab the existing name handle and signal the Load to continue once the Save is complete.
I also have a redundancy where it times out and logs each 10 seconds block by the save operation. As I said, the default is meant to be 20 seconds between an entity being removed form MemoryCache and it being conscious of it to fire the event which in turn saves the entity.
Thank you to anyone who followed my ramblings through all that. Given the nature of the sync requirements was the EventWaitHandle lock the best solution?
For completeness I wanted to post what I did to address the issue. I made multiple changes to the design to create a tidier solution which did not require a named sync object and allowed me to use a simple lock instead.
First the data entity repository is a singleton which was stored in the request cache. This front end of the repository is detached from the cache's themselves. I changed it to reside in the session cache instead which becomes important below.
Second I changed the event for the expired entity to route through the data entity repository above.
Third I changed the MemoryCache event from RemovedCallback to UpdateCallback**.
Last, we tie it all together with a regular lock in the data entity repository which is is the user's session and the gap-less expiry event routing through the same allowing the lock to cover load and save (expire) operations.
** These events are funny in that A) you can't subscribe to both and B) UpdateCallback is called before the item is removed from the cache but it is not called when you explicitly remove the item (aka myCache.Remove(entity) won't call event but UpdateCallback will). We made the decision if the item was being forcefully removed from the cache that we didn't care. This happens when the user changes company or clears their shopping list. So these scenarios won't fire the event so the entity may never be saved to the DB's cache tables. While it might have been nice for debugging purposes it wasn't worth dealing with the limbo state of an entity's existence to use the RemovedCallback which had 100% coverage.
I am trying to update a QTreeWidget every 60 seconds. I have it on a QTimer right now but my concern is that when it updates it will disrupt the users progress(for example if they have a parent opened up so you can see the children, when I update it completely resets the structure). Is there a model or anything I can do to prevent this from hurting their progress?
It is not possible for the view to update and remember the previously expanded items. You could cache the state of your view and reconstruct it with QTreeWidget::expand and QTreeWidget::scrollTo after an update, but the user will still see how the view closes and expands again. Also, it would not be enough to store the currently expanded item's index because that might change after the update. So you'd have to cache some unique identifier and search it in the upated widget afterwards.
What you are trying to do, is quite unusual, since normally you'd only update the widget when the data changes instead of using a fixed interval of time.
Consider using a QTreeView in combination with a QAbstractListModel instead of a QTreeWidget because the latter is designed to hold constant data. Then you can emit dataChanged on the model which automatically updates the QTreeView.
I'm using RestKit 0.20.2 in combination with MagicalRecord (important for any eventual context problems that you can think of).
My app is a saving tickets(entity) with items(entity) and each item has a tax(entity). My use case is this : I need to sync core data with my webserver when the ipad reconnects to the internet after a long period of time beeing unable to send the data (for whatever reason)
My problem is to be able to sync a lot of objects (it can go from 100 to a 1000 to even more), to be able to post a lot of objects without timeouts I set the restkit concurrency :
[RKObjectManager sharedManager].operationQueue.maxConcurrentOperationCount = 3;
Now this is working absolutely fine. But my problem is that i have alot of redundant entities that syncs with every item.
For instance each item has a tax, but i only have two taxes in my model that need to be synced with the web service and then sent as a relationship with the item (i only put the id of the tax). So to circumvent that problem for each postItem i check if the related Tax has an ID, if yes, then i can parse the item directly with the tax relationship in it, if not, i need to sync the tax first then the item with the returned taxID.
The work around is working as expected too. But again there is a problem, because between each postItem RestKit isn't saving the newly TaxID between two requests, so instead of sending it once, it sends it every time it encounters it inside an item and when all the operations are done it saves the newly created taxIDs
To improve that I digged a bit in restkit and found
- (void)enqueueBatchOfObjectRequestOperations:(NSArray *)operations
progress:(void (^)(NSUInteger numberOfFinishedOperations, NSUInteger totalNumberOfOperations))progress
completion:(void (^)(NSArray *operations))completion
So now i'm building RKManagedObjectRequestOperations for my tax entities and batch them.
Then i sync the items. It's more efficient, and i don't need to set a dependency between operations (because i need them to be executed in a certain order, tax then items then the whole ticket.)
PROBLEM Between the two enqueueBatchOperations, RestKit doesn't map the result of the first batch immediately, even if i explicitly call
[[RKManagedObjectStore defaultStore].mainQueueManagedObjectContext saveToPersistentStore:&error]
It isn't mapped because after the first batch of taxes, i send all the items, and i can see that the taxID's aren't set, but after all the batches are done, i can clearly see them mapped correctly in my core data file. So I thought it was a context issue, but when i digg into RestKit and more specifically in
appropriateObjectRequestOperationWithObject:(id)object
method:(RKRequestMethod)method
path:(NSString *)path
parameters:(NSDictionary *)parameters
I can see line 580 :
NSManagedObjectContext *managedObjectContext = [object respondsToSelector:#selector(managedObjectContext)] ? [object managedObjectContext] : self.managedObjectStore.mainQueueManagedObjectContext;
That sets the mainQueueContext (and not the object context) for the operations (i've checked with breakpoints), so calling save or saveToPersistentStore should propagate the changes from child contexts to the mainQueue and... it's where i've lost hope and turned to stackoverflow ;)
As it usually happens i found the solution after posting on SO :)
The problem was that the RKManagedObjectRequestOperations where all created before restkit actually sent the information. So the context was the same for all the requests (as mentioned in the appropriateObjectRequestOperationWithObject method, and the changes weren't propagated because the context reference was the "old" reference.
To have the information on the next requests i just build the RKManagedObjectRequestOperations in the enqueueBatchOfOperations completion block, and now all is working fine with the newly created taxID ;)
I have entities that are managed by Core Data and have several cases where, within a single method, I set some attribute values that will result graph changes that Core Data will enforce and perform additional actions that (logically) depend on uptodate state for the graph.
Is there any reason not to call processPendingChanges after each time a relationship is set, to ensure that the graph is always fully uptodate? Everything works as it should when I do this, but, clearly, it's a bit "noisy", and breaks up some processing that would otherwise be notifications (e.g, fetched results controllers that end up sending lots of controllerWillChangeContent/controllerDidChangeContent to their deligates when one would otherwise have happened).
ADDITION:
Will the graph always be up-to-date after a return from any method that makes changes to an entity?
I found it the hard way that you should call processPendingChanges before inspecting deletedObjects of NSManagedObjectContext. At least if some relationships have deleteRule set to NSCascadeDeleteRule.
If you don't call processPendingChanges then deletedObjects may not contain objects that will be deleted by cascade at the end of current event.
processPendingChanges is most often used on iOS with multiple context operating on seperate threads. It plays a bigger and more common role under MacOS.
You usually don't have to call it under iOS in most circumstances. Doing so doesn't really give you much of an advantage and it can cause lags in the UI when executed on the main thread if you have a complex graph.
I wouldn't bother with it unless testing reveals you are loosing graph integrity for some reason.
I have a Silverlight app where I've implemented the M-V-VM pattern so my actual UI elements (Views) are separated from the data (Models). Anyways, at one point after the user has gone and done some selections and possible other input, I'd like to asyncronously go though the model and scan it and compile a list of optiions that the user has changed (different from the default), and eventually update that on the UI as a summary, but that would be a final step.
My question is that if I use a background worker to do this, up until I actually want to do the UI updates, I just want to read current values in one of my models, I don't have to synchronize access to the model right? I'm not modifying data just reading current values...
There are Lists (ObservableCollections), so I will have to call methods of those collections like "_ABCCollection.GetSelectedItems()" but again I'm just reading, I'm not making changes. Since they are not primitives, will I have to synchronize access to them for just reads, or does that not matter?
I assume I'll have to sychronize my final step as it will cause PropertyChanged events to fire and eventually the Views will request the new data through the bindings...
Thanks in advance for any and all advice.
You are correct. You can read from your Model objects and ObservableCollections on a worker thread without having a cross-thread violation. Getting or setting the value of a property on a UI element (more specifically, an object that derives from DispatcherObject) must be done on the UI thread (more specifically, the thread on which the DispatcherObject subclass instance was created). For more info about this, see here.