How to sync records between Core Data and CloudKit efficiently - core-data

I'm currently learning how to use CloudKit Framework and lack of documentation or examples showing how to sync Core Data and CloudKit.
I have watched all WWDC videos (2014, 2015, 2016) Dedicated to CloudKit, but none of them telling us how to implement syncing with Core Data. I can't find any fresh examples, tutorial or books, showing how to implement this syncing.
I know that it is effective to use Operations API by CloudKit (not Convenience API) and to Subscribe to changes as it said in the new WWDC 2016 videos, dedicated to CloudKit, but mapping with CoreData is a real problem.
For example, let's say I would like to create an app similar to Notes app. while offline, user can create his notes and work with them saving them to his core data database. When the device going online the app checks what changed on the server and saves newly created records to server (CloudKit).
When the app starts, it also fetches for changes from the CloudKit and if there are changes , it updates local cache (Core Data) with the new changes.
I would appreciate to have a common pattern of syncing. Where to place syncing with Core Data methods and how they should look like?
Would appreciate any information or help about this.
I'm using Swift 3, Xcode 8 , iOS 10.

As of iOS 13, there are new API's that simplify this synchronization for developers. I would recommend you to watch the WWDC19 session about the new synchronization between CoreData and CloudKit. Please note that these new API's only work for iOS 13+.
Video: https://developer.apple.com/videos/play/wwdc2019/202/
In short, you need to start using NSPersistentCloudKitContainer instead of NSPersistentContainer. This will let the syncing work automatically using automatic conflict resolution with a last-writer-wins merge strategy. If you want to build a good working app, you'll also need to do some modifications to improve the syncing for your app.
Official documentation can be found at:
Setting Up Core Data with CloudKit
Syncing a Core Data Store with CloudKit
Data modeling for collaboration (conflict-free replicated data type)
At the end of the session they also demonstrated an example of better sync merging than the default 'last-writer-wins merge strategy'. The usage of Causal Trees allow multiple users to edit the same string (and to some extend other types of data) without losing any data. I would really recommend everyone to read this article from Archagon that describes how this works and how to implement it (also with CloudKit syncing, but not the new automatic one). As demonstrated in the session, you can also implement this with the new automatic syncing between CoreData and CloudKit.

Core Data already provides the user with the ability to sync to iCloud. There's no need to use CloudKit.
Design For Core Data In iCloud
But yes, Core Data with iCloud has been deprecated. Even so, it has not been discontinued, and there are no immediate plans at apple to discontinue it, they just want to discourage its use. But it also has problems with rationalising updates from multiple devices.
In any case, I have been looking into the question of how to do this with cloud kit myself. Two answers; the first is to use the following;
Seam in GitHub
The second is to do it manually;
Designing for CloudKit
The key here is that Cloud Kit needs the record metadata to be able to handle record updates reliably, so you have to save that metadata in your Core Data database. The CKRecord class includes a method encodeSystemFields(with:) which will encode those fields into a Data record that can be stored in your database, and then your can use the appropriate decoder when you need to restore the CKRecord.
Anyway, I am about to start doing this myself. I'll update this with more information when I have it.

Apple has recently published a guide that seems to answer this question. Check out Apple's Maintaining a Local Cache of CloudKit Records to see how to store CloudKit data on device.
While this guide doesn't provide sample code to write to the device, it does answer the rest of the question. This tells you how to fetch changes from CloudKit and create data which can be stored on the device.

Related

Sync multiple devices with Core Data

I have a coredata app that I would love to be able to share the same data with multiple devices, possibly with iCloud/cloud kit. I am not sure where to start, or how to go about it? The only thing I can think of, but still not sure how to do, would be to sync the SQLite files with iCloud? Not sure if thats a good idea or not? I just recently converted my app over to swift 3 and iOS10 core data code. The only way I am able to share data between devices currently is thru iTunes files sharing.
For whatever reason this topic is hard to find modern info on.
Core Data doesn't have support for this. Except for the built-in iCloud sync, but that's deprecated as of iOS 10.
You could use CloudKit to sync data, but you'll have to write your own code to convert between Core Data's persistent store and CloudKit's online store. It's not impossible but it's certainly not automatic.
Syncing the SQLite file is not a good idea unless you really want to corrupt the data.

AFIncrementalStore with Parse

I am developing an social app on iOS that have many-to-many relation, local persistency, and user interaction. I have tried using native Parse API in iOS and find it too cumbersome to do all the client-server logic. So my focus shifted to finding a syncing solution.
After some research I found AFIncrementalStore quite easy to use and it's highly integrated in CoreData. I just started to work on this and I have two questions to ask:
1) How to do the authentication process? Is it in AFRESTClient?
2) How to set up AFRESTClient to match Parse's REST API? (an example would be great!)
P.S. I also found FTASync, which seems to be another solution. Any thought on this framework?
Any general suggestion on client-server syncing solutions will be highly appreciated!
Thanks,
Lei Zhang
Back with iOS 5 Apple silently rolled out NSIncrementalStore to manage connection between APIs and persistent stores. Because I couldn't word it better myself:
NSIncrementalStore is an abstract subclass of NSPersistentStore designed to "create persistent stores which load and save data incrementally, allowing for the management of large and/or shared datasets". And while that may not sound like much, consider that nearly all of the database adapters we rely on load incrementally from large, shared data stores. What we have here is a goddamned miracle.
Source: http://nshipster.com/nsincrementalstore/
That being said, I've been working on my own NSIncrementalStore (built specifically for Parse and utilizing the Parse iOS/OS X SDK) and you're welcome to check out/use/contribute to the project at https://github.com/sbonami/PFIncrementalStore.
Take a look at this StackOverflow question and at Chris Wagner's article on raywenderlich.com.
The linked SO question has examples for how to include the authentication token with each request to Parse. So you'll just need to have the user log in first, and store their token to include it with each subsequent request.
Chris Wagner's tutorial has a sample AFHTTPClient named SDAFParseApiClient to communicate with the Parse REST API. You'd have to adapt it to be an AFRESTClient subclass, but it should give you a start.
Some other thoughts between the two solutions you're considering:
AFIncrementalStore does not allow the user to make any changes without a network connection, while FTASync keeps a full Core Data SQLite store locally and syncs changes to the server when you tell it to.
FTASync requires you to make all your synched managed objects subclasses of FTASyncParent, with extra properties for sync metadata. AFIncrementalStore keeps its metadata behind the scenes, not in your model.
FTASync appears not to be widely used and hasn't been updated in over a year; if you use it you will likely be maintaining it.

Checking iCloud for existing content

What is the best way to check iCloud for existing data?
I need to check that data doesn't exist on the local device, or iCloud so I can then download it.
Since you included the core-data tag I'm assuming you mean that you're using Core Data rather than iCloud file APIs or the ubiquitous key-value store.
With Core Data's built-in iCloud support, you check on existing data in exactly the same way as if you were not using iCloud. Once you create your Core Data stack, you check what data exists by doing normal Core Data fetches. There's no (exposed) concept of local vs. cloud data, there's just one data store that happens to know how to communicate with iCloud. You don't explicitly initiate downloads-- they happen automatically.
At app launch time when you call addPersistentStoreWithType:configuration:URL:options:error:, Core Data internally initiates the download of any data that's not available locally yet. As a result this method may block for a while. If it returns successfully, then all current downloads can be assumed to be complete.
If new changes appear while your app is running, Core Data will download and import them and, when it's finished, will post NSPersistentStoreDidImportUbiquitousContentChangesNotification to tell you what just happened.
This all describes how Core Data's iCloud is supposed to work. In practice you'll probably find that it doesn't always work as intended.
Thanks to #Tom Harrington for pointing out, this error is nothing to do with the developer/coding - it's purely to do with iCloud/Apple/connection issues.
More on this SO answer I found.

Update crashes after coredata automatic lightweight migration

I recently submitted an upgrade of my app which included a lightweight coredata migration (including new fields in existing tables and a couple of new tables). I followed every tip regarding this migration, including some I found on this site.
I thoroughly tested the update on three different devices and it all went ok!!!
However, this update is crashing an all my devices and probably on all my customers. I can't explain why this is happening.
Could you please help me understand this debacle?
To truly test your app and migration, you need to run your original app to create data store according to the original data model. Then you need to run your new app, opening data store that was generated with original app. This can be a real pain and is easier (at least initially) to do in Simulator because you have more control over the file system and can swap in a saved original data store. On iDevice you need to regenerate original data store for each test.
If you are testing on your own development devices then you have already migrated your data store. Is it possible that your test devices created their data stores with new data model - and never actually performed a migration?
I only generally use automatic migration during beta testing, for quick revisions, other than that I always use a mapping model, so that you have control.
the other issue is that if your model shifts far enough between releases, auto migration from v1-v2 could be fine, and v2-v3 could be ok, but v1-v3 could be too drastic to be inferred. by making maps for them, you retain control of the migration.

Help understanding saving data please. Core data vs plist

Is every app that allows users to input data built with core data?
I've built a "grocery list" type of table view app where you name the list and then in a detail view add items to the list. Simple.
What I don't get is this, based on an iphone development book the example saves the data to a plist using dictionaries.
I've learned that it works on the simulator but not the device because the data is saved to the application bundle not the document directory (which was new to me!)
On the device the app works great except-it won't HOLD the data.
Is core data or sqlite the only solution?
Is every app that allows users to input data built with core data?
Note that your question as posed is incorrect, as it assumes that CoreData is tied to SQLite and is an alternative to plists.
CoreData is a framework for object lifecycle and graph management. It provides implementation of common tasks like changes tracking and propagation, consistency enforcement, data validation and so on.
The CoreData framework is a separate from the object persistence layer and can use different serialization implementations, including SQLite and XML (plists).
For more details, read Core Data Programming - Persistent Store Features.
The decision whether you should use CoreData should be based on whether you need any of the features it provides. If you need to serialize simple object graphs, without consistency requirements, you can use standard NSDictionary to serialize your data in a simple plist file in any of the application-writable folders. Otherwise, use CoreData, and choose the proper persistent store based on the type of data you will be storing.
From what I've seen around the internet, you can use Core Data (which gives you the options of SQLite, atomic, and XML), you can use NSKeyedArchivers and NSKeyedUnarchivers (http://www.vimeo.com/1454094) or you can store the data inside the local application folder (possibly using a serialization method). It looks like Core data is the best solution, but a more complex one to implement. For a simple app, as yours is, I think serializing data and storing it in the local app directory would be perfect.
I am surprised that your book is showing an example where user data is written to the app bundle. Actually, I'm a little surprised that that is even possible.
You should be able to write your data to an NSDictionary (or NSMutableDictionary) and then write that to your app's Documents directory, using -writeToFile:atomically:
Reading data back in should also be straightforward, using -initWithContentsOfFile:.
For someone just getting started, I would recommend keeping it simple. Working NSDictionary is very simple, though you have to manage things like the list of lists and how to name lists that are stored in Documents directory, etc.
Ultimately, using Core Data would probably be a better approach. It offers more flexibility and more power - but, as ever, those advantages come at a cost.
Your question is very important to the community in the respect that
you are asking a strategic question: which technology do I use, when?
Core Data is best for the day-to-day work of a list-based app. Core data is built to mirror the storage of data, similar to how databases work. Relational structures, sorting, key indexing and other row-based attributes are best supported by Core Data.
Property Lists (*.plist) is best suited to one-time updates to critical environmental settings. The user, for example, can optionally set .plist attributes through IOS Settings app. So passwords, account settings, email addresses, and configuration options can be set here nicely. This kind of data is very different from frequently-updated, transactional data.
XML Persistence is closely related to .plist, in that the property list (or .plist) is an xml file in itself. Hence, you could download a stream of xml data, then use it in your app using the same programming rubric as you would, adjusting a property list. Hence, receiving xml data from the web, or uploading such a list, maps nicely to xml persistence.
AWS also proposed the AWS-Persistence library, to support synchronizing your core data collections with their online databases. This could provide helpful by 1) having a user populate data locally via Core Data, then lazily/opportunistically uploading the list. For your purposes (grocery shopping list), this could provide immediacy to the user, while giving your server an interesting big-data opportunity (analyze user transactions, provide recommendations, sell ads, etc).
Hope this gets future visitors tapping into the wealth of what IOS provides -- peace!

Resources