I have several different apps that I'd like to share a personal core data store with on the same device and across devices. I've been experimenting with NSPersistentCloudKitContainer, which seems to work, but it can take a while to sync and it's not clear when it has. I have also experimented with using app groups where the SQLite file is placed in a group folder so that changes are immediately available to other apps on the device.
My question is whether it is better to do one, or the other or both. Will two apps open on the same device accessing the shared SQLite cause conflict issues? Do I even need to sync data with cloudKit if I'm already syncing with a shared group? Using NSPersistentCloudKitContainer alone means that each app will be syncing its own copy of the data locally, which doesn't seem very efficient and again I don't know if it's synched when I open the other app up. But I have seen conflicts when I have both apps open at the same time. Is there a foolproof way to prevent conflicts when one app goes into the background?
App groups and CloudKit are orthogonal to each other. They solve different problems and you can use one or the other or both depending on what you want to do. App groups are good for sharing data on one device, and CloudKit works across multiple devices. You can use both if you want both of those things to happen. It's not one or the other, it's either one or both depending on the app.
The only foolproof way to avoid conflicts would be to never edit the same data in more than one place. Since that probably won't happen with either of these, it's better to look at how to resolve conflicts when they arise. Core Data has several built in merge policies to resolve conflicting changes, and you can write your own if you want something different.
With CloudKit your app will be notified when new changes are available. Merge changes then, and let the merge policy deal with conflicts. With app groups there's no notification of external changes, so the best approach is to check for new data whenever the app comes to the foreground (in case it was edited somewhere else).
If you're seeing conflicts you don't know how to resolve, post a separate question with details of the problem and someone can probably help with it.
Related
I am trying to understand the use of deployment slots for hosting my web app using the Azure app service.
I am particularly confused with the ideal ways to deal with the database while the swap is performed.
While maintaining two database versions seems like a solution, it adds the complexity of maintaining data across multiple databases to make them consistent.
What are the recommended ways for dealing with database schema and migrations while using blue/green deployments and in particular deployment slots?
Ideally you'll stage / production would share the same database, so it would not be an issue.
But if you have more slots, then you'd better also work with different databases and handle migrations during the release phase
We've worked through various solutions to this problem for a few years. There's not a toolset that provides a magic bullet for all cases. There are a few solutions:
Smaller databases/trivial changes
If it is possible to execute a migration script on a database that will complete in a second or two, and you can have an easy fallback script, you can execute the script concurrently with the swap. This can also be automated. But it's a higher stress situation and not one I'd recommend. This can even be done with EF Migrations.
Carefully ensure database compatibility between versions
Since we're dealing with a few hundred GB of data that cannot go down, we've just made it a rule that the database has to work with both versions of our application. It's not as awful or impossible as it sounds. For example, net new tables and fields can oftentimes be added before you even perform the swap. We test rollback between versions as part of our QA. If some fields need to be dropped, we wait until after the new version has been deployed and burned in, then run another script to perform the drops after we're sure we won't need rollback. We'll create new stored procedures when one needs to be upgraded so that the new version has its own. Example: sp_foo and sp_foo2.
We've had a lot of success with this strategy.
Slots are a feature specifically for App Services and not for DBs, if you want to use a specific DB with a specific slot then you setup the slot like this:
https://learn.microsoft.com/en-us/azure/app-service/deploy-staging-slots
Now when using Slots and swapping it also swaps App Configurations\Settings, and in App Settings you can have two DB connections strings but each with its own slot name and setting enabled. You can see it has been shown in this example here as well: https://learn.microsoft.com/en-us/azure/app-service/deploy-staging-slots#swap-two-slots
I'm new to pouch/couch and looking for some guidance on handling conflicts. Specifically, I have an extension running pouchdb (distributed to two users). Then the idea is to have a pouchdb-server or couchdb (does it matter for this small a use case?) instance running remotely. The crux of my concern is handling conflicts, the data will be changing frequently and though the extensions won't be doing live sync, they will be syncing very often. I have conflict handling written into the data submission functions, however there could still be conflicts when syncing occurs with multiple users.
I was looking at the pouch-resolve-conflicts plugin and see immediately the author state:
"Conflict resolution should better be done server side to avoid hard to debug loops when multiple clients resolves conflicts on the same documents".
This makes sense to me, but I am unsure how to implement such conflict
resolution. The only way I can think would be to place REST API layer
in front of the remote database that handles all updates/conflicts etc with custom logic.
But then how could I use the pouch sync functionality? At that point I
may as well just use a different database.
I've just been unable to find any resources discussing how to implement conflict resolution server-side, in fact the opposite.
With your use case, you could probably write to a local pouchdb instance and sync it with the master database. Then, you could have a daemon that automatically resolve conflicts on your master database.
Below is my approach to solve a similar problem.
I have made a NodeJS daemon that automatically resolve conflicts. It integrates deconflict, a NodeJS library that allows you to resolve a document in three ways:
Merge all revisions together
Keep the latest revisions (based on a custom key. Eg: updated_at)
Pick a certain revision (Here you can use your own logic)
Revision deconflict
The way I use CouchDB, every write is partial. We always take some changes and apply them to the latest document. With this approach, we can easily take the merge all revision strategy.
Conflict scanner
When the daemon boot, two processes are executed. One that go through all the changes. If a conflict is detected, it's added to a conflict queue.
Another process is executed and remain active: Continuous changes scanner.
It listen to all new changes and add conflicted documents to the conflict queue
Queue processing
Another process is started and keeps polling the queue for new conflicted documents. It gets conflicted documents in batch and resolve them on by one. If there's not documents, it just wait a certain period and starts the polling again.
Having worked a little bit with Redux I realized that the same concept of unidirectional flow would help me avoid the problem of conflicts altogether.
Redux flows like this...
So, my clientside code never write definitive data to the master database, instead they write insert/update/delete requests locally which PouchDB then pushes to the CouchDB master database. On the same server as the master CouchDB I have PouchDB in NodeJS replicating these requests. "Superviser" software in NodeJS examines each new request, changes their status to "processing" writes the requested updates, inserts and deletes, then marks the request "processed". To ensure they're processed one at time the code that receives each request, stuffs them into a FIFO. The processing code pulls them from the other end.
I'm not dealing with super high volume, so the latency is not a concern.
I'm also not facing a situation where numerous people might be trying to update exactly the same record at the same time. If that's your situation, your client-side update requests will need to specify the rev number and your "supervisors" will need to reject change requests that refer to a superseded version. You'll have to figure out how your client code would get and respond to those rejections.
I have an app with multiple updates on the AppStore already, funny thing happened, I thought that the lightweight migration happens automatically, however, my recent discovery that I need to add the
NSDictionary *storeOptions = #{NSMigratePersistentStoresAutomaticallyOption:#YES, NSInferMappingModelAutomaticallyOption:#YES};
to my persistentStoreCoordinator shook my confidence when I realized I already have 5 core data models.
The question is: when I add the above line to the next version of the app, is it going to work for everyone when they update? Because right now everything that happens when they open the app .. is a fancy CRASH.
Thx
It will work if automatic lightweight migration is possible for the migration you're trying to perform. Whether this will work depends on the differences between the model used for the existing data and the current version of the model. Many common changes permit automatic lightweight migration but not all. You'll need to review the docs on this kind of migration and decide whether it will work in your case.
If it doesn't work, there are other ways to handle it, for example by creating a mapping file to tell Core Data how to make changes that it can't infer automatically.
I'm curious how auto scaling works with an Azure web application, specifically how each new instance gets a copy of the source code for the application.
Is this what happens, or is the source stored somewhere and each new instance points to it?
The entire virtual machine is duplicated. So usually you might have just one database but multiple apps receiving and processing the requests. If you need an "autoscaling" database too, then there are database solutions that handle syncronization across multiple machines, but in that case you're probably better off using Azure's native database which takes care of that.
I believe that the mvc mini profiler is a bit of a 'God-send'
I have incorporated it in a new MVC project which is targeting the Azure platform.
My question is - how to handle profiling across server (role instance) barriers?
Is this is even possible?
I don't understand why you would need to profile these apps any differently. You want to profile how your app behaves on the production server - go ahead and do it.
A single request will still be executed on a single instance, and you'll get the data from that same instance. If you want to profile services located on a different physical tier as well, that would require different approaches; involving communication through internal endpoints which I'm sure the mini profiler doesn't support out of the box. However, the modification shouldn't be that complicated.
However, would you want to profile physically separated tiers, I would go about it in a different way. Specifically, profile each tier independantly. Because that's how I would go about optimizing it. If you wrap the call to your other tier in a profiler statement, you can see where the problem lies and still be able to solve it.
By default the mvc-mini-profiler stores and delivers its results using HttpRuntime.Cache. This is going to cause some problems in a multi-instance environment.
If you are using multiple instances, then some ways you might be able to make this work are:
to change the Http Cache to an AppFabric Cache implementation (or some MemCached implementation)
to use an alternative Storage strategy for your profile results (the code includes SqlServerStorage as an example?)
Obviously, whichever strategy you choose will require more time/resources than just the single instance implementation.