Friday May 31, 2013

Operational Transforms And Core Data, In The Wild

I love this. Milen Dzhumerov of Clear walks through their clever use of operational transforms to sync changes to a Core Data database over iCloud:

It should be noted that the system described above is probabilistic, stemming from the distributed nature of iCloud – a peer only ever has a local snapshot of the global state. For example, a device cannot know whether there is some other device that has not synced yet or whether a device has gone offline forever. Consequently, all algorithms employed must be designed to take these scenarios into account. In Clear, the system handles any such cases by simply never losing data – i.e., taking a union defined on the object graph. In very rare cases, it might end up with the undesirable behaviour of a deleted object re-appearing but that is preferable to losing data.

Back at CocoaConf San Jose, Kevin Kim and I were talking about doing this exact thing—recording and playing back transforms to the Core Data object graph and sending those individual transactions over the wire (iCloud in this case). It’s the same way that Google Wave worked. I miss Wave. :(

I’m curious how this technique holds up across different model versions. One of the biggest drawbacks to treating Core Data as an object storage and syncing framework is what to do when two apps of different versions try to access the same store. If models could be migrated through these operational transforms, but done in a backward compatible way, then older clients wouldn’t have to break. But now we’re glossing over the specific data architecture decisions that affect this resiliency. I like Clear’s approach overall, but I suspect that mixing object persistence and data syncing is not a very good general purpose solution. You’d have to craft your app around the limitations, like Clear did.

(via Drew McCormack)