One of the popular soundbites from Steve Jobs’ keynote at this week’s WWDC was “the truth is in the cloud.” Some have suspected Jobs of adopting metaphysical hyperbole in pitching Apple’s new iCloud service, but the ”truth” of the matter (sorry) is a bit more nuanced. Jobs was using “truth” both to make an insider’s reference to an old Apple technology, and also to look forward to brand new technology in iCloud.
Truth as used here has an inside technical meaning — the master or “truth” database in a distributed data system is the authoritative record for resolving conflicts. The master store in Apple’s existing Sync Services is the “truth database.” Here’s an image from Apple’s public documentation for Sync Services that illustrates the term.
In the existing Mac-centric sync model, the truth database is stored on the computer. Applications that want to interact with Sync Services don’t talk directly to the underlying database itself, but rather communicate through the Sync Engine, part of Mac OS X, which maintains the integrity, or truth, of the authoritative master data.
The other side to the “truth in the cloud” comment is that iCloud is more than just online document storage. Jobs made reference to this on Monday, saying:
Some people think the cloud is just a hard disk in the sky. We think it’s way more than that. iCloud is integrated with your apps, so everything happens automatically. It just works.
After a demonstration of iWork apps saving documents to the cloud, Jobs quickly added that iCloud Storage APIs would also include key value data. If Apple follows some of the existing conventions of Sync Services, developers would not interact directly with the master store, but would rely on system-level services, the APIs, to maintain the integrity of the cloud data. These system services will remove a tremendous burden from developers, making it much simple to integrate iOS apps with iCloud.
Syncing key-value pairs is also more flexible than syncing documents, and perfectly suited for several common uses. For example, to keep track of which levels are unlocked in a game like Angry Birds, the developer could update the key-value pair for each level independently from the others. In this scenario, the key might be the level id (“world-3-2″), and the value would be “locked” or “unlocked” as appropriate. It is simpler to fast sync just the one key-value pair that has changed instead of updating the entire settings file each time. This key-value pair approach also makes it much easier for developers to avoid situations where updating settings on one device would overwrite or erase all the settings on another device.
Some might be skeptical since Sync Services has suffered from a number of issues like mistaken duplication of contacts and calendar events. Jobs poked fun at Apple’s missteps with their previous cloud efforts when he said joked that users might ask “why should I believe them? They’re the ones that brought me MobileMe,” but he continued finished by admitting that while “it wasn’t [Apple's] finest hour,” the company “learned a lot.” Apple will have to put all of that learning to use to build a rock-solid online key-value store that can deal with billions of transactions coming from hundreds of millions of devices. That learning will be especially important when safe-guarding all that customer data in the key-value store.
Key-value stores have gained a lot of recent attention in web development circles because they offer a flexible, and fast, way to access small bits of data like user preferences. Solutions like MongoDB, Redis, Tokyo Cabinet, Memcache, Hadoop, MapReduce, Cassandra, Google’s BigTable , and Amazon’s Dynamo are all examples of web-friendly datastores that work with key-value pairs. Derrick Harris, over on our Structure channel, broke down how some of these technologies are being used to power the gear in Apple’s new data center, and highlighted a job listing which called for experience with key-value store technologies like Hadoop, Memcache and Cassandra.
iCloud is not just about online document storage, but a whole infrastructure for syncing data and preferences with an online key-value store — a new Sync Services built around the idea of keeping the truth database in the cloud instead of on a computer. The cloud is the new center of the Apple universe, in other words.
A ‘key’ difference (again, so sorry) between iCloud and other key-value stores is that Apple is focused on providing infrastructure for native iOS and Mac apps while others are working on platforms for web apps that run in the browser. Apple wants to make things “just work” for iOS apps without placing unnecessary burdens on developers as they incorporate iCloud services. This tension between open and fragmented and closed and integrated, which we’ve already seen in the Android and iPhone platforms, is being played out again here. But the fight is a good one for consumers, since it should lead to ever-better apps that test the limits of what we thought we could do with mobile devices.