Analyst Report: Hybrid application design: balancing cloud-based and edge-based mobile data

1 Summary

We’re now seeing an explosion in the number and types of devices, the number of mobile users, and the number of mobile applications, but the most impactful long-term changes in the mobile space will occur in mobile data as users increasingly interact with larger volumes and varieties of data on their devices. More powerful devices, better data-sync capabilities, and peer-to-peer device communications are dramatically impacting what users expect from their apps and which technologies developers will need to utilize to meet those expectations.

As this report will demonstrate, the rules are changing quickly, but the good news is that, because of more cross-platform tools like Xamarin and database-sync capabilities, the game is getting easier to play. CIOs, CTOs, application developers, architects, and cloud service providers need to understand that:

  • Tablets and smartphones are becoming more powerful every year, yet most mobile apps remain server-centric and network-intensive.
  • As more power gets pushed to the edge, both users and developers will want to leverage that power. As more device types come into being, cross-platform, data-centric app design will become crucial.
  • Peer-to-peer communication and client-side storage will free apps of network latency and connectivity concerns, extending interactions beyond today’s thin client-network dependence. They also will pose significant challenges to security and data integrity.
  • The drastic and foundational changes coming in all layers of the mobile space—network, hardware, software, and, most importantly, data storage—require us to revisit many chapters of our app-design playbook.

Thumbnail image courtesy of flickr user Pays de Montfort en Brocéliande

2 More users, devices, and apps

In just a brief 30 years, computing has gone through three major architecture shifts and is now entering a fourth, the somewhat ambiguously named “internet of things.”

  • Through the 1980s, computing was the exclusive domain of large enterprises, and there were only a few thousand mainframes hosting perhaps a few thousand apps run by a few million users.
  • The mid ‘90s ushered in the client/server age, which subsequently gave birth to the internet and hundreds of millions of users running tens of thousands of apps. The number of unique device types both serving and consuming all of these apps and services could probably be counted in the low hundreds, if not the dozens.
  • Mobile devices are largely credited with driving the next iteration of “web scale,” where we also began to leverage highly scalable cloud-based backends. Just two years ago, we broke the million-app milestone, and 2013 is now on track to double that number. Currently billions of users consume millions of applications on thousands of distinct device types.

None of us—software engineers included—can escape our history. Architectures that we redesigned for web scale tried to leverage low-cost, infinitely scalable public clouds. This worked well as long our users were always connected to the network so they could access that scale. Unfortunately, we know that, for both the present and maybe even the long-term, ubiquitous network connectivity is a lie.

Despite our best plans, natural disasters have demonstrated for us many times in recent years that the bandwidth we have can be taken away abruptly and without notice. Even without nature’s meddling, realities like battery life, infrastructure, and cost always will limit what needs to be constantly connected to “all the other things.”

Although we’re unlikely to see an explosion of always-on connectivity, the internet of things will give us an explosion of distinct devices—sensors, security cameras, and wearables—as well as the further proliferation of phones and tablets. When all is said and done, the distinct device count could number in the hundreds of thousands or even millions.

3 Next-gen devices

The votes are in, and the future is decided. PCs are exiting stage right, and their replacements—wirelessly connected mobile devices—are getting starring roles in our work and personal lives. By the end of 2013, tablet shipments are scheduled to overtake desktop PC shipments; smartphone shipments are already outpacing PCs by roughly 10 to one. The trend shows no signs of slowing down either. “In context, there are currently 800 million smartphones versus 5.5 billion non-smartphones,” relayed Cimarron Buser, VP of development at Apperian. “Projecting out into the future, it’s pretty clear that the smart devices will take over.”

The fast-moving mobile device market also looks nothing like its PC predecessor. New mobile devices are released every year, but PCs (and yes, we’ll pick on Macs too) haven’t changed much in decades despite touting “innovations,” which are generally only incremental advances in slightly different screen sizes and resolutions, peripheral quality, or performance.

Although it’s tempting to break mobile down into its two major players—Android and Apple—the number of unique devices, especially on the more open Android platform, is staggering. “Samsung alone has 27 different screen sizes, and the top 100 Android devices only cover about one-third of the users,” said Nat Friedman, CEO of Xamarin. “The top 250 only cover about one-half.”

And it’s not just screen sizes and hardware specs that are changing rapidly; it’s anything and everything we can do in the digital or analog world. Although the new iPhones got the most attention from Apple’s September 2013 announcement, history may well look back and judge differently. Long after the plastic phone is forgotten, it’s likely that we’ll remember that Touch ID, the M7, and the iBeacon were the real announcements that day. These three technologies marked the mobile device’s real entrance into the world of physical interaction, and it’s hard to overestimate the impact that will have. In the very near future, mobile devices will diagnose illnesses and possibly prescribe or even administer medication. The ecosystem necessary to support identity, security, and data integrity will be enormous and transformative.

It’s also pretty clear that phones and tablets won’t be the only game in town. For example, e-readers like the Kindle and Nook are still extremely popular, and each has its own browser. Wearables like Google Glass, the Pebble smartwatch, and personal sensors are already with us and evoking some strong reactions. Although no one knows which of these devices will ignite, nearly everyone agrees that one or more will catch on. As Friedman reminded us, “We haven’t yet seen successful form factors outside of phones and tablets, but it’s going to happen; it’s clear.”

We’re also finding new ways to connect these devices to one another. Hardware advances like Wi-Fi Direct, which enables peer-to-peer networks out to 200 feet versus Bluetooth’s 30, and near-field communication (NFC), which allows users to “bump” their phones to exchange info, open up new possibilities for ad-hoc interconnectivity and data-sharing. We should expect to see more devices supporting these standards, and when the hardware is widespread, we’ll also see more device-to-device apps that leverage that hardware.

Hardware creation too is becoming increasingly democratized. With easy access to Arduinos, 3D printers, Kickstarter, and communities like local hacker spaces, armies of hardware hackers are already creating some viable, interesting products for mobile. While some of these may only provide insight into your snowboarding and surfing, others could potentially revolutionize the market.

With all these variables in the mix, it’s impossible to imagine which devices will emerge as the winners, but we can be reasonably certain of a few things:

  • These devices are not far off. Within one to two years, you’ll likely see most of these devices mentioned on a daily basis.
  • Fragmentation and freedom will increase dramatically in coming years.
  • Every one of these devices will need software, and that software may be very different from what we’re used to.

4 Next-gen apps

Although the “millions of devices” age is quite a way off, we’ve had access to millions of apps since 2011, and we add more than 1,000 per day. Quality, as we know, rarely equals quantity, and apps are no exception. Only 40 percent of the available apps break even on their development costs, and the top 1 percent make 30 percent of the overall revenue.

A good deal of this fumbling could be attributed to the young age of the space. Companies and the humans who work for them rarely get anything right on the first attempt, but we do learn, and we do improve. One of the biggest lessons we’ve learned so far is that mobile needs to be treated as the unique platform it is. “A lot of companies say, ‘Let’s take what we’ve got on the PC and just put it on a mobile device,’” said Buser. “That does not work. To make mobile apps work, be effective, and get adoption, you have to boil down the essentials—make the experience crisp and easy for folks.”

Getting the basics down is key, and the first wave of mobile taught us many valuable lessons.

  • Single-purpose apps are king. Friedman drives this home with the story of DoubleDutch, which provides an app for mobile salespeople. “When they finish a meeting, salespeople get a push notification to their device asking them to answer just two questions via sliders: ‘How well did the meeting go?’ and ‘How much revenue do we think we can get from this customer?’” Friedman said. Drop-dead simple to use, and it turns out that fresh feedback is also a highly accurate predictor.
  • Given the choice of convenience or privacy, most users choose the former. Car insurer Progressive has a device called Snapshot that users can put in their car that monitors driving habits and automatically adjusts rates. Snapshot is available in 42 states and, in just a few years of deployment, has signed up one million users. A recent study by Forbes shows that more than 50 percent of millennials (those ages 18 to 34) willingly give away personal info for coupons or deals. “There are big social implications about all of this,” warned Jeff Hartley, VP of services at Sencha. “If Aetna’s tracking my fitness information, are they going to be adjusting my rates along the line?” If other verticals are any indication, users won’t much care about giving up even health details if they get some tangible benefit in return.
  • Cross-platform design is a requirement. With multiple options for hybrid as well as native development and the dizzying and ever-expanding number of operating systems and form factors, siloed, parallel native development no longer makes any sense. Lorin Beer from PhoneGap/Adobe said, “The future of mobile is multi-platform, no doubt. Targeting one single environment, investing heavily in that platform, and then having that platform disappear in terms of your user base wanting that device would be disaster.”
  • There is no “enterprise-only” device. Whether it’s bring-your-own-device (BYOD) or an increasingly rare company-provided device, the end-user will mix personal and company data, and any policies or rules that a company tries to lay down are both completely unenforceable and easy to route around. Instead of fighting a losing battle, enterprises should focus on building useful, empowering apps that allow their employees to do their jobs better. In cases in which control over corporate data is still needed, enterprise mobility management vendors provide a wide range of containerization options. One example is Sencha Space, which allows corporate apps to run in a remotely wipeable isolated container.
  • Data needs to be shared across devices. When transitioning from their phones to their tablets to their wearables to their car interfaces or any other number of “things,” users expect the experience with their data to remain the same. Our calendars, documents, contact lists, and other data are viewed as static, inviolable, and completely independent from our apps. The window through which we view and interact with that data will depend heavily on the device’s display window and interface, but portability of that data is key to an app’s success.
  • The network is undependable. One of the key design considerations in any app must be locally cached data, allowing the app to work just fine even in offline mode. Friedman noted, “Customers assume the network is always there, but even in richly connected cities, it’s just not true.” Smart companies such as McKesson are taking this into account, even with such sensitive data as bedside pharmacist records. McKesson’s app allows doctors to record—even without network connectivity—which patient received which medication at what time. When the network becomes available, the records are synced from the device to the central system and back out to other caretakers’ mobile devices.

5 Next-gen infrastructure

With the rapid adoption of smartphones, we should not be surprised that mobile data traffic has doubled each year for the past few years. What may be surprising is just how unconnected most of the world still is. As Buser pointed out, “People in China, India, and much of the developing world have these devices but no internet connectivity.” For the vast majority of the world, 2G networks (GSM and CDMA) are as good as it gets. Until the rest of the world catches up with network speeds, app developers must be cognizant and careful when hitting network resources, especially if their audience could include users with poor connectivity.

Although LTE adoption in the industrialized world has been brisk (already estimated to currently cover about 10 percent of the world’s population), it’s not expected to become widespread until 2018.

6 Data at the core

Although devices and apps generally receive the bulk of the public’s love and the media’s attention, it’s really the data that drives those app downloads; it’s the capture and/or transmission of that data that makes users buy new hardware. We’ve previously seen this with forward-facing cameras, improved image capture, better microphones enabling better speech recognition, and easier data sharing via NFC—all technologies that enabled us to better capture or share data. We’re likely to see this much more with biometric sensors, self-forming networks, and an explosion of peripherals. As Jonathan Rende, VP of products at Appcelerator, said, “The rich experience isn’t just the UI of the app; it’s also enhanced by the data.” Arguably, the data space offers the most exciting prospect for the future of our mobile devices.

Regardless of the data they deal with, apps and hardware fly or flop based on two things—how easy they make it to capture and share that data and the intrinsic value of the data itself (or the utility gained from it). What we now know from years of big data exploration is that, given enough of it, all data is valuable. We also know that, year after year, the volume and variety of data we’re generating and consuming grows by leaps and bounds.

Right now for mobile, that data looks fairly mundane: recommendations, weather updates, traffic delays, and status updates. Yawn. But that’s all about to change.

Our physical world is really just a collection of measurable sensory data; temperatures, temporal data, geospatial data, and biometric readings are just a few of the innumerable data points that our phones, our tablets, our wearables, and our sensor arrays will be collecting. Rather than entering the internet of things, a better descriptive might be the “internet of sensors.” And those sensors are going to be kicking off data like never before.

As our devices continually become more powerful and able to handle bigger workloads, how that data is stored, synced, and shared across all these disparate devices is what will have the biggest impact on mobile adoption and utility in the coming years. With network connectivity looking to remain spotty, slow, and unreliable, the ability for devices to work in offline mode and sync their collected data when the network is available is becoming more important.

Two big trends that show no signs of slowing down are more power being pushed to the edge and higher user expectations that their apps will function well even without a stable connection. Luckily, these two trends play off of one another; as our devices become more powerful, we’re able to push more of the storage and processing needs down to them, occasionally syncing or reconciling data to consolidated cloud-based backends when connections are re-established.

Even today, many apps are currently supplying this capability to their users. For example, the travel organizer app TripCase has an offline mode that lets users access their rental car and hotel reservations, weather, and itineraries without needing to be connected. YouTube is also finding enough storage space on current hardware to allow users to temporarily cache and later view videos offline.

The number of apps that can benefit from sync or client-side caching is staggering. According to Friedman, data synchronization is the single biggest issue app makers face today. “We have tons of customers who say ‘data sync, data sync, data sync—this is our No. 1 problem in mobile,’” he said. Until recently, if you wanted data sync, you had to roll your own with custom client- and cloud-side storage.

There have been several approaches for packaging the data-sync problem and making device-side caching easier for developers to implement. Mobile-backend-as-a-service (MBaaS) provider Parse made a respectable effort with its wildly popular “SaveEventually” feature, which lets users store files to a local device and then upload to the cloud when connectivity is restored. But the Parse solution is only meant to store and sync individual objects, not something more complicated like a full-on database with tables, columns, and rows.

For client-side database needs, SQLite has long been the go-to solution. It performs well, supports nearly all of the SQL-92 standard SQL interactions, and, best of all, it’s pre-loaded on every mobile device out there. In early 2013, it was blessed with sync capabilities via Zumero. Using either the Zumero client SDK or Zumero for Xamarin, a developer can quickly and easily add a true synchronizable database to an app. If your data model is unlikely to change and your data fits into a nice table/row/column paradigm, your client-side app needs would probably be delivered using Zumero and SQLite.

For a great number of both current and present apps, however, those are two assumptions that don’t hold true.

After they’ve solved their data-sync issue, many app developers next struggle with keeping ahead of the app’s evolving data needs. They want to add or remove fields, or they want to re-organize the data, but every change involves the nightmare of sending out app updates and supporting multiple versions of the data model. They’re forced to fit their data into “rectangles,” while the data may want to be hierarchies that look more like triangles. This is when a distributed, synchronizable NoSQL engine would come in handy, and the market is just beginning to step up with options.

Couchbase, a recent upstart in the now fairly mature NoSQL engine space, just released two mobile-focused products that provide both sync support and allow for fluid data models:

  • Couchbase Lite: A lightweight, local native NoSQL (document plus key-value) database designed specifically for iOS and Android. The value portion of the data model is optimized to work with JSON documents, allowing easy modification.
  • Cloud Sync Gateway: Allows Couchbase Lite clients to sync their data with remote, cloud-based Couchbase Servers when network connectivity becomes available, ensuring that all clients have the same picture of the data.

Couchbase Lite users claim that it is lightweight, fast, reliable, and easy to use. “The Couchbase API is phenomenally easy, and we were impressed with how easy it is to integrate Couchbase with our app,” Friedman touted. At the recent Couchbase San Francisco event, where the products were first introduced, Couchbase Chief Architect J. Chris Anderson demonstrated the full code required for a functioning, fully self-synchronizing app in just about 10 minutes.

The Couchbase solution is well-thought out, including a layer of security via “channels,” attaching access controls to individual pieces of data and allowing certain data to only flow to certain devices or users. Because Couchbase Lite is a document-plus-key-value datastore, it can store any type of data required— binary, textual, structured, or not—and changing the document model to accommodate new data is as simple as adding that new data to your next document. It would appear that the only thing that Couchbase Lite currently lacks is competition, but that is destined to change as both developers and users begin demanding more options for client-side, unstructured data stores.

Distributed, synchronizable NoSQL engines like Couchbase are likely to play a starring role in both our present and future worlds when we can predict neither the number or types of devices nor the exact types or volumes of data that we’ll generate. These engines, in combination with more and more powerful devices, also open up a whole new dimension of applications.

7 Power on the edge

Despite innumerable claims of its demise, Moore’s Law (computers double in power every 12 to 18 months) continues to chug along just fine. Several decades of Moore’s Law holding true now find us living in a world where each of our phones are dozens of times as powerful as the supercomputers of just two or three generations ago. With the breakneck adoption in mobile devices and the capabilities of each doubling every year or so, we could conceivably find ourselves with more collective computing capacity on the edge than we have in public clouds.

Although device-to-device technology is still lamentably immature and underutilized, it’s a near certainty that we’ll soon be connecting our devices up in small local networks, even when the larger internet is unavailable.

Couchbase gives us a glimpse of this future with its recent Checkers Lite game, which lets large groups of people play the old-time favorite. The twist on the game is that it uses Couchbase Lite under the hood to sync everyone’s moves. You play on one team, and although your teammates may move differently, arrows on the board highlight the trending moves. After a handful of seconds, the most popular move is taken, and the other team’s clock starts.

Imagine now a chess game with a grandmaster on one side and tens of thousands of players on the other. Imagine natural disaster-response apps that establish ad-hoc, local device-to-device networks to enable communication between response team members. Imagine a collaboration engine installed on BYOD devices that lets employees vote on company direction or management decisions in real-time during the company all-hands.

It won’t take much to get us from here to there—mainly just waiting for more devices to support technologies like Wi-Fi Direct and more app builders to leverage data-sync technologies like Zumero and Couchbase.

Even if the internet of things doesn’t arrive as soon or as dramatically as some suspect:

  • Our networks will continue to be spotty,
  • Our devices are going to continue kicking off larger volumes of more varied data,
  • Our data will need to be shared across multiple devices, and
  • Our mobile devices will continue to become more powerful.

The apps of today require not only integration of cross-platform design, data sync, and data evolution but also that their maintenance teams keep their eyes peeled for new device, hardware, and local connectivity advances. Businesses and apps built with multi-device, data-centric, local-connectivity design considerations will, at the very least, be cheaper to build, easier to maintain, and offer better user experiences. If the internet of things turns out to be everything that some are hoping, those same businesses and apps will be uniquely poised to leapfrog their competition.

8 Key takeaways

  • Device types, form factors, and functionality will continue to explode, and with the onset of wearables and the maker movement, apps will need to work on an even more fragmented list of hardware and operating systems.
  • Apps are likely to change a great deal based on the new hardware and onset of the internet of things, but the trends are for hyper-focus, cross-platform design, data sync, and offline operation.
  • Infrastructure is not likely to change much, aside from the non-industrialized world achieving 3G and 4G technologies in the coming years.
  • Data storage and sync are becoming more important, and technologies like Zumero and Couchbase Lite enable a great deal of flexibility and scale.
  • As more and more computing power moves to the edge, new architectures and applications will emerge that challenge old-school “centralized” control.

9 About Rich Morrow

Rich Morrow is a 20-year open-source technology veteran who enjoys writing code, teaching, writing, and speaking. His current passions are cloud technologies (mainly Amazon Web Services and OpenStack) and big data (Hadoop and NoSQL), and he spends about half of his work life travelling around the country training other technologists and business leaders on their use. He leads the Denver/Boulder Cloud Computing Group, as well as quicloud, a Boulder, Colorado-based cloud and big data consultancy.

10 About Gigaom Research

Gigaom Research gives you insider access to expert industry insights on emerging markets. Focused on delivering highly relevant and timely research to the people who need it most, our analysis, reports, and original research come from the most respected voices in the industry. Whether you’re beginning to learn about a new market or are an industry insider, Gigaom Research addresses the need for relevant, illuminating insights into the industry’s most dynamic markets.

Visit us at: research.gigaom.com.

11 Copyright

© Knowingly, Inc. 2013. "Hybrid application design: balancing cloud-based and edge-based mobile data" is a trademark of Knowingly, Inc. For permission to reproduce this report, please contact sales@gigaom.com.

Tags