What seems to be missing from all of these discussions, though, is a dialogue about how to steer around this bottleneck and make big data directly accessible to business leaders. We have done it before in the software industry, and we can do it again.

To accomplish this goal, it’s helpful to understand the data scientist’s role in big data. Currently, big data is a melting pot of distributed data architectures and tools like Hadoop, NoSQL, Hive and R. In this highly technical environment, data scientists serve as the gatekeepers and mediators between these systems and the people who run the business – the domain experts.

While difficult to generalize, there are three main roles served by the data scientist: data architecture, machine learning, and analytics. While these roles are important, the fact is that not every company actually needs a highly specialized data team of the sort you’d find at Google or Facebook. The solution then lies in creating fit-to-purpose products and solutions that abstract away as much of the technical complexity as possible, so that the power of big data can be put into the hands of business users.

By way of example, think back to the web content management revolution at the turn of the century. Websites were all the rage, but the domain experts were continually banging their heads against the wall – we had an IT bottleneck. Every new piece of content had to be scheduled and sometimes hard-coded by the IT elite. So how was it resolved? We generalized and abstracted the basic needs into web content management systems and made them easy for non-techies to use. As long as you didn’t need anything too crazy, the problem was solved easily, and the bottleneck averted.

Let’s dig a little deeper into the three main roles of today’s data scientist, using online commerce as a backdrop.

Data Architecture

The key to reducing complexity is to limit scope. Nearly every ecommerce business is interested in capturing user behavior – engagements, purchases, offline transactions and social data – and almost every one of them has a catalog and customer profiles.

Limiting scope to this basic functionality would allow us to create templates for the standard data inputs, making both data capture and connecting the pipes much simpler. We’d also need to find meaningful ways to package the different data architectures and tools, which currently include Hadoop, Hbase, Hive, Pig, Cassandra and Mahout. These packages should be fit for purpose. It comes down to the 80/20 rule: 80 percent of big data use cases (which is all most ecommerce businesses need), can be achieved with 20 percent of the effort and technology.

Machine Learning

Surely we need data scientists in machine learning, right? Well, if you have very customized needs, perhaps. But most of the standard challenges that require big data, like recommendation engines and personalization systems, can be abstracted out. For example, a large part of the job of a data scientist is crafting “features,” which are meaningful combinations of input data that make machine learning effective. As much as we’d like to think that all data scientists have to do is plug data into the machine and hit “go,” the reality is people need to help the machine by giving it useful ways of looking at the world.

On a per domain basis, however, feature creation could be templatized, too. Every commerce site has a notion of buy flow and user segmentation, for example. What if domain experts could directly encode their ideas and representations of their domains into the system, bypassing the data scientists as middleman and translator?

Analytics

It’s never easy to automatically surface the most valuable insights from data. There are ways to provide domain-specific lenses, however, that allow business experts to experiment – much like a data scientist. This seems to be the easiest problem to solve, as there are a variety of domain-specific analytics products already on the market.

But these products are still more constrained and less accessible to domain experts than they could be. There is definitely room for a friendlier interface. We also need to take into consideration how the machine learns from the results that analytics deliver. This is the critical feedback loop, and business experts want to provide modifications into that loop. This is another opportunity to provide a templatized interface.

As we learned in the CMS space, these solutions won’t solve every problem every time. But applying a technology solution to the broader set of data issues will relieve the data scientist bottleneck. Once domain experts are able to work directly with machine learning systems, we may enter a new age of big data where we learn from each other. Maybe then, big data will actually solve more problems than it creates.

Scott Brave is co-founder and CTO of Baynote, an e-tail and e-commerce advisory business. He is also an editor of the “International Journal of Human-Computer Studies” (Amsterdam: Elsevier) and co-author of “Wired for speech: How voice activates and advances the human-computer relationship” (Cambridge, MA: MIT Press).

Photo courtesy of Sergey Nivens/Shutterstock.com

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A near-term outlook for big data
• 2012: The Hadoop infrastructure market booms
• Infrastructure Q1: Cloud and big data woo enterprises

While matching the impact of the first wave may be difficult, a new wave of MVNOs is reaching the market now with equally disruptive business models, such as dramatically reduced acquisition and service costs, low-cost voice and data services, and exciting new capabilities. Here’s how they’re doing it:

Low-cost model

Handset subsidies can be a large part of the acquisition cost for traditional carriers (in the case of the iPhone, for instance, several hundred dollars per unit). In contrast, many new MVNOs are adopting Bring Your Own Device (BYOD) strategies, with SIM-only MVNOs like Simple Mobile, Red Pocket and Ultra on the GSM side, and a new BYO Sprint Device solution for MVNOs like Kajeet, Ting and others on the CDMA network. Sprint’s BYOSD program has the added benefit that no SIM kit or installation is required; the handset is activated simply via its serial number.

With BYOSD, for example, Kajeet offers network-based parental controls, web filtering and location services on recycled handsets. BYOD solves two problems for the MVNO – eliminating handset subsidies and reducing logistics cost (kitting, shipping, warranty repairs and returns). Even where customized handsets are used, MVNOs sell them above cost, eliminating costly subsidies.

Not sold in stores

Many new MVNOs bypass the retailer and dealer channel altogether and save a bundle by embracing online distribution, web marketing, social media, viral and multi-level marketing. In lieu of paying retailers high commissions and sales incentives while still fighting for shelf space, these MVNOs rely on newer, lower-cost targeting. SEO and SEM are just the beginning. MVNOs like Ting sponsor selected podcasts and weekly Facebook caption contests to reach their target audience. Solavei uses multi-level marketing, Facebook, and tried-and-true referral incentives.  And Kajeet uses a “Mom Sales Team” referral program (that interestingly relies on old-fashioned word-of-mouth among parents).

Service – airtime and data – costs can also be reduced. With increasing data usage, many MVNOs utilize dual-mode phones (cellular and Wi-Fi) to offload voice and data traffic to Wi-Fi networks, which is increasingly available in homes, offices and businesses. And an added benefit for providers: offloading to Wi-Fi turns off the carrier’s meter.

MVNOs to watch

Ting appeals to early adopters and Internet-savvy folks with a completely different take on pricing. Self-described as “Geek-powered,” Ting lets customers design their own rate plan, buying only as many minutes, messages and megabytes as needed, with plan sharing for just $6/device. Alternatively, customers can simply pay for actual usage at the end of the month. Is it prepaid or postpaid? Ting’s answer: “We call it fair, and trust you.” Customers manage usage from an online dashboard, and customer care is friendly, unscripted and helpful. Bring your Sprint device to Ting, or select from a range of Android handsets.

When Republic Wireless introduced its $19/month unlimited plan as a beta trial, everyone asked how they planned to do it. Republic relies heavily on Wi-Fi networks at home and work, using “hybrid calling” or cellular offload where traffic only rolls to Sprint’s cellular network when Wi-Fi is unavailable. Republic is now shipping a Motorola Android smartphone, running proprietary Republic software (for $259), which completes the no-contract package. And apparently the beta trial worked just fine: the same $19 plan is now available to all.

FreedomPop guarantees 500MB of free 4G mobile broadband data every month, with no data caps or throttling, and attractive plans ($17.99/month for 2GB of data, a cent per MB additional). Customers can earn additional data for each friend referred or unlimited data by engaging in partner promotions. The Freedom Hub Burst, a 4G Wi-Fi router that offloads cellular to wireline and supports up to 10 devices, is free with security deposit. They also offer the Freedom Sleeve Rocket, an iPod Touch case that turns it into an iPhone. Plans include trading bandwidth with other FreedomPop users, and creating bandwidth-sharing communities. Launched on Clearwire, FreedomPop will add Sprint’s LTE network next year.

Voyager Mobile , a Tennessee-based company with just nine employees, nonetheless thinks big. With unlimited talk at $17 to unlimited everything for $39, Voyager’s shrinking service plan program drops monthly rates by $1 for every six months of on-time payments. Last month, Voyager announced Project Global Voyager for calling “across the world, without any roaming charges.” Using dual CDMA-GSM handsets, on Sprint here and GSM everywhere else, Voyager promises international voice, messaging and data worldwide “without a penny of roaming charges” in first-half 2013. Voyager says demand is strong. Two other MVNOs are also addressing international roaming costs – Karma in the U.S. and GlobalGig in London.

And everybody is watching Solavei, a T-Mobile MVNO that, under an elaborate compensation plan, pays bounties to members for signing up new customers. Members are encouraged to share with family and friends, and post on social networks in a marketing scheme described as “more like Amway or Tupperware.” Last month, Solavei announced it reached 65,000 members just six weeks after launch and had paid more than $1 million in commissions. This would be a roaring start, but not losing steam is often the challenge for multi-level marketing.

These and other new MVNOs are introducing attractive pricing with innovative technology and business models. As always though, execution and deep pockets will determine winners and losers.

Whitey Bluestein, a 25-year telecom veteran, is a strategic advisor and corporate development specialist focused on prepaid, mobile applications, payments and roaming services. Visit whiteybluestein.com.

Photo courtesy of Michaelstockfoto/Shutterstock.com.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Mobile 2012 and beyond
• CES 2012: a recap and analysis
• What to watch in mobile in 2013

Change Tracking

The chief problem for me was that Pages did not show any sort of change tracking, making the app useless if your workflow relies on this feature. The good news is version 1.7 adds change tracking. The bad news: it’s so poorly implemented that it’s still almost useless.

While you now have the ability to review and accept changes on your iOS device, you still cannot use comments. For me, that is the most important part of the review process. Usually, that’s where a reviewer or editor asks questions and there’s a sidebar discussion in-line about the change. So, while I can accept the change someone made, I can’t see the commentary. I pretty much tend to usually “accept all changes” anyway. (I’m not in the legal profession, so your usage may vary).

The current state of change tracking, where comments aren’t handled, isn’t even something I can give Apple partial credit on.

Editing on the iPhone 5

Before the iPhone 5, editing in Pages on an iPhone was a complete mess. The screen was too small to display a lot of text and the keyboard covered up too much of the page. With the iPhone 5, well, it’s better, but sadly not by much. The chief advantage now is that I can fudge the margins a little bit and have my rows of text span the width of the screen in landscape so I’m not scrolling from side to side to see the text. While I can adjust the zoom level when reading with double-taps, once I edit the text, it zooms in.

Life would be so much easier if I could edit text in the zoomed-out view. So, for the most part I use Pages to refer to documents — meeting agendas, notes, etc. Performing more detailed tasks on a screen smaller than my iPad is something I might need to look at an iPad mini for.

iOS version still lags behind OS X

Styles still remain partially implemented. While you can choose from a list of styles, you can’t create your own. You also cannot update a style if you want the font to be different, though you can change the font manually.

Tables of contents also remain elusive, which can be a problem if you’ve made significant edits to a document and need to update the TOC.

Final thoughts

Pages for iOS still lags far behind the OS X version of the app.  Since we’re clearly past the myth of iPads existing only as consumption devices, and more professionals are eschewing laptops for iPads, I think Apple needs to seriously step up its game with its iOS iWork offerings. While I can accept that OS X Pages does not have feature parity with Microsoft Word, the lack of parity between iOS and OS X iWork apps is beginning to become tough to handle.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Survey: How apps can solve photo management
• Analyzing the wearable computing market
• The future of mobile: a segment analysis by GigaOM Pro

Here are the 10 most-popular posts I’ve written this year on the topic, as well as 10 of my personal favorites. Maybe they’ll jog your memory about some ideas you’ve forgotten, or perhaps you’re reading them for the first time. Either way, enjoy this look back at in 2012 in data.

The most-popular posts of the year

• 5 low-profile startups that could change the face of big data (January 28): These companies aren’t so low-profile anymore (or in all cases using the same name), but they’re they’re just as cool.
• What it really means when someone says Hadoop (February 6): I should revisit this regularly. The Hadoop ecosystem just gets more confusing.
• Exclusive: The brains behind Hive launch on-demand Hadoop service (June 6): This might be the first of many startups from Facebook’s big data experts.
• How Facebook keeps 100 petabytes of data online (June 13): Speaking of Facebook, it’s storing a lot of data. Here’s how it migrated it to a new data center without downtime.
• How India’s favorite TV show uses data to change the world (August 11): The show, Satyamev Jayate, and its host, Aamir Khan, are tackling difficult social issues and generating incredible audience response.
• How Disney built a big data platform on a startup budget (September 16): Or at least a startup mindset. It used lots of open source — databases, Hadoop and then some — to serve an entire company.
• Forget your fancy data science, try overkill analytics (September 21): The winner of our inaugural Kaggle challenge used simple models and supplemented with lots of cloud resources to predict what readers want.
• Why becoming a data scientist might be easier than you think (October 14): Coursera and its online education brethren are teaching the foundational skills needed for some new-age data analysis.
• 5 trends that are changing how we do big data (November 3): Machine learning, Hadoop applications and artificial intelligence are among the things advancing how we consume big data technologies.
• A programmer’s guide to big data: 12 tools to know (December 18): Just what it sounds like. This is a collection of tools for easily analyzing application data or building data-centric applications.

The best of the rest

• Supreme Court sidesteps digital privacy … for now (January 24): United States v. Jones was an important decision for citizens’ Fourth Amendment rights, and might have set the stage for even broader protections.
• Forget the EU: How to really empower users on privacy (January 26): Google, Facebook and others had some big privacy snafus this year. Here are some ideas for overcoming the tension between free services and privacy.
• Under the covers of eBay’s big data operation (January 31): A look at the technologies eBay uses to do its big data work, ranging from Teradata to HBase and all in between.
• Satellite imagery and Hadoop mean $70 for Skybox (April 17): This is one of the coolest startups around — at least in terms of shooting for the stars (literally).
• Hey, Los Angeles, Xerox thinks it can clear traffic on I-10 (July 20): It’s not easy to predict traffic patterns or parking availability, but Xerox and its partners are trying in one of the toughest places around.
• Why data should be our guiding light on public policy (July 27): Researchers are building computer models and algorithms to help analyze all sorts of difficult problems, yet they remain ignored.
• 5 ideas to help everyone make the most of big data (September 17): This is a handful of bright ideas from bright people on how to think about data analysis and exploration in an age of big data.
• A startup asks, ‘What if you didn’t have to analyze data at all?’ (November 20): Although it claims to not use machine learning, BeyondCore shows what’s possible when software finds the connections first before users start exploring.
• Data isn’t just the new oil, it’s the new money. Ask Zoë Keating (November 20): The idea that data can be as valuable as money is gaining acceptance with everyone from major retailers to independent musicians.
• How Obama’s data scientists built a volunteer army on Facebook (December 8): We covered a lot about data science and politics in 2012, but Obama for America’s ability to predict how to reach which voters helped deliver the ultimate prize.

Feature image courtesy of Shutterstock user tuulijumala.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A near-term outlook for big data
• The importance of putting the U and I in visualization
• Don’t hold your breath for a single big data stack

The X Phone, as it’s currently dubbed, could have some unique features to make it stand out from the crowded smartphone market. Reports suggest a possible bendable display, ceramics for the casing and some type of gesture recognition. After the release of such a phone, Google is expected to offer an X Tablet. If the news is accurate, I’d expect these “X” products to be introduced at Google’s next annual I/O developer conference, scheduled for May 15 to 17 next year.

While waiting for an X Tablet, current Android slates haven’t had a version of Flipboard to call their own until now. This week, a tablet-optimized version of the popular news reading application appeared in the Google Play store. It’s surprising that it took so long for this app to appear on tablets, mainly because the reading experience of the magazine-style is well suited for larger screens. Indeed, the app debuted in 2010 on Apple’s iPad and later found its way to smaller screens.

Bigger screens are the current trend, however, and not just for tablets. Samsung is leveraging its Galaxy brand for a new low-cost phone with 5-inch display. The Galaxy Grand only has an 800 x 480 screen resolution, but does include a dual-core processor, 8 GB of internal storage, full gigabyte of memory and support for 21 Mbps 3G networks. Support for two SIM cards indicate that Samsung is looking to bring the large screen experience to Asia and Africa at a relatively low price point; something that can help the manufacturer stay at or near the top of Android sales in those regions.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Is Android broken and if so, will Google fix it?
• Tablets wars: Apple is from Venus, Amazon is from Mars
• What the Google-Motorola deal means for Android, Microsoft and the mobile industry

• The doctor, the CIA and the blood of Bin Laden: A really great old-school, behinds-the-scene story told by Matthieu Aikins in the pages of GQ.
• Social media and the devolution of friendship: As you sit with your family and friends this holiday week, it might be a good time to reevaluate and rethink online relationships and the meaning they carry.  I don’t necessarily agree with the writer, but don’t disagree with her either, for her observations do ring true.
• Tick Tock Tick Tock: Really great story about master horologist John Metcalfe and how he repairs clocks. This is good stuff, especially if you are a fan of watches and people who make things with their hands.
• Pop goes the art bubble: And just when you were thinking things were bad in Startupland!
• Light… what is it good for?, asks Anton Desyatnikov in Aeon magazine. Wonderful piece for all of us who love physics and lasers and light and simply questioning everything.
• The future of Special Ops: The rise of endless irregular wars means that special ops are going to have to be redefined, argues Linda Robinson, Adjunct Senior Fellow for U.S. National Security and Foreign Policy at the Council on Foreign Relations.
• Love means never wanting to say sorry. I think you should read this piece rather than me interpret this for you.

Happy holidays and a very happy new year.