FedEX CIO Robert Carter, who spoke at the IT Expo in Austin on Thursday, explained that this act of attaching bits to real-world atoms creates opportunities galore.

“The information we apply to the physical world creates an incredible opportunity for us,” Carter said. “When we apply more bits to the atoms, we create more opportunity for interactions, more opportunities to do business, and opportunities to change how you see the world.” In later conversation with a few reporters, he explained how this lofty vision affects IT, and how changes in IT bring about this vision.

Paving the way for “epic data”

FedEx’s home page in 1994

FedEx has a history of embracing technology that gets it closer to its customers. Its first web site was put in place in 1994 and was just a basic HTML page that asked for customers to enter their tracking number, then communicated that info back to a mainframe. The mainframe figured out where the package was and shot the info back to the customer. Today, it offers a basic native app on all platforms that’s basically a superficial “skin” that talks back to myriad FedEx services to give customers the tools and information they need.

As FedEx has grown, the backend infrastructure to support the organization has adapted accordingly, with FedEx using gear form Teradata and Greenplum to handle today’s data warehousing and analytics. Carter didn’t say how much data the company generates a day, but noted that it has exabytes and exabytes of data that it generates from the 9 million shipments it averages daily. His so-called “epic data” is then kept and stored indefinitely.

And as FedEx adds its SenseAware platform, it is adding real-time data and notifications to its infrastructure at a more granular level. That platform, which launched in 2009, contains a variety of sensors and radio chips that allow it to detect temperature, location, light and report back if a package (and its contents) hits a problem. The SenseAware device, which is a roughly 6 inches by 6 inches, gets dropped into high-value packages like diamonds or human organs and can proactively monitor the package for 96 hours and then alert recipients and senders if something endangers or waylays the package.

Because the SenseAware device has a radio, it is constantly broadcasting information back to FedEx, and can generate a lot of data that must be acted on in real time. Without the right infrastructure, that might be overwhelming. Still, Carter notes that the bulk of FedEx’s exabytes of data are structured and sent from different services to the same message bus where decisions or further analytics can happen.

In the future, databases are for archival purposes

“There is so much coming online that allows us to look at large data sets. From the technical mindset, what’s happening fundamentally is a shift from the reason databases even existed,” Carter explained. Databases were built because memory was precious and operations and IT staff had to allocate when and what made it into memory at any given time. But in modern data centers and computing architectures there’s plenty of addressable memory and cabinets of non-volatile flash memory available for applications. “Databases will become archival rather than a system of record,” Carter said.

Carter compares it to having a file cabinet versus having a more brain-like process with a matrix of information the computer can harness. But FedEx has some advantages in building that matrix. For example, much of its data comes from pre-determined processes such as existing routes or metrics affecting its business and, thus, is mostly structured. Carter says that only a few elements of data, such as the monitoring of Facebook and Twitter to talk to customers, are relatively unstructured.

He anticipates the future of his many exabytes of data as being dumped into a pool of storage with some metadata attached to the files so it can be analyzed. It sounds closer to a key-value store or even some of the NoSQL efforts, although he said FedEx isn’t using many of the new open source data stores or analytics out there, relying in Greenplum’s Hadoop distribution for analysis today.

As companies seek to embrace and use big data, it’s clear that the way data is stored and analyzed is changing. Bt understanding that the application of data to physical items like packages can generate huge opportunities shouldn’t be forgotten either. Big data needs to be used to produce big (or even little) insights.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Will the Real-Time Web Bring High Performance to a System Near You?
• How emerging technologies will influence collaboration
• The importance of putting the U and I in visualization

Don’t call off the data deluge yet

That represents a 78 percent compound annual growth rate in mobile data traffic over the next five years, which in 2011 topped out at 0.6 exabytes. According to Cisco, we ain’t seen nuthin’ yet. Just the incremental data traffic added to mobile networks in 2015 and 2016 will be three times larger than the entirety of the mobile Internet this year, Cisco predicts. The total number of global connections will top 10 billion, far exceeding the world’s projected population of 7.3 billion. Average connection speeds to mobile devices will increase by a factor of nine, from 1.3 Mbps sent down to a smartphone in 2011 to 5.2 Mbps in 2016. By that year, 71 percent of all traffic will be dominated by a single application: video.

Last year, Cisco predicted an annual growth rate of 92 percent between 2010 and 2015, and at first glance it might appear that Cisco is adjusting its numbers downward. But director of service provider marketing Douglas Webster said that is not the case. The phenomenal growth in 2010 is now factored in to this year’s projections, he explained: As global mobile data use swells, each year’s percentage growth will be smaller than the previous. In fact, Cisco has bumped up its projections for global consumption this year over last, revising the projected monthly run rate in 2015 from 6.3 exabytes to 6.9 exabytes.

“It’s very much a matter of large numbers,” Webster said. “If history is a guide then overall growth is likely to be greater than what we’re estimating.”

What’s AT&T got to do with it?

Cisco’s VNI numbers have become one of the industry’s standard measurements for projecting future mobile data demands, and they have been cited by carriers, infrastructure vendors and even the U.S. government as justification for clearing massive amounts of new spectrum for mobile broadband use.

But in the past month, AT&T revealed that its data growth rate is now running at about 40 percent, far smaller than you would expect in an exploding mobile broadband market. That has led several critics, including GigaOM contributor Tim Farrar, to question whether the spectrum crisis the industry supposedly faces is a myth.

AT&T’s current growth rate, however, isn’t the best snapshot of the industry as a whole. Cisco’s figures are a global average, not just for the U.S., which experienced the smartphone boom far before other regions. Meanwhile, AT&T, by virtue of its years of iPhone exclusivity, is well ahead of the U.S. curve, with an industry-leading 56.8 percent smartphone penetration. Unlike its competitors, AT&T can no longer double its smartphone base. Its future mobile data growth will increasingly depend more on its existing subscribers than on new ones.

The next big growth spurt is from the 99 percent

Cisco is projecting a 74 percent annual growth rate in the U.S., only slightly less than the overall global growth rate. But mobile data growth hardly slowed down in the U.S. last year. Using real network data to validate its numbers, Cisco found that traffic over cellular networks increased by 172 percent in 2011, compared with a 171 percent increase in 2010. AT&T may be slowing down, but the rest of the wireless industry is not.

The biggest check on U.S. growth that Cisco found was among the top 1 percent of users, which traditionally consume the lion’s share of mobile network capacity. In 2010, the top 1 percent of mobile data users were responsible for an astounding 51 percent of all traffic. This year that top 1 percent consumed only 24 percent of traffic, a likely result of tiered data plans and throttling by all the major operators save Sprint, Webster said. Still, that didn’t stop the remaining 99 percent from boosting their overall consumption:

• The average mobile connection in the U.S. generated 319 MB of traffic per month in 2011, up 156 percent from 125 MB per month in 2010.
• The average smartphone generated 201 MB of traffic, up 152 percent from 80 MB per month in 2010.
• Laptops are still by far the biggest mobile broadband hogs, generating 2,507 MB of traffic per month in 2011, up 88 percent from 1,336 MB per month in 2010.
• While still not the most prevalent devices connected to the cellular network, tablets generated an average of 382 MB of traffic per month in 2011, up from 198 MB per month in 2010.

Cellular-connected tablets already consume nearly twice what the typical smartphone does, and their average consumption is increasing at a faster pace. As the smartphone data boom starts to taper off, it is easy to envision how the tablet could kick off the next big data growth spurt in the U.S. — that is, when consumers finally start connecting them to mobile broadband networks.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Carrier IQ and the continued erosion of operator trust
• The future of mobile: a segment analysis by GigaOM Pro
• New solutions for the evolving mobile network

Cisco anticipates that in 2015, the average mobile user will consume 1,118 megabytes of traffic per month. For perspective, today the average mobile connection generates 65 megabytes of traffic per month, equivalent to about 15 MP3 music files. While the average growth is impressive, those megabytes will be spread across a wider number of devices, from tablets (big bandwidth hogs) to e-readers, which consume much less bandwidth, making it all the more significant. The Cisco study predicts that by 2015, more than 5.6 billion personal devices will be connected to mobile networks, and there will also be 1.5 billion machine-to-machine nodes — nearly the equivalent of one mobile connection for every person in the world. Cisco doesn’t give the current number of connections, but the GSM Association puts it at about 5 billion.

Cisco anticipates that mobile network-connected tablets will generate 248 petabytes per month in 2015 while machine-to-machine (M2M) traffic will reach 295 petabytes per month in 2015. The numbers are interesting because they are on opposite ends of the spectrum when it comes to data demand and profitability. Consumer video is in high demand thanks to devices such as tablets, but it’s also something operators are watching because they are worried it will clog their networks while not bringing higher margins. Meanwhile, much of the M2M connectivity will be small amounts of traffic, but extremely profitable.

The question operators must answer is how soon these dynamics will reach some sort of equilibrium, or perhaps, if those dynamics will reach an equilibrium. Cisco notes in a presentation that, “in 2010, global mobile data traffic experienced a 2.6 times growth for the third year in a row, despite a slow economic recovery, increased traffic offload, and the advent of tiered pricing.”

Operators are trying to cut back on data use, whether because their networks are overwhelmed or simply because they want to keep their nice margins on wireless data. However, that growth is still coming, and operators are undoubtedly trying to shape their customer base to ensure that a consumer-heavy subscriber portfolio doesn’t drag down their bottom line. So tiered pricing, pricing that ignores the reality of consuming more data at faster speeds, and possibly limits on what one can download in cheaper plans are all likely to make an appearance.

Crazy Stats (and One Chart) to Beef Up Your Mobile Broadband Reports

• Mobile video is forecast to represent 66 percent of all mobile data traffic by 2015, increasing 35-fold from 2010 to 2015, the highest growth rate of any mobile data application tracked in the Cisco VNI Global Mobile Data Traffic Forecast.
• Mobile traffic originating from tablet devices is expected to grow 205-fold from 2010 to 2015, the highest growth rate of any device category tracked.
• Global mobile data traffic increased 159 percent from calendar year 2009 to calendar year 2010 to 237 petabytes per month, or the equivalent of 60 million DVDs.
• Global mobile data traffic grew 4.2 times as fast as global fixed broadband data traffic in 2010.
• Global mobile data traffic in 2010 was three times the size of all global Internet traffic (fixed and mobile) in the year 2000.
• Smart phones, laptops, and other portable devices will drive more than 87 percent of global mobile traffic by 2015.

Related GigaOM Pro content:

• Mobile Broadband: Pricing for Profits
• Metered Mobile Data Is Coming and Here’s How
• Everybody Hertz: The Looming Spectrum Crisis