For instance, application design principles changed dramatically when the PC, x86 architecture, and client/server paradigm were born in the ’80s. And  it happened again with the advent of the web and open-source technology in the mid ’90s. Whenever such abrupt changes arise, application developers are forced to rethink how they build and deploy their software.

Today, we’re seeing a huge leap in infrastructure capability, this time pioneered by Amazon Web Services. It’s clear that to take full advantage of the new cloud infrastructure, applications that run successfully on AWS must be inherently different than applications that were built to run successfully on a corporate server – even a virtualized one. But there are a number of other particular ways in which today’s (and tomorrow’s) cloud applications will need to be designed differently than in the past. Here are the most crucial ones, and how the ways of the old world have been changed in the new one :

Scaling 

In the old world, scaling was accomplished by scaling up – to accommodate more users or data, you simply bought a bigger server.

In the new world, scaling is typically done by scaling out. You don’t add a bigger machine, you add multiple machines of the same sort. In the cloud world, those machines are virtual machines, and their instantiations in the cloud are instances.

Resilience 

Before, software was seen as unreliable, and resilience was built into the hardware layer.

Today, the underlying infrastructure – the hardware – is seen as the weak link, and it is up to applications to accommodate for this. There is no guarantee that a virtual machine instance will always function. It can disappear at any moment and the application must be prepared for this.

By way of example, Netflix, arguably the most advanced user of the cloud today, has gone the farthest in adopting this new paradigm. They have a process called ChaosMonkey that randomly kills virtual machine instances from underneath the application workloads. Why on earth do they do this on purpose? Because they are ensuring uptime and resilience: By exposing their applications to random loss of instances, they force application developers to build more resilient apps. Brilliant.

Bursting

In the old world – think accounting and payroll applications – the application workload was reasonably stable and predictable. It was known how many users a system had, and how many records they were likely to process at any given moment.

In the new world, we see variable and unpredictable workloads. Today’s software systems have to reach farther out in the world, to consumers and devices that demand services at unpredictable moments and unpredictable loads. To accommodate such unforeseen fluctuations in individual application workloads required a new software architecture. We now have it in the cloud, but clearly it is still in its infancy.

Software variety

In the past we didn’t have much software variety. Each application was written in one language and used one database. Companies standardized on a single, or at least very few operating systems. The software stack was boringly simple and uniform (at least now in retrospect).

In the new world of cloud, the opposite is happening. Within a single application, many different languages can be used, many different libraries and toolkits can be employed, and many different database products can be used. And because in a cloud you can create and spin up your own image, tailored to your and your application’s specific needs, applications within one company must be able to operate under a spectrum of configurations.

From VM to cloud 

Even between the relatively new technology of hypervisors and the modern cloud thinking, there are differences. VMware, the pioneer and leader in virtualization, built its hypervisors to essentially behave the way physical machines did before.

But in the cloud world, the virtual machine is not a representation of a physical server; it’s a representation of units of compute. (Steve Bradshaw wrote about this topic in depth.)

User patience

In the old world, users were taught to be patient. The system may have needed a long time to respond to simple retrieval or update requests, and new features were added slowly to the application (if at all).

In the new cloud world, users have no patience. They hardly tolerate latency or wait times, and they look for improvements in the service every week, if not every day. Evidence of this can be found in self-service IT. Rather than file a ticket with IT and wait for a response several days later, users of IT can self-provision the resources they need.

Do these observations rhyme with what you are experiencing and taking action on in your organization? I look forward to comments and debate on this topic.

Marten Mickos is the CEO of Eucalyptus Systems. He previously served as CEO of MySQL AB, which was acquired by Sun Microsystems. He is a member of the board of directors of Nokia.

Have an idea for a post you’d like to contribute to GigaOm? Click here for our guidelines and contact info.

Photo courtesy of Mike Flippo/Shutterstock.com.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Infrastructure Q1: Cloud and big data woo enterprises
• What Enterprise Software Vendors Could Learn from the Consumer Space
• Continuous delivery and the world of devops

Marketing becomes new revenue arm

Traditionally, CMOs have dealt with the “soft skills” of marketing. They headed up cost centers filled with branding, advertising and campaigns that were expensive endeavors, producing benefits that were often difficult to measure. In this current shift, CMOs might not bring CIOs to their knees, but if corporate budgets could talk they would certainly favor the CMO. To wit: Gartner predicts that by 2017 the CMO will spend more on IT than the CIO. Why? Mobile, big data and real-time analytics are transforming the modern CMO’s organization from a cost center to a critical revenue-driving arm to reach and engage the customer base.

Realizing that CIOs and CMOs probably hate stories of a battle raging between them, there is actually a peaceful, and logical, middle ground in which the two work together to harness the vastness of big data to create real-time – and importantly, actionable – analytics. While the CMO brings the marketing skills to the table, the CIO has the technical chops to deal with capturing, processing and integrating data to make it useful.

Mobile transition requires new tools

The CMO is in a particularly prime position in the increasingly important mobile channel, as the feedback loop is uniquely personal and immediate. The ability to measure, predict and act upon an end-user action has never been more precise.

However, mobile is a different beast and legacy systems designed for the web rely on cookies to track user interaction. As we go more mobile, we enter a world devoid of cookies. New tools are required to match mobile users to their actions across multiple channels. Along with new tools, there needs to be a new partnership – the CMO and CIO united as a Dynamic Duo – working diligently together to deliver the cloud-based and back-office infrastructure required to pull actionable information from big data across all channels: bricks and mortar, web and mobile.

New focus on data-driven outreach

To get a view into what makes the CMO so potentially powerful, let’s turn to another battle, the one that happens on a quarterly basis across corporate America and beyond: The fight to make the quarterly earnings number. Typically, a CEO faced with a sales or earnings shortfall will turn to the CFO and the head of sales and say: “What can you do?”

The honest answer is: short term, not much. They can jam the channel, which will make the next quarter’s number even harder to achieve. Or shovel product out the door at prices that wouldn’t otherwise make sense. But by the time the word gets to the field and actions kick into gear, there can be quite a lag in results. And, again, these results may have value only for the short-term.

In the very near future – one that has already arrived for some organizations – the CEO turns to the CMO, who looks up from his or her laptop and says: “No problem. We should make that number by noon.”

The CFO and head of sales turn and say in unison: “How?”

“While you were talking I found an overstock of our Zing42 and sent an offer to 234,000 customers who have purchased this in the past, and statistically are ready to buy again, and to another 341,000 customers who haven’t purchased this yet, but show a high probability of doing so at the price point we are offering.”

No hurried sales calls to the field asking them to stuff the channel. No broadcasting of sales and discounts that can tarnish an image or weaken a brand. But rather, an analytically-driven offer to a curated digital audience delivered via a simple mobile notification, or in-application offer, to thousands of people who are analyzed and selected to be eager buyers.

This is the look of the CMO ascending: Powered by a cloud of mobile, big data and real-time analytics.

Rob Lilleness is the CEO of Medio, a Seattle-based provider of analytics solutions for mobile computing. Follow him on Twitter @Robmedio.

Have an idea for a post you’d like to contribute to GigaOm? Click here for our guidelines and contact info.

Photo courtesy Pressmaster/Shutterstock.com.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• The importance of putting the U and I in visualization
• What Amazon’s new Kindle line means for Apple, Netflix and online media
• The Internet of Things: What It Is, Why It Matters

It’s important to note that Twitter  is a platform, not a news service, and also that regardless, no one outlet can control the internet anyway: That’s the beauty and curse of the beast. So when news breaks it’s about two things: accuracy and distribution.  Right now we’re stuck with a drunk leaf blower in a flour factory. I’ll be the first to champion these tools as platforms for change, opportunity and knowledge sharing, but it has become clear we – and especially the reeling news media – are in need of a system that helps Twitter et al sort through the haze of breaking news and get the facts straight, faster. The current model and tools are not clearing things up – they are adding to the mess.

A centralized, collaborative evidence table

Sifting through the mountains of analysis on the bombings alone, I couldn’t help but think of the customized Twitter Nascar hashtag page that was put together in 2011, and how it tied in nicely with the New Yorker digital “evidence table” it describes for Reddit users. So one first step I propose is a one-stop place or system in times of important news or mass emergency. Literally just a single agreed upon place to gather the facts, or a system of checks and balances free from speculation. Like a central hub for help to be channeled and extra information provided – a bit like an active Storify stream.

This feels like a simple fix to a complicated problem at the source. Clearly there are huge questions about who administers it, but one thing is clear: It must be solitary and held to a strict code that is pre-agreed upon, possibly among a cross-collaboration of the major newspapers. For instance, each might host the same page so traffic stays where the trust is with the user.  There is no speculation: Simple fact dissemination and information being released – only after  it is verified – so that the news-consuming public has a go-to source that is consistent.

The ability to deal with errors

Imagine if Twitter or Facebook could lower the relevancy of an incorrect tweet or post in real-time so that bad information was less likely to be seen. Reddit and pals is a more difficult kettle of fish because of their very nature.  We will need to help them help themselves by providing clear information in order for them to do what they do best – engage with it.

The Atlantic wrote recently about the need to undo things on Twitter because it is currently a one-way system that, while capable of self-correcting over time, is pretty flawed when it comes to doing so in the moment. And so we need a system that enables users to revoke or modify what has been said so that it is instantly identifiable. Internet fundamentalists find this idea uncomfortable but many I suspect would cherish the ability to be alerted to incorrect information so the continued dissemination of knowingly false info can be minimized.  A technical nightmare sure, but something to work towards.

A need for innovation and cooperation

Right now big data is not being used or harnessed by news organization beyond visualization or longer-form pieces but I imagine a time (and not too far in the future either) when we see news outlets using Twitter and company in a much smarter fashion than simply looking at volume spikes and “first-grabs.”  So for instance, outlets might soon use data to predict, locate and activate “sleeper-unit” journalists (and trained citizen journalists) who are armed with Facetime technology – or simply volunteer individuals streaming through a phone that a news outlet is able to instantly locate via GPS. It’s interesting to note that Twitter has just appointed its first Data Editor who is charged with “explain[ing] how this phenomenon works.”

We have two options when it comes to “fixing” truly crucial news and real-time mass events: 1) Assume that what we’re doing now works but will need a few tweaks, or;  2) Realize that our current system is no longer tenable and so needs a complete overhaul. Any honest appraisal will quickly come to the brutal truth that the current system is failing, and so needs to be rebuilt from the ground up. In the short term, we will need to move toward something like a system of “Flash tweets:” site-wide notifications, that simultaneously tweet, post, alert, offer a donation system, etc to news sites when major events transpire. I’d like to see a system like this fleshed out by the social juggernauts as they further flex their news muscles.

Paul Armstrong is founder of  Digital Orange Consulting. Contact him via paularmstrong.net, or on Twitter @paul__armstrong or @TheMediaIsDying.

Have an idea for a post you’d like to contribute to GigaOm? Click here for our guidelines and contact info.

Photo courtesy Edward Meyer.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• The 2013 task management tools market
• How consumer media will change in 2013
• Examining the rise of crowd labor platforms in 2012

I recently met up with my friend and one-time business partner, Steve Lee, who is product director on the Google Glass project, and before that, ran product management on Google Maps for Mobile. Other than a quick tour of the device, Steve basically let me dive in, so as to experience Glass with a beginner’s mind. I won’t bother reviewing the basic capabilities and specs, which have been covered exhaustively already. Instead I want to focus on some of the points that are in debate, and whether I believe that Glass is destined to succeed.

Glass is translucent; designed to be invisible

In “Waves of Power,” David Moschella shows how new disruptive industries begin as verticals, since the complete product solution requires one provider to deliver the whole enchilada. The new industry continues on this path until the solutions finally reach the “good enough” stage, when the larger trend becomes horizontal orientation, so as to achieve ubiquity, commoditization and the broadest possible ecosystem. (In passing, one can see the battle between Apple’s iOS and Google’s Android in this light.) The endgame, so to speak, is that the technology becomes persistent, embedded and ever-present to the point of being “invisible.”

It’s a paradoxical concept to be sure. On the one hand, the technology is everywhere; how can it be invisible? On the other, it’s because it’s everywhere that we no longer think about it as exceptional – and, equally, grand solutions can anticipate and incorporate its ever-presence.

Take for instance the evolution of social mores around cellphones. Every day on my morning bus ride to work, virtually everyone is peering into some device, immersed in another world – a concept that once would have been considered rude and shocking. Similarly, I recently endured a ride near a phone-yapping lawyer who was advising a prospective client on their legal rights – casually and unconcerned, within full earshot of others. This is the new normal.

I think that in the not very distant future, the new forms of interactions that come from using Google Glass – or a very close version of them – will not only be accepted, but commonplace. Google Glass is going to be the NEW, new normal.

Designing a new kind of native experience

To further the point, many have suggested that wearing Google Glass out in public will carry a negative stigma, implying rudeness at a minimum, and privacy invasion at worst. My gut tells me that those people are flat out wrong for two reasons. One, that particular cow has already left the barn (my morning bus ride is emblematic of this truth.)

Two, Google got the design ethos exactly right. It’s a device that is designed for everyday use, but also an adornment that is designed to look good when worn as an accessory. For instance, I never post pictures of myself in my articles, yet I specifically wanted to post a picture of myself wearing Glass:

Why? I think it looks good in the same way a merino wool Zegna sweater looks good.

That in itself is a key narrative: Google has taken the ultimate in geekery and made it feel cool.

The hard technical problems solved

In the age of mobility, connectivity and apps, native experiences will flower and bloom prodigiously. Seen in this light, Google Glass is a credible new flower, growing a little bit every day. So is it ready for prime time? In the continuum from alpha to beta to mass-consumer ready, I’d call it a pretty advanced beta.

The bottom line is that it’s clear Google has solved the hard technical problems, the way they think about the complete solution is well thought out, and I can see a clear segmentation path for how they will take this to market.

As such, if you believe that using your voice, simplified touch actions and augmented visuals is a logical native modality for being social, creative, curious or communicative, then Glass is worth a look.

That brings me to the screen, which is neither obtrusive nor ineffective. It’s there when you need it, and it works. That’s analogous to being embedded to the point of invisibility – until, you have a native moment, and then Glass is at the ready. That in itself is a triumph.  Moreover, its voice-directed interface, interaction with smartphones (for 3G service) and touch controls are mightily impressive.

What is a bit pedestrian, though, is the experiential richness of the actual services that you can access through the system’s card like screens – both Google’s and third party ones. For the device to evolve from missionary to mission-critical, this is the area needing the greatest improvement (although, to be fair, we are at the earliest of days of Glass as a developer platform).

The road to mainstream

For me, the key variables start with pricing. The Explorer release is $1500, which obviously targets a very select niche. I can easily see such a device going for $600-800, since there is no carrier subsidy to lean on. Positioned as a fashion accessory at that price point, Glass should grab a Louis Vuitton-esque slice of the market. That’s single-digit millions of units annually. It’s not until such a device gets to $300 or less when one can expect tens of millions of devices selling annually. But in a five-year horizon, that scenario is not hard to see playing out.

I haven’t yet decided if Glass is a device that I would use everyday all the time, or on spot occasions. Then again, who says I need to? This is more about viability and heartbeat, and the fact that there are lots of jobs for such a device in personal, interpersonal, and industry vertical job categories.

On this front, my eyes don’t lie.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• What the Google-Motorola deal means for Android, Microsoft and the mobile industry
• Bluetooth to Feel Blue as Personal Area Network Battles Loom
• Call it Real-Time, Squared, or NewNet, The Web Is Changing

I feel like we are on the cusp of a similarly fundamental shift in business software. Once again, the change is about rethinking business applications, but this time it is with a mobile lens. When talking with companies that don’t have a specific mobile strategy, I keep hearing about how mobile is just a feature of cloud-based applications. Yet when we meet with entrepreneurs who are building “mobile-first” business apps, we can see a completely different way of thinking: Leveraging the unique capabilities of mobile devices is at the core of every decision they make.

At Emergence, we have put together a first cut at defining the emerging mobile business app landscape based on conversations with over 100 early stage companies. Here’s a look at our findings:

Vertical apps: field-based, on the go

Vertical apps are defined by their singular focus on industries that have a substantial cohort of non-desk workers. Real Estate is a great example. The industry has been notoriously slow to adopt new technology, but in the last two years there has been an explosion in mobile-first technology solutions. For example, Cartavi’s mobile solution for real estate transactions gives agents, buyers, sellers, and related parties (mortgage banks, title companies, etc.) the ability to store, access, update and share transaction documents in the cloud.

On the construction side of real estate, PlanGrid is eliminating reams of paper-based blueprints and change orders with a rich mobile app built specifically for the iPad. Given the field-based nature of the work and its traditional reliance on paper-based solutions, it makes sense that real estate is one of the most crowded boxes on the Emergence mobile business app landscape.

Healthcare and education are other verticals that are particularly well-suited to mobile business solutions. In these sectors, most mobile business apps are enabling entirely new forms of communication, rather than replacing paper-based solutions. Doximity offers a professional networking solution for physicians that hasn’t existed before (Disclosure: The author’s firm, Emergence Capital Partners, is an investor in Doximity). More than 160,000 doctors have joined the network and over 75 percent of their engagement is through mobile devices.

Other vertical categories including restaurants, transportation and hospitality have begun to attract attention, and we predict that we will soon see more apps in the manufacturing, retail and agribusiness verticals, given the “always on their feet” nature of the workers in these sectors.

Horizontal apps: innovating on existing services

Looking at horizontal apps, most start-ups emerge in three sub-sectors: productivity; mobile marketplaces; and sales, marketing and services. A defining characteristic of the companies in the productivity space is their focus on leveraging the native integration of email, desk and calendar in mobile devices. For example, Tylr Mobile has recently launched a new mobile app for salespeople called Workinbox that directly connects salesforce.com data with email. Another young company, Cloudmagic, has developed mobile search capability that allows users to find names, documents, or other files stored in any of your cloud-based applications (e.g. email, calendars, Box, Evernote).

In the mobile marketspace sector, many business apps share a heavy reliance of location-based services. There are several promising car service companies such as Uber, Flywheel, Hailo, and Taxi Magic that enable drivers to find passengers through a marketspace leveraging GPS data. Another interesting company, Quri, has developed a mobile platform that uses both GPS data and mobile cameras to crowdsource in-store intelligence for consumer brands.

Within the sales, marketing & service, ServiceMax, has developed a mobile solution that helps companies manage the entire field service process including scheduling, parts and contracts (Disclosure: The author’s firm, Emergence Capital Partners, is an investor in ServiceMax). The company sells into over 20 verticals, and its bookings increased over 150 percent relative to last year. As we think about the other horizontal sub-sectors, some companies that stand-out have found innovative ways to use mobile cameras (Expensify, Flint), alerts (PagerDuty), and even the headset jack (Square).

Unsolved challenges remain

At Emergence, we feel optimistic about the tremendous growth in mobile business applications that we have seen over the last 12 months. However, like all nascent categories, there are many unsolved challenges. Distribution issues come up during every conversation we have with entrepreneurs. All mobile business app entrepreneurs want to figure out how to leverage mobile app stores more effectively, but most struggle to do so.

In addition, founders are working through the question of whether to native-build mobile business apps or to use a browser-based approach. To date, I have found that native apps tend to provide a better user experience. Further, business users often like access to their applications even when they don’t have connectivity – which makes native apps a better choice today. However, developing and maintaining native apps across multiple platforms is challenging and costly. I am hopeful that the evolution of HTML5 will help entrepreneurs achieve the benefits of both.

As we look ahead, we know the mobile business app landscape will continue to evolve. We realize that we have likely missed some great companies, so please let us know if we missed you. Or if you are thinking about other ways to categorize the companies, it would be great to share ideas.

Kevin Spain is a General Partner at Emergence Capital Partners, a Silicon Valley venture capital firm focused on Enterprise SaaS. He sits on the board of several companies, including Doximity, Viglink, and Welltok.

Have an idea for a post you’d like to contribute to GigaOm? Click here for our guidelines and contact info.

Photo courtesy Praisaeng/Shutterstock.com.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Bluetooth to Feel Blue as Personal Area Network Battles Loom
• How HR can make the case for workforce analytics
• Podcast: Mobile winners and losers in 2012 and what to expect in 2013

To get ahead of these growing demands we will need a smart grid that interconnects power plants and natural energy sources with consumers to intelligently optimize the consumption and distribution of power.

Data centers will become hubs

In order to take full advantage of innovations in smart grid technology, we must first transition data centers into the center of our power and computing activities to monitor and maximize efficiency and plan future energy and cooling sources – all while keeping an increasingly centralized society working, connecting, living and communicating.

Historically, communities were built around religious structures, such as the town church, which brought people together and became an irreplaceable pillar of the community. Today, we are experiencing a similar phenomenon wherein the data center functions as that same pillar but for an online community of enterprises and consumers, positioned at the heart of the digital economy. This will become even more pronounced in the near future, as researchers predict that nearly 75 percent of the world’s population will live in cities by the year 2050.

Efficient distribution of greener power

Many countries are already making strides to develop smart grids within a given region. The primary benefit of smart grid technology is that it allows power from, say, wind farms in Spain, to be distributed to consumers in Denmark with very little energy lost in transport and at a manageable cost. That way, areas that may not have strong natural resources to generate sustainable energy aren’t forced to rely on coal or natural gas to power their infrastructure. This will become an even more important structure as data centers proliferate to match growing global compute and power demands.

Extending the smart grid’s role

By positioning data centers as the epicenter of everything high-tech, we can extend the scope and role of the smart grid beyond efficient structures and smart power distribution. The current concept of the smart grid ensures that buildings are running as efficiently as possible as standalone structures, but also in combination with a green energy source that is distributed via a smart power infrastructure. By constantly measuring and planning future energy sources that require very little energy to generate and have little impact on the environment, data center operators could power their facilities with significantly fewer resources.

Further, by adding an autonomic planning element to the smart grid that can track energy availability and usage throughout the day, operators would be able to rely on the smart grid to determine when to switch between solar, wind or tidal energy, to name a few. This way operators can rely on the smart grid to determine the best time of day to use different types of power, streamlining the switch from one to another and contributing to a more sustainable environment by using clean energy sources.

For instance, if wind farms create the most energy in the late afternoon whereas solar panels pointed east are most useful in the morning, the smart grid can automatically switch from one power source to the other when it is the most viable and beneficial. With this level of intelligence incorporated into data centers and the smart grid linking them with one another, it’s possible that we could support continuous compute growth over many decades.

The need for a global initiative

While this may seem like a radical concept in territories like the United States or India (where the current electrical grids appear to be duct-taped together rather than planned), Europe is a great example of a region that has made significant progress to unite its energy sources and consumers. New resources are being discovered and explored regularly, offering additional juice to power the growing European smart grid, and consumers in large cities like London, Paris and Madrid are already able to benefit from energy gathered from turbines in the ocean and wind farms in the countryside.

In order to achieve true energy efficiency that goes beyond sustainable power sources, though, this must be a global initiative that is collaborated on by universities, governments, non-profits, corporations and even individuals. With global cooperation, we could see solar panels in China powering nightlights in New York City, orchestrated by a network of data centers strategically positioned around the globe.

Lex Coors is vice president, data center technology and engineering group and chief engineering officer for Interxion. 

Have an idea for a post you’d like to contribute to GigaOm? Click here for our guidelines and contact info.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Smart Grid Apps: Six Trends That Will Shape Grid Evolution
• Report: An Open Source Smart Grid Primer
• Mobility on Demand Takes Aim at Transport Networks’ “Last Mile”

Yes. So would alchemy, but neither exist. Inbox Zero, we discovered, is a mirage. My team and I learned a lot in the process about what contributes to real productivity gains. They informed our decisions about product design but they aren’t tied to any specific technology choice: We think these insights apply universally.

Inbox Zero: The myth

You could always open your email client, select all, hit delete and, in theory at least, experience a moment of peace. Inevitably though it would be followed by panic, because you know there were important things in that pile you just vaporized: all those opportunities, responsibilities, duties and interactions in your work and personal life now left unresolved.

The alternative, which most of us struggle with every day, is keeping up with an inbox in a uniform, disciplined way. Yes, you can take an opportunity out of your inbox or your brain and write it down (as David Allen suggests), but the real problem is that, so long as opportunities exist or work is in progress, your backlog of to-dos will always be greater than zero, no matter how you track them, define them, or how quickly you complete them. Your subconscious is always keenly aware of it. Even with delegation, you can pass a hot potato to someone else to move it forward, but it always comes back, often hotter the second time around.

Peace of mind

The first step in achieving peace of mind is facing — and accepting — the fact that we will never be “done.” Our perfect inboxes are fleeting. Our task lists will never be empty. The Twittersphere will keep fluttering. And that’s a good thing, believe it or not. If you admit and surrender, you can embrace the world as your oyster. Suddenly, you’ve entered a place of endless possibility for learning and doing.

How, then, does one find peace? Here are three things that we learned in our journey.

Know what you MUST do

Not what you should do, or want to do, or what someone else wants you to do, but what you MUST do to be able to live with yourself, according to your own goals and ambitions and higher purpose. That may involve keeping your job. It may not. It may involve helping a friend, or doing a stranger a favor. Each moment is a value judgment, and only yours to make.

In that moment, seek clarity. We’ve found through both empirical and anecdotal research that identifying MUSTs and organizing around them is the ultimate secret to most successes.

Make sure the list is complete.

We’ve learned that incompleteness is what hurts people’s productivity most. It’s about more than just missing a deadline, however. Not having MUSTs captured in a trusted system leads to difficulty focusing on tasks at hand. Our subconscious interrupts and forces a context switch.

So if there’s paper in folders, voicemails, texts, meeting notes, scribbles on stickies, etc. that represent a MUST, you MUST go find them. Granted, there will always be surprises, even some that you MUST deal with. But incompleteness is nothing more than human error – your error – and is completely avoidable.

Make a plan.

In our company book club, we came across an interesting explanation for the subconscious interruption called the Zeigarnik effect. The gist is that when an objective is incomplete and no plan is in place, the mind interrupts itself with intrusive thoughts until the situation is remedied.

MUSTs take blocks of time and heavy mental energy – they won’t fit into small gaps in between emails. So placing MUSTs in the context of time helps focus, and more importantly, makes sure those MUSTs actually get done. (But don’t go too far out; over-scheduling tasks has risks too. Commitments with ourselves are the easiest to break, after all.) The key seems to lie in these steps: scripting just today where visibility is best, leaving standing whitespace in your calendar most days, and having a complete accompanying list that gets consistent review and prioritization.

With your MUSTs done, booked, and queued up for review, your mental space is freed to focus on the payoff. More days can be good, even great. Temptations still strike during booked MUST windows and we, too, often succumb. It is a journey for us and for us all, but one we’re making progress on by the week.

Shawn is co-founder and CEO of Handle, a priority engine for making people more effective, and runner-up TC Disrupt NY 2013. He is also Managing Director at Menlo Ventures, where he led the first investment in Siri, and currently sits on the Boards of IMVU, PlayPhone, Roku, Telenav and YuMe. 

Have an idea for a post you’d like to contribute to GigaOm? Click here for our guidelines and contact info.

Photo courtesy of  REATISTA/Shutterstock.com.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• GigaOM Research highs and lows from CES 2013
• How HR can make the case for workforce analytics
• The 2013 task management tools market

Well, we may have gotten a preview of that recently with Jawbone’s acquisition of BodyMedia (Disclosure: the author’s company, Comcast Ventures, is an investor in BodyMedia). Coming on the heels of Zeo closing the doors on its innovative sleep tracking device, is it possible the pendulum is already swinging from proliferation and expansion to consolidation in just four short months since CES?

I think it’s still too soon to make that call, but clearly market leaders like Jawbone are looking forward, identifying what still needs to be solved in this category, and acting strategically. It’s going to be a land grab for the right talent, intellectual property and data that can help.

Below are three key components that companies in the wearable technology segment still need to address in order to produce positive results for today’s consumers and stay relevant:

1. Accuracy and efficacy: The core sensing technology for many of these products is a three-axis accelerometer of the same sort found in our smartphones. A lot can be inferred from one sensor, but having a multitude of sensors sharpens the accuracy of what we think the body is doing. That’s why we are seeing additional sensors such as heart rate, skin temperature, galvanic skin response and heat flux becoming more prevalent in these devices. It’s also critical to appreciate the increased order of magnitude in difficulty when going from one sensor to multiple sensors.  Sensor fusion is the future, but sensor fusion is hard.  Only true data scientists need apply.When dealing with personal health, “close enough” is not good enough, and this class of product could be leading its consumers astray. The healthcare industry is accustomed to conducting clinical studies and publishing research findings. While that model is perhaps a little heavyweight for fitness trackers, there is still room for greater transparency on the accuracy and efficacy of these devices. Simply put, accuracy shouldn’t be in the eye of the beholder, and can only be achieved by benchmarking against voluminous amounts of data over a sustained period of time. In order to establish and justify widespread consumer trust, there needs to be independent analysis against gold standards, shared in a public setting.
2. The right form factor for the occasion: At CES 2013, we witnessed new wristbands from Fitbit and Fitbug (Disclosure: see below), new forearm bands from Scosche and Wahoo, new clip-on units from Withings and GeoPalz, and new watches from Basis and Mio. The technology has also moved to T-shirts, headbands, hats and shoes and BodyMedia showed off a more jewelry-like form factor. Clearly there’s a lot of experimentation with form factor going on, and the question remains whether one dominant type will emerge that rules them all, or if instead manufacturers evolve to a product line strategy with a SKU that accompanies each corresponding daily activity.We’re accustomed to changing our clothes during the course of the day, so some would argue it’s not unreasonable to think we may some day adopt the habit of change our assemblage of devices, too. But a strong case can be made too that passive tracking may be the preferred long-term model over premeditated, active tracking – we’ve all witnessed too many New Year’s resolutions fall by the wayside to assume otherwise.
3. Coaching and counseling: At its core, all these devices encourage behavioral change in an individual to lead a more active, healthy lifestyle. Any weight loss or fitness professional will tell you that maintaining a behavioral change is extremely difficult and so when left as a solo endeavor, the chances of long-term success are less than promising.Thus several companies have wisely paired their products with fitness services to increase consumers’ chances for success: Philips Active Link is available to Weight Watchers members, BodyMedia is a long-standing partner with 24 Hour Fitness, and Fitbit collaborates with Retrofit. Incorporating the use of these devices in a larger, more holistic program makes a lot of sense and may become the predominant model of engagement in the future. For sure, the data scientists can make the data dashboards coming off of these apps more intuitive, more actionable, more DIY, but the successful system of tomorrow may just need to expand its scope into “services” such as coaching and counseling.

We are experiencing more consumerization of healthcare and this class of technologies engenders more knowledge and more transparency. For this category to have the long lasting affect we all hope for, however, the technology needs to evolve beyond simply measuring calories burned or steps walked. The market winners and losers are starting to take shape.

Disclosure: Fitbit is backed by True Ventures, a venture capital firm that is an investor in the parent company of this blog, Giga Omni Media. Om Malik, founder of Giga Omni Media, is also a venture partner at True.

Michael Yang is Managing Director of Comcast Ventures. 

Have an idea for a post you’d like to contribute to GigaOm? Click here for our guidelines and contact info.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Connected world: the consumer technology revolution
• The future of mobile: a segment analysis by GigaOM Pro
• What the Google-Motorola deal means for Android, Microsoft and the mobile industry

But the fact remains that big data can only ever be as good as the machine learning that is used to provide insight, and even the most sophisticated machine learning techniques aren’t omniscient – the old adage “garbage in, garbage out” sums up this dilemma perfectly. Businesses planning to invest in big data science, with the hopes of reaping the potential wealth of insights available, must at all costs avoid introducing bias into the process – or risk jeopardizing everything.

Data bias syndrome

Data bias comes in many forms. It can come from poorly defined business domain objectives. Or, it can come from opting to gather data that are easy to collect rather than data that are most informative. Data scientists can also receive data that have been biased by incorrect assumptions by the domain experts. (And as a footnote, the recent example of the austerity economics Excel scandal shows how a minute data error can have cascading and devastating effects.)

Likewise, data scientists themselves are not immune to bias. Some can run afoul of their own preconceived notions about business domain – too much knowledge can cause one to filter out data that may actually be helpful.  Scientists with deep experience in a particular data set may develop too much reliance on pre-existing algorithms without re-examining validity for a particular use case.

Finally, data quantity is a common problem. Intelligent learning requires abundant data, and often the data available are not sufficient to draw accurate conclusions – a problem known as data sparsity. This may sound unbelievable considering that data volume is doubling every two years according to an EMC study,  but there’s a difference between a dense data set populated by similar data points, and the far more diverse sets of user data points we find in the real world. In these cases, the gaps in the data are filled by machine learning algorithms that may inherently be biased, based on assumptions made by the data scientist when designing the algorithm. The trick is to find the right balance between unbiased data exploration and data exploitation.

Removing bias

As companies bring data science in-house or purchase tools that act as a data abstraction layer, the need to address data bias becomes more immediate. The smart move is to build bias-quelling tactics into the data science process itself. Here’s how:

• Employ domain experts Rely on them to help select relevant data and explore which features, inputs and outputs produce the best results. If heuristics are used to gain insights into smaller data sets, the data scientist will work with the domain expert to test the heuristics and ensure they actually produce better results. Like a pitcher and catcher in a baseball game, they are on the same team, with the same goal, but each brings different skill sets to complementary roles.
• Look for white spaces  Data scientists who work with one data set for periods of time risk complacency, making it easier to introduce bias that reinforces preconceived notions. Don’t settle for what you have; instead, look for the “white spaces” in your data sets and search for alternate sources to supplement “sparse data.”
• Open a feedback loop This will help data scientists react to changing business requirements with modified models that can be accurately applied to the new business conditions. Applying Lean Startup like continuous delivery methodologies to your big data approach will help you keep your model fresh.
• Encourage your data scientists to explore.  If you can afford your own team of data scientists, be sure they have the space and autonomy to explore freely. Some equate big data to the solar system, so get out there and explore this uncharted universe!

Whatever you do, don’t ignore the issue: The last thing you want to do is implement a system that develops and propagates data, only to learn it’s hopelessly biased. If you don’t solve this problem sooner rather than later, your organization will miss out on what many analysts are calling the next frontier for innovation.

Haowen Chan is currently a principal scientist at Baynote,  a provider of personalization solutions for online retailers. Robin D. Morris is a senior data scientist at Baynote; he is also associate adjunct professor in the Department of Applied Math and Statistics at the University of California, Santa Cruz.

Have an idea for a post you’d like to contribute to GigaOm? Click here for our guidelines and contact info.

Photo courtesy pzAxe/Shutterstock.com.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Listening platforms: finding the value in social media data
• The importance of putting the U and I in visualization
• A near-term outlook for big data

Like a bad episode of Hoarders, people love to store all things digital, most of which will never be accessed again. And, like a bad episode of Storage Wars, our love of storing crap means we need more places to store it. Today’s content has outgrown even the hydro-electric dam powered Mega Data Centers built just yesteryear. Increasingly, operators are turning to distributing their information across multiple geographically dispersed data centers. As the number, size, and distances between the data centers have steadily grown, timing distribution and accuracy has likewise grown in importance in keeping the data centers in sync.

In a previous article I discussed new standards being developed to increase the accuracy of timing for the internet and other IP-based networks. Current systems and protocols offer milliseconds of accuracy. But that just isn’t enough as we depend more on real-time information and compute, storage and communications networks become more distributed. While people often cite the importance of timing on mobile backhaul networks in the next-genration LTE-Advanced networks,there has been less publicity around the need for these new timing technologies in the continued growth of data centers.

The rise of Hadoop in an age of digital garbage

Massive storage of data appears to occur in periods, very analogous to dinosaur evolution. A database architecture will rise to the forefront, based upon its advantages, until it scales to the breaking point and is completely superseded by a new architecture. At first, databases were simply serial listed values with row/column arrangements. Database technology leapt forward and became a self-sufficient business with the advent of relational databases. It appeared for a while relational databases would be the end word in information storage, but then came Web 2.0, social media, and the cloud. Enter Hadoop.

A centralized database works, as the name suggests, by having all the data located in a single indexed repository with massive computational power to run operations on it. But a centralized database cannot hope to scale to the size needed by today’s cloud apps. Even if it could, the time needed to perform a single lookup would be unbearable to an end user at a browser window.

Hadoop de-centralizes the storage and lookup, as well as computational power. There is no index, per se. Content is distributed across a wide array of servers, each with their own storage and CPU’s, and the location and relation of each piece of data mapped. When a lookup occurs, the map is read, and all the pieces of information are fetched and pieced together again. The main benefit of Hadoop is scalability. To grow a database (and computational power), you simply keep adding servers and growing your map.

Even Hadoop is buried under mounds of digital debris

It looked like Hadoop would reign supreme for generations to come, with extensions continuously breathing new life into the protocol. Yet, after only a decade, databases based upon Hadoop such as Facebook are at the breaking point. Global traffic is growing beyond exponential, and most of it is trash. Today’s databases look more like landfills than the great Jedi Archives. And recently hyped trends such as lifelogging suggest the problem will get much worse long before it gets better.

The main limitation of Hadoop is that it works great within the walls of a single massive data center, but is less than stellar once that database outgrows the walls of a single data center and has to be run across geographically separated databases. It turns out the main strength of Hadoop is also its Achilles heel. With no index to search, every piece of data must be sorted through, a difficult proposition once databases stretch across the globe. A piece of retrieved data might be stale by the time it reaches a requester, or mirrored copies of data might conflict with one another.

Enter an idea keep widely dispersed data centers in sync — Google True Time. To grossly oversimplify the concept, True Time API adds time attributes to data being stored, not just for expiration dating, but also so that all the geographically disparate data centers’ content can be time aligned. For database aficionados, this is sacrilegious, as all leading database protocols are specifically designed to ignore time to prevent conflicts and confusion. Google True Time completely turns the concept of data storage inside out.

Introducing Spanner

In True Time, knowing the accurate “age” of each piece of information, in other words where it falls on the timeline of data, allows data centers that may be 100ms apart to synchronize not just the values stored in memory locations, but the timeline of values in memory locations. In order for this to work, Google maintains an accurate “global wall-clock time” across their entire global Spanner network.

Transactions that write are time stamped and use strict two phase locking (S2PL) to manage access. The commit order is always the timestamp order. Both commit and timestamp orders respect global wall-clock time. This simple set of rules maintains coordination between databases all over the world.

However, there is an element of uncertainty introduced into each data field, the very reason that time has been shunned in database protocols since the dawn of the data itself.

Google calls this “network-induced uncertainty”, denoted with an epsilon, and actively monitors and tracks this metric. As of summer 2012, this value was running 10ms for 99.9 percent (3 nines) certainty. Google’s long term goal is to reduce this below 1ms. Accomplishing this will require a state of the art timing distribution network, leveraging the same technologies being developed and deployed for 4G LTE backhaul networks.

A modest proposal

While True Time was most likely developed to improve geographic load balancing, now that accurate time stamping of data exists, the possibilities are profound. The problems associated with large databases go beyond simply managing the data. The growth rate itself is unsustainable. Data storage providers must do more than grow their storage, they must also come up with ways to improve efficiencies and ebb the tsunami of waste that is common in the age of relatively free storage.

It’s a dangerous notion, one simply must challenge the basic tenet that all data is forever. Our minds don’t work that way, why should computers? We only hold on to key memories, and the further the time from an event, the fewer the details are held. Perhaps data storage could work similarly. Rather than delete a picture that hasn’t been accessed in a while, a search is performed for similar photos and then only one kept. And as time passes, perhaps rather than simple deletion, a photo is continuously compressed, with less information kept, until the photo memory fades into oblivion. Like that old Polaroid hung on the refrigerator door.

Jim Theodoras is director of technical marketing at ADVA Optical Networking, working on Optical+Ethernet transport products.

Dinosaur image courtesy of Flickr user Denise Chen.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A near-term outlook for big data
• Dissecting the data: 5 issues for our digital future
• 9 Companies that Pushed the Infrastructure Discussion in 2010