The massive amount of data that is emerging from connected, digital systems, is fundamentally changing everything, from Internet search to entertainment, to disease management, to energy consumption. Here’s 10 case studies that highlight the power of big data.


Big data is now your company’s virtual assistant

By Bobbie Johnson

Big data is empowering an entire generation of smart businesses to gain fresh insights into their customers and build new products. But it can also revolutionize the way a company looks at itself, too.

That’s the premise of Autopilot, the flagship product from Frankfurt-based automation expert Arago. The system — a virtual assistant of sorts — uses a combination of data and artificial intelligence to take over the most boring and repetitive tasks of managing a large IT infrastructure, effectively becoming a new member of a sysadmin team.

After Autopilot is given access to the streams of information being logged by your servers and is taught about common problems your administrators encounter, it can use what it knows to ensure your services run smoothly.

For example, say a monitoring system reports a potential problem: An error is being thrown out by one of the company’s Oracle database nodes, which seems to have a nearly full tablespace. Autopilot picks up the ticket immediately, connects to the server, examines the database connections, retrieves data about the tablespace, and then adjusts it or reorganizes the database storage to resolve the problem. It has taken immediate action and completed its task in seconds, spitting out a full record of what it has done.

“People are used to tools, but this is more like a colleague,” says Chris Boos, the company’s CEO. “It writes tickets, it closes them, it asks you for help.”

The difference between this and other systems is that it is based on algorithms that learn how a problem gets resolved; it’s not just performing workflows that somebody else has input into it. That means it actually makes decisions for itself, interrogating previous data in order to understand and learn what’s happening now and what it can do to help.

Boos admits this prospect can be terrifying for some people who see Autopilot as a threat, and it can seem outlandish to others who struggle to believe it can make a substantial impact.

But the results can be significant. One commercial implementation of the software — which runs across a string of virtual machines at Amazon or VMware and sends data back to Arago’s own cloud-based cluster — achieves a 93 percent level of automation for IT administration.

“That means something previously done with 70 people is now done with 8,” says Boos. “And that allows those people to go and tackle more interesting problems, rather than deal with boring and repetitive tasks.”

But Autopilot is not just allowing Arago to help companies free up their talent from tedium. It is also helping Arago turn big data back on itself.

Now that the system is learning what happens inside large IT deployments, the company is starting to explore what that means, whether that is understanding the common problems faced by particular setups, helping CIOs avoid expensive mistakes, or simply giving customers the chance to benchmark their infrastructure costs.

“Since we collect all of this data on how IT systems run, we’ve started to think about how we can use it,” says Boos. “It turns out we can basically predict how much effort you have to put in to operate an environment, all without knowing anything specific — all we need is a very lightweight model of your system.”

All of this comes despite the fact that Boos doesn’t really think of his business as a big data company.

Sure, it uses Cassandra and Hadoop — what he refers to as “all that big data stuff” — but for him, the secret is in the artificial intelligence. The data is essentially a tool that helps people make better, more informed decisions. But it is a means to an end that can change not only the way you operate but also the products you offer.

You’re subscribed! If you like, you can update your settings

firstpage of 11
  1. Reblogged this on Dots Of Color and commented:
    Big data big money!

    1. I don’t get it, what does Big Data have to do with a video card…or is this some lamesauce ad post?

    2. The emergence of this so-called big data phenomenon is also fundamentally changing everything from the way companies operate

      1. Yes, it does. Who controls the most data wins. At least Facebook would like to think so. ;-)

  2. Is gigabytes bytes more then a gigabyte?

    1. Katie Fehrenbacher gil Monday, March 12, 2012

      nope just a typo, fixed that, thanks!

      1. Typos happen.
        Gil’s “more thEn a gigabyte” is just plain ignorant.

    2. Grammar Police gil Thursday, March 15, 2012

      If you’re going to complain about a typo, make sure you don’t have any in your immature comment. When you have full mastery of the language, then you’ll be allowed to comment.

  3. infotech ideas Monday, March 12, 2012

    Great info! Bring the expo to SFO as well!

  4. remedy2020@gmail.com Monday, March 12, 2012

    Advertorial ! Advertorial! so fast you sold your soul!

  5. DataStax, more specifically Cassandra, can solve all big data problems.
    And its open sourced.

  6. SAP HANA to the rescue!

  7. Reblogged this on <i>cu Lì!</i> and commented:
    great info :)

  8. why do we have to click through so many pages. can you at least provide a way to read it in a single page? (like businessinsider) there is not even a print option and it doesn’t work with readability. i thought more of gigaom. disappointed.

  9. idiots…
    «“We want to unlock the black box of how an artist becomes a star,” White said»
    what makes the charts is good music, not $$$$$ pumped into it 8-X
    just like m$$$$$ can keep wasting billion$ on WP trying to make it a success, it won’t work. its crap, it doesn’t sell

  10. Steven Brown Monday, March 19, 2012

    Big Data is a tactical problem. Content and Business Analytics and Intelligence is the logistical problem. One must pay particular attention to Business Process Models, Entity-Relationship Models, and Data Modeling to be able to use ETL and Data Integration Technologies for developing your data storage organization, retrieval, formatting, clustering, Web Caching; and backup, recovery, and archival retention strategies.

  11. Edwin Ritter Wednesday, May 9, 2012

    Reblogged this on Ritter's Ruminations & Ramblings and commented:
    As 2012 reaches the half way mark, here is a quick view on how this hot topic. This is the first of three. Posts on the other trends will follow.

    So ‘big data’ is a hot topic. What is it? Simply stated, everything you do on the web is tracked and creates data. So much data is collected that 90% of the online data was created in just the last two years. This data is stored, sliced, diced and analyzed. The growth in data is due to several things such as proliferation of smart phones and tablets, lower storage costs and improved analytical tools. This article reveals 10 ways in which big data will have an impact.

  12. Mission Impossible Wednesday, May 30, 2012

    Thank you for the thoughts on this so far.
    The kate broadwell

  13. Mission Impossible Thursday, May 31, 2012

    Thanks instead of the article. Blogging is replacing main onslaught news for various black people.
    The vpn port

Comments have been disabled for this post