20 Comments

Summary:

The massive amount of data that is emerging from connected, digital systems, is fundamentally changing everything, from Internet search to entertainment, to disease management, to energy consumption. Here’s 10 case studies that highlight the power of big data.

feature-waterfall

How Twitter data-tracked cholera in Haiti

By Mathew Ingram

Sifting through the massive amounts of information that flow through the Twitter network is no easy task, since more than 250 million tweets are posted every day, according to a recent estimate from the company. But within that stream are some valuable pieces of information — data that could be used to track the spread of disease, for example, and more accurately identify its victims. A recent study by medical researchers at Harvard showed that Twitter was substantially faster at tracking the spread of cholera in Haiti following the earthquake in 2010 than any traditional diagnostic methods.

In fact, the study (PDF), which was authored by Dr. Rumi Chunara, a research fellow at Harvard Medical School who also works with the online data-oriented HealthMap project, showed that by using information from Twitter, researchers were able to pinpoint outbreaks of the deadly disease more than two weeks before they were identified by traditional methods. The study was released in January, on the second anniversary of the Haitian earthquake.

In an interview with GigaOM, Chunara said she and the other researchers got the idea for the study after noticing a lot of cholera-related messages flowing through Twitter in the aftermath of the quake. These tweets were highlighted by the social media–tracking service Ushahidi, a platform used by aid agencies, government workers and others during events like the Haiti quake as a way to track victims and other incidents. Other studies have shown that Ushahidi and similar tools can be a much more effective means of communication in such circumstances than official channels.

“We noticed that the volume of tweets over time correlated quite strongly with the official reports of cholera in Haiti, so we decided to take a closer look,” Chunara said. The researchers collected and scanned 4,697 reports via the HealthMap service, along with almost 200,000 individual tweets, which she said was “quite a large data set compared with similar studies of this kind.” The point of the research was to show that information from real-time sources like Twitter and other social tools could be an effective supplement to official methods of locating and diagnosing outbreaks like cholera.

“Official case reports have to get verified by hospitals, so it often takes a couple of weeks for that information to be posted and available to health workers,” said Chunara. “Informal sources like Twitter are obviously much more real-time.” The study concluded that by analyzing Twitter data, researchers would not only have been able to pinpoint the location of cholera cases but could also have determined the reproductive rate of the outbreak more quickly, something that can be a crucial element for health workers in stemming the spread of an infectious disease.

Armed with data from the study, Chunara said health workers and nongovernmental agencies would theoretically be able to apply these principles in future disasters and outbreaks and get a faster reading of where to focus their efforts. “The use of social media is growing so quickly, and provides so much interesting data, that it behooves us to figure out how to use these kinds of tools for research,” she said.

What the Harvard and HealthMap study shows is that analyzing the data from large sets like the tweets around Haiti isn’t just good at tracking patterns or seeing connections after an event has occurred, but can actually be of use to researchers on the ground while those events are underway. Other projects aimed at Twitter data have tried to isolate stock-related activity or predict purchasing behavior, but the HealthMap research is one of the first to show that mining social networks could have a real — and real-time — impact on health and social welfare.

You’re subscribed! If you like, you can update your settings

firstpage of 11
  1. Reblogged this on Dots Of Color and commented:
    Big data big money!

    1. I don’t get it, what does Big Data have to do with a video card…or is this some lamesauce ad post?

    2. The emergence of this so-called big data phenomenon is also fundamentally changing everything from the way companies operate

      1. Yes, it does. Who controls the most data wins. At least Facebook would like to think so. ;-)

  2. Is gigabytes bytes more then a gigabyte?

    1. Katie Fehrenbacher gil Monday, March 12, 2012

      nope just a typo, fixed that, thanks!

      1. Typos happen.
        Gil’s “more thEn a gigabyte” is just plain ignorant.

    2. Grammar Police gil Thursday, March 15, 2012

      If you’re going to complain about a typo, make sure you don’t have any in your immature comment. When you have full mastery of the language, then you’ll be allowed to comment.

  3. infotech ideas Monday, March 12, 2012

    Great info! Bring the expo to SFO as well!

  4. remedy2020@gmail.com Monday, March 12, 2012

    Advertorial ! Advertorial! so fast you sold your soul!

  5. DataStax, more specifically Cassandra, can solve all big data problems.
    http://cassandra.apache.org/
    And its open sourced.

  6. SAP HANA to the rescue!

  7. Reblogged this on <i>cu Lì!</i> and commented:
    great info :)

  8. why do we have to click through so many pages. can you at least provide a way to read it in a single page? (like businessinsider) there is not even a print option and it doesn’t work with readability. i thought more of gigaom. disappointed.

  9. idiots…
    «“We want to unlock the black box of how an artist becomes a star,” White said»
    what makes the charts is good music, not $$$$$ pumped into it 8-X
    just like m$$$$$ can keep wasting billion$ on WP trying to make it a success, it won’t work. its crap, it doesn’t sell
    period

  10. Steven Brown Monday, March 19, 2012

    Big Data is a tactical problem. Content and Business Analytics and Intelligence is the logistical problem. One must pay particular attention to Business Process Models, Entity-Relationship Models, and Data Modeling to be able to use ETL and Data Integration Technologies for developing your data storage organization, retrieval, formatting, clustering, Web Caching; and backup, recovery, and archival retention strategies.

  11. Edwin Ritter Wednesday, May 9, 2012

    Reblogged this on Ritter's Ruminations & Ramblings and commented:
    As 2012 reaches the half way mark, here is a quick view on how this hot topic. This is the first of three. Posts on the other trends will follow.

    So ‘big data’ is a hot topic. What is it? Simply stated, everything you do on the web is tracked and creates data. So much data is collected that 90% of the online data was created in just the last two years. This data is stored, sliced, diced and analyzed. The growth in data is due to several things such as proliferation of smart phones and tablets, lower storage costs and improved analytical tools. This article reveals 10 ways in which big data will have an impact.

  12. Mission Impossible Wednesday, May 30, 2012

    Thank you for the thoughts on this so far.
    The kate broadwell

  13. Mission Impossible Thursday, May 31, 2012

    Thanks instead of the article. Blogging is replacing main onslaught news for various black people.
    The vpn port

Comments have been disabled for this post