Blog Post

The increasingly blurry line between Big Data and Big Brother

Stay on Top of Enterprise Technology Trends

Get updates impacting your industry from our GigaOm Research Community
Join the Community!

The potential benefits of “big data” have been well described, both by us and others: the ability to spot flu trends earlier and potentially save lives, for example, or to make it easier for companies to provide services in a more personalized way. But these same tools could also be used for more disturbing purposes that smack of Orwell’s Big Brother, and two prominent digital skeptics — Nicholas Carr and Evgeny Morozov — recently raised warning flags about that prospect. Which kind of future will we get?

Carr looked at a recent speech from PayPal (s ebay) co-founder Max Levchin at the DLD conference in Germany (one Om also attended, where he conducted an in-depth interview with Levchin) and clearly didn’t like what he saw. Levchin’s view of people, according to Carr, is that they are just resources that are not being utilized efficiently, and the technology of sensors and real-time information can be used to improve that, in much the same way that programmers try to optimize the clock cycles of a microprocessor. To take one example, Levchin said:

“How about dynamic pricing for brain cycles? We have been maximizing utilization of very high-value, very low-frequency specialists — today you can already rent the brain of a data-mining genius via Kaggle by the hour, tomorrow by brain-hour. Just like the [email protected] screensaver ‘steals’ CPU cycles to sift through cosmic radio noise for alien voices, your brain plug firmware will earn you a little extra cash while you sleep, by being remotely programmed to solve hard problems.”

More efficient for users, or just creepy?

data center photo-210x140

If you are a geek, this might sound like something with a lot of potential, but Carr describes it as “Clay Shirky’s ‘cognitive surplus’ idea taken to its logical, fascistic extreme.” Levchin goes on to paint a picture of a future in which his insurance company learns — via sensors in his car — that he is taking his children to work, and boosts his insurance premium by a few dollars for the extra risk (Note: we’ll be talking more about the potential of big data at our Structure:Data conference in New York).

Levchin no doubt sees this as efficient, but Carr sees the looming shadow of Big Brother: What if those same sensors detected that you were overweight, or had eaten too much pizza, he asks — would they report that to your insurance company? Maybe the company would boost your rates a little, or maybe you would be “scheduled for a brief re-education session down at the local office of the Bureau for Internal Resource Optimization.” As he puts it:

“This is the nightmare world of Big Data, where the moment-by-moment behavior of human beings — analog resources — is tracked by sensors and engineered by central authorities to create optimal statistical outcomes. We might dismiss it as a warped science fiction fantasy if it weren’t also the utopian dream of the Max Levchins of the world. They have lots of money and they smell even more.”

In a recent piece for Slate, Carr’s fellow digital skeptic Evgeny Morozov looked at the potential implications of banks and other credit-issuing agencies using big data to determine who deserves a loan. Although he says the idea of big data is “mostly big hype,” Morozov talks about several companies that are trying to use data from all kinds of sources — including social networks such as Facebook (s fb) and Twitter — to figure out who is credit-worthy.

Hong Kong-based Lenddo and U.S.-based LendUp look at an applicant’s connections on Facebook and Twitter, Morozov says, and “the key to getting a successful loan is having a handful of highly trusted individuals in your social networks.” A British payday-loan company called Wonga even considers the time of day and how a user clicks around a website in order to determine whether they deserve a loan (although Morozov doesn’t mention it, PayPal uses similar methods to gauge credit-worthiness).

The key is who controls the use of the information

Big Brother is watching you

Morozov also mentions ZestFinance, founded by former Google chief information officer Doug Merrill (who we had at our Structure: Data conference in New York last year), whose company looks at more than 70,000 signals and 10 different models to assess credit risk. And he draws a direct link between this and Big Brother, saying: “If only East Germany’s Stasi — the true pioneers of ‘big data’ — had the same model for assessing potential dissidents!”

Despite that comment, however, in the end he (somewhat surprisingly) seems concerned mostly that these companies will use all this information to market things to people who don’t need them, rather than turning them in to the government or their insurance company:

“What happens once these firms, having figured out that all data are credit data, realize that all data are also marketing data? Given how much they know about their clients, it would be very hard for such lending companies not to use this information to sell their existing customers on yet another loan or, perhaps, encourage them to use the loan to take advantage of some unique online sales offer.”

The common thread in both of these dystopian visions is a world in which our data is transmitted without our knowledge, and/or used against us in some way. Where Levchin seems to see an efficient exchange of data between user and service, one with benefits for both — and presumably a level (and secure) playing field in terms of who has access to it — Carr and Morozov see companies and governments misusing this data for their own nefarious purposes, while we remain powerless.

What makes it difficult to argue with either one is that we’ve already seen the building blocks of this potential future emerge, whether it’s Facebook playing fast and loose with the privacy settings of a billion people, or companies aggregating information and creating profiles of us and our activities and desires. What happens when the sensor-filled future that Levchin imagines becomes a reality? Who will be in control of all that information?

Images courtesy of Shutterstock / Lightspring and Flickr / Thomas Leuthard.

11 Responses to “The increasingly blurry line between Big Data and Big Brother”

  1. Daniel Comp

    Thanks Matthew.

    I see the same trend emerging with Google, Nasa, Darpa, Singularity University, etc. Bill Joy (Sun Microsystems Co-Founder) talked about the responsibility and our best bet on managing the threats and risks in his TED talk in 2005 or so. And, here we are so much closer.

    I wrote about a similar emergence with the hiring of Ray Kurzweil by Google as Director of Engineering. You can watch excepts of these here:

  2. Spot-on and timely article. As anecdotal (and probably also real?) evidence suggests, skimming twitter can help one to spot houses that can be broken into (because the residents just tweeted about their holiday). Or mobile location data can tell you where one stays, where one works, and who else (from his/her social network) works in the neighborhood etc. Indeed Big data = Big brother!

  3. Jacques T

    I think the most important question is – who really benefits from big data?
    The axis of venture capital, banks, media companies and service providers points to an enormous concentration of power. While most of these player scream about intrusive big government, these private powers are going un-checked with “users” unwittingly buying a digital dream that is likely to become a nightmare.

  4. Martin Bergstrom

    There are some fine lines in this discussion that will inevitably be toed, stretched, and occasionally crossed. I think it really comes down to users and their preferences. A lot of the potential customization and efficiency increases from technology and especially big data are going to come from a loss of privacy and I think it’s the individuals who will ultimately have to decide where they draw the line. Companies are going to have to be conscious of this and provide opt out options to avoid worrying the public and potentially facing regulation in the form of legislation. There are some fine lines, but I think they can be negotiated successfully.

  5. Good one. With all the data, the banks couldn’t even figure out how derivatives worked, their own creation. Or how interlinked and depended they where. Let alone when they would go bankrupt. They still don’t know how complex systems work, like most “big data” proponents

  6. A dystopian future is one view that assumes the worst. I too assume that men will always use technology for bad things. They always have. Yet that doesn’t mean tech stops. It just means that people come to grips with it and then move on to the next thing.

    For me personally, I have come to grips with it all by justifying lying to big brother. Currently, I don’t think any technology can determine whether I am lying or not. If the data they collect is wrong, so too will the conclusions.

    Unless coercion is used to determine whether or not I am lying, I am safe from the rise of the machines.

  7. Ascend Innovations

    Ok, get ready for a counter view and why so-called Big Data is overrated. That’s not to say its not being chased by the investment bulls who see it as the next band-wagon. The point here is that 99%ish of the information collected in Big Data is noise.

    In the past we measured what we could and derived rules based on it. In time these rules were discarded because new tools emerged which allowed new measurements to be made, and so on…….

    We are now at the point when vast amounts of data can be collected and analysed cheaply.

    A few fortunes will be made. However there is much hyperbole, but in the end not much more is going to come out of it. Curious mistakes will be made. The data will be misused.

    Take it from me, we can do a zillion tests and collect all the information, and interrogate it whichever way, but if a patient finds it difficult to walk into a room, they’ll struggle through major surgery. On the other hand, if they walk in, they’ll almost always walk out afterward.

    Instead of mega-crunching TB of data, lets start slimming down and do things smartly.

    From the doctors of