Blog Post

Is Carrier IQ a big data mercenary?

A video posted by an Android(s goog) developer has turned into a scandal that could envelop the whole wireless industry. Since developer Trevor Eckhart first revealed the details of how a mysterious keystroke-logging application created by Carrier IQ tracked every action performed on Android phones, operators, handset vendors and even the almighty Apple(s aapl) have been implicated in the plot. But it’s not just the obvious wireless players that see value in Carrier IQ’s covertly collected data.

Media-measurement company Nielsen (s nlsn) is tapping into that information pipeline as well, which raises the questions of how many other companies may be buying information from Carrier IQ. Providing a carrier with anonymous performance metrics is one thing, but selling compiled customer data to a third-party with no relationship to the customer or the network Carrier IQ is monitoring is another altogether. Carrier IQ claims to be offering a service to the operators to help them optimize their networks, but it may well be a big data mercenary selling information on all kinds of mobile consumer behavior to the highest bidder. It might even be playing both sides.

The story so far

AT&T(s T) and T-Mobile USA are just the latest to admit they received data from Carrier IQ on the behavior of their customers’ smartphones. However, like Sprint (s S), they claimed it used that information solely for network optimization purposes. Verizon is the only major U.S. operator untarnished. Apple copped to installing Carrier IQ’s software on all its iPhones before the release of iOS 5. HTC and Samsung acknowledged implementing Carrier IQ, but only at the behest of their carrier customers, which didn’t prevent them from getting slapped with class action lawsuits.

Lawyers have compared Carrier IQ’s covert digital snooping to illegal wiretapping. Carrier IQ has even attracted the attention of Congress. U.S. Senator Al Franken (D-Minn.), who chairs the Senate subcommittee on Privacy, Technology and the Law, sent a letter Thursday to Carrier IQ President and CEO Larry Lenhart asking some poignant questions about how and for whom Carrier IQ collects its data.

Taking a cue from Franken, let’s ask some of those same questions. Moving beyond what data Carrier IQ is collecting, which has been covered extensively by Eckhart and subsequent stories, lets’ explore why Carrier IQ is collecting information from millions of smartphones and more importantly who its selling that data to.

A bizarre big-data triangle

Based on the patterns of admissions and denials we’re seeing around the industry, as well as some background conversations with some industry sources, it looks like Carrier IQ is two-headed beast: one head being its covert handset software and the other being its measurement and analytics service. Certain handset makers, like HTC and Samsung, are installing the app on many, if not all, of their smartphones at the root layer, but those handsets aren’t necessarily the customers for the analytics service. In fact, both HTC and Samsung deny they receive any of the data collected.

Then who does? AT&T, T-Mobile and Sprint are three, as was Apple, but another is Nielsen. In October, Nielsen signed on as a Carrier IQ partner saying it would use the company’s technology to help “measure the performance of mobile services, networks, and devices” and “gather actionable intelligence on the performance of mobile devices and networks.”

That sounds very much like what both AT&T and Sprint are saying. We reached out to Nielsen to ask what exactly they’re doing with Carrier IQ data. Here’s the email response from VP of Global Communications Marivi Lerdo de Tejada:

“Nielsen and Carrier IQ announced an alliance in October 2011 to explore potential ways to measure mobile services, networks and devices, exclusively using opt-in panels and in accordance with Nielsen’s stringent privacy standards. To date, we continue to explore these opportunities, with neither any work for clients initiated, nor any panels created.”

If Nielsen sticks with its opt-in policies, it won’t get into the trouble the carriers and handset vendors appear to be in, but are there other market research companies that might not be so scrupulous? Carrier IQ has no qualms with selling carriers their own customers’ data without those customers’ permission. Could it sell the data it collects from AT&T, Sprint and T-Mobile’s customers to another market analytics firms. Could it sell AT&T’s data to Sprint and vice versa?

Late Thursday, Carrier IQ broke its silence, putting out a statement explaining what data it collects and what it does with it. Here are some excerpts:

“While a few individuals have identified that there is a great deal of information available to the Carrier IQ software inside the handset, our software does not record, store or transmit the contents of SMS messages, email, photographs, audio or video. For example, we understand whether an SMS was sent accurately, but do not record or transmit the content of the SMS. We know which applications are draining your battery, but do not capture the screen.

“… Carrier IQ acts as an agent for the Operators. Each implementation is different and the diagnostic information actually gathered is determined by our customers – the mobile Operators. Carrier IQ does not gather any other data from devices.”

Carrier IQ claims to count the frequency of actions, not the content of the actions themselves, and say whatever data it sends back to is servers is encrypted and personal information is protected. But Carrier IQ isn’t being quite so honest about who its customers or potential customers are. Right there on its home page, Carrier IQ says it gives handset manufacturers as well as wireless operators “unprecedented insight into their customers’ mobile experience.” Nielsen certainly isn’t a wireless operator.

This isn’t just about a few carriers keeping tabs on their customers. This is big data for the mobile world – massive databases of consumer behavior delving into when, how and in what manner we use our devices. By Carrier IQ’s own admission, its software is embedded in more than 150 million handsets. There are plenty of companies that would find that information enormously useful. The problem is Carrier IQ never got permission from all these smartphone users to collect that data, never told them it was gathering it, and never provided a way of opting out.

Who gave Carrier IQ permission?

Carrier IQ couldn’t just do this on its own, covertly installing rootkit software into millions of phones without anyone’s knowledge. It had to have the cooperation of operators like Sprint and AT&T and of the handset manufacturers that built their devices. HTC and Samsung are pointing fingers directly at the operators. Take the statement circulating from HTC, which we first saw on Bright Side of the News:

“Carrier IQ is required on devices by a number of U.S carriers so if consumers or media have any questions about the practices relating to, or data collected by, Carrier IQ we’d advise them to contact their carrier. It is important to note that HTC is not a customer or partner of Carrier IQ and does not receive data from the application, the company, or carriers that partner with Carrier IQ. HTC is investigating the option to allow consumers to opt-out of data collection by the Carrier IQ application.”

But if the operators are entirely to blame, how is that Nokia(s nok), Google and RIM (s RIM) can claim they don’t use Carrier IQ’s software? It may have been harder to drill down into Nokia and RIMs’ operating system, sure, but Carrier IQ was able to work with Apple to embed its software deep into the iPhone. Plus, if RIM and Nokia can turn down Sprint and AT&T, why can’t HTC and Samsung? It’s not as if Nokia market share is so strong in the U.S. it can casually deny a software customization request from AT&T, one of the world’s largest GSM operators. And though RIM claims to be Carrier IQ-free, that hasn’t stopped AT&T, T-Mobile and Sprint from selling plenty of BlackBerry devices.

I’m not fully convinced Carrier IQ’s intent is as evil as the deluge of recent coverage makes it out to be, though its methods are both suspect and scary. But if it’s a conspiracy you’re looking for, then there are plenty of possible conspirators. Carrier IQ didn’t do this alone. To drill this far down into the habits of mobile consumers, companies up and down the wireless value needed to be complicit.

Image courtesy of  flickr user alancleaver

11 Responses to “Is Carrier IQ a big data mercenary?”

  1. I can see both sides of this. As a user I would like an opt-in/out option. I also work for a software company(in a completely different industry), and i know that the only way you get to the bottom of issues is with logging. If what they’re logging is really as “generic” as it sounds then i dont see any real harm in it.

    • Kevin Fitchard

      Thanks for commenting, Cathy. I have to agree. General anonymous data or data collected for diagnostic purposes would be perfectly valid and probably already covered in the operators privacy policies. The question I was trying to raise with this story is whether the entity operators hired to provide that diagnostic data is making an extra buck on the side by reselling that data to others.

  2. Vladimir Rodionov

    “The problem is Carrier IQ never got permission from all of these smartphone users to collect that data, never told them it was gathering it, and never provided a way of opting out.”
    Read CIQ latest PR – they explain everything. Carriers:

    1. Install CIQ software
    2. Enable/ disable data collection
    3. decide what type of metrics they want to collect
    4. Only carriers have access to the data they collected.
    5 CIQ does not sell data to third parties (because it does not have access to this data in a first turn)

    Last but not least, we all dislike the small print on papers we usually sign. I am pretty sure that every carrier has corresponding section in their contracts which allows them to collect intelligent metrics information from customer’s devices.
    Read all small prints and make your decision. Its up to you to buy or not to buy new shiny smartphone with a great discount.
    Stop looking for a black cat in dark room. Nothing there.

    • Kevin Fitchard

      Hi Vladimir, thanks for commenting. I agree with you, there’s nothing wrong with diagnostic tools, and if that was it then I would say non-story. But it doesn’t appear that CIQ is selling data only to carriers regardless of what they say in the press release. I think that is a cause for concern, don’t you?

  3. IMO the wireless community (operators, vendors) believe that this kind of stats gathering is useful. This is reflected in the fact that 3GPP which is the international body for wireless cellular standards has recently completed a document that specifies collection of kinds of stats. The idea is to improve the radio network faster (vs. the alternative, drive tests).

    People need to understand *wireless cellular* is NOT a computer like a PC. It needs to signal, manage resources, which is very complex. Operators have needs, and all CarrierIQ was doing was supplying a product. There are other vendors competing with CarrierIQ out there.

    People just like to take on Carriers and this company is IMO being unfairly picked on, when they are actually sitting on information that can be used for targetting and not doing it, because the legal status is unclear. Meanwhile Facebook and Google are collecting lots of information on you and me. And to me that is fine, since they provide valuable services too.

    Privacy is a nice topic for news media. However in practice very tricky. People talk about it all the time, yet 99% probably have not botherred to tinker with the Facebook settings. Google collects *huge* data. But I don;t know anyone who does not use Google because of that.

    Even your grocery store knows the everything you buy from prescription medication to food to contraceptives, because they give you a stupid card which gives you a 1-2% discount.

    • Kevin Fitchard

      Hi Anon, thanks for commenting. All good points, but there’s a difference between on operator collecting diagnostic data and a company selling information about you without your permission. If a CIQ is only doing the former, perhaps that’s harmless and even beneficial, but if it’s selling that info to media analytics companies to help make better advertising, that’s not exactly diagnostics either.

      • Kevin,
        You’re making a big assumption about the nature of the relationship with Nielsen. There’s nothing in the Nielsen announcement that says that CarrierIQ would sell data to Nielsen, or that CarrierIQ (the company) even has access to any data.

      • Kevin Fitchard

        Hi Sam, thanks for commenting. True, I am making some assumptions, but I’m mainly trying to ask questions about the full extent of Carrier IQ’s business model. As for the Nielsen deal, you’re right: nothing says Carrier IQ would “sell” data to Nielsen. They’ve termed it an alliance. Nielsen seems to be taking ever precaution to ensure that everyone opts in as well. The point I’m making is Nielsen isn’t a carrier. Carrier IQ is using data it gathered supposedly at the behest of carriers to work with Nielsen’s media measurement service. I they’re working with Nielsen, they could just as easily be working with others.