Our mobile devices are amazing at capturing real-world data — location, temperature, movement, sound — that just goes to waste if we don’t put it to use. It’s easy enough to get a personalized experience on the web, but these types of data might make it possible to get one in traditionally more-static places such as retail and radio as well. At the least, perhaps we can expect content, price tags and experiences that cater more to our actual tastes than those of station programmers and a fashion designer’s idea of what people should be willing to pay.

Location is the key to everything

Retailers already have a pretty good sense of what people are buying and even how they’re moving through stores, but they don’t really know where customers are going once they leave. This knowledge could be very useful, however: If you want to improve your store or figure out how to market your company, knowing what else your customers are up to could go a long way. This type of data is starting to become available thanks in part to a Seattle-based startup called Placed.

We’ve been covering Placed for about a year, since it launched its first product targeting developers interested in learning where users were accessing their applications and mobile sites. The company has since expanded its operations to include a Panels service that lets the company track around the clock, on behalf of paying businesses, the physical location of customers who have downloaded the app (usually in exchange for a small monetary reward). It also has its own Panels app, unaffiliated with commercial customers, that allows Placed to provide market data on the physical movements of some 70,000 consumers.

This week, the company released a report highlighting some national findings from the first quarter, including, for example, what departments stores are most popular with what demographics, what business categories experienced the most increases in traffic, and what businesses have the highest and lowest affinities (i.e., people who visit one also visit, or don’t visit, the other). If you’re willing to pay, Placed will tell you pretty much anything you want to know, founder and CEO David Shim told me, broken down by geographic region, business type, demographic, you name it.

Shim noted a couple of actual users and potential users that I think highlight why this type of data is so valuable. One is a high-end retail business that found out that while female millenials enter its stores a lot, they don’t buy a lot. Rather, the stores they visit next are usually discount retailers such as Burlington Coat Factory and Ross. The suggestion is clear: These shoppers want to see what’s hot and then buy a reasonable facsimile at a lower price.

He also noted that some Las Vegas casinos are interested in running their own Placed panels to figure out what restaurants their guests are eating at once they leave the casino grounds. Now, if casinos can figure out where else on the Strip people are spending their money, they can make better choices when it comes time to swap out their own restaurants and shops.

In both cases, it’s possible the answer to the question of how to get more of these customers’ money is to drop prices. If a 10 percent price reduction leads to a 14 percent increase in sales, that’s a win-win situation.

Rethinking radio

Location data becomes even more valuable when combined with other data, though, such as sound. Consider the implications of knowing not just what radio stations people are hearing — which is essentially how the Arbitron ratings work — but what songs they’re actually listening to. Just because you hear the Latino station for an hour at the taco shop during lunchtime or the top 40 station at the gym, that doesn’t mean you’re listening to them or like listening to them.

But the songs you choose to listen to in your car, for example, probably tell a lot about what you actually like. And the technology exists to determine that. Last month, I wrote about how Gracenote is able to use the internal microphones on tablets and smartphones to recognize the songs playing on people’s televisions or stereos. It can also detect reactions such as cheering or booing, and likely whether someone turns up the volume.

Arbitron’s Portable People Meter

Now, all of a sudden, one can envision a world in which programming managers at radio stations can figure out on a song-by-song (or artist-by-artist) basis what people are actually listening to, and when and where they’re listening. If all it involves is someone downloading an app, they can presumably do it at a larger scale than requiring people to wear special additional sensors or fill out a diary. Broadcast radio can never be as personalized as something like Pandora, but it could start sounding a lot more like what listeners would choose if left to their own devices.

Digital radio could get downright great, even better than what Pandora can currently offer. I might never add Disney theme songs or the Sesame Street favorites to my preferences list, but if that’s all I listen to when I’m in my car between 4:30 p.m. and 5 p.m. — and it is — maybe a service could hook me up with some new songs every day. If I’ve turned up the volume on a Taylor Swift song three times this week while I was at home, maybe I actually like it and want to hear more even if I won’t admit it.

Not just data, but good data

As great as all this might sound (it does to me), it’s the advent of big data that makes it all possible. Placed’s analytics are so accurate because it has special algorithms to determine where a person actually is — even if there are numerous options within a small area — and its models are constantly being trained. Shim said his company gets 15,000 responses a day to surveys asking Panels users whether it had them at the right location, and it has already validated 3.5 million of the the 13 billion locations in its database.

Gracenote, for its part, has audio and metadata for millions of songs that it keeps in memory so it can access them in a hurry for the sake of real-time recognition. It can group music into dozens of categories based on genre, artist, geography or even just how the songs sound. It wants to build an in-car system that can change songs based on driving conditions fed to the stereo from the car itself.

I acknowledge this all sounds a little creepy, but, ironically, it also sounds like the beginning of the end for some concerns over privacy. Heck, Shim said, about 500,000 people have already downloaded the Placed Panels app.

Really, it all comes down to value. If handing over a little bit of data actually provides value in return — in the form something better than just targeted ads — it appears people will be willing to do so. People tell Amazon about their purchases, let Google Now access their email and tell Placed which store they’re at out of five possibilities because they think they’re getting a worthwhile service in exchange.

The data-collection genie is already out of the bottle. Now it’s just a matter of making it work for us instead of at our expense.

Feature image courtesy of Shutterstock user Vadim Georgiev.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Connected world: the consumer technology revolution
• NewNet Q4: Platform mania and social commerce shakeout
• NewNet Q4: Platform mania and social commerce shakeout

A few years into Steve Jobs’s second stint as Apple’s CEO, the company hadn’t yet reinvented itself as one of the world’s most-important technology companies, but it was a big-enough distribution channel for the two-year-old Gracenote. At that point, Gracenote had built a respectable business collecting and providing metadata for the compact discs that people were ripping onto their computers, and it relied on software partners to get in front of the music consumers doing the uploading. One of those partners was a popular Mac jukebox application called SoundJam MP.

Ty Roberts Source: Gracenote

So, Gracenote Co-founder and CTO Ty Roberts told me during a recent interview, his company heeded Apple’s warning and bought more servers. At some point around that time (details on the date of the acquisition are sketchy), Apple bought SoundJam MP. Then, at MacWorld in January 2001, Apple released the first version of iTunes (based on the SoundJam technology) and grew Gracenote’s footprint by putting it on more machines. In October 2001, Apple released the iPod and changed Gracenote’s life forever.

The holiday season — particularly Christmas morning — provides a clear example of how stark the change was. “We used to call it iPod day,” Roberts explained, because the company’s servers would go crazy as people opened up their new iPods and immediately began ripping CDs onto their computers. The company’s chief scientist would stay up 20 hours a day for 5 days straight to make sure the database didn’t crash under the load.

From that point on, Roberts explained, a graph showing the rate at which people were uploading music to Gracenote would go from a steady incline into a vertical line. At one point the company was getting metadata from — by Roberts’s estimate — literally every CD being ripped onto personal computers. There was so much database traffic — both writing and reading — because Apple didn’t release the first version of the iTunes Store until April 2003; if users wanted to use their iPods, they had to upload music first.

Scaling like the big boys

Today, of course, Gracenote (which Sony acquired for $260 million in 2008) is pretty much ubiquitous, at least when it comes to metadata. It has metadata for about 130 million songs — and growing — from all over the world and provides metadata to everything from iTunes to Path to your car’s entertainment console. Even if they’re not available for sale as MP3, if someone somewhere at some point ripped a CD and entered its information, Gracenote has data on those artists and songs.

Its database now gets 15 billion queries a month, or 500 million a day (“We’re probably bigger than Bing,” Roberts joked), and the company’s infrastructure has scaled a few times to meet this demand. What began as a small web database running on a few servers grew into an Oracle environment that provided better performance. And when Oracle became cost-prohibitive because of Gracenote’s expanding scale, it shifted again into a highly optimized system that spans thousands of cores in four global data centers.

Now, GM and VP of Automatic Content Recognition Michael Jeffrey noted, almost everything from the chip level up is optimized specifically for Gracenote.

There’s no “world music” when you’re in the “world”

And this setup lets Gracenote do a lot more than just recognize music listeners’ files and give them the album art. For one, Roberts explained, it lets Gracenote be a global company. “We want to have all the music in the world,” Roberts said, “… because our customers ship their products globally.” In fact, part of the reason it’s now part of Sony is that Sony was distributing Gracenote so widely as part of the music player in its Vaio line of laptops.

In order to ensure that everyone has a natural experience wherever they’re accessing Gracenote, part of the job of the company’s 100-person editorial team is to categorize music hierarchically by locality. So, when a user in Japan uploads a CD and Gracenote returns the metadata, it’s categorized as “rock and roll,” for example, rather than a catch-all category like “world music” that a U.S. user might see.

“We want music to feel like a person in your country actually organized it,” Roberts said, “not some dude from California.”

Better music and television through data science

All that data also makes Gracenote a natural fit for recommending new music, although right now the company prefers to let partners handle the algorithms because recommendations tend to be highly product-specific. For example, the iTunes Genius feature is a pretty run-of-the-mill recommendation engine, but, Roberts explained, Apple places a premium on accuracy because its recommendations cost users 99 cents (or more) a shot. With a subscription service like Spotify, though, trying new music is risk-free, so it can play a little faster and looser with its algorithms.

Because Gracenote is present in so many cars — about 35 million — the company has put a lot thought into how to optimally deliver services there, too. Until drivers can bring their interest graphs and music libraries with them to their cars, he explained, any sort of in-car recommendation engine has to be pretty simple and non-distracting — perhaps like thumbs-up or thumbs-down button on the display that will eventually be able to recognize someone’s tastes.

The company has even developed what Roberts calls “machine listening,” which is the ability of an algorithm to recognize the mood, tempo and other audio attributes of music. This is comparable to what Pandora offers, but Gracenote has data on pretty much any song someone could possibly have, which means it can make even your personal music library that much smarter. One idea the company is tinkering is something Roberts describes as “audio coffee.” Depending on any variety of factors — time of day, location, driving conditions or behavior — the stereo system could pick music that either picks up a driver’s pulse or maybe relaxes him.

For Gracenote’s next chapter, the company is banking tablets to deliver a kick like the iPod did last decade. Gracenote is already working with television partners on real-time ad-swapping and intelligent content recommendations, and now it wants to dive deep into the second-screen world. Its new product called Entourage uses a tablet’s internal sensors to hear the television show or music playing in a room and then surface related content, perhaps from the web — like what Entourage user Zeebox provides — or perhaps produced, interactive material like the SyFy channel delivers via its Sync app.

Two ads for two different viewers.

Later this year, GM and VP Jeffrey said, Gracenote will be doing pilots with some large sports broadcasters around a “cheer and jeer” feature that measures how hard people in a room are cheering for or booing their favorite sports teams. If you’re elated, you might see an ad for season tickets. If you’re sad, maybe it’s an an for beer.

Even Roberts is impressed, especially considering that the company’s first use of audio recognition was to make sure users got the right data for their exact version of a song: “I never thought the recognition would break open these kind of new fields.”

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Connected world: the consumer technology revolution
• Forecast: the future of the digital music industry
• Social-TV apps and consumer behavior

Developers can make use of these resources to identify music through audio fingerprinting, access additional metadata about music and even retrieve the cover art for CDs. Gracenote’s music database contains information about a total of 130 million songs, which are classified by more than 2,000 musical genres.

“The company has a history of supporting innovation,” Gracenote President Stephen White said during a phone interview Thursday. Gracenote’s original CD database used to offer developers free access for non-commercial applications, but White said that Gracenote moved away from some of these principles when the company got acquired by Sony. Now Gracenote wants to reclaim its place in the music tech developer ecosystem.

Gracenote’s developer program soft launched at the Stockholm Music Hack Day last month, and 300 developers have already signed up to make use of the company’s resources. One of them is an iOS app called Stream That Song that uses Gracenote’s music recognition to identify a song that’s playing in the background (think Shazam) and then adds it to a user’s Spotify or Deezer library.

Resources of the program are limited to non-commercial app development. White argued that this gives developers a chance to start working on their app and apply for a commercial license once it’s proven to be successful. “There’s no need to strike licenses before you start working,” he said.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Analyzing the wearable computing market
• Takeaways from connected consumer’s second quarter
• CES 2012: a recap and analysis

Content recognition is a key component in many second screen efforts: The technology helps apps on tablets or mobile phones to serve context-relevant content by telling them what’s going on on the TV set. Early content recognition efforts mostly used a tablet’s microphone to “listen in” on a viewer’s TV consumption and then identify programming based on audio fingerprinting. Gracenote has been using this approach as well, but White told me Tuesday that the direct engagement with broadcasters has made him realize that audio recognition isn’t always enough.

Instead, Gracenote now wants to offer a combination of audio and video content recognition. White said that consumers will see this come to their TV sets early next year. The company is also targeting set-top boxes, and has already been powering apps in the second screen space. “The beauty of our platform is that we can support all of those deployments,” said White.

Of course, Gracenote isn’t the only one trying to make a splash in the second-screen space. Tuesday morning, ConnecTV announced the launch of its second screen app in 40 local markets, targeting viewers of local broadcasters with companion content on their iPads. White acknowledged that there is lively competition going on in the space, but said that Gracenote supported the use of other recognition engines in its platform as well. It made perfect sense to use some of these technologies in concert and see them evolve into a common standard, he said. “We are seeing a real desire for standardization,” explained White.

Check out my earlier video of White demonstrating Gracenote’s audio content recognition:

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• How consumer media will change in 2013
• Controversy, courtrooms and the cloud in Q1
• Q4 Wrap-up: SOPA and the future of digital content

Some numbers out from eMarketer (via Memeburn) seem to imply that up to now, advertisers have not ignored connected TV. But nor has interest in it grown:

Looking just at investment in just in digital video advertising (itself only a part of overall spend in digital ads), only eight percent of advertisers said they invested in connected TV video ads in 2011. That’s a modest number, and smaller than other mediums like mobile, where respectively 42 and 41 percent of advertisers said they would spend on iPhone and iPad platforms, and 31 percent said they would invest in ads for Android devices.

Perhaps more tellingly, that eight percent was flat on the year before — that implies that interest in the medium of connected TV seems to have stalled.

Publishers, meanwhile, seem to also be trying to play catch-up in this area: in 2011, 17 percent said they supported video advertising for connected TVs, while only 11 percent supported it in 2010. According to eMarketer’s numbers that puts connected TV video ads on par with video advertising support for non-Apple (NSDQ: AAPL) tablets, but still a ways behind iPhone and iPad (both 35 percent) and Android smartphones (28 percent).

Judging by announcements last week during CES, consumer electronics companies still laying the groundwork for how advertising on connected TV platforms might work in the years ahead:

– Samsung is forging its Smart TV strategy on its own, proprietary platform (complete with Kinect-like gesture recognition and lots of content options). And last week it also launched its own AdHub connected TV advertising services. Premium Interactive Advertising (PIA) is Samsung’s very first ad initiative for its Smart TV platform, and offers advertisers banner advertising on its home screen with functions like click to microsite (interactive multi pages), click to video, and click to application.

– And while Google (NSDQ: GOOG) had a mixed showing for its Google TV connected platform at CES, it continues to make headway into how it is helping more traditional broadcasters sell and manage their advertising inventory. (Their latest customer win: Cox Media.) As those broadcasters begin to consider how they offer advertising on connected TVs, I think it’s a safe bet to say that Google will step up to offer their services there, too.

– Sony’s music and media technology company Gracenote, meanwhile, launched an interesting-looking new service called Entourage, which uses “audio fingerprinting technology” to listen to short clips of content via a mobile device that can trigger the service to then look up content on a smart TV, be it a film to purchase or an advertisement. Gracenote, of course, will need buy-in from smart TV players to make this into an actual service; given that Gracenote is owned by Sony (NYSE: SNE), their might be the first TVs, mobiles and other products where you will see it work first.

– Meanwhile, Sony inked another interactive TV ad deal: Yahoo’s connected TV services, which serve content related to programs and advertisements on regular TV, will now be embedded in Sony Bravia sets. Like Gracenote, it offers an audio-based trigger to deliver advertising (its technology comes from its own subsidiary, IntoNow). Yahoo (NSDQ: YHOO) says its 180-odd connected TV apps already work on connected TV services from Samsung, VIZIO, Toshiba, Haier and HiSense. Yahoo is also starting to roll out new ad units around its these: it says that Toyota is the first company to use such a service, with display ads placed into Yahoo’s Fantasy Football TV app.

– Ooyala is another online video player that is linking up with a vendor, in this case Panasonic for its Viera line of connected TVs. Advertisers and content publishers now using its platform will now also be able to publish their content on Panasonic’s TVs as they are now able to do on mobile and Internet platforms. The Country Network will be the first broadcaster to use the service to offer selections of its content on-demand and live through Panasonic’s platform.

Considering this is CES we are talking about, it’s unsurprising to have a profusion of device- and platform-led announcements, although in the bigger scheme of things broadcasters will inevitably also be playing a role.

“I think traditional broadcasters want to be seen to be involved in new media today,” says Simon Murray, an analyst with Digital TV Research in the UK. “They were slow before and now want to be involved in new technologies and providing information and serving advertisers in different ways.” But he also notes that “Broadcasters have a bit of a dilemma. Obviously they want to diversify but they don’t want to alienate existing advertisers and traditional ways of advertising because it’s still their bread and butter. Puttng too many ads on to programs outside their traditional market might upset their ad base.”

But regardless of who controls the game, perhaps the biggest issue of all is that, unlike internet or even mobile penetration, we are still a long way from critical mass in connected TV services. Figures from Strategy Analytics estimate that by the end of 2011, there were only 12 million connected TV devices distributed globally by the end of 2011. The U.S. had a penetration of only eight percent while Europe’s connected TV device penetration was seven percent.

That, too, appears to be changing, though. Digital TV Research forecasts that it will take until 2016 for there to be a global penetration of 20 percent in connected TV services — that includes not just connected sets, but also other devices such as blu-ray players; games consoles; retail set-top boxes and pay-TV set-top boxes. Highly anticipated updates from the likes of Apple and its Apple TV could also advance the market.

While today games consoles are the most common way for making a TV internet-friendly, Murray says he thinks that connected TVs will overtake games consoles by the end of this year (2012) as the main route to accessing Internet services via televisions worldwide, driven by more widely-available

“The thing about advertisers is that they want traffic. Some will experiment earlier. It might appeal to advertisers selling highly priced items such as cars, where you might want to demonstrate features of, say, a new model of the BMW, but I don’t know how much P&G or Unilever would benefit TV interactive advertising today,” Murray told paidContent. “I think it will be a fairly minor and experimental feature for at least the next five years, until the market becomes more established.”

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Who and what to watch in the new era of the living room
• Welcome to the New Paradigm: TV Makers Rule
• CES 2012: a recap and analysis

Gracenote’s advanced content recognition technology makes it possible to identify both on-demand movies as well as live TV content. Gracenote President Stephen White gave me a quick demo of the technology last week in San Francisco. Check it out in the following video:

Gracenote’s service is comparable to IntoNow in that it uses a tablet’s microphone to listen to the audio track of what’s playing on TV. It then checks the resulting fingerprint against a growing database of video content to deliver information to the tablet — a process that takes five seconds or less, according to White.

He told me Gracenote’s service will eventually be able to deliver rich scene-level metadata for tens of thousands of movies, going as far as offering links to buy any of the products shown on-screen. Information for hundreds of movies will be available when the service launches in earnest next March.

Contextual information for live TV isn’t quite as deep, but Gracenote uses its partnership with Tribune Media Services to cross-reference TV Guide data with what a user is currently watching. Gracenote wants to offer its advanced content recognition platform to CE manufacturers, broadcasters and developers of second screen apps, White explained.

This is technically not the first time the company has been powering this kind of second-screen experiences: Gracenote’s subsidiary Gravity Mobile built the technology behind the ABC second-screen apps that are powered by Nielsen. However, those apps are based on watermarks incorporated into the shows ABC build its apps for, which obviously requires the cooperation of the broadcaster.

Gracenote’s fingerprinting approach, on the other hand, works with any kind of content. That means third parties could also use this kind of technology to sell ads against content they don’t own or control. “It’s definitely disruptive,” admitted White. However, he said broadcasters for the most part are very interested in second-screen solutions, if only to defend their own ad dollars. Said White: “The broadcasters realize that it’s gonna happen with our without them.”

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• What the shift to the cloud means for the future EPG
• Report: The Ongoing Battle for the Digital Home
• OTT technologies and strategies for broadcasters

But Sony Ericsson faces more problems than most in that its phones don’t do anything particularly well. If it wants to avoid further losses, it needs to focus, notably on improved OS design, open systems, and flexible media integration.

Apart from the Nokia-styled G705u, Sony Ericsson has failed to push the design envelope, impairing its phones with stale ‘candybar’ designs and small screen sizes. Most critically, its phones’ best features have been sabotaged by uneven executions. For example, 2008’s Xperia X1 phone has sleek media menus and direct YouTube support, but transitioning between productivity-heavy applications on its WinMo 6.1 OS is a mess. Similarly, the W890i has excellent audio fidelity and music features (like a MusicDJ that helps you compose ringtones), but it’s WiFi-free and its keys are uncomfortably tiny.

In order to truly find success in the mobile market, Sony Ericsson needs to:

• Build media-friendly phone UIs that take better advantage of its Gracenote metadata software. Easier integration with wireless hardware (from cars to stereo systems) will allow people to stream content between all gadgets, not just Sony devices (See: PSP with PS3). While you can already do this to some extent, existing Sony Ericsson apps don’t make the process any smoother. Flexible apps would prompt people to use them more and increase a phone’s value.
• Include visually dynamic software, like Microsoft’s MixView music recommendation engine. It would give people a direct incentive to use Sony Ericsson’s Play Now Plus download service, which is also hampered with DRM music.
• Build a new operating system. Other phone makers have bypassed Sony Ericsson with a simpler OS and faster transitions between apps, as mentioned above. The company has promised an Android OS phone by the end of the year as well as an improved Windows Mobile 7.
• Keep all handset prices competitively low. With their sub-$200 Blu-ray players, Sony and Samsung found out this winter that affordable products from big-name brands are the most successful in a bad economy.

Sony Ericsson also needs to simplify by cutting its number of phone models. Counting CES announcements, there will be 24 phones laced with its Project Capuchin API. That’s too many. The company succeeded with mid-market customers when ‘high-end’ features were the province of business users, but that ended when the iPhone brought multimedia and web access to the mainstream. Its newest phones do include high-end features but they’re split between the Cyber-shot and Walkman lines. One phone has an 8-megapixel camera, but doesn’t have gesture control; another has shake-shuffling without camera face detection. Placing the best features in one high-end phone would create an excellent competitor for the Palm Pre and the iPhone.

Sadly, Sony’s business plan centers around maintaining legacy systems and saturating the market with multiple phones (even if evidence in the last few years suggests customers don’t care about brand loyalty and expect the full feature suite), so don’t expect such an approach to change anytime soon. At the very least, Sony Ericsson should create one major Walkman phone, one photo phone, and one low-priced web/business phone that combines the improvements already mentioned. The rest should be discontinued.

Greater openness also means taking a risk with gaming applications. Recently, Sony rejected Ericsson’s idea for a PSP phone, probably to avoid diluting the strength of the console. But a new focus on open-source, App Store-like gaming development needs to extend to Sony Ericsson phones. If not, Nintendo might get there first and jump further ahead of its rival.

If Sony Ericsson doesn’t make some of these changes, it will continue its recent pattern of bringing features a year behind its competitors — and its decline will continue.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• CES 2012: a recap and analysis
• Connected world: the consumer technology revolution
• The future of mobile: a segment analysis by GigaOM Pro