Blog Post

New vision algorithm will label you ‘goth,’ try to sell you black makeup

Stay on Top of Enterprise Technology Trends

Get updates impacting your industry from our GigaOm Research Community
Join the Community!

University of California, San Diego researchers have developed an algorithm that’s quite impressive from a computer science standpoint, but might give some people flashbacks to high school. The technique uses machine learning to figure out the most-common visual characteristics of certain groups of people (e.g., hipsters, goths and surfers) and trains an algorithm that can label group photos accordingly.

This way, you don’t have to tell a site like Facebook that you’re into The Cure or many-zippered pants — it will know just from looking at your pictures. When it comes to labeling people, though, the algorithm is an equal opportunity offender. It’s trained to identifty eight popular subcultures, according to Wikipedia: biker, country, goth, heavy metal, hip hop, hipster, raver and surfer. However, “star quarterback with a new Mustang” isn’t one of them.

Jokes aside, this is a pretty interesting project from a computer vision perspective, even if the algorithm is only 48 percent accurate at this point (the researchers found that using just chance resulted in a 9 percent success rate). It’s arguably higher level than techniques like facial recognition, yet more granular at the same time. Rather than just making the step from recognizing a human to recognizing a specific person (or from recognizing an animal to recognizing an angora rabbit), the algorithm is actually looking at the entire context of an image — all the people in it, as well as the venue it which it was taken — to say something about a person.

The project website has more information about the technique, including related research papers.

Source: UCSD
Source: UCSD

Thanks to advances in (and lower costs for) computing power and data analysis, as well as the glut of images available across Google, Flickr, Facebook and everywhere else online, we have seen a lot of activity (and signs of activity) in the computer vision space lately that will continue to affect our digital experience.

Google has arguably been leading the charge — using deep learning techniques to learn the contents of untagged photos in Google Images and users’ Google+ accounts — but it’s certainly not alone anymore. Microsoft is doing research into image recognition, as are Yahoo and Dropbox, assuming some recent acquisitions are indicative of their plans. Earlier this week, Facebook hired computer vision and deep learning expert Yann LeCun to head up its new artificial intelligence lab.

One thing that all of these efforts have in common is a recognition of the commercial benefits that advanced computer vision capabilities can provide. For the web companies, it means a better user experience around the countless photos they upload, while the companies are also able to detect trends in image content. Oh, and there’s prospect of better targeted advertising. Even the UCSD researchers working on the subculture algorithm seem excited about, or at least aware of, that possibility.

Source: UCSD
Source: UCSD

Of course, there are also potentially much greater benefits to this kind of research — everything from detecting illegal behavior online to studying the demographics of public places using surveillance camera footage. Photographs (and even video) are a great untapped source of information that as of yet we really haven’t exploited like we have other online data.

When it comes to this latest algorithm, though, I’ll know it’s being used commercially — and has expanded its scope — when I start seeing deals for trips to the Land of Make Believe. Balding, graying hair? Cardigan sweaters? Clearly a children’s television host.