If this research pans out, don’t bring a bag of potato chips to the secret meeting


Credit: Aniphoto / Shutterstock

MIT, Microsoft and Adobe researchers recently took a video of a room from 15 feet away through a pane of soundproof glass. Even though they obviously couldn’t hear, they were able to tell exactly what was said inside the room. Their secret? A vibrating bag of potato chips caught on film.

“When sound hits an object, it causes the object to vibrate,” lead paper author Abe Davis said in a release. “The motion of this vibration creates a very subtle visual signal that’s usually invisible to the naked eye. People didn’t realize that this information was there.”

The team created an algorithm that can translate those tiny vibrations back into audio, allowing them to extract intelligible speech from objects. The researchers successfully pulled sound from videos of tinfoil, water in a glass and a plant.

Their work could be useful to determine the composition of different types of objects. As Davis put it, “different objects are going to respond to sound in different ways,” potentially allowing people to use a camera to analyze what a material is made of. The technology could also be used in law enforcement and forensics to recover or record audio.

The best way to capture sound with a video is to use a high-speed camera that capture frames faster than audio signals move. In this case, the researchers used cameras that captured footage at 2,000 to 6,000 frames per second. But they were also able to determine the gender, number and voice characteristics of speakers from cameras as slow as 60 frames per second–within reach of mobile phones like the iPhone 5s and basic point and shoot cameras.

The researchers will present their results at the Siggraph conference this week.

“We’re scientists, and sometimes we watch these movies, like James Bond, and we think, ‘This is Hollywood theatrics. It’s not possible to do that. This is ridiculous.’ And suddenly, there you have it. This is totally out of some Hollywood thriller,” Alexei Efros, associate professor of electrical engineering and computer science at the University of California-Berkeley, said in the release. “I’m sure there will be applications that nobody will expect. I think the hallmark of good science is when you do something just because it’s cool and then somebody turns around and uses it for something you never imagined.”



Naive comment. For all you know a government (ours or another) could already be using this info, bringing it to light is a good thing for all. Regulation and oversight are necessary to ensure no abuse but you aren’t likely to stop the accumulation of information. I for one find this enlightening and intriguing (though anyone who has ever looked at how a speaker works shouldn’t be surprised). I would never discourage or disparage discovery or dissemination of information on methodologies!


I think the sign that you are douchebag tool is when you “think the hallmark of good science is when you do something just because it’s cool and then somebody turns around and uses it for something you never imagined.” – and that something is a tyrannical government using your cool thing for fascist surveillance!

Thanks a lot MIT!

Comments are closed.