Extract audio from the vibration of objects

Six years ago in the Eagle Eye (2008), the Aria robot could tell what the room was saying just by looking at the vibration of the water in a nearby coffee shop. And now, scientists have proven that it is completely grounded. One can use a high-speed camera to record the vibration of any object in the room, and then use the algorithm to transfer the vibrations back to the sound that the human ear can hear.

Sound nature is the vibration of air , the air when vibrating with different intensity will produce different sounds. When we say or open the music in the room, the sound will create vibrations that spread throughout the room, exerting a force on the whole object nearby, whichever is closest to the sound source. The greater the impact force, the greater the impact vibration.

Based on this, a team of researchers from two major companies, Adobe, Microsoft, and the MIT Institute, have experimented together using a high-speed camera capable of recording up to 6,000 fps. to record the smallest vibration of objects when the sound waves impact. Then use the computer's algorithm to convert the vibrations back into audible form . In this way, it is possible to record sound remotely even when the room is soundproofed by dedicated glass doors.

In the video below, you'll find the following 3 scenarios:

1. Record sound from the vibrations of the leaves.
2. Captures sound from the vibration of a bag in a room using soundproof glass.
3. Record sound from the vibration of the diaphragm of the headset.

These experiments show a very interesting result because one can clearly hear the music being played in the room or the computer headset, which can also hear the human voice but The amount is quite bad, hard to recognize at the level of one word. In the last experiment, people did not use a 6000fps camera anymore, they used a regular DSLR camera and recorded at 60 frames per second, sound quality at this time is worse but still can hear the music. pretty good.