I reverse engineered Deepinder Goyal's Temple so my laptop can hear me without a microphone.
Not exactly (I don't have the device) but I wanted to test his claim that physical micro-vibrations generated through the skin and skull when speaking can be converted into text. I thought it should be possible at some level with my laptop because it has an accelerometer, a type of motion sensor.
Sound is vibration, so when there is sound, the body of the laptop shakes a little. My question was whether the accelerometer could record that shaking, and whether I could turn the recording back into audio and transcribe it.
Talking at the laptop did nothing. I shouted, I hummed, I even talked while resting my chin on it. The sensor picked up nothing useful. It all comes down to frequency. In my tests, it only picked up a usable signal from vibrations below about 200 Hz, and anything higher got lost in the sensor’s own noise. The frequencies that make speech intelligible (like when we speak) sit much higher, between ~300 and 3,400 Hz, so my voice never showed up in the readings.
So I took a short sentence and slowed it down 40x, which lowers every frequency in it by the same factor. I played that through the laptop’s own speakers with the microphone input off, and let the accelerometer record the vibrations in the laptop’s body in a silent environment. Then I sped the accelerometer’s recording back up 40x and gave it to a speech-to-text model.
It came back with every word right.
But this only worked on slowed audio played from the laptop speakers directly. Which makes sense because Temple sits pressed against your head with what I'd guess is a much faster bone-conduction sensor with higher sensitivity than this. This is also why Temple can't 'hear' people around/near you because the sensor can only detect physical surface vibrations.