Transforming Keystroke Noise to Text: Self-Supervised Acoustic Eavesdropping Attacks on Keyboards

Transforming Keystroke Noise to Text: Self-Supervised Acoustic Eavesdropping Attacks on Keyboards. Okada and colleagues ask whether passive audio can reveal English text typed in physical spaces or during online meetings. Here, self-supervised means the attack does not begin with labeled examples linking sounds to characters on the target keyboard. By the end, you should understand how the attack combines sound patterns with language context, what the experiments demonstrated, and which assumptions limit the threat.

Keyboard audio can act as a side channel because different keys produce distinguishable sound patterns. Earlier supervised acoustic approaches commonly depended on labeled recordings from the target keyboard or a closely matched setup. Those recordings are hard for an attacker to obtain, and performance can be sensitive to changes in the keyboard or recording channel. Okada and colleagues instead group similar sounds without initially knowing their characters, then use language context to resolve uncertain sound-to-letter relationships.

The operational question is stricter than whether keyboard audio contains information. The attacker is supposed to reconstruct text using only recorded keystrokes, with minimal setup, no labeled target-device data, and limited observations. The primary target is English natural-language text. The researchers also examine random strings as a stand-in for passwords, while acknowledging that passwords and credit-card numbers do not follow ordinary language patterns.

The pipeline first finds likely keystrokes by looking for peaks in short-window audio energy. It standardizes the sound features, then groups acoustically similar events together. An operator manually identifies a single space-key sound to provide an anchor. Next, a language model that works one character at a time uses context on both sides of each position. It keeps probabilistic links between sound groups and possible characters, rather than forcing an early choice. A feedback loop then combines those links with the language model’s predictions. The system repeats this process, using its own provisional labels to refine the result.

In the close-proximity experiment, a smartphone near the target laptop captured enough audio for more than 99% reconstruction accuracy from 100–150 observed keystrokes. Okada and colleagues also evaluated multiple laptop platforms and several recording conditions: across a desk, through a wall with a contact microphone, and through online-conferencing audio. Reconstruction accuracy remained high across those evaluated scenarios.

The evidence covers several ways sound might reach an eavesdropper: direct nearby recording, more distant airborne audio, through-the-wall recording with a contact microphone, and keystrokes carried in meeting audio. The researchers used multiple laptop platforms and recording paths to test robustness across different conditions. One condition needs special attention: the meeting experiment disabled noise suppression and cancellation because strong suppression can substantially attenuate keystroke sounds. That meeting condition therefore represents transmission without those suppression features.

Language context creates an important boundary. The evaluation primarily concerns English sentences, while passwords and credit-card numbers may not resemble natural language. Random-string estimation was only a proxy for password recovery, and exact password recovery remained harder, although the acoustic representation reduced uncertainty. The audio segmentation also depends on consecutive keystrokes not overlapping, and the pipeline requires a person to identify a space-key sample. These conditions leave identical performance on overlapping typing, arbitrary secrets, or a fully automatic attack unproven.

For defenders, the measurements support treating microphone paths and keyboard vibration as possible information-leakage channels. Strong meeting noise suppression can attenuate keystrokes. Vibration-damping materials can reduce the signal captured by contact microphones, although they may not eliminate it. A reasonable defensive interpretation is to evaluate audio processing and physical damping together wherever sensitive typing occurs near active microphones or accessible surfaces. The researchers likewise argue that mitigation requires coordination across multiple system layers.

Okada and colleagues demonstrated that unlabeled keyboard sounds can be grouped and then resolved into typed text with language context and repeated refinement. Security architects, collaboration-platform owners, and side-channel researchers should account for this attack path when microphone or contact recording is plausible. But the evaluated conditions do not establish equal exposure across all keyboards or meetings. The evaluation covered multiple laptop platforms, and the meeting setup disabled noise suppression. Natural-language reconstruction should not be equated with exact password recovery, which remained more difficult.