Can data be stolen from a language model via a side channel?

Whisper Leak: a side-channel attack on Large Language Models, Yonatan and Mike’s Daily Paper Review: 26.11.25

In this paper, two researchers from Microsoft (the Israeli Jonathan Bar Or and the Canadian Geoff McDonald) describe a side-channel attack against Large Language Model (LLM) systems.

The motivation is that cloud-based language models communicating with users, even if the communication is encrypted, may expose sensitive information through traffic patterns, specifically packet sizes and inter-packet arrival times. This information allows an attacker to obtain details about the prompt that was sent.

The paper defines and describes an attack called “Whisper Leak.” It demonstrates that it is possible to identify “sensitive topics” and “regular queries” from encrypted LLM traffic. Meaning: even without seeing the text of the prompt or the model’s response, significant information can be inferred.

In the paper, the researchers describe how LLMs use the TLS protocol for end-to-end encryption. The TLS protocol performs a cryptographic key exchange and then switches to symmetric encryption, where the encryption key and the decryption key are identical. The symmetric encryption in all LLMs tested is a Stream Cipher, where the size of the encrypted message equals the size of the plaintext message. Therefore, a strong correlation is possible between the size of an encrypted token and the size of a visible (plaintext) token. Additionally, LLMs tend to return tokens to the user as quickly as possible, creating a correlation between the time differences of encrypted data packets and the time differences of token generation.

The paper reports on experiments conducted with 28 different LLMs, showing that an attack exploiting the aforementioned patterns exists and succeeds at significant rates. The attack relies on collecting features from the communication, such as packet sizes, duration between sending a prompt and receiving a response, etc. After collecting the data, statistical analysis or machine learning models are used to determine if the prompt belongs to a “sensitive” or “regular” topic. The paper tested three types of models:

• BERT Model: Implementing a large language model trained to handle sequential data.

• LSTM Model: Implementing a neural network adapted for sequential data.

• LightGBM Model: Based on decision trees.

These models were trained on sequential data based on packet sizes and the timing between packets. This data was collected from 10,000 different benign prompts sent to the attacked language model, as well as 1,000 different prompts containing a sensitive topic. In the paper, all three models achieved impressive results of approximately 98% success in classifying a “sensitive prompt” versus a “non-sensitive prompt,” given only the sequential data based on packet sizes and inter-packet timing.

The implication of such an attack is relevant to an attacker with the ability to eavesdrop on network traffic (such as governments or a hacker running code on a router), for example, the ability to identify regime opponents, terrorism, and various other activities.

Finally, the paper suggests several ways to mitigate the problem. Microsoft worked with numerous industry leaders in a process lasting nearly a year, during which several companies (OpenAI, Mistral) added an option to their APIs to include obfuscation, random data generated between tokens that prevents the attack.

The paper emphasizes the need to design LLMs and their surrounding systems to successfully minimize data leakage through such side channels.

https://arxiv.org/abs/2511.03675