14 Mar 2024 | Roy Weiss, Daniel Ayzenshteyn, Guy Amit, Yisroel Mirsky
This paper presents a novel side-channel attack on AI assistants, specifically targeting the token-length side-channel. The authors identify that many vendors, including OpenAI and Microsoft, transmit responses as token sequences over the web, which can be used to infer sensitive information. They propose a token inference attack that leverages large language models (LLMs) to translate token-length sequences into legible sentences, providing context to narrow the search space, and performing a known-plaintext attack by fine-tuning the model on the target model's writing style. The attack successfully reconstructs 29% of responses and infers the topic from 55% of them. The paper evaluates the attack on OpenAI's ChatGPT-4 and Microsoft's Copilot, demonstrating its effectiveness in both browser and API traffic. The contributions of the paper include identifying a novel side-channel and offering a comprehensive framework for understanding and mitigating the risks associated with the token-length side-channel.This paper presents a novel side-channel attack on AI assistants, specifically targeting the token-length side-channel. The authors identify that many vendors, including OpenAI and Microsoft, transmit responses as token sequences over the web, which can be used to infer sensitive information. They propose a token inference attack that leverages large language models (LLMs) to translate token-length sequences into legible sentences, providing context to narrow the search space, and performing a known-plaintext attack by fine-tuning the model on the target model's writing style. The attack successfully reconstructs 29% of responses and infers the topic from 55% of them. The paper evaluates the attack on OpenAI's ChatGPT-4 and Microsoft's Copilot, demonstrating its effectiveness in both browser and API traffic. The contributions of the paper include identifying a novel side-channel and offering a comprehensive framework for understanding and mitigating the risks associated with the token-length side-channel.