Adaptive Text Watermark for Large Language Models

Adaptive Text Watermark for Large Language Models

2024 | Yepeng Liu, Yuheng Bu
The paper addresses the challenge of generating high-quality watermarked text while maintaining robustness, security, and detectability without prior knowledge of the prompt and model. It proposes an adaptive text watermarking strategy that adaptively adds watermarks to token distributions with high entropy, leaving low-entropy distributions untouched. To enhance security, the output logits are scaled up based on the semantic embedding of previously generated text using a semantic mapping model, rather than using a fixed green/red list. The method is evaluated on various LLMs, demonstrating comparable robustness to existing watermark methods while maintaining text quality comparable to unwatermarked LLM-generated text. The proposed approach is also shown to be resistant to paraphrasing attacks and has superior security compared to other methods. The paper concludes by discussing the impact of the watermarking technique on transparency, accountability, and public trust in AI technologies.The paper addresses the challenge of generating high-quality watermarked text while maintaining robustness, security, and detectability without prior knowledge of the prompt and model. It proposes an adaptive text watermarking strategy that adaptively adds watermarks to token distributions with high entropy, leaving low-entropy distributions untouched. To enhance security, the output logits are scaled up based on the semantic embedding of previously generated text using a semantic mapping model, rather than using a fixed green/red list. The method is evaluated on various LLMs, demonstrating comparable robustness to existing watermark methods while maintaining text quality comparable to unwatermarked LLM-generated text. The proposed approach is also shown to be resistant to paraphrasing attacks and has superior security compared to other methods. The paper concludes by discussing the impact of the watermarking technique on transparency, accountability, and public trust in AI technologies.
Reach us at info@study.space
[slides] Adaptive Text Watermark for Large Language Models | StudySpace