**MARKLLM: An Open-Source Toolkit for LLM Watermarking** **Authors:** Leyi Pan, Aiwai Liu, Zhwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Xuming Hu, Lijie Wen, Irwin King, Philip S. Yu **Abstract:** The emergence of Large Language Models (LLMs) has significantly enhanced various tasks but also raised concerns about potential misuse. Watermarking LLM outputs to identify LLM-generated text is crucial. However, the complexity of existing watermarking algorithms and evaluation procedures pose challenges. To address these issues, MARKLLM is introduced as an open-source toolkit for LLM watermarking. It offers a unified and extensible framework, user-friendly interfaces, and automatic visualization of underlying mechanisms. The toolkit includes 12 evaluation tools covering detectability, robustness, and text quality impact, along with two automated evaluation pipelines. MARKLLM aims to support researchers and the public in understanding and advancing LLM watermarking technology. **Key Contributions:** 1. **Unified Implementation Framework:** MARKLLM provides a flexible and standardized framework for implementing LLM watermarking algorithms, supporting nine specific algorithms from two families: KGW and Christ. 2. **User-Friendly Interfaces:** Consistent and user-friendly interfaces for loading algorithms, generating watermarked text, detecting watermarks, and gathering visualization data. 3. **Visualization Solutions:** Custom visualization solutions for both algorithm families, enabling users to understand the mechanisms visually. 4. **Evaluation Module:** Comprehensive suite of 12 evaluation tools and two automated evaluation pipelines, facilitating flexible and thorough assessments. **Design and Experimental Results:** - **Design Perspective:** MARKLLM is designed with a modular, loosely coupled architecture for scalability and flexibility. - **Experimental Perspective:** In-depth evaluations of nine included algorithms demonstrate their effectiveness and efficiency. - **Ecosystem Perspective:** MARKLLM provides a comprehensive set of resources, including a Python package, installation instructions, and an online Jupyter notebook demo. **Conclusion:** MARKLLM is a comprehensive toolkit for LLM watermarking, offering easy implementation, visualization, and evaluation. It aims to foster a collaborative ecosystem for advancing LLM watermarking technology.**MARKLLM: An Open-Source Toolkit for LLM Watermarking** **Authors:** Leyi Pan, Aiwai Liu, Zhwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Xuming Hu, Lijie Wen, Irwin King, Philip S. Yu **Abstract:** The emergence of Large Language Models (LLMs) has significantly enhanced various tasks but also raised concerns about potential misuse. Watermarking LLM outputs to identify LLM-generated text is crucial. However, the complexity of existing watermarking algorithms and evaluation procedures pose challenges. To address these issues, MARKLLM is introduced as an open-source toolkit for LLM watermarking. It offers a unified and extensible framework, user-friendly interfaces, and automatic visualization of underlying mechanisms. The toolkit includes 12 evaluation tools covering detectability, robustness, and text quality impact, along with two automated evaluation pipelines. MARKLLM aims to support researchers and the public in understanding and advancing LLM watermarking technology. **Key Contributions:** 1. **Unified Implementation Framework:** MARKLLM provides a flexible and standardized framework for implementing LLM watermarking algorithms, supporting nine specific algorithms from two families: KGW and Christ. 2. **User-Friendly Interfaces:** Consistent and user-friendly interfaces for loading algorithms, generating watermarked text, detecting watermarks, and gathering visualization data. 3. **Visualization Solutions:** Custom visualization solutions for both algorithm families, enabling users to understand the mechanisms visually. 4. **Evaluation Module:** Comprehensive suite of 12 evaluation tools and two automated evaluation pipelines, facilitating flexible and thorough assessments. **Design and Experimental Results:** - **Design Perspective:** MARKLLM is designed with a modular, loosely coupled architecture for scalability and flexibility. - **Experimental Perspective:** In-depth evaluations of nine included algorithms demonstrate their effectiveness and efficiency. - **Ecosystem Perspective:** MARKLLM provides a comprehensive set of resources, including a Python package, installation instructions, and an online Jupyter notebook demo. **Conclusion:** MARKLLM is a comprehensive toolkit for LLM watermarking, offering easy implementation, visualization, and evaluation. It aims to foster a collaborative ecosystem for advancing LLM watermarking technology.