This paper introduces and systematically explores the phenomenon of "glitch tokens" in large language models (LLMs), which are anomalous tokens produced by established tokenizers that can compromise the models' response quality. The study investigates the unexpected behaviors caused by glitch tokens in LLMs, categorizes the types of glitch tokens, analyzes their frequency in real-world datasets, and proposes an efficient detection method. The research focuses on seven top popular LLMs using three distinct tokenizers, involving a total of 182,517 tokens. The study identifies 7,895 glitch tokens and categorizes them into five types based on their characteristics and the unexpected behaviors they induce in LLMs. The research also proposes GLITCHHUNTER, a novel iterative clustering-based technique for efficient glitch token detection. The evaluation shows that GLITCHHUNTER significantly outperforms three baseline methods on eight open-source LLMs. The study provides the first comprehensive analysis of glitch tokens and offers valuable insights into mitigating tokenization-related errors in LLMs. The findings indicate that glitch tokens tend to cluster in the embedding space, and GLITCHHUNTER effectively detects them by leveraging this clustering behavior. The results demonstrate that glitch tokens can lead to unexpected behaviors such as spelling mistakes, hallucinatory completions, and random character outputs. The study also highlights the prevalence of glitch tokens in real-world datasets, emphasizing the importance of addressing this issue to ensure the reliability and safety of LLMs. The proposed method, GLITCHHUNTER, is efficient and effective in identifying glitch tokens, reducing the number of queries required and accelerating the detection process. The study contributes to the understanding of glitch tokens in LLMs and provides a practical solution for their detection.This paper introduces and systematically explores the phenomenon of "glitch tokens" in large language models (LLMs), which are anomalous tokens produced by established tokenizers that can compromise the models' response quality. The study investigates the unexpected behaviors caused by glitch tokens in LLMs, categorizes the types of glitch tokens, analyzes their frequency in real-world datasets, and proposes an efficient detection method. The research focuses on seven top popular LLMs using three distinct tokenizers, involving a total of 182,517 tokens. The study identifies 7,895 glitch tokens and categorizes them into five types based on their characteristics and the unexpected behaviors they induce in LLMs. The research also proposes GLITCHHUNTER, a novel iterative clustering-based technique for efficient glitch token detection. The evaluation shows that GLITCHHUNTER significantly outperforms three baseline methods on eight open-source LLMs. The study provides the first comprehensive analysis of glitch tokens and offers valuable insights into mitigating tokenization-related errors in LLMs. The findings indicate that glitch tokens tend to cluster in the embedding space, and GLITCHHUNTER effectively detects them by leveraging this clustering behavior. The results demonstrate that glitch tokens can lead to unexpected behaviors such as spelling mistakes, hallucinatory completions, and random character outputs. The study also highlights the prevalence of glitch tokens in real-world datasets, emphasizing the importance of addressing this issue to ensure the reliability and safety of LLMs. The proposed method, GLITCHHUNTER, is efficient and effective in identifying glitch tokens, reducing the number of queries required and accelerating the detection process. The study contributes to the understanding of glitch tokens in LLMs and provides a practical solution for their detection.