This paper highlights a critical security vulnerability in large language model (LLM)-based recommendation systems (RS). The authors demonstrate that attackers can significantly increase the exposure of target items by subtly altering their textual content during the testing phase, without affecting the overall recommendation performance. This attack is stealthy and difficult to detect, as the modifications to the text are subtle. The study uses four mainstream LLM-based recommendation models and extensive experiments to validate the efficacy and stealthiness of the textual attack paradigm. The results show that traditional shilling attacks are ineffective in LLM-based RS, while word-level text attacks are more effective. The paper also explores the impact of model fine-tuning and item popularity on the attack, and evaluates the transferability of the attack across different models and tasks. Finally, a simple rewriting defense strategy is proposed to mitigate the issue to some extent. The findings emphasize the need for enhanced security measures in LLM-based RS to protect against such attacks.This paper highlights a critical security vulnerability in large language model (LLM)-based recommendation systems (RS). The authors demonstrate that attackers can significantly increase the exposure of target items by subtly altering their textual content during the testing phase, without affecting the overall recommendation performance. This attack is stealthy and difficult to detect, as the modifications to the text are subtle. The study uses four mainstream LLM-based recommendation models and extensive experiments to validate the efficacy and stealthiness of the textual attack paradigm. The results show that traditional shilling attacks are ineffective in LLM-based RS, while word-level text attacks are more effective. The paper also explores the impact of model fine-tuning and item popularity on the attack, and evaluates the transferability of the attack across different models and tasks. Finally, a simple rewriting defense strategy is proposed to mitigate the issue to some extent. The findings emphasize the need for enhanced security measures in LLM-based RS to protect against such attacks.