This paper explores the potential risks of adversarial attacks against language agents powered by large language models (LLMs). Language agents, which can perform complex tasks using LLMs or large multimodal models (LMMs), have shown great promise in real-world applications. However, their development and deployment have outpaced our understanding of their safety risks, raising concerns about their vulnerability to adversarial attacks. The paper presents a unified conceptual framework for language agents, consisting of three major components: Perception, Brain, and Action. Under this framework, the authors discuss 12 potential attack scenarios targeting different components of an agent, covering various attack strategies such as input manipulation, adversarial demonstrations, jailbreaking, and backdoors. These attack strategies are connected to previously applied methods on LLMs, highlighting the need for a thorough understanding of language agent risks before their widespread deployment. The paper also discusses specific attack scenarios across different components of language agents. For example, in the Perception component, attackers can manipulate product descriptions and images to mislead agents in online shopping scenarios. In the Brain component, attackers can manipulate environmental feedback to influence the agent's reasoning and planning processes. In the Action component, attackers can exploit vulnerabilities in external tools and APIs to perform malicious actions. The paper emphasizes the importance of addressing these risks through further research and development. It calls for a deeper understanding of the safety risks associated with language agents and the promotion of responsible practices in their development and use. The authors conclude that language agents, while powerful, are not immune to adversarial attacks and require careful consideration of their security and safety.This paper explores the potential risks of adversarial attacks against language agents powered by large language models (LLMs). Language agents, which can perform complex tasks using LLMs or large multimodal models (LMMs), have shown great promise in real-world applications. However, their development and deployment have outpaced our understanding of their safety risks, raising concerns about their vulnerability to adversarial attacks. The paper presents a unified conceptual framework for language agents, consisting of three major components: Perception, Brain, and Action. Under this framework, the authors discuss 12 potential attack scenarios targeting different components of an agent, covering various attack strategies such as input manipulation, adversarial demonstrations, jailbreaking, and backdoors. These attack strategies are connected to previously applied methods on LLMs, highlighting the need for a thorough understanding of language agent risks before their widespread deployment. The paper also discusses specific attack scenarios across different components of language agents. For example, in the Perception component, attackers can manipulate product descriptions and images to mislead agents in online shopping scenarios. In the Brain component, attackers can manipulate environmental feedback to influence the agent's reasoning and planning processes. In the Action component, attackers can exploit vulnerabilities in external tools and APIs to perform malicious actions. The paper emphasizes the importance of addressing these risks through further research and development. It calls for a deeper understanding of the safety risks associated with language agents and the promotion of responsible practices in their development and use. The authors conclude that language agents, while powerful, are not immune to adversarial attacks and require careful consideration of their security and safety.