Malla: Demystifying Real-world Large Language Model Integrated Malicious Services

Malla: Demystifying Real-world Large Language Model Integrated Malicious Services

19 Aug 2024 | Zilong Lin, Jian Cui, Xiaojing Liao, XiaoFeng Wang
The paper "Demystifying Real-world Large Language Model Integrated Malicious Services" by Zilong Lin, Xiaojing Liao, Jian Cui, and XiaoFeng Wang from Indiana University Bloomington explores the growing trend of using large language models (LLMs) for malicious purposes, known as *Malla*. The authors conduct a systematic study of 212 real-world *Mallas* to understand their proliferation, operational modalities, and impact on public LLM services. They uncover eight backend LLMs used by *Mallas* and 182 prompts that bypass protective measures of public LLM APIs. The study reveals that *Mallas* are increasingly being hosted on public LLM hosting platforms like Poe, and they offer services such as generating malicious code, phishing emails, and creating deceptive websites. The research highlights the ethical and security concerns surrounding the misuse of LLMs and provides insights into countermeasures against this cybercrime. Key findings include the rapid growth of *Mallas* in underground marketplaces, the economic viability of these services, and the effectiveness of *Mallas* in producing high-quality malicious content. The study also identifies two primary techniques used by *Mallas*: exploiting uncensored LLMs and jailbreaking public LLM APIs. The authors release a set of artifacts, including prompts and backend LLMs, to support further research and understanding of *Malla* operations.The paper "Demystifying Real-world Large Language Model Integrated Malicious Services" by Zilong Lin, Xiaojing Liao, Jian Cui, and XiaoFeng Wang from Indiana University Bloomington explores the growing trend of using large language models (LLMs) for malicious purposes, known as *Malla*. The authors conduct a systematic study of 212 real-world *Mallas* to understand their proliferation, operational modalities, and impact on public LLM services. They uncover eight backend LLMs used by *Mallas* and 182 prompts that bypass protective measures of public LLM APIs. The study reveals that *Mallas* are increasingly being hosted on public LLM hosting platforms like Poe, and they offer services such as generating malicious code, phishing emails, and creating deceptive websites. The research highlights the ethical and security concerns surrounding the misuse of LLMs and provides insights into countermeasures against this cybercrime. Key findings include the rapid growth of *Mallas* in underground marketplaces, the economic viability of these services, and the effectiveness of *Mallas* in producing high-quality malicious content. The study also identifies two primary techniques used by *Mallas*: exploiting uncensored LLMs and jailbreaking public LLM APIs. The authors release a set of artifacts, including prompts and backend LLMs, to support further research and understanding of *Malla* operations.
Reach us at info@study.space
Understanding Malla%3A Demystifying Real-world Large Language Model Integrated Malicious Services