This study investigates the persuasive capabilities of large language models (LLMs) in direct conversations with human participants, focusing on the impact of personalization. The researchers conducted a randomized controlled trial where participants engaged in short, multiple-round debates with either a human or an LLM, with or without access to personal information about their opponent. The results show that GPT-4, when paired with personalization, significantly increased the odds of participants agreeing with their opponent by 81.7% compared to debates with humans. Without personalization, GPT-4 still outperformed humans but the effect was less pronounced and statistically non-significant. The study highlights the importance of personalization in enhancing LLMs' persuasive power and suggests that online platforms and social media should implement measures to counter the spread of LLM-driven persuasion.This study investigates the persuasive capabilities of large language models (LLMs) in direct conversations with human participants, focusing on the impact of personalization. The researchers conducted a randomized controlled trial where participants engaged in short, multiple-round debates with either a human or an LLM, with or without access to personal information about their opponent. The results show that GPT-4, when paired with personalization, significantly increased the odds of participants agreeing with their opponent by 81.7% compared to debates with humans. Without personalization, GPT-4 still outperformed humans but the effect was less pronounced and statistically non-significant. The study highlights the importance of personalization in enhancing LLMs' persuasive power and suggests that online platforms and social media should implement measures to counter the spread of LLM-driven persuasion.