Evaluating the persuasive influence of political microtargeting with large language models

Evaluating the persuasive influence of political microtargeting with large language models

June 7, 2024 | Kobi Hackenburg and Helen Margettts
This study evaluates the persuasive influence of political microtargeting using large language models (LLMs), specifically GPT-4. Researchers developed a custom web application to integrate self-reported demographic and political data into GPT-4 prompts in real-time, enabling the live creation of personalized messages for political issues. A preregistered randomized control experiment with 8,587 participants tested whether access to individual-level data increases the persuasive influence of GPT-4. The study found that while GPT-4-generated messages were broadly persuasive, the persuasive impact of microtargeted messages was not statistically different from that of non-microtargeted messages. These findings suggest that the influence of current LLMs may not lie in their ability to tailor messages to individuals but rather in the persuasiveness of their generic, nontargeted messages. The study also examined the effect of varying the number and type of attributes used to tailor messages. Results showed that increasing the number of attributes used for tailoring did not significantly enhance persuasiveness. Additionally, different political and demographic attributes did not significantly affect the persuasive power of messages. Participants were also asked to identify the author of the messages they received, and those in the accurate targeting condition were more likely to identify the message as AI-generated compared to the best message condition. The study highlights the potential of LLMs in political microtargeting but also raises concerns about the accuracy of LLMs in reflecting the opinions of specific demographic groups. While the study found no significant persuasive advantage of microtargeting, it suggests that the persuasiveness of nontargeted messages remains substantial. The study also emphasizes the need for further research to explore the effectiveness of personalized messaging in different contexts and the potential risks of AI-driven political microtargeting. The study provides a dataset, GPTarget2024, for future research on the persuasive influence of LLMs in political microtargeting.This study evaluates the persuasive influence of political microtargeting using large language models (LLMs), specifically GPT-4. Researchers developed a custom web application to integrate self-reported demographic and political data into GPT-4 prompts in real-time, enabling the live creation of personalized messages for political issues. A preregistered randomized control experiment with 8,587 participants tested whether access to individual-level data increases the persuasive influence of GPT-4. The study found that while GPT-4-generated messages were broadly persuasive, the persuasive impact of microtargeted messages was not statistically different from that of non-microtargeted messages. These findings suggest that the influence of current LLMs may not lie in their ability to tailor messages to individuals but rather in the persuasiveness of their generic, nontargeted messages. The study also examined the effect of varying the number and type of attributes used to tailor messages. Results showed that increasing the number of attributes used for tailoring did not significantly enhance persuasiveness. Additionally, different political and demographic attributes did not significantly affect the persuasive power of messages. Participants were also asked to identify the author of the messages they received, and those in the accurate targeting condition were more likely to identify the message as AI-generated compared to the best message condition. The study highlights the potential of LLMs in political microtargeting but also raises concerns about the accuracy of LLMs in reflecting the opinions of specific demographic groups. While the study found no significant persuasive advantage of microtargeting, it suggests that the persuasiveness of nontargeted messages remains substantial. The study also emphasizes the need for further research to explore the effectiveness of personalized messaging in different contexts and the potential risks of AI-driven political microtargeting. The study provides a dataset, GPTarget2024, for future research on the persuasive influence of LLMs in political microtargeting.
Reach us at info@futurestudyspace.com