22 Mar 2024 | Luyang Lin, Lingzhi Wang, Jinsong Guo, Kam-Fai Wong
This study investigates the presence and nature of bias within large language models (LLMs) and its impact on media bias detection. The research explores whether LLMs exhibit inherent biases, particularly in political bias prediction and text continuation tasks, and how these biases affect the accuracy of media bias detection. The study also examines bias across various topics and proposes debiasing strategies, including prompt engineering and model fine-tuning. The findings reveal that LLMs, such as GPT-3.5, exhibit a left-leaning bias in political bias prediction, which can lead to misclassification of articles. Additionally, the study shows that LLMs may exhibit varying degrees of bias across different topics, with some topics showing more pronounced bias than others. The research also evaluates the effectiveness of debiasing methods, such as prompt engineering and fine-tuning, in reducing bias and improving performance. The results indicate that while some debiasing methods are effective, others may inadvertently introduce more bias. The study also compares the bias tendencies of various LLMs, including open-source and closed-source models, and finds that while some models exhibit more pronounced bias, others show less. Overall, the study highlights the importance of addressing bias in LLMs to ensure fair and equitable AI systems.This study investigates the presence and nature of bias within large language models (LLMs) and its impact on media bias detection. The research explores whether LLMs exhibit inherent biases, particularly in political bias prediction and text continuation tasks, and how these biases affect the accuracy of media bias detection. The study also examines bias across various topics and proposes debiasing strategies, including prompt engineering and model fine-tuning. The findings reveal that LLMs, such as GPT-3.5, exhibit a left-leaning bias in political bias prediction, which can lead to misclassification of articles. Additionally, the study shows that LLMs may exhibit varying degrees of bias across different topics, with some topics showing more pronounced bias than others. The research also evaluates the effectiveness of debiasing methods, such as prompt engineering and fine-tuning, in reducing bias and improving performance. The results indicate that while some debiasing methods are effective, others may inadvertently introduce more bias. The study also compares the bias tendencies of various LLMs, including open-source and closed-source models, and finds that while some models exhibit more pronounced bias, others show less. Overall, the study highlights the importance of addressing bias in LLMs to ensure fair and equitable AI systems.