Large Language Models are Geographically Biased

Large Language Models are Geographically Biased

5 Feb 2024 | Rohin Manvi, Samar Khanna, Marshall Burke, David Lobell, Stefano Ermon
Large Language Models (LLMs) carry inherent biases from their training corpora, which can perpetuate societal harm. This paper explores the geographic biases of LLMs, defining geographic bias as systemic errors in geospatial predictions. The study demonstrates that LLMs can make accurate zero-shot geospatial predictions, showing strong monotonic correlation with ground truth data. However, LLMs exhibit biases across various topics, particularly on sensitive subjective topics such as attractiveness, morality, and intelligence, where they are biased against locations with lower socioeconomic conditions. The paper introduces a bias score to quantify these biases, finding significant variation among existing LLMs. The results highlight the need for researchers to address geographic biases when training LLMs to avoid harmful stereotypes in their applications.Large Language Models (LLMs) carry inherent biases from their training corpora, which can perpetuate societal harm. This paper explores the geographic biases of LLMs, defining geographic bias as systemic errors in geospatial predictions. The study demonstrates that LLMs can make accurate zero-shot geospatial predictions, showing strong monotonic correlation with ground truth data. However, LLMs exhibit biases across various topics, particularly on sensitive subjective topics such as attractiveness, morality, and intelligence, where they are biased against locations with lower socioeconomic conditions. The paper introduces a bias score to quantify these biases, finding significant variation among existing LLMs. The results highlight the need for researchers to address geographic biases when training LLMs to avoid harmful stereotypes in their applications.
Reach us at info@study.space
[slides and audio] Large Language Models are Geographically Biased