This paper explores the concept of representational harms in algorithmic systems, emphasizing the need to expand the definition beyond behavioral effects to include cognitive and affective impacts. It argues that current definitions of representational harms focus on observable behaviors, neglecting internal changes in cognitive states, beliefs, and emotional responses. The authors propose a broader understanding of representational harms that includes changes in cognitive, affective, and emotional dimensions of mental representations, which can lead to physical, psychological, or social harm. They highlight the unique vulnerabilities of large language models (LLMs) to perpetrate representational harms, especially when these harms are not measured or mitigated.
The paper outlines high-level requirements for measuring and mitigating representational harms, supported by a case study involving LLMs in a conversational setting. It discusses the challenges of measuring representational harms, including the difficulty of identifying and quantifying subtle, diffuse effects. The authors also explore the challenges specific to LLMs, such as seamless design and the ubiquity of deployment, which can exacerbate representational harms. They propose mitigations such as seamful design, counter-narratives, and measurement frameworks to address these harms.
The paper concludes by emphasizing the importance of measuring representational harms at various granularities and the need for institutional and organizational processes to ensure accountability and appropriate reparations. It also discusses morally defensible cases where representational harms may be permissible, highlighting the complexity of balancing different considerations in ethical decision-making. Overall, the paper aims to establish a framework for broadening the definition of representational harms and translating insights from fairness research into practical measurement and mitigation practices.This paper explores the concept of representational harms in algorithmic systems, emphasizing the need to expand the definition beyond behavioral effects to include cognitive and affective impacts. It argues that current definitions of representational harms focus on observable behaviors, neglecting internal changes in cognitive states, beliefs, and emotional responses. The authors propose a broader understanding of representational harms that includes changes in cognitive, affective, and emotional dimensions of mental representations, which can lead to physical, psychological, or social harm. They highlight the unique vulnerabilities of large language models (LLMs) to perpetrate representational harms, especially when these harms are not measured or mitigated.
The paper outlines high-level requirements for measuring and mitigating representational harms, supported by a case study involving LLMs in a conversational setting. It discusses the challenges of measuring representational harms, including the difficulty of identifying and quantifying subtle, diffuse effects. The authors also explore the challenges specific to LLMs, such as seamless design and the ubiquity of deployment, which can exacerbate representational harms. They propose mitigations such as seamful design, counter-narratives, and measurement frameworks to address these harms.
The paper concludes by emphasizing the importance of measuring representational harms at various granularities and the need for institutional and organizational processes to ensure accountability and appropriate reparations. It also discusses morally defensible cases where representational harms may be permissible, highlighting the complexity of balancing different considerations in ethical decision-making. Overall, the paper aims to establish a framework for broadening the definition of representational harms and translating insights from fairness research into practical measurement and mitigation practices.