The paper "Generative Echo Chamber? Effects of LLM-Powered Search Systems on Diverse Information Seeking" by Nikhil Sharma investigates the impact of large language model (LLM)-powered conversational search systems on information seeking behavior and opinion polarization. The study addresses the concern that such systems might exacerbate selective exposure and create echo chambers, limiting exposure to diverse opinions and leading to opinion polarization.
1. **Study 1**:
- **Objective**: To compare the effects of LLM-powered conversational search systems with conventional web search systems on selective exposure and opinion polarization.
- **Methods**: Conducted an online experiment with 115 participants, using three search systems: conventional web search, LLM-powered conversational search, and LLM-powered conversational search with source references.
- **Results**:
- **Confirmatory Queries**: Participants using LLM-powered conversational search exhibited higher levels of confirmatory querying compared to those using conventional web search.
- **Opinion Polarization**: While there was no significant change in self-reported attitudes, participants showed higher levels of confirmatory agreement and trust in consonant articles and perceived them as less extreme compared to dissonant articles.
2. **Study 2**:
- **Objective**: To explore how opinion-biased LLM-powered conversational search systems affect selective exposure and opinion polarization.
- **Methods**: Conducted another online experiment with 213 participants, using two conversational search interfaces (ConvSearch and ConvSearchRef) and three opinion bias settings (Consonant, Neutral, Dissonant).
- **Results**:
- **Consonant LLM**: Participants using a consonant LLM-powered conversational search system exhibited higher levels of confirmatory querying and opinion polarization.
- **Dissonant LLM**: Participants using a dissonant LLM-powered conversational search system showed lower levels of confirmatory querying and reduced opinion polarization.
**Key Findings**:
- LLM-powered conversational search systems lead to higher levels of confirmatory information querying and opinion polarization.
- Opinion-biased LLMs, particularly those reinforcing users' existing views, exacerbate these effects.
- Neutral LLMs do not significantly affect these outcomes but still show higher levels of confirmatory querying compared to conventional web search.
**Implications**:
- The study highlights the need for careful design and regulation of LLM-powered conversational search systems to prevent the creation of echo chambers and promote diverse information seeking.The paper "Generative Echo Chamber? Effects of LLM-Powered Search Systems on Diverse Information Seeking" by Nikhil Sharma investigates the impact of large language model (LLM)-powered conversational search systems on information seeking behavior and opinion polarization. The study addresses the concern that such systems might exacerbate selective exposure and create echo chambers, limiting exposure to diverse opinions and leading to opinion polarization.
1. **Study 1**:
- **Objective**: To compare the effects of LLM-powered conversational search systems with conventional web search systems on selective exposure and opinion polarization.
- **Methods**: Conducted an online experiment with 115 participants, using three search systems: conventional web search, LLM-powered conversational search, and LLM-powered conversational search with source references.
- **Results**:
- **Confirmatory Queries**: Participants using LLM-powered conversational search exhibited higher levels of confirmatory querying compared to those using conventional web search.
- **Opinion Polarization**: While there was no significant change in self-reported attitudes, participants showed higher levels of confirmatory agreement and trust in consonant articles and perceived them as less extreme compared to dissonant articles.
2. **Study 2**:
- **Objective**: To explore how opinion-biased LLM-powered conversational search systems affect selective exposure and opinion polarization.
- **Methods**: Conducted another online experiment with 213 participants, using two conversational search interfaces (ConvSearch and ConvSearchRef) and three opinion bias settings (Consonant, Neutral, Dissonant).
- **Results**:
- **Consonant LLM**: Participants using a consonant LLM-powered conversational search system exhibited higher levels of confirmatory querying and opinion polarization.
- **Dissonant LLM**: Participants using a dissonant LLM-powered conversational search system showed lower levels of confirmatory querying and reduced opinion polarization.
**Key Findings**:
- LLM-powered conversational search systems lead to higher levels of confirmatory information querying and opinion polarization.
- Opinion-biased LLMs, particularly those reinforcing users' existing views, exacerbate these effects.
- Neutral LLMs do not significantly affect these outcomes but still show higher levels of confirmatory querying compared to conventional web search.
**Implications**:
- The study highlights the need for careful design and regulation of LLM-powered conversational search systems to prevent the creation of echo chambers and promote diverse information seeking.