Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration This paper proposes MODULAR PLURALISM, a framework for pluralistic alignment of large language models (LLMs) through multi-LLM collaboration. The framework allows a base LLM to "plug into" a pool of smaller, specialized community LMs, enabling collaboration in three modes: Overton, Steerable, and Distributional. These modes support diverse values, perspectives, and cultural representations. MODULAR PLURALISM is compatible with black-box LLMs and allows for modular control of adding new community LMs to better represent underrepresented communities. The framework is evaluated on six tasks and four datasets, showing improvements in pluralistic alignment across six black-box and open-source LLMs. Results demonstrate that MODULAR PLURALISM improves coverage of diverse values by 68.5%, offers greater steerability towards values and demographic attributes in 26.6% and 10.4% of cases, and better reflects the distributional nature of moral scenarios and global perspectives by at least 10.9%. Analysis reveals that LLMs are generally faithful to inputs from smaller community LMs, allowing seamless patching by adding new community LMs. MODULAR PLURALISM is evaluated on six tasks with four datasets, including Overton, Steerable, and Distributional alignment objectives. Results show that MODULAR PLURALISM outperforms baselines in all three objectives, with the highest performance in Overton alignment. The framework is also shown to improve alignment with cultural and demographic attributes, and to better model nationality distributions. The framework is compared with three baselines: vanilla LLMs, prompting for pluralism, and mixture-of-experts. Results show that MODULAR PLURALISM outperforms these baselines in all three alignment objectives. The framework is also shown to improve alignment with underrepresented communities, such as Asian and African cultures. The framework is evaluated on a variety of tasks, including value-based alignment, steerable alignment, and distributional alignment. Results show that MODULAR PLURALISM improves alignment with diverse values, perspectives, and cultural representations. The framework is also shown to improve alignment with underrepresented communities, such as Asian and African cultures. The framework is evaluated on a variety of tasks, including value-based alignment, steerable alignment, and distributional alignment. Results show that MODULAR PLURALISM improves alignment with diverse values, perspectives, and cultural representations. The framework is also shown to improve alignment with underrepresented communities, such as Asian and African cultures.Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration This paper proposes MODULAR PLURALISM, a framework for pluralistic alignment of large language models (LLMs) through multi-LLM collaboration. The framework allows a base LLM to "plug into" a pool of smaller, specialized community LMs, enabling collaboration in three modes: Overton, Steerable, and Distributional. These modes support diverse values, perspectives, and cultural representations. MODULAR PLURALISM is compatible with black-box LLMs and allows for modular control of adding new community LMs to better represent underrepresented communities. The framework is evaluated on six tasks and four datasets, showing improvements in pluralistic alignment across six black-box and open-source LLMs. Results demonstrate that MODULAR PLURALISM improves coverage of diverse values by 68.5%, offers greater steerability towards values and demographic attributes in 26.6% and 10.4% of cases, and better reflects the distributional nature of moral scenarios and global perspectives by at least 10.9%. Analysis reveals that LLMs are generally faithful to inputs from smaller community LMs, allowing seamless patching by adding new community LMs. MODULAR PLURALISM is evaluated on six tasks with four datasets, including Overton, Steerable, and Distributional alignment objectives. Results show that MODULAR PLURALISM outperforms baselines in all three objectives, with the highest performance in Overton alignment. The framework is also shown to improve alignment with cultural and demographic attributes, and to better model nationality distributions. The framework is compared with three baselines: vanilla LLMs, prompting for pluralism, and mixture-of-experts. Results show that MODULAR PLURALISM outperforms these baselines in all three alignment objectives. The framework is also shown to improve alignment with underrepresented communities, such as Asian and African cultures. The framework is evaluated on a variety of tasks, including value-based alignment, steerable alignment, and distributional alignment. Results show that MODULAR PLURALISM improves alignment with diverse values, perspectives, and cultural representations. The framework is also shown to improve alignment with underrepresented communities, such as Asian and African cultures. The framework is evaluated on a variety of tasks, including value-based alignment, steerable alignment, and distributional alignment. Results show that MODULAR PLURALISM improves alignment with diverse values, perspectives, and cultural representations. The framework is also shown to improve alignment with underrepresented communities, such as Asian and African cultures.