The paper introduces the Collective Predictive Coding (CPC) hypothesis, which posits that symbol emergence, particularly language, arises from decentralized Bayesian inference in a multi-agent system. The hypothesis emphasizes the interdependence between forming internal representations through physical interactions with the environment and sharing and utilizing meanings through social semiotic interactions. The total system dynamics are theorized from the perspective of predictive coding, drawing inspiration from computational studies grounded in probabilistic generative models and language games, such as the Metropolis–Hastings naming game. The CPC hypothesis suggests that symbol emergence follows the society-wide free-energy principle and provides a new explanation for why large language models (LLMs) appear to possess knowledge about the world based on experience, despite lacking sensory organs or bodies. The paper reviews past approaches to symbol emergence systems, offers a comprehensive survey of related prior studies, and discusses future challenges and potential cross-disciplinary research avenues. The main contributions of the study include a general framework for symbol emergence systems, a computational understanding of symbol emergence, and a theoretical connection between predictive coding, the free-energy principle, and symbol emergence.The paper introduces the Collective Predictive Coding (CPC) hypothesis, which posits that symbol emergence, particularly language, arises from decentralized Bayesian inference in a multi-agent system. The hypothesis emphasizes the interdependence between forming internal representations through physical interactions with the environment and sharing and utilizing meanings through social semiotic interactions. The total system dynamics are theorized from the perspective of predictive coding, drawing inspiration from computational studies grounded in probabilistic generative models and language games, such as the Metropolis–Hastings naming game. The CPC hypothesis suggests that symbol emergence follows the society-wide free-energy principle and provides a new explanation for why large language models (LLMs) appear to possess knowledge about the world based on experience, despite lacking sensory organs or bodies. The paper reviews past approaches to symbol emergence systems, offers a comprehensive survey of related prior studies, and discusses future challenges and potential cross-disciplinary research avenues. The main contributions of the study include a general framework for symbol emergence systems, a computational understanding of symbol emergence, and a theoretical connection between predictive coding, the free-energy principle, and symbol emergence.