This paper provides a comprehensive survey of Semantic Communication Networks (SemComNet), focusing on its architecture, security, and privacy. SemComNet, an advanced communication paradigm, aims to convey essential meanings of source data rather than just bit transmission, enabling efficient spectrum utilization and flexible networking among intelligent agents. The paper introduces a three-layer architecture of SemComNet: the control layer for resource management and orchestration, the semantic transmission layer for efficient semantic information delivery, and the cognitive sensing layer for environmental perception and knowledge enrichment. It discusses the working modes of SemComNet, including paired, clustered, and networked communication, and the enabling technologies such as AI, networking, knowledge representation, and ubiquitous computing. The paper also delves into the security and privacy challenges in SemComNet, categorizing threats into three layers: control, semantic transmission, and cognitive sensing. Key threats include sponge examples attacks, knowledge base poisoning, desynchronization of knowledge bases, and mismatches of knowledge bases. Each threat is analyzed in detail, and potential defense approaches are discussed to ensure the secure and reliable operation of SemComNet. Finally, the paper outlines future research directions, emphasizing the need for robust and green SemComNet to support intelligent applications like autonomous driving and the Metaverse. This survey is the first to comprehensively cover the fundamentals of SemComNet and its security and privacy aspects, making it a valuable resource for researchers and practitioners in the field.This paper provides a comprehensive survey of Semantic Communication Networks (SemComNet), focusing on its architecture, security, and privacy. SemComNet, an advanced communication paradigm, aims to convey essential meanings of source data rather than just bit transmission, enabling efficient spectrum utilization and flexible networking among intelligent agents. The paper introduces a three-layer architecture of SemComNet: the control layer for resource management and orchestration, the semantic transmission layer for efficient semantic information delivery, and the cognitive sensing layer for environmental perception and knowledge enrichment. It discusses the working modes of SemComNet, including paired, clustered, and networked communication, and the enabling technologies such as AI, networking, knowledge representation, and ubiquitous computing. The paper also delves into the security and privacy challenges in SemComNet, categorizing threats into three layers: control, semantic transmission, and cognitive sensing. Key threats include sponge examples attacks, knowledge base poisoning, desynchronization of knowledge bases, and mismatches of knowledge bases. Each threat is analyzed in detail, and potential defense approaches are discussed to ensure the secure and reliable operation of SemComNet. Finally, the paper outlines future research directions, emphasizing the need for robust and green SemComNet to support intelligent applications like autonomous driving and the Metaverse. This survey is the first to comprehensively cover the fundamentals of SemComNet and its security and privacy aspects, making it a valuable resource for researchers and practitioners in the field.