This paper introduces the concept of knowledge circuits in pretrained transformers, aiming to understand how language models store and process knowledge. The authors analyze the computation graph of language models to identify knowledge circuits that are crucial for articulating specific knowledge. Using experiments with GPT2 and TinyLLAMA, they observe how information heads, relation heads, and MLPs collaborate to encode knowledge within the model. They also evaluate the impact of current knowledge editing techniques on these circuits, providing insights into their functioning and limitations. The study further uses knowledge circuits to analyze and interpret language model behaviors such as hallucinations and in-context learning. The paper proposes a new perspective on knowledge storage in transformers, focusing on knowledge circuits as critical subgraphs that reveal the mechanisms of knowledge representation. The authors explore the cooperation between different components in transformers, such as attention heads, MLPs, and embeddings, to understand how knowledge is stored and expressed. They find that knowledge circuits can uncover implicit neural knowledge representations, elucidate internal mechanisms for knowledge editing, and facilitate the interpretation of complex language model behaviors. The study also investigates the effectiveness of knowledge circuits in various domains, including factual, bias, linguistic, and commonsense knowledge. The authors use GPT-2 and TinyLLAMA to explore the potential knowledge representations and utilization mechanisms in these models. They find that knowledge circuits can maintain a significant portion of the model's performance with only a small subset of the original knowledge circuit's subgraph. The results show that knowledge circuits can enhance model performance on various tasks, including factual recall. The paper also discusses the impact of different knowledge editing methods on the model's performance. It evaluates the effectiveness of methods such as ROME and fine-tuning, finding that they can successfully edit knowledge but may also introduce unintended effects. The study highlights the importance of understanding the internal mechanisms of knowledge editing to improve the safety and reliability of language models. Overall, the paper contributes to the understanding of how knowledge is stored and processed in transformers, providing insights into the functioning of knowledge circuits and their potential for improving language model design and editing. The findings suggest that knowledge circuits can be a valuable tool for analyzing and interpreting language model behaviors, enhancing the model's ability to handle complex tasks and reduce errors.This paper introduces the concept of knowledge circuits in pretrained transformers, aiming to understand how language models store and process knowledge. The authors analyze the computation graph of language models to identify knowledge circuits that are crucial for articulating specific knowledge. Using experiments with GPT2 and TinyLLAMA, they observe how information heads, relation heads, and MLPs collaborate to encode knowledge within the model. They also evaluate the impact of current knowledge editing techniques on these circuits, providing insights into their functioning and limitations. The study further uses knowledge circuits to analyze and interpret language model behaviors such as hallucinations and in-context learning. The paper proposes a new perspective on knowledge storage in transformers, focusing on knowledge circuits as critical subgraphs that reveal the mechanisms of knowledge representation. The authors explore the cooperation between different components in transformers, such as attention heads, MLPs, and embeddings, to understand how knowledge is stored and expressed. They find that knowledge circuits can uncover implicit neural knowledge representations, elucidate internal mechanisms for knowledge editing, and facilitate the interpretation of complex language model behaviors. The study also investigates the effectiveness of knowledge circuits in various domains, including factual, bias, linguistic, and commonsense knowledge. The authors use GPT-2 and TinyLLAMA to explore the potential knowledge representations and utilization mechanisms in these models. They find that knowledge circuits can maintain a significant portion of the model's performance with only a small subset of the original knowledge circuit's subgraph. The results show that knowledge circuits can enhance model performance on various tasks, including factual recall. The paper also discusses the impact of different knowledge editing methods on the model's performance. It evaluates the effectiveness of methods such as ROME and fine-tuning, finding that they can successfully edit knowledge but may also introduce unintended effects. The study highlights the importance of understanding the internal mechanisms of knowledge editing to improve the safety and reliability of language models. Overall, the paper contributes to the understanding of how knowledge is stored and processed in transformers, providing insights into the functioning of knowledge circuits and their potential for improving language model design and editing. The findings suggest that knowledge circuits can be a valuable tool for analyzing and interpreting language model behaviors, enhancing the model's ability to handle complex tasks and reduce errors.