This paper explores new philosophical questions raised by recent progress in large language models (LLMs), focusing on interpretability, multimodal and modular extensions, consciousness, and reproducibility. It argues that understanding LLMs' internal mechanisms is crucial for assessing their relevance to human cognition. The paper discusses the limitations of benchmarking LLMs, noting that performance on benchmarks does not necessarily reflect true competence. It highlights the challenges of interpreting LLMs, emphasizing the need for mechanistic explanations that reveal causal structures. The paper introduces intervention methods such as ablation, nullspace projection, and activation patching to uncover how LLMs process information. These methods help identify causal relationships between internal representations and behavior. The paper also discusses the importance of mechanistic interpretability in understanding how LLMs transform inputs into outputs, highlighting the role of circuits and features in this process. Case studies illustrate how mechanistic interpretability can reveal specific circuits, such as induction heads, that enable pattern completion and sequence generalization in LLMs. The paper concludes that mechanistic interpretability is essential for understanding LLMs and their potential relevance to human cognition.This paper explores new philosophical questions raised by recent progress in large language models (LLMs), focusing on interpretability, multimodal and modular extensions, consciousness, and reproducibility. It argues that understanding LLMs' internal mechanisms is crucial for assessing their relevance to human cognition. The paper discusses the limitations of benchmarking LLMs, noting that performance on benchmarks does not necessarily reflect true competence. It highlights the challenges of interpreting LLMs, emphasizing the need for mechanistic explanations that reveal causal structures. The paper introduces intervention methods such as ablation, nullspace projection, and activation patching to uncover how LLMs process information. These methods help identify causal relationships between internal representations and behavior. The paper also discusses the importance of mechanistic interpretability in understanding how LLMs transform inputs into outputs, highlighting the role of circuits and features in this process. Case studies illustrate how mechanistic interpretability can reveal specific circuits, such as induction heads, that enable pattern completion and sequence generalization in LLMs. The paper concludes that mechanistic interpretability is essential for understanding LLMs and their potential relevance to human cognition.