Patchscopes is a unifying framework for inspecting hidden representations of language models (LLMs). The paper introduces Patchscopes, a modular framework that allows for decoding information from LLM representations by "patching" them into a different inference pass that encourages the extraction of that information. Patchscopes can be configured to query various kinds of information from LLM representations, and it can be used to answer a wide range of questions about an LLM's computation. The framework is shown to encompass many prior interpretability methods, including those based on projecting representations into the vocabulary space and intervening on the LLM computation. Patchscopes also enables new possibilities such as using a more capable model to explain the representations of a smaller model and multihop reasoning error correction. The paper shows that many existing methods, including those that rely on vocabulary projections and computation interventions, can be cast as Patchscopes. Moreover, new configurations of the framework introduce more effective tools in addressing the same questions, while mitigating several limitations of prior approaches. Patchscopes also enables addressing underexplored questions, such as fine-grained analysis of the input contextualization process and the extent to which a more expressive model can be used to inspect hidden representations of a smaller model. The paper presents experiments evaluating the benefits and opportunities introduced by Patchscopes, focusing on auto-regressive LLMs. The results show that Patchscopes outperforms existing methods in several tasks, including decoding next-token predictions and extracting attributes from LLM representations. Patchscopes is also shown to be effective in analyzing the contextualization of entity names in early layers and in leveraging stronger models for inspection via cross-model patching. The paper also demonstrates how Patchscopes can be used to correct multi-hop reasoning errors, particularly when the model is capable of conducting each reasoning step correctly but fails when they need to be composed in-context. The paper concludes that Patchscopes is a general modular framework for decoding information from the hidden representations in LLMs. It shows that prominent interpretability methods can be viewed as instances of Patchscopes, and new configurations result in more expressive, robust across layers, and training-data-free alternatives that mitigate their shortcomings. In addition, novel configurations introduce unexplored possibilities of stronger inspection techniques, as well as practical benefits, such as correcting multi-hop reasoning errors.Patchscopes is a unifying framework for inspecting hidden representations of language models (LLMs). The paper introduces Patchscopes, a modular framework that allows for decoding information from LLM representations by "patching" them into a different inference pass that encourages the extraction of that information. Patchscopes can be configured to query various kinds of information from LLM representations, and it can be used to answer a wide range of questions about an LLM's computation. The framework is shown to encompass many prior interpretability methods, including those based on projecting representations into the vocabulary space and intervening on the LLM computation. Patchscopes also enables new possibilities such as using a more capable model to explain the representations of a smaller model and multihop reasoning error correction. The paper shows that many existing methods, including those that rely on vocabulary projections and computation interventions, can be cast as Patchscopes. Moreover, new configurations of the framework introduce more effective tools in addressing the same questions, while mitigating several limitations of prior approaches. Patchscopes also enables addressing underexplored questions, such as fine-grained analysis of the input contextualization process and the extent to which a more expressive model can be used to inspect hidden representations of a smaller model. The paper presents experiments evaluating the benefits and opportunities introduced by Patchscopes, focusing on auto-regressive LLMs. The results show that Patchscopes outperforms existing methods in several tasks, including decoding next-token predictions and extracting attributes from LLM representations. Patchscopes is also shown to be effective in analyzing the contextualization of entity names in early layers and in leveraging stronger models for inspection via cross-model patching. The paper also demonstrates how Patchscopes can be used to correct multi-hop reasoning errors, particularly when the model is capable of conducting each reasoning step correctly but fails when they need to be composed in-context. The paper concludes that Patchscopes is a general modular framework for decoding information from the hidden representations in LLMs. It shows that prominent interpretability methods can be viewed as instances of Patchscopes, and new configurations result in more expressive, robust across layers, and training-data-free alternatives that mitigate their shortcomings. In addition, novel configurations introduce unexplored possibilities of stronger inspection techniques, as well as practical benefits, such as correcting multi-hop reasoning errors.