This paper provides a comprehensive review of mechanistic interpretability (MI) for transformer-based language models (LMs). MI aims to understand the internal workings of LMs by reverse-engineering their computations into human-understandable mechanisms. The paper outlines fundamental objects of study in MI, techniques used for its investigation, approaches for evaluating MI results, and significant findings and applications. It also presents a roadmap for beginners to navigate the field and leverage MI for their benefit. The paper identifies current gaps in the field and discusses potential future directions. MI is a bottom-up approach that interprets LMs by decomposing them into smaller components and more elementary computations. It focuses on three areas: the study of features, circuits, and their universality. Features are human-interpretable input properties encoded in a model's activation. Circuits are meaningful computational pathways that connect features. Universality refers to the extent to which similar features and circuits are formed across different LMs and tasks. The paper reviews various techniques used in MI, including logit lens, probing, sparse autoencoder (SAE), visualization, automated feature explanation, knockout/ablation, and causal mediation analysis (CMA). These techniques help in understanding the internal workings of LMs and their behavior. The paper also discusses the evaluation of MI results, including faithfulness, completeness, minimality, and plausibility. It highlights findings on features, circuits, and universality, as well as applications of MI in model enhancement, AI safety, and other downstream tasks. The paper concludes with a discussion on future work and challenges in the field of MI.This paper provides a comprehensive review of mechanistic interpretability (MI) for transformer-based language models (LMs). MI aims to understand the internal workings of LMs by reverse-engineering their computations into human-understandable mechanisms. The paper outlines fundamental objects of study in MI, techniques used for its investigation, approaches for evaluating MI results, and significant findings and applications. It also presents a roadmap for beginners to navigate the field and leverage MI for their benefit. The paper identifies current gaps in the field and discusses potential future directions. MI is a bottom-up approach that interprets LMs by decomposing them into smaller components and more elementary computations. It focuses on three areas: the study of features, circuits, and their universality. Features are human-interpretable input properties encoded in a model's activation. Circuits are meaningful computational pathways that connect features. Universality refers to the extent to which similar features and circuits are formed across different LMs and tasks. The paper reviews various techniques used in MI, including logit lens, probing, sparse autoencoder (SAE), visualization, automated feature explanation, knockout/ablation, and causal mediation analysis (CMA). These techniques help in understanding the internal workings of LMs and their behavior. The paper also discusses the evaluation of MI results, including faithfulness, completeness, minimality, and plausibility. It highlights findings on features, circuits, and universality, as well as applications of MI in model enhancement, AI safety, and other downstream tasks. The paper concludes with a discussion on future work and challenges in the field of MI.