Fine-tuning enhances existing mechanisms in language models, as demonstrated through a case study on entity tracking. The study shows that fine-tuning on tasks like arithmetic improves a model's ability to track entities within context. The entity tracking circuit in the base model and its fine-tuned versions is largely the same, with the original model's circuit performing better in fine-tuned versions. The circuit functions by tracking the position of the correct entity in the input context. Fine-tuned models improve performance by better handling augmented positional information. Techniques like Patch Patching, DCM, and CMAP were used to analyze the circuits. The findings suggest that fine-tuning enhances, rather than fundamentally alters, the model's mechanisms. The entity tracking circuit remains consistent across base and fine-tuned models, with performance gains attributed to improved core mechanisms. The study also shows that fine-tuned models use the same circuit for entity tracking, with enhanced functionality in certain components. The results indicate that fine-tuning improves the model's ability to track entities by enhancing the circuit's ability to process positional information. The study concludes that fine-tuning enhances existing mechanisms without fundamentally changing the model's operation.Fine-tuning enhances existing mechanisms in language models, as demonstrated through a case study on entity tracking. The study shows that fine-tuning on tasks like arithmetic improves a model's ability to track entities within context. The entity tracking circuit in the base model and its fine-tuned versions is largely the same, with the original model's circuit performing better in fine-tuned versions. The circuit functions by tracking the position of the correct entity in the input context. Fine-tuned models improve performance by better handling augmented positional information. Techniques like Patch Patching, DCM, and CMAP were used to analyze the circuits. The findings suggest that fine-tuning enhances, rather than fundamentally alters, the model's mechanisms. The entity tracking circuit remains consistent across base and fine-tuned models, with performance gains attributed to improved core mechanisms. The study also shows that fine-tuned models use the same circuit for entity tracking, with enhanced functionality in certain components. The results indicate that fine-tuning improves the model's ability to track entities by enhancing the circuit's ability to process positional information. The study concludes that fine-tuning enhances existing mechanisms without fundamentally changing the model's operation.