The paper "The Pitfalls of Next-Token Prediction" by Gregor Bachmann and Vaishnavh Nagarajan explores the limitations of next-token prediction models in modeling human intelligence. The authors argue that while next-token prediction models have achieved significant success, they may not be capable of planning and executing complex tasks as humans do. They distinguish between two phases of next-token prediction: autoregressive inference and teacher-forcing training, and show that teacher-forcing can lead to specific failures in certain tasks. Key points of the paper include: 1. **Autoregressive Inference and Teacher-Forcing Training**: The authors clarify that autoregressive inference and teacher-forcing training are distinct processes. Autoregressive inference involves generating tokens one at a time based on previous outputs, while teacher-forcing training involves feeding the model prefixes of the ground truth response to predict the next token. 2. **Clever Hans Cheat**: In teacher-forcing, the model can use shortcuts (Clever Hans cheat) to fit later tokens by leveraging the revealed prefix of the ground truth answer, but this can lead to poor planning for earlier tokens. 3. **Indecipherable Token**: After the Clever Hans cheat is perfected, the model is left with incomplete supervision for earlier tokens, making it difficult to learn the true mechanism. 4. **Experimental Validation**: The authors demonstrate these failures using a path-finding task on a graph, where both Transformer and Mamba models fail to learn the correct solution despite being trained on the same distribution. 5. **Teacherless Training**: They propose a modification to teacher-forcing where the model predicts multiple future tokens in advance, which helps avoid the Clever Hans cheat and allows the model to learn the correct solution. The paper concludes by highlighting the need to differentiate between autoregressive inference and teacher-forcing training to address the limitations of next-token prediction models and inspire future research in this area.The paper "The Pitfalls of Next-Token Prediction" by Gregor Bachmann and Vaishnavh Nagarajan explores the limitations of next-token prediction models in modeling human intelligence. The authors argue that while next-token prediction models have achieved significant success, they may not be capable of planning and executing complex tasks as humans do. They distinguish between two phases of next-token prediction: autoregressive inference and teacher-forcing training, and show that teacher-forcing can lead to specific failures in certain tasks. Key points of the paper include: 1. **Autoregressive Inference and Teacher-Forcing Training**: The authors clarify that autoregressive inference and teacher-forcing training are distinct processes. Autoregressive inference involves generating tokens one at a time based on previous outputs, while teacher-forcing training involves feeding the model prefixes of the ground truth response to predict the next token. 2. **Clever Hans Cheat**: In teacher-forcing, the model can use shortcuts (Clever Hans cheat) to fit later tokens by leveraging the revealed prefix of the ground truth answer, but this can lead to poor planning for earlier tokens. 3. **Indecipherable Token**: After the Clever Hans cheat is perfected, the model is left with incomplete supervision for earlier tokens, making it difficult to learn the true mechanism. 4. **Experimental Validation**: The authors demonstrate these failures using a path-finding task on a graph, where both Transformer and Mamba models fail to learn the correct solution despite being trained on the same distribution. 5. **Teacherless Training**: They propose a modification to teacher-forcing where the model predicts multiple future tokens in advance, which helps avoid the Clever Hans cheat and allows the model to learn the correct solution. The paper concludes by highlighting the need to differentiate between autoregressive inference and teacher-forcing training to address the limitations of next-token prediction models and inspire future research in this area.