Chain-of-Thought Reasoning without Prompting

Chain-of-Thought Reasoning without Prompting

23 May 2024 | Xuezhi Wang and Denny Zhou
The paper explores the inherent reasoning capabilities of large language models (LLMs) without the need for prompting techniques. The authors investigate whether LLMs can generate chain-of-thought (CoT) reasoning paths by simply altering the decoding process. They find that by considering alternative top-$k$ tokens during decoding, LLMs can naturally elicit CoT paths, which often lead to more accurate answers. This approach bypasses the limitations of greedy decoding and allows for a more truthful assessment of the models' intrinsic reasoning abilities. The paper introduces a method called CoT-decoding, which selects CoT paths based on the model's confidence in the final answer. Empirical studies on various reasoning benchmarks show that CoT-decoding significantly improves the LLMs' reasoning performance, even across different model scales. The authors also discuss the limitations and future directions, including the potential for combining CoT-decoding with existing prompting techniques.The paper explores the inherent reasoning capabilities of large language models (LLMs) without the need for prompting techniques. The authors investigate whether LLMs can generate chain-of-thought (CoT) reasoning paths by simply altering the decoding process. They find that by considering alternative top-$k$ tokens during decoding, LLMs can naturally elicit CoT paths, which often lead to more accurate answers. This approach bypasses the limitations of greedy decoding and allows for a more truthful assessment of the models' intrinsic reasoning abilities. The paper introduces a method called CoT-decoding, which selects CoT paths based on the model's confidence in the final answer. Empirical studies on various reasoning benchmarks show that CoT-decoding significantly improves the LLMs' reasoning performance, even across different model scales. The authors also discuss the limitations and future directions, including the potential for combining CoT-decoding with existing prompting techniques.
Reach us at info@study.space