The paper explores the inherent reasoning capabilities of large language models (LLMs) without the need for prompting techniques. The authors investigate whether LLMs can generate chain-of-thought (CoT) reasoning paths by simply altering the decoding process. They find that by considering alternative top-$k$ tokens during decoding, LLMs can naturally elicit CoT paths, which often lead to more accurate answers. This approach bypasses the limitations of greedy decoding and allows for a more truthful assessment of the models' intrinsic reasoning abilities. The paper introduces a method called CoT-decoding, which selects CoT paths based on the model's confidence in the final answer. Empirical studies on various reasoning benchmarks show that CoT-decoding significantly improves the LLMs' reasoning performance, even across different model scales. The authors also discuss the limitations and future directions, including the potential for combining CoT-decoding with existing prompting techniques.The paper explores the inherent reasoning capabilities of large language models (LLMs) without the need for prompting techniques. The authors investigate whether LLMs can generate chain-of-thought (CoT) reasoning paths by simply altering the decoding process. They find that by considering alternative top-$k$ tokens during decoding, LLMs can naturally elicit CoT paths, which often lead to more accurate answers. This approach bypasses the limitations of greedy decoding and allows for a more truthful assessment of the models' intrinsic reasoning abilities. The paper introduces a method called CoT-decoding, which selects CoT paths based on the model's confidence in the final answer. Empirical studies on various reasoning benchmarks show that CoT-decoding significantly improves the LLMs' reasoning performance, even across different model scales. The authors also discuss the limitations and future directions, including the potential for combining CoT-decoding with existing prompting techniques.