Boosting of Thoughts: Trial-and-Error Problem Solving with Large Language Models

Boosting of Thoughts: Trial-and-Error Problem Solving with Large Language Models

2024 | Sijia Chen, Baochun Li, Di Niu
This paper introduces Boosting of Thoughts (BoT), an automated prompting framework for large language models (LLMs) that iteratively explores and self-evaluates multiple trees of thoughts to generate trial-and-error reasoning experiences. Starting from a simple prompt without human annotations, BoT iteratively refines reasoning steps by analyzing errors and incorporating feedback, leading to improved problem-solving performance. The framework uses a weighted binary tree structure for thought generation, allowing for efficient exploration and aggregation of reasoning paths. BoT aggregates multiple thought chains to form a single, more logical reasoning chain, which is then evaluated to generate feedback for further refinement. This process is repeated across iterations, gradually improving the prompt and leading to accurate solutions. Experiments on various benchmark datasets, including GSM8K, AQuA, MMLU, and MATH, demonstrate that BoT achieves higher or comparable problem-solving rates compared to other advanced prompting approaches. Notably, BoT outperforms the Tree of Thoughts (ToT) approach on the challenging Game of 24 task, achieving a 9.7% higher solve rate. The framework's effectiveness is attributed to its ability to iteratively refine prompts using error analysis and feedback, enabling LLMs to generate more accurate reasoning steps without relying on human annotations. BoT's scalability and efficiency make it a promising approach for enhancing LLM performance across diverse tasks.This paper introduces Boosting of Thoughts (BoT), an automated prompting framework for large language models (LLMs) that iteratively explores and self-evaluates multiple trees of thoughts to generate trial-and-error reasoning experiences. Starting from a simple prompt without human annotations, BoT iteratively refines reasoning steps by analyzing errors and incorporating feedback, leading to improved problem-solving performance. The framework uses a weighted binary tree structure for thought generation, allowing for efficient exploration and aggregation of reasoning paths. BoT aggregates multiple thought chains to form a single, more logical reasoning chain, which is then evaluated to generate feedback for further refinement. This process is repeated across iterations, gradually improving the prompt and leading to accurate solutions. Experiments on various benchmark datasets, including GSM8K, AQuA, MMLU, and MATH, demonstrate that BoT achieves higher or comparable problem-solving rates compared to other advanced prompting approaches. Notably, BoT outperforms the Tree of Thoughts (ToT) approach on the challenging Game of 24 task, achieving a 9.7% higher solve rate. The framework's effectiveness is attributed to its ability to iteratively refine prompts using error analysis and feedback, enabling LLMs to generate more accurate reasoning steps without relying on human annotations. BoT's scalability and efficiency make it a promising approach for enhancing LLM performance across diverse tasks.
Reach us at info@study.space
[slides] Boosting of Thoughts%3A Trial-and-Error Problem Solving with Large Language Models | StudySpace