27 Mar 2024 | Siyu Yuan, Kaitao Song, Jiangjie Chen, Xu Tan, Yongliang Shen, Kan Ren, Dongsheng Li, Deqing Yang
EASYTOOL is a framework that transforms diverse and lengthy tool documentation into concise and unified tool instructions to improve the performance of LLM-based agents in using tools. The framework purifies essential information from extensive tool documentation and provides standardized tool descriptions and functionalities for LLM-based agents. Extensive experiments on multiple tasks demonstrate that EASYTOOL can significantly reduce token consumption and improve the performance of LLM-based agents on tool utilization in real-world scenarios. The framework consists of two stages: first, re-organizing the original tool documentation by eliminating irrelevant information and keeping the function description of each tool. Second, designing a functional-guideline instruction for LLMs and enabling LLMs to refine the tool documentation by providing parameters of each tool and examples to instruct LLMs for usage. The results show that EASYTOOL can effectively improve the capability of LLMs in tool utilization. The framework is evaluated on three datasets from distinct domains, showing that EASYTOOL effectively and efficiently improves the capability of LLMs in tool utilization. The framework is also tested on real-world web services and numerical reasoning tasks, demonstrating its effectiveness in enhancing tool utilization capabilities. The framework is limited to single documentation and cannot handle tool dependencies. It also requires models with instruction-following ability. The framework is ethical and follows the ACL Code of Ethics and the Code of Conduct. The framework is also tested on different LLMs, showing that EASYTOOL can significantly reduce incorrect tool usage and improve the performance of LLM-based agents in real-world scenarios. The framework is also tested on different retrieval methods, showing that EASYTOOL can greatly improve the retrieval performance. The framework is also tested on different tool instructions, showing that EASYTOOL can significantly improve the accuracy of tool selection and execution. The framework is also tested on different numerical reasoning tasks, showing that EASYTOOL can significantly improve the accuracy of tool utilization in complex math problems. The framework is also tested on different tool benchmarks, showing that EASYTOOL can significantly improve the performance of LLM-based agents in real-world scenarios. The framework is also tested on different tool instructions, showing that EASYTOOL can significantly improve the accuracy of tool selection and execution. The framework is also tested on different numerical reasoning tasks, showing that EASYTOOL can significantly improve the accuracy of tool utilization in complex math problems. The framework is also tested on different tool benchmarks, showing that EASYTOOL can significantly improve the performance of LLM-based agents in real-world scenarios.EASYTOOL is a framework that transforms diverse and lengthy tool documentation into concise and unified tool instructions to improve the performance of LLM-based agents in using tools. The framework purifies essential information from extensive tool documentation and provides standardized tool descriptions and functionalities for LLM-based agents. Extensive experiments on multiple tasks demonstrate that EASYTOOL can significantly reduce token consumption and improve the performance of LLM-based agents on tool utilization in real-world scenarios. The framework consists of two stages: first, re-organizing the original tool documentation by eliminating irrelevant information and keeping the function description of each tool. Second, designing a functional-guideline instruction for LLMs and enabling LLMs to refine the tool documentation by providing parameters of each tool and examples to instruct LLMs for usage. The results show that EASYTOOL can effectively improve the capability of LLMs in tool utilization. The framework is evaluated on three datasets from distinct domains, showing that EASYTOOL effectively and efficiently improves the capability of LLMs in tool utilization. The framework is also tested on real-world web services and numerical reasoning tasks, demonstrating its effectiveness in enhancing tool utilization capabilities. The framework is limited to single documentation and cannot handle tool dependencies. It also requires models with instruction-following ability. The framework is ethical and follows the ACL Code of Ethics and the Code of Conduct. The framework is also tested on different LLMs, showing that EASYTOOL can significantly reduce incorrect tool usage and improve the performance of LLM-based agents in real-world scenarios. The framework is also tested on different retrieval methods, showing that EASYTOOL can greatly improve the retrieval performance. The framework is also tested on different tool instructions, showing that EASYTOOL can significantly improve the accuracy of tool selection and execution. The framework is also tested on different numerical reasoning tasks, showing that EASYTOOL can significantly improve the accuracy of tool utilization in complex math problems. The framework is also tested on different tool benchmarks, showing that EASYTOOL can significantly improve the performance of LLM-based agents in real-world scenarios. The framework is also tested on different tool instructions, showing that EASYTOOL can significantly improve the accuracy of tool selection and execution. The framework is also tested on different numerical reasoning tasks, showing that EASYTOOL can significantly improve the accuracy of tool utilization in complex math problems. The framework is also tested on different tool benchmarks, showing that EASYTOOL can significantly improve the performance of LLM-based agents in real-world scenarios.