This paper provides a comprehensive survey of the use of tools in language models (LMs), addressing the question of what tools are and how they assist LMs. The authors define tools as external programs used by LMs and review various tooling scenarios and approaches. They empirically study the efficiency of different tooling methods by measuring their required compute and performance gains on various benchmarks, highlighting challenges and potential future research directions. The paper covers a broad range of tool categories, including perception, action, and computation tools, and discusses advanced methods such as multi-tool selection, complex tool usage, and tool creation. It also evaluates existing testbeds and evaluation metrics, identifying missing aspects and suggesting concrete metric improvements. The authors conclude with practical guidance on when and how to use tools effectively, emphasizing the importance of efficiency, quality, reliability, and reproducibility in tool usage.This paper provides a comprehensive survey of the use of tools in language models (LMs), addressing the question of what tools are and how they assist LMs. The authors define tools as external programs used by LMs and review various tooling scenarios and approaches. They empirically study the efficiency of different tooling methods by measuring their required compute and performance gains on various benchmarks, highlighting challenges and potential future research directions. The paper covers a broad range of tool categories, including perception, action, and computation tools, and discusses advanced methods such as multi-tool selection, complex tool usage, and tool creation. It also evaluates existing testbeds and evaluation metrics, identifying missing aspects and suggesting concrete metric improvements. The authors conclude with practical guidance on when and how to use tools effectively, emphasizing the importance of efficiency, quality, reliability, and reproducibility in tool usage.