ROUTERBENCH is a benchmark designed to evaluate the performance of multi-LLM routing systems. As the application of Large Language Models (LLMs) expands, the need for efficient and cost-effective serving solutions becomes critical. While individual LLMs may not be optimal for all tasks, routing systems can combine the strengths of multiple models to enhance performance and reduce costs. However, the lack of a standardized benchmark for evaluating routing systems has hindered progress in this area. ROUTERBENCH addresses this gap by providing a comprehensive evaluation framework and a large dataset of over 405,000 inference results from various LLMs. It also introduces a theoretical framework for assessing routing systems based on cost and performance metrics. The benchmark includes a diverse range of tasks and domains, allowing for the systematic evaluation of different routing strategies. The results demonstrate that while some routing mechanisms struggle with complex tasks, others show promising performance. ROUTERBENCH also introduces a mathematical formulation for evaluating routing systems, including linear interpolation and non-decreasing convex hulls, to compare different routing approaches. The benchmark includes a dataset of eight tasks and a RAG dataset, covering a wide range of LLMs and tasks. The results show that the Oracle router, which always selects the best-performing LLM, achieves near-optimal performance at a low cost. However, the study also highlights the potential of predictive and cascading routers to improve performance while maintaining cost efficiency. The benchmark provides a standardized framework for evaluating routing systems, enabling further research and development in this area.ROUTERBENCH is a benchmark designed to evaluate the performance of multi-LLM routing systems. As the application of Large Language Models (LLMs) expands, the need for efficient and cost-effective serving solutions becomes critical. While individual LLMs may not be optimal for all tasks, routing systems can combine the strengths of multiple models to enhance performance and reduce costs. However, the lack of a standardized benchmark for evaluating routing systems has hindered progress in this area. ROUTERBENCH addresses this gap by providing a comprehensive evaluation framework and a large dataset of over 405,000 inference results from various LLMs. It also introduces a theoretical framework for assessing routing systems based on cost and performance metrics. The benchmark includes a diverse range of tasks and domains, allowing for the systematic evaluation of different routing strategies. The results demonstrate that while some routing mechanisms struggle with complex tasks, others show promising performance. ROUTERBENCH also introduces a mathematical formulation for evaluating routing systems, including linear interpolation and non-decreasing convex hulls, to compare different routing approaches. The benchmark includes a dataset of eight tasks and a RAG dataset, covering a wide range of LLMs and tasks. The results show that the Oracle router, which always selects the best-performing LLM, achieves near-optimal performance at a low cost. However, the study also highlights the potential of predictive and cascading routers to improve performance while maintaining cost efficiency. The benchmark provides a standardized framework for evaluating routing systems, enabling further research and development in this area.