Quantum computer benchmarking is essential for assessing the performance of quantum hardware and software. Benchmarks help scientists, engineers, and users understand a quantum computer's capabilities, but poor benchmarks can mislead research. This perspective reviews the science of quantum computer benchmarking, discussing the role of benchmarks, how they measure progress toward "quantum utility," and open research questions. Quantum computer benchmarks measure the performance of different parts of a quantum computer, such as quantum gates, compilers, and error correction. They can be low-level, measuring individual components, or high-level, evaluating the overall performance of a quantum system. High-level benchmarks, like IBM's quantum volume benchmark, are increasingly used to assess the performance of integrated quantum computers. However, they may not fully capture the complexities of quantum utility, which requires reducing hardware errors and achieving reliable, scalable quantum systems. Benchmarks must be well-motivated, well-defined, implementation-robust, system-robust, efficient, and technology-independent. Current high-level benchmarks often lack these properties, leading to potential misdirection in research. For example, quantum volume is a popular metric but may not reliably measure progress toward quantum utility. Quantum computing systems consist of multiple layers, including hardware and software, often referred to as the quantum computing "stack." Benchmarks interact with this stack to test different components and performance metrics. High-level benchmarks test the entire system, while low-level benchmarks focus on individual components like qubits and gates. Benchmarking problem-solving capabilities involves testing how well a quantum computer solves computational problems. These benchmarks can be applied to any computing system, enabling comparisons between different quantum computers and classical systems. However, they must be carefully designed to measure the quality and efficiency of solutions. Benchmarking circuit execution capabilities tests how well a quantum computer runs quantum programs. These benchmarks can be high-level or low-level, depending on the complexity of the programs tested. High-level benchmarks often use structured programs, while low-level benchmarks focus on individual gates and circuits. Component benchmarks measure the performance of fundamental logic operations, such as single-qubit and two-qubit gates. These benchmarks are crucial for understanding the reliability and efficiency of quantum systems. As quantum computing moves toward fault-tolerant architectures, component benchmarks will become increasingly important for evaluating the performance of physical qubits and gates. Measuring progress toward quantum utility requires a range of complementary benchmarks that address specific challenge problems. These problems are computational tasks that, if solved by a quantum computer, would demonstrate quantum utility. Resource estimates are needed to determine the computational resources required to solve these problems, helping to define and assess quantum utility. In summary, quantum computer benchmarking is a critical tool for evaluating the performance of quantum systems. Effective benchmarks must be well-motivated, well-defined, and robust to ensure they accurately measure progress toward quantum utility. As quantum computing technology advances, the development of reliable and comprehensive benchmarks will be essential for guiding research and development in the field.Quantum computer benchmarking is essential for assessing the performance of quantum hardware and software. Benchmarks help scientists, engineers, and users understand a quantum computer's capabilities, but poor benchmarks can mislead research. This perspective reviews the science of quantum computer benchmarking, discussing the role of benchmarks, how they measure progress toward "quantum utility," and open research questions. Quantum computer benchmarks measure the performance of different parts of a quantum computer, such as quantum gates, compilers, and error correction. They can be low-level, measuring individual components, or high-level, evaluating the overall performance of a quantum system. High-level benchmarks, like IBM's quantum volume benchmark, are increasingly used to assess the performance of integrated quantum computers. However, they may not fully capture the complexities of quantum utility, which requires reducing hardware errors and achieving reliable, scalable quantum systems. Benchmarks must be well-motivated, well-defined, implementation-robust, system-robust, efficient, and technology-independent. Current high-level benchmarks often lack these properties, leading to potential misdirection in research. For example, quantum volume is a popular metric but may not reliably measure progress toward quantum utility. Quantum computing systems consist of multiple layers, including hardware and software, often referred to as the quantum computing "stack." Benchmarks interact with this stack to test different components and performance metrics. High-level benchmarks test the entire system, while low-level benchmarks focus on individual components like qubits and gates. Benchmarking problem-solving capabilities involves testing how well a quantum computer solves computational problems. These benchmarks can be applied to any computing system, enabling comparisons between different quantum computers and classical systems. However, they must be carefully designed to measure the quality and efficiency of solutions. Benchmarking circuit execution capabilities tests how well a quantum computer runs quantum programs. These benchmarks can be high-level or low-level, depending on the complexity of the programs tested. High-level benchmarks often use structured programs, while low-level benchmarks focus on individual gates and circuits. Component benchmarks measure the performance of fundamental logic operations, such as single-qubit and two-qubit gates. These benchmarks are crucial for understanding the reliability and efficiency of quantum systems. As quantum computing moves toward fault-tolerant architectures, component benchmarks will become increasingly important for evaluating the performance of physical qubits and gates. Measuring progress toward quantum utility requires a range of complementary benchmarks that address specific challenge problems. These problems are computational tasks that, if solved by a quantum computer, would demonstrate quantum utility. Resource estimates are needed to determine the computational resources required to solve these problems, helping to define and assess quantum utility. In summary, quantum computer benchmarking is a critical tool for evaluating the performance of quantum systems. Effective benchmarks must be well-motivated, well-defined, and robust to ensure they accurately measure progress toward quantum utility. As quantum computing technology advances, the development of reliable and comprehensive benchmarks will be essential for guiding research and development in the field.