This paper presents a benchmark suite for continuous control tasks in reinforcement learning, addressing the lack of a commonly adopted benchmark in this domain. The benchmark includes a variety of tasks such as classic problems like cart-pole swing-up, high-dimensional locomotion tasks, partially observable tasks, and hierarchical tasks. The authors implement several reinforcement learning algorithms and evaluate their effectiveness on these tasks. The benchmark and reference implementations are released to facilitate reproducibility and encourage further research. The results show that while some algorithms perform well on certain tasks, they struggle with more complex tasks, highlighting the need for new algorithms to handle hierarchical and high-dimensional control problems.This paper presents a benchmark suite for continuous control tasks in reinforcement learning, addressing the lack of a commonly adopted benchmark in this domain. The benchmark includes a variety of tasks such as classic problems like cart-pole swing-up, high-dimensional locomotion tasks, partially observable tasks, and hierarchical tasks. The authors implement several reinforcement learning algorithms and evaluate their effectiveness on these tasks. The benchmark and reference implementations are released to facilitate reproducibility and encourage further research. The results show that while some algorithms perform well on certain tasks, they struggle with more complex tasks, highlighting the need for new algorithms to handle hierarchical and high-dimensional control problems.