This paper presents a comprehensive study on using deep reinforcement learning (RL) to create dynamic locomotion controllers for bipedal robots. The goal is to develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. The proposed RL-based controller incorporates a novel dual-history architecture, utilizing both a long-term and short-term input/output (I/O) history of the robot. This control architecture, when trained through the proposed end-to-end RL approach, consistently outperforms other methods across a diverse range of skills in both simulation and the real world. The study also delves into the adaptivity and robustness introduced by the proposed RL system in developing locomotion controllers. The resulting control policies can be successfully deployed on Cassie, a torque-controlled human-sized bipedal robot. This work pushes the limits of agility for bipedal robots through extensive real-world experiments. The paper demonstrates a diverse range of locomotion skills, including robust standing, versatile walking, fast running with a demonstration of a 400-meter dash, and a diverse set of jumping skills, such as standing long jumps and high jumps. The paper also discusses related work in model-based optimal control and model-free reinforcement learning for bipedal locomotion. The proposed framework is evaluated in simulation and real-world experiments, showing its effectiveness in enabling a wide range of dynamic locomotion skills. The framework is designed to be general and adaptable to different locomotion skills, with a focus on robustness and adaptability. The paper concludes with a discussion of the insights and lessons learned during the development of this work.This paper presents a comprehensive study on using deep reinforcement learning (RL) to create dynamic locomotion controllers for bipedal robots. The goal is to develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. The proposed RL-based controller incorporates a novel dual-history architecture, utilizing both a long-term and short-term input/output (I/O) history of the robot. This control architecture, when trained through the proposed end-to-end RL approach, consistently outperforms other methods across a diverse range of skills in both simulation and the real world. The study also delves into the adaptivity and robustness introduced by the proposed RL system in developing locomotion controllers. The resulting control policies can be successfully deployed on Cassie, a torque-controlled human-sized bipedal robot. This work pushes the limits of agility for bipedal robots through extensive real-world experiments. The paper demonstrates a diverse range of locomotion skills, including robust standing, versatile walking, fast running with a demonstration of a 400-meter dash, and a diverse set of jumping skills, such as standing long jumps and high jumps. The paper also discusses related work in model-based optimal control and model-free reinforcement learning for bipedal locomotion. The proposed framework is evaluated in simulation and real-world experiments, showing its effectiveness in enabling a wide range of dynamic locomotion skills. The framework is designed to be general and adaptable to different locomotion skills, with a focus on robustness and adaptability. The paper concludes with a discussion of the insights and lessons learned during the development of this work.