Gymnasium: A Standard Interface for Reinforcement Learning Environments

Gymnasium: A Standard Interface for Reinforcement Learning Environments

24 Jul 2024 | Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U. Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, Rodrigo Perez-Vicente, Andrea Pierré, Sander Schulhoff, Jun Jet Tai, Hannah Tan, Omar G. Younis
Gymnasium is an open-source library designed to provide a standardized API for reinforcement learning (RL) environments. Its primary contribution is a central abstraction that facilitates interoperability between benchmark environments and training algorithms. The library includes various built-in environments and utilities to simplify research, and it is supported by many training libraries. This paper outlines the design decisions, key features, and differences from alternative APIs. The introduction highlights the evolution of RL, from the Deep Q-Networks (DQN) to the widespread adoption of deep neural network-based RL, which has led to significant advancements in tasks like Go, DoTA 2, and Starcraft 2. OpenAI Gym, the first widely adopted API, inspired Gymnasium, which offers improvements and updates to enable continued use in open-source RL research. Gymnasium focuses on the environment side of RL research, abstracting away agent design and implementation. It supports environment versioning, reproducibility, and easy customization through wrappers. The library also supports environment vectorization for efficient sampling and parallelization. The paper details the structure of a valid Gymnasium environment, including observation and action spaces, episode initiation, step functions, and rendering capabilities. It also describes the metadata and environment specification, which include static details and initialization specifications. Gymnasium includes a suite of implemented and tested environments, such as CartPole, Lunar Lander, and MuJoCo-based environments, along with third-party environments like Arcade Learning Environments and Safety Gymnasium. The paper also discusses related work, including competing environment APIs and projects that enhance environment performance and parallelization. In conclusion, Gymnasium serves as a robust and versatile platform for RL research, offering a unified API that enables compatibility across a wide range of environments and training algorithms. Its future will be shaped by the active community, aiming to continue its role as a centerpiece of the open-source RL research community.Gymnasium is an open-source library designed to provide a standardized API for reinforcement learning (RL) environments. Its primary contribution is a central abstraction that facilitates interoperability between benchmark environments and training algorithms. The library includes various built-in environments and utilities to simplify research, and it is supported by many training libraries. This paper outlines the design decisions, key features, and differences from alternative APIs. The introduction highlights the evolution of RL, from the Deep Q-Networks (DQN) to the widespread adoption of deep neural network-based RL, which has led to significant advancements in tasks like Go, DoTA 2, and Starcraft 2. OpenAI Gym, the first widely adopted API, inspired Gymnasium, which offers improvements and updates to enable continued use in open-source RL research. Gymnasium focuses on the environment side of RL research, abstracting away agent design and implementation. It supports environment versioning, reproducibility, and easy customization through wrappers. The library also supports environment vectorization for efficient sampling and parallelization. The paper details the structure of a valid Gymnasium environment, including observation and action spaces, episode initiation, step functions, and rendering capabilities. It also describes the metadata and environment specification, which include static details and initialization specifications. Gymnasium includes a suite of implemented and tested environments, such as CartPole, Lunar Lander, and MuJoCo-based environments, along with third-party environments like Arcade Learning Environments and Safety Gymnasium. The paper also discusses related work, including competing environment APIs and projects that enhance environment performance and parallelization. In conclusion, Gymnasium serves as a robust and versatile platform for RL research, offering a unified API that enables compatibility across a wide range of environments and training algorithms. Its future will be shaped by the active community, aiming to continue its role as a centerpiece of the open-source RL research community.
Reach us at info@study.space
[slides and audio] Gymnasium%3A A Standard Interface for Reinforcement Learning Environments