Gymnasium: A Standard Interface for Reinforcement Learning Environments

Gymnasium: A Standard Interface for Reinforcement Learning Environments

24 Jul 2024 | Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U. Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, Rodrigo Perez-Vicente, Andrea Pierré, Sander Schulhoff, Jun Jet Tai, Hannah Tan, Omar G. Younis
Gymnasium is an open-source library providing an API for reinforcement learning (RL) environments. Its main contribution is a central abstraction for wide interoperability between benchmark environments and training algorithms. Gymnasium includes various built-in environments and utilities to simplify researchers' work and is supported by most training libraries. This paper outlines the main design decisions for Gymnasium, its key features, and the differences to alternative APIs. Gymnasium is a maintained fork of OpenAI Gym, bringing many improvements and API updates to enable its continued usage for open-source RL research. It focuses entirely on the environment side of RL research, abstracting away the aspect of agent design and implementation. The only restriction on the agent is that it must produce a valid action as specified by the environment's action space. Gymnasium's environment interface is agnostic to the internal implementation of the environment logic, enabling the use of external programs, game engines, network connections, etc. Reproducibility is crucial in academia, and Gymnasium has several features to support this, including environment versioning, recreating environments, and episodic seeding. Gymnasium provides a suite of wrappers that can be easily applied to an existing environment, along with an interface that allows users to implement their own wrappers. This significantly simplifies many pre- and post-processing steps useful in research. Environment vectorization is common practice in RL research, where multiple copies of the same environments are run concurrently, making it possible to batch the policy inference and improve effective sampling performance. Gymnasium supports an explicit gym.VectorEnv base class which includes some environment-agnostic vectorization implementations, but also makes it possible for users to implement arbitrary vectorization schemes, preserving compatibility with the rest of the Gymnasium ecosystem. Gymnasium includes a suite of implemented, extensively tested, and researched environments such as CartPole, Lunar Lander, and MuJoCo-based environments. It also supports various third-party environments compatible with it. Gymnasium serves as a robust and versatile platform for RL research, offering a unified API that enables compatibility across a wide range of environments and training algorithms. By focusing on key aspects such as reproducibility, easy customization through wrappers, and environment vectorization, Gymnasium ensures a streamlined and efficient workflow for researchers. The future of Gymnasium will be shaped by its active community, hopefully continuing to serve as a centrepiece of the open-source RL research community for many years to come.Gymnasium is an open-source library providing an API for reinforcement learning (RL) environments. Its main contribution is a central abstraction for wide interoperability between benchmark environments and training algorithms. Gymnasium includes various built-in environments and utilities to simplify researchers' work and is supported by most training libraries. This paper outlines the main design decisions for Gymnasium, its key features, and the differences to alternative APIs. Gymnasium is a maintained fork of OpenAI Gym, bringing many improvements and API updates to enable its continued usage for open-source RL research. It focuses entirely on the environment side of RL research, abstracting away the aspect of agent design and implementation. The only restriction on the agent is that it must produce a valid action as specified by the environment's action space. Gymnasium's environment interface is agnostic to the internal implementation of the environment logic, enabling the use of external programs, game engines, network connections, etc. Reproducibility is crucial in academia, and Gymnasium has several features to support this, including environment versioning, recreating environments, and episodic seeding. Gymnasium provides a suite of wrappers that can be easily applied to an existing environment, along with an interface that allows users to implement their own wrappers. This significantly simplifies many pre- and post-processing steps useful in research. Environment vectorization is common practice in RL research, where multiple copies of the same environments are run concurrently, making it possible to batch the policy inference and improve effective sampling performance. Gymnasium supports an explicit gym.VectorEnv base class which includes some environment-agnostic vectorization implementations, but also makes it possible for users to implement arbitrary vectorization schemes, preserving compatibility with the rest of the Gymnasium ecosystem. Gymnasium includes a suite of implemented, extensively tested, and researched environments such as CartPole, Lunar Lander, and MuJoCo-based environments. It also supports various third-party environments compatible with it. Gymnasium serves as a robust and versatile platform for RL research, offering a unified API that enables compatibility across a wide range of environments and training algorithms. By focusing on key aspects such as reproducibility, easy customization through wrappers, and environment vectorization, Gymnasium ensures a streamlined and efficient workflow for researchers. The future of Gymnasium will be shaped by its active community, hopefully continuing to serve as a centrepiece of the open-source RL research community for many years to come.
Reach us at info@study.space