Gymnasium is an open source Python library for developing and comparing reinforcement learning algorithms by providing a standard API to communicate between learning algorithms and environments, as well as a standard set of environments compliant with that API. This is a fork of OpenAI's Gym library by the maintainers (OpenAI handed over maintenance a few years ago to an outside team), and is where future maintenance will occur going forward
The documentation website is at gymnasium.farama.org, and we have a public discord server (which we also use to coordinate development work) that you can join here: https://discord.gg/bnJ6kubTg6
Gymnasium includes the following families of environments along with a wide variety of third-party environments
- Classic Control - These are classic reinforcement learning based on real-world problems and physics.
- Box2D - These environments all involve toy games based around physics control, using box2d based physics and PyGame-based rendering
- Toy Text - These environments are designed to be extremely simple, with small discrete state and action spaces, and hence easy to learn. As a result, they are suitable for debugging implementations of reinforcement learning algorithms.
- MuJoCo - A physics engine based environments with multi-joint control which are more complex than the Box2D environments.
- Atari - A set of 57 Atari 2600 environments simulated through Stella and the Arcade Learning Environment that have a high range of complexity for agents to learn.
- Third-party - A number of environments have been created that are compatible with the Gymnasium API. Be aware of the version that the software was created for and use the
apply_env_compatibilityingymnasium.makeif necessary.
To install the base Gymnasium library, use pip install gymnasium
This does not include dependencies for all families of environments (there's a massive number, and some can be problematic to install on certain systems). You can install these dependencies for one family like pip install gymnasium[atari] or use pip install gymnasium[all] to install all dependencies.
We support and test for Python 3.7, 3.8, 3.9 and 3.10 on Linux and macOS. We will accept PRs related to Windows, but do not officially support it.
The Gymnasium API models environments as simple Python env classes. Creating environment instances and interacting with them is very simple- here's an example using the "CartPole-v1" environment:
import gymnasium as gym
env = gym.make("CartPole-v1")
observation, info = env.reset(seed=42)
for _ in range(1000):
action = env.action_space.sample()
observation, reward, terminated, truncated, info = env.step(action)
if terminated or truncated:
observation, info = env.reset()
env.close()Please note that this is an incomplete list, and just includes libraries that the maintainers most commonly point newcommers to when asked for recommendations.
- CleanRL is a learning library based on the Gym API. It is designed to cater to newer people in the field and provides very good reference implementations.
- Tianshou is a learning library that's geared towards very experienced users and is design to allow for ease in complex algorithm modifications.
- RLlib is a learning library that allows for distributed training and inferencing and supports an extraordinarily large number of features throughout the reinforcement learning space.
- PettingZoo is like Gym, but for environments with multiple agents.
Gymnasium keeps strict versioning for reproducibility reasons. All environments end in a suffix like "_v0". When changes are made to environments that might impact learning results, the number is increased by one to prevent potential confusion. These inherent from Gym.
We have a roadmap for future development work for Gymnasium available here: Farama-Foundation#12