RL environment list

A comprehensive list of categorized reinforcement learning environments.

Started and maintained by Andrew Szot and Youngwoon Lee.

Related Collections

Two other resources for RL environments:

Table of Contents

Environments are listed alphabetically.

Robotics

Assistive-gym
  • 6 assistive tasks (ScratchItch, BedBathing, Feeding, Drinking, Dressing, and ArmManipulation).
  • 4 commercial robots (PR2, Jaco, Baxter, Sawyer).
  • 2 human states: static or active (takes actions according to a separate control policy).
  • Customizable female and male human models. 40 actuated human joints (head, torso, arms, waist, and legs).Realistic human joint limit.
Dexterous Gym
  • Extensions of the OpenAI Gym Dexterous Manipulation Environments.
  • Multiple environments requiring cooperation between two hands (handing objects over, throwing/catching objects).
  • "Pen Spin" Environment - train a hand to spin a pen between its fingers.
DoorGym
  • Train a policy to open up various doors.
  • Unity integration.
  • Random door knob generator and door knob dataset.
Gym Gazebo 2
  • Toolkit for developing and comparing reinforcement learning algorithms using ROS 2 and Gazebo.
Gym Ignition
  • Provides the capability of creating reproducible robotics environments for reinforcement learning research.
  • Accelerated and multiprocess execution
IKEA Furniture Assembly
  • Complex long-horizon manipulation tasks.
  • Includes 80+ furniture models, customizable background, lighting and textures.
  • Features Baxter, Sawyer, and more robots.
Meta-World
  • 50 diverse robot manipulation tasks on a simulated Sawyer robotic arm.
  • Also includes a variety of evaluation modes varying the number of training and testing tasks.
Playroom
  • Variety of tasks in desk scenario.
  • Evaluation code and play dataset will be included soon.
RAISIM
  • Raisim is a physics engine for rigid-body dynamics simulation. Although it is a general physics engine, it has been mainly used/tested for robotics and reinforcement learning so far. It features an efficient implementation of recursive algorithms for articulated system dynamics (Recursive Newton-Euler and Composite Rigid Body Algorithm). RaisimLib is an exported cmake package of raisim.
RLBench
  • 100 unique, hand designed tasks.
  • Vision-guided manipulation, imitation learning, multi-task learning, geometric computer vision and few-shot learning.
Robosuite
  • A set of standard benchmarking tasks in robots.
  • Defines a framework for easily creating new tasks and environments.
Roboschool
  • Control robots in simulation.
  • Can use other physics engines other than MuJoCo.
  • Alternative to standard OpenAI Gym mujoco environments.
  • Easy to train multiple agents at once.
Rex-Gym
  • OpenAI Gym environments for an open-source quadruped robot (SpotMicro)

Games

Bomberland
  • Multi-agent 2D grid environment based on Bomberman.
Coin-Run
  • Training environment which provides a metric for an agent’s ability to transfer its experience to novel situations.
Gym Retro
  • Gym Retro lets you turn classic video games into Gym environments for reinforcement learning and comes with integrations for ~1000. games.
Holodeck
  • High Fidelity Simulator for Reinforcement Learning and Robotics Research.
MarLÖ : Reinforcement Learning + Minecraft
  • A high level API built on top of Project MalmÖ to facilitate Reinforcement Learning experiments with a great degree of generalizability, capable of solving problems in pseudo-random, procedurally changing single and multi agent environments within the world of the mediatic phenomenon game Minecraft.
Minecraft
  • Data API for the MineRLv0 dataset.
  • Also has minecraft environment simulator with basic built in tasks.
PHYRE
  • Benchmark for physical reasoning that contains a set of simple classical mechanics puzzles in a 2D enviroment.
Soccer Simulator
  • Can control one or all football players at a time.
  • Includes football academy for diverse scenarios such as various passing scenarios.
StarCraft 2
  • Provides an interface for RL agents to interact with StarCraft 2, getting observations and sending actions.
SuperMario
  • Gym wrapper for the Super Mario levels. Includes many levels.
TorchCraft
  • Python interface for playing "StarCraft: Brood War".
VizDoom
  • ViZDoom allows developing AI bots that play Doom using only the visual information (the screen buffer).

Multi-Task Learning

Meta-World
  • 50 diverse robot manipulation tasks on a simulated Sawyer robotic arm.
  • Also includes a variety of evaluation modes varying the number of training and testing tasks.
Multiworld
  • Variety of Gym GoalEnvs that return the goal in the observation.
Playroom
  • Variety of tasks in desk scenario.
  • Evaluation code and play dataset will be included soon.
RoboDesk
  • Multi-task RL benchmark that comes with tasks from easy to hard, with dense and sparse rewards.
  • Based on the Playroom desk env, with more robust physics settings and controls that are suitable for RL.
RLBench
  • 100 unique, hand designed tasks.
  • Vision-guided manipulation, imitation learning, multi-task learning, geometric computer vision and few-shot learning.

Suites

DeepMind Control Suite
  • A variety of benchmarking continuous control tasks.
OpenAI Gym Atari
  • 59 Atari 2600 games.
OpenAI Gym Classic
  • Control theory problems from the classic RL literature.
OpenAI Gym Mujoco
  • Continuous control tasks, running in a fast physics simulator.
OpenAI Gym Robotics
  • Simulated goal-based tasks for the Fetch and ShadowHand robots.
Unity Agents
  • A number of control tasks in the Unity engine.
  • Includes example of parallel learning.
PettingZoo
  • It's akin to a multi-agent version of OpenAI's Gym library.
  • Includes Atari, Classic Games, Particle Environments and many more.

Generalization

Cartpole Generalization
  • Test generalization through varying the mass and length of the pole in CartPole.
Natural RL Environment
  • Play common gym tasks with randomly generated backgrounds to test generalization.
DMControl Generalization Benchmark
  • Generalization benchmark for continuous control tasks from DeepMind Control Suite. Includes hundreds of environments with randomized colors and dynamic video backgrounds of varying difficulty.
Procgen
  • 16 simple-to-use procedurally-generated environments which provide a direct measure of how quickly a reinforcement learning agent learns generalizable skills.
  • The environments run at high speed (thousands of steps per second) on a single core.
Animal-AI Testbed
  • 900 tasks reflecting various cognitive skills of animals.
  • Powered by Unity ml-agent.
Crafter
  • Open world survival game that evaluates many agent abilities within one environment.
  • Faster and easier than Minecraft but poses some of the same challenges.
  • Can be used to evaluate reward-based or unsupervised agents (e.g. artificial curiosity).

Navigation

DeepMind Lab
  • Provides a suite of challenging 3D navigation and puzzle-solving tasks for learning agents.
gym-maze
  • A simple 2D maze environment where an agent (blue dot) finds its way from the top left corner (blue square) to the goal at the bottom right corner (red square).
  • The objective is to find the shortest path from the start to the goal.
gym-minigrid
  • Lightweight and fast grid world implementation with various included tasks.
  • Easily modifable and extendable.
gym-miniworld
  • Minimalistic 3D interior simulator as an alternative to VizDoom or DMLab.
  • Easily modifable and extendable.
Obstacle Tower
  • Traverse through procedurally generated floors which get progressively harder.
  • Challenging visual inputs.

Home (More Navigation)

AI2THOR
  • An Interactive 3D Environment for Visual AI
Gibson
  • 3d navigation in indoor scans
Habitat
  • AI Habitat enables training of embodied AI agents (virtual robots) in a highly photorealistic & efficient 3D simulator, before transferring the learned skills to reality
HoME: a Household Multimodal Environment
  • A platform for agents to learn from vision, audio, semantics, physics, and interaction with objects and other agents, all within a realistic context.
House3D
  • House3D is a virtual 3D environment which consists of thousands of indoor scenes equipped with a diverse set of scene types, layouts and objects sourced from the SUNCG dataset
  • It consists of over 45k indoor 3D scenes, ranging from studios to two-storied houses with swimming pools and fitness rooms
  • All 3D objects are fully annotated with category labels
  • Multiple observation modalities
  • Fast rendering at thousands of frames per second
MINOS
  • MINOS is a simulator designed to support the development of multisensory models for goal-directed navigation in complex indoor environments.
  • MINOS leverages large datasets of complex 3D environments and supports flexible configuration of multimodal sensor suites.
Nvidia ISAAC simulator
  • A virtual robotics laboratory and a high-fidelity 3D world simulator
VirtualHome
  • A 3D environment allowing to simulate and generate videos of activities as sequences of actions and interaction.

Multi-Agent

Massive Multi Agent Game Environment
  • We consider MMORPGs (Massive Multiplayer Online Role Playing Games) the best proxy for the real world among human games: they are complete macrocosms featuring thousands of agents per persistent world, diverse skilling systems, global economies, complex emergent social structures, and ad-hoc high stakes single and team based conflict.
Multi-agent Particle Environment
  • A simple multi-agent particle world with a continuous observation and discrete action space, along with some basic simulated physics
OpenAI Multi-Agent Competition Environments
  • Contains many continous control, multi-agent tasks.
OpenAI Multi-Agent Hide and Seek
  • A team of seekers and a team of hiders.
  • Both teams can use tools to achieve their objective.
RoboSumo
  • Sumo-wrestling between two ants using continuous control.
SUMO-RL
  • Multi-agent traffic signal control using SUMO simulator.

Safety

Assistive-gym
  • 6 assistive tasks (ScratchItch, BedBathing, Feeding, Drinking, Dressing, and ArmManipulation).
  • 4 commercial robots (PR2, Jaco, Baxter, Sawyer).
  • 2 human states: static or active (takes actions according to a separate control policy).
  • Customizable female and male human models. 40 actuated human joints (head, torso, arms, waist, and legs).Realistic human joint limit.
DeepMind AI Safety Gridworlds
  • This is a suite of reinforcement learning environments illustrating various safety properties of intelligent agents.
Safety Gym
  • Tools for accelerating safe exploration research.

Autonomous Driving

Autonomous Vehicle Simulator
  • Open source simulator for autonomous vehicles built on Unreal Engine / Unity, from Microsoft AI & Research
BARK-ML
  • Open source environments and reinforcement learning agents for autonomous driving and behavior generation.
CARLA
  • CARLA has been developed from the ground up to support development, training, and validation of autonomous driving systems
DeepDrive Self Driving Car Simulator
  • End-to-end simulation for self-driving cars
DeepMind StreetLearn
  • A C++/Python implementation of the StreetLearn environment based on images from Street View, as well as a TensorFlow implementation of goal-driven navigation agents solving the task published in “Learning to Navigate in Cities Without a Map”, NeurIPS 2018
DeepGTAV v2
  • A plugin for GTAV that transforms it into a vision-based self-driving car research environment.
DuckieTown
  • Self-driving car simulator for the Duckietown universe.
Highway-Env
  • A collection of environments for autonomous driving and tactical decision-making tasks
SVL Simulator
  • Simulation software to accelerate safe autonomous vehicle development
  • Custom environment to support openai gym interface
TORCS
  • TORCS, The Open Racing Car Simulator is a highly portable multi platform car racing simulation
  • Many tracks, opponents and cars available
  • Easy to modify

Humanoid

Full Body Muscle Simulator
  • A basic simulation and control for full-body Musculoskeletal system
Osim-rl
  • Reinforcement learning environments with musculoskeletal models. Task: learning to walk/move/run using musculoskeletal models.
Roboschool
  • Control robots in simulation.
  • Can use other physics engines other than MuJoCo.
  • Alternative to standard OpenAI Gym mujoco environments.
  • Easy to train multiple agents at once.

Text

Jericho
  • A learning environment for man-made Interactive Fiction games.
TextWorld
  • TextWorld is a sandbox learning environment for the training and evaluation of reinforcement learning (RL) agents on text-based games.

Misc

Reco Gym
  • Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising.
RecSim
  • A Configurable Recommender Systems Simulation Platform from Google.
Gym-ANM
  • Environments that model Active Network Management (ANM) tasks in electricity distribution networks.

Physics Simulators

Disclaimer

The list is not comprehensive, so please let us know if there is any environment that is missing, miscategorized, or needs a different description or image. Please submit an issue or open a pull request.