A standalone library to randomize various OpenAI Gym Environments.
Domain Randomization is a idea that helps with sim2real transfer, but surprisingly has no general open source implementations. This library hopes to fill in that gap by providing a standalone library that you can use in your own work.
Within our library, you can find randomized-environment implementations for:
LunarLander-v0Pusher3DoF-v0Humanoid-v2HalfCheetah-v2FetchPush-v1ResidualPushEnv-v0ResidualPickAndPlaceEnv-v0
import randomizer
import gym
from randomizer.wrappers import RandomizedEnvWrapper
env = RandomizedEnvWrapper(gym.make('ResidualPickAndPlaceRandomizedEnv-v0'), seed=123)
env.randomize()
'''
List of the parameters that you wish to randomize.
The elements in the list can either be a string ("default") or a number between 0 and 1. (refer below example)
Important Note : Please maintain the same order as in the corresponding .json file.
'''
env.randomize(["default", 0.1])
env.reset()We also provide an implementation for vectorized environments, which is useful in reinforcement learning for parallel training or evaluation.
There are three main file to get your randomized environment to work correctly: environment file, configuration file, and environment registration.
Since the standardized gym environments (MuJoCo, Box2D, Pybullet) don't include randomization, we'll need to create our own randomization files. For Box2D, the instructions are quite similar (and you can use Lunar Lander as a starting example), but in this example, we'll look at MuJoCo.
For MuJoCo, our library rewrites the XML (some parameters however, can be changed after the XML is compiled, but we find it inconvenient to do this when some parameters can and can't). We load the XML and then locate the randomization parameters we're interested in. The main function is called update_randomized_params(), which gets called by our library on reset.
To set the ranges that you can randomize, you'll need a configuration file, which just is a list of JSON objects that tells us what dimensions to randomize, and the ranges (specified by a default value + min/max multipliers).
To use the environment, you'll need to register the environment. While this looks like a normal environment file, you'll want to notice the kwargs, which our library uses to find the correct configuration file to pull the randomized values from.