Rainbow SpaceY is a game in which a rocket learns how to take off from one planet (the origin planet) and lands successfully on a target planet. The rocket can only learn these by trial and error which means this problem can’t be done in a supervised way and it must be solved using Reinforcement Learning. For the RL formulation, we have utilized some physical measurements as our state, engineered our own reward function based on distances and orientation, and rockets moves in four directions as our actions. In order to solve this problem, we have implemented the famous DQN, DDQN, Dueling DQN, and used the Prioritized DQN from an open source project. At the end, we integrated them into one agent called Semi-Integrated agent. Although all the algorithms succeeded in achieving the task, the Semi-Integrated agent achieved the highest success rate. This indicates that combining different Extensions of DQN can result in better performance, although it’s not guaranteed.
Implementation Details We implemented the game from scratch using the OpenAI Gym Python interface. Our algorithms were implemented using the PyTorch library version 1.1 and Python 3.7. We utilized the GPUs of our laptops for training the models.