This repository describes a lightweight first person point of view environment
called roundworld and the reinforcement learning environment rlbug, which
implements a series of lightweight tasks designed to explore
offline and online continual reinforcement learning.
This environment was developed in the context of DARPA's Lifelong Learning Machines program and it is geared to exploring online and continual learning for agents interacting in an open world environment.
As of 2020 we couldn't find an environment that was 1) lightweight, 2) easily reconfigurable, and 3) could be used to build curricula or sequences of tasks where we can control which specific information is reused over the lifetime of a single agent.
This repository contains the following:
roundworld: a package implementing a lightweight "engine" for a first person point of view environment.rlbug: a module withinroundworldimplementing a RL environment based onroundworld. Its interface is gym-compatible, thoughgymnasiumis not a dependency in order to keep the lightweight philosophy. A separate implementation fully integrated intogymnasiumis in the works.
rlbug was used as a testbed for the general policy algorithm inspired
on the architecture of the insect brain. This work was presented at the
Offline Reinforcement Learning Workshop at the
Neural Information Processing Systems Conference (NeurIPS) 2022.
There are currently three tasks in rlbug:
-
SingleTargetis a task where the agent has to reach an object. The object's position and size is randomly generated during each episode. The task can be initialized with a number between 1 and 27 representing the type of object. If a number is used the same type of object is used in all episodes. Otherwise, a random object is selected for each episode. -
Target1in5is a task where the agent need to reach one target among five objects. Each task is initialized by a tuple of three numbers, corresponding to the target, a decoy, and background objects. The position of the objects is chosen randomly at the start of each episode to prevent the agent from learning to move in a predetermined trajectory. -
SaliencyTargetis a task where the agent needs to reach an object using a saliency input. This can be used as a baseline example.
Check the examples folder for some trivial examples.
Copyright © 2020, UChicago Argonne, LLC
roundworld is distributed under the terms of BSD License. See LICENSE