Gym Novel Gridworlds are OpenAI Gym environments for developing and evaluating AI agents that can detect and adapt to unknown sudden novelties in their environments. In each environment, the agent needs to craft objects using multiple recipes, which requires performing certain steps in some sequence.
git clone https://github.com/gtatiya/gym-novel-gridworlds.git
cd gym-novel-gridworlds
pip install -e .
If you prefer, you can install directly from PyPI:
pip install gym-novel-gridworlds
@article{goel2021novelgridworlds,
title={NovelGridworlds: A Benchmark Environment for Detecting and Adapting to Novelties in Open Worlds},
author={Shivam Goel* and Gyan Tatiya* and Matthias Scheutz and Jivko Sinapov},
journal={International Foundation for Autonomous Agents and Multiagent Systems, AAMAS},
year={2021}
}
Python 3.7.9 and following packages were used for development:
pip install gym==0.18.0
pip install keyboard==0.13.5
pip install matplotlib==3.3.3
pip install numpy==1.19.4
pip install stable-baselines==2.10.1
pip install tensorflow-gpu==1.15.0
import gym
import gym_novel_gridworlds
env = gym.make('NovelGridworld-Pogostick-v1')
done = False
while not done:
env.render()
action = env.action_space.sample()
obs, reward, done, info = env.step(action)
env.close()Play the game using keyboard interface. To change the environment and to inject a novelty you must edit keyboard_interface.py.
python tests/keyboard_interface.py
In each environment, the agent needs to craft objects using multiple recipes, which requires performing certain steps in some sequence.
The default observation space is of type gym.spaces.Dict that contains information about the map, agent_location, agent_facing_id, and inventory_items_quantity.
Feel free to use Observation Wrappers to change observation space and Wrappers to limit action space and more.
NovelGridworld-Bow-v0
|
NovelGridworld-Bow-v1
|
NovelGridworld-Pogostick-v0
|
NovelGridworld-Pogostick-v1
|
Note that deprecated environments do not support any wrappers.
NovelGridworld-v0
|
NovelGridworld-v1
|
NovelGridworld-v2
|
NovelGridworld-v3
|
NovelGridworld-v4
|
NovelGridworld-v5
|
If the trained agent exists, then you can see it in action using:
python enjoy.py -env <env id> -episodes <number of episodes>
For example, enjoy PPO2 on NovelGridworld-v0 during 10 episodes:
python enjoy.py -env NovelGridworld-v0 -episodes 10
import gym
import gym_novel_gridworlds
from gym_novel_gridworlds.novelty_wrappers import inject_novelty
env_id = 'NovelGridworld-Pogostick-v1'
env = gym.make(env_id)
novelty_name = 'axe'
novelty_arg1 = 'iron'
novelty_arg2 = 'true'
difficulty = 'medium'
env = inject_novelty(env, novelty_name, difficulty, novelty_arg1, novelty_arg2)| Novelty Name | Novelty Details | Easy | Medium | Hard | novelty_arg1 | novelty_arg2 |
|---|---|---|---|---|---|---|
| addchop | A new action (Chop) is added. It's like break action, but instead of 1 item, agent will get 2 items, but step_cost will be higher (1.2 times). | - | - | - | - | - |
| additem | A new item is introduced in the map. | 1-10% of the free locations are occupied by the new item. | 10-20% of the free locations are occupied by the new item. | 20-30% of the free locations are occupied by the new item. | Any item name (e.g. arrow, spring) | - |
| addjump | A new action (Jump) is added, when it's executed, the agent jumps 2 blocks forward. | - | - | - | - | - |
| axe | An axe is introduced. Using axe reduces the step_cost when Break action is used. If novelty_arg2 is set to true, the agent will get 2 items in inventory after break action instead of 1. There are 12 variants of this novelty: 3 difficulties x 2 attributes of axe x 2 quantities on breaking. | An axe is in the inventory. The agent must select axe to use it. | An axe in the map. The agent must go near the axe to grab it. | A recipe to craft axe is given. The agent must craft the axe. | Attribute of axe (e.g. wooden, iron) | Optional: true or false.If true, the agent will get 2 items in inventory after break action otherwise 1. |
| axetobreak | An axe is introduced and axe required to break items. Using axe reduces the step_cost when Break action is used. There are 6 variants of this novelty: 3 difficulties x 2 attributes of axe. | An axe is in the inventory. The agent must select axe to use it. | An axe in the map. The agent must go near the axe to grab it. | A recipe to craft axe is given. The agent must craft the axe. | Attribute of axe (e.g. wooden, iron) | - |
| breakincrease | The agent gets 2 items in the inventory when the agent breaks that item instead of 1. | - | - | - | Optional: any existing item (e.g. tree_log), Default: all items | - |
| crate | A new item, crate, is introduced in the map. When the crate is broken, some ingredients of the goal item to craft are added in the inventory. | 100% of the ingredients of the goal item to craft are added in the inventory. | 50-90% of the ingredients of the goal item to craft are added in the inventory. | 10-50% of the ingredients of the goal item to craft are added in the inventory. | - | - |
| extractincdec | The agent gets more/less items in the inventory when the agent extracts it instead of the default quantity. | - | - | - | increase or decrease | - |
| fence | Items in the map are blocked by fence. The agent must break the fence to reach the blocked items. There are 6 variants of this novelty: 3 difficulties x 2 attributes of fence. | 20-50% items in the map are blocked by fence. | 50-90% items in the map are blocked by fence. | 90-100% items in the map are blocked by fence. | Attribute of fence (e.g. wooden, iron) | - |
| fencerestriction | Restrict breaking an item around fence until fence(s) are broken. All fences are always breakable. 50-90% items in the map are blocked by fence (fence medium novelty). | Breaking one fence next to the item, will enable breaking that item. This is same as fence novelty. | Breaking fences on one side of the item, will enable breaking that item. | Breaking all fences around the item, will enable breaking that item. | Attribute of fence (e.g. wooden, iron) | - |
| firewall | Walls are replaced by fire-walls, agent dies when it's next to fire-wall. | - | - | - | - | - |
| remapaction | Actions are remapped randomly | Only manipulation actions (e.g. Forward, Left, Right, Break, Place_tree_tap, Extract_rubber) are remapped. | Manipulation actions and craft actions (e.g. Craft_plank, Craft_stick, Craft_tree_tap, Craft_pogo_stick) are remapped. | Manipulation actions, craft actions and select actions (e.g. Select_crafting_table, Select_plank, Select_pogo_stick, Select_rubber, Select_stick, Select_tree_log, Select_tree_tap) are remapped. | - | - |
| replaceitem | Replace an existing item with a new item. | 5-10% of existing item are replaced with the new item. | 40-90% of existing item are replaced with the new item. | 100% of existing item are replaced with the new item. | Item to replace (e.g. wall). Must be an existing item. | Item to replace with (e.g. stone-wall). Must be a new item. |
addchop
|
additem
|
addjump
|
axe
|
axetobreak
|
breakincrease
|
crate
|
extractincdec
|
fence
|
fencerestriction
|
firewall
|
remapaction
|
replaceitem
|
import gym
import gym_novel_gridworlds
from gym_novel_gridworlds.observation_wrappers import LidarInFront
env_id = 'NovelGridworld-Pogostick-v1'
env = gym.make(env_id)
env = LidarInFront(env, num_beams=8)| Observation Name | Observation Details |
|---|---|
| AgentMap | Agent's local view within a range (agent_view_size), agent_facing_id, inventory_items_quantity. |
| LidarInFront | Send several beans (num_beams) at equally spaced angles in 360 degrees in front of agent + agent's current inventory. |
import gym
import gym_novel_gridworlds
from gym_novel_gridworlds.wrappers import LimitActions
env_id = 'NovelGridworld-Bow-v0'
env = gym.make(env_id)
env = LimitActions(env, {'Forward', 'Left', 'Right', 'Break', 'Craft_bow'})| Wrapper Name | Wrapper Details |
|---|---|
| LimitActions | Limit the actions in the environment. |
| SaveTrajectories | Save agent trajectories in the environment. |