DRLViz is an interactive tool visualizing the hidden recurrent state of an RL agent and its effect on affordances and policy.
DRLViz: Understanding Decisions and Memory in Deep Reinforcement Learning
Authors: Theo Jaunet, Romain Vuillemot, Christian Wolf
This repository contains the front-end of DRLViz, a visual analytics tool to explore the memory of agents trained with Deep Reinforcement learning.
(Designed to work on Google Chrome, at 1920*1080)
This tool is accessible using the following link: https://sical.github.io/drlviz/
To run this interface locally, download or clone this repository
git clone https://github.com/sical/drlviz.git
Open the downloaded directory and start any server. For demonstration sake we used: SimpleHTTPServer
python -m SimpleHTTPServer 8000
Once the server is launched, you should be able to access DRLViz at: http://localhost:8000/.
To use DRLViz with your own data, you must first down setup a local instance of DRLViz as described in the previous section.
Then, you can either:
Change line#64 in main.html and add your on file as it follows:
let datafile = "mydata.json" // Change "mydata.json" with your own dataset
Each episode must be at the JSON's root, and their key name must start with "episode" concatened with their incremented id (e.g. 'episode0', then, 'episode1' and so on ..). Your file must contain attributes as presented bellow.
{
episode0: {
actions: [0,4,5,...], # interger [0-7] corresponding to the index in ['left','Right','Forward','Backward','Left + Forward','Left + Backward', 'Right + forward','Right + backward']
fov: [ # array of objects corresponding to the objects in the agent's field of view
[
{object_position_y:320.3, # float indicating Y position of the observed object in the environment
object_position_x: 310.2, # float indicating X position of the observed object in the environment
object_id:, # interger indicating the object's id one per object
object_x:, # position X in pixels of the object in the current observation (e.g. 0 if it is on the left)
object_name:"" }, # String indicating the agent's type (e.g. health pack)
{..}
],
[
{...}
]
],
health: [100.0,90.0,52.6,...], # float indicating the agent's health status
hiddens: [[0.95456,0.6446,...],[1.0,0.544,...],...] , Array of floats corresponding to a hidden state.
inputs: ['','',...], # Image corresponding to the agent's observation. Those images are encoded in base64 strings.
orientations[270.3,285.6,...], # float indicating the agent's orientation scaled on 360 degrees
positions [[680.5,210.3],[682.3,215.6]], # array of float corresponding to the agent's position in the environment.
probabilities [[,],[,]], # float correpsoning the actions probabilities with index matching those of actions.
saliencies: ['','',...], # Image encoded in base64 strings, they will be used as background of inputs.
scores: [0.0,1.0,...] # float corresponding to either the agent's performance or its reward. This value must be accumalated.
},
episode1: {...}
}
In order to produce your own memory reductions, you can can use the script reduce.py. Such a script, is designed to work on python >= 3.5 and linux, it requires some python dependencies you can install with the following command.
pip install -r requirements.txt
This script allows you to select the memory order you want, and the number of elements to preserve. The outcome of this script is a file 'result.json' which you can then use in DRLViz.
If you find this usefull, consider citing the following:
Theo Jaunet, Romain Vuillemot, Christian Wolf DRLViz: Understanding Decisions and Memory
in Deep Reinforcement Learning url: https://diglib.eg.org/handle/10.1111/cgf13962
@article {10.1111:cgf.13962,
journal = {Computer Graphics Forum},
title = {{DRLViz: Understanding Decisions and Memory in Deep Reinforcement Learning}},
author = {Jaunet, Theo and Vuillemot, Romain and Wolf, Christian},
year = {2020},
publisher = {The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {10.1111/cgf.13962}
}