Add training from coloured images
Closed this issue · 2 comments
In the version of PAAC that we use, the agent is trained with images of the emulator in grayscale. We want to add an option to train the agent with RGB images. Consequently, the convolutional layers of the network will need to be adapted to the RGB images. See branch PlayInColours
We made some test over a set of 3 Atari games : Ms Pacman, Breakout and Space Invaders.
For each test, we plotted the mean of the last 50 rewards at the end of a training episode. On the first graph, the x axis is in millions of steps. On the second graph, the x axis is in hours.
Test on Ms Pacman :
In green, the RGB training. In grey, greyscale training.
With the greyscale training, the agent achieves better performances, in a smaller amount of time or steps.
Test on Breakout :
In green, the RGB training. In red, greyscale training.
With the greyscale training, the agent achieves the same performances with the same number of steps but needs less time than RGB training.
Test on Space Invaders :
In orange, the RGB training. In blue, the greyscale training.
With the greyscale training, the agent achieves the same performances with the same number of steps but needs less time than RGB training.
Overall, the training is slower when using RGB images and it doesn't change the performances a lot.
However, as the greyscale formula changes depending on the image module or emulator your are using, with RGB images it's easier to compare results .