openai/gym

Difference between Breakout-v0, Breakout-v4 and BreakoutDeterministic-v4?

tianlinxu312 opened this issue · 8 comments

tianlinxu312 commented

Can anyone explain the difference between Breakout-v0, Breakout-v4 and BreakoutDeterministic-v4?

xysun commented

If you look at the atari_env source code, essentially:

  • v0 vs v4: v0 has repeat_action_probability of 0.25 (meaning 25% of the time the previous action will be used instead of the new action), while v4 has 0 (always follow your issued action)
  • Deterministic: a fixed frameskip of 4, while for the env without Deterministic, frameskip is sampled from (2,5) (code here)

There is also NoFrameskip-v4 with no frame skip and no action repeat stochasticity.

  • 👍44
  • 😄1
  • 🎉1
  • ❤️9
tianlinxu312 commented

If you look at the atari_env source code, essentially:

  • v0 vs v4: v0 has repeat_action_probability of 0.25 (meaning 25% of the time the previous action will be used instead of the new action), while v4 has 0 (always follow your issued action)
  • Deterministic: a fixed frameskip of 4, while for the env without Deterministic, frameskip is sampled from (2,5) (code here)

There is also NoFrameskip-v4 with no frame skip and no action repeat stochasticity.

Thank you very much. That's very clear.

xysun commented

Glad to help! Can you close the issue now it's solved?

christopherhesse commented

Thanks for answering @xysun!

rfali commented

Breakout-v0 vs Breakout-v4
v0: repeat_action_probability of 0.25 (meaning 25% of the time the previous action will be used instead of the new action)
v4: repeat_action_probability of 0 (always follow issued action)

Breakout-v4 vs BreakoutDeterministic-v4 vs BreakoutNoFrameskip-v4
game-vX: frameskip is sampled from (2,5), meaning either 2, 3 or 4 frames are skipped [low: inclusive, high: exclusive]
game-Deterministic-vX: a fixed frame skip of 4
game-NoFrameskip-vX: with no frame skip

Breakout-v4 vs Breakout-ram-v4
game-ram-vX: Observation Space (128,). Observation is given as a vector of size 128.
game-vX: Observation Space (210,160,3). Observation is given as pixels of Width:210, Height:160, Depth:3

The atari environment source code has been removed from Gym [AFAIK] and you can see it on the ALE's GitHub. See source code here

Other useful references:
This video
ALE v0.7 Blog which introduces the v5 versions of the game [Sep 2021].

alexxcollins commented

This thread has helped me out a lot.

@rfali - I think you may have a typo and your post should read:

Breakout-v4 vs Breakout-ram-v4
game-ram-vX: Observation Space (128,). Observation is given as a vector of size 128.
game-vX: Observation Space (210,160,3). Observation is given as pixels of Width:210, Height:160, Depth:3

basictask commented

Dear rfali,
I have checked, what you have written in game-v4 / game-ram-v4 is swapped up as alexxcollins said

rfali commented

@alexxcollins @kknydnai thanks for noticing that typo, I have updated my comment!