Suggested max_episode_length
Closed this issue · 1 comments
Nice environment!
Just wonder are there any suggested values for max_episode_length according to the previous runs? Or the environment will eventually terminate due to sticky actions? Thanks!
Thanks, I'm glad you like MinAtar!
It's hard to suggest a max_episode_length since besides freeway, which has a built-in timeout, in most of the game a strong agent can continue for an essentially arbitrary amount of time.
As you say, even with perfect play, termination should eventually occur due to sticky actions in most cases. Though in seaquest an agent could potentially continue forever by simply never descending (though this would also result in 0 reward). This is true to the original Atari game but potentially a bit inconvenient from an evaluation perspective if your agent gets stuck in this strategy.
If the concern is with how to measure training data given unknown episode length I'd recommend focusing on the number of training frames, not episodes. Further discussion on this point can be found in section 3.1.3 of this paper:
https://www.jair.org/index.php/jair/article/download/11182/26388