huggingface/deep-rl-class

[HANDS-ON BUG] The environment ``CartPole-v1`` used in unit 4 has weird configurations

k0T0z opened this issue · 9 comments

k0T0z commented

Describe the bug

Try to run the original notebook failed.

  1. The env.reset() returns a tuple of two items for CartPole-v1 only. We expect only the state.
  2. The env.step(action) returns a tuple of 5 items for CartPole-v1 only. We expect only 4 items in the tuple.
  3. The softmax function has a dimension of 1 which is wrong, it should be 0.
  4. The record_video() function fails for CartPole-v1 environment only for some reason.

Material

I didn't use Google Colab and here is my OS info:

  • Windows 10
k0T0z commented

Hmmmmm, it seems that the code runs well on Colab but not locally!!

simoninithomas commented

Hey there 👋 sorry for the delay. But you're the first to encounter this issue. That might be also the version of Cartpole you use. For this we use Gym not Gymnasium "You may be wondering why we install gym and not gymnasium, a more recent version of gym? Because the gym-games we are using are not updated yet with gymnasium."

k0T0z commented

No problem at all, as I said the problem comes when running this code locally, dk why, maybe I made a mistake.

simoninithomas commented

I think it's because you might not use the same environment. Unit 4 uses gym == 0.21 not gymnasium 😄

k0T0z commented

Hmmm, you may be right yeah, I will check and confirm.

simoninithomas commented

Hey there 👋 Did you found if it was because of that?

k0T0z commented

Sorry for being late, yeah, it is definitely because of inconsistent versions so I am closing this.
Thank you.

ra9hur commented

Even I have been running this locally and using gymnasium. I made respective changes for env.reset() and env.step(action).

Training and evaluation went fine.

However, I am struck at record_video() while pushing to Hub. I get an error "The image must have at least two spatial dimensions".

I did change eval_env to include render_mode.
eval_env = gym.make(env_id, render_mode="rgb_array")

Any suggestions ?

unit4.zip

ra9hur commented

Restarted the system / notebook and this error does not show up. However, "push to hub" process takes up all the memory and finally crashes.