[HANDS-ON BUG] Error reading hyperparameters on Unit3.ipynb

[rodrigoclira]

Describe the bug

I faced an error trying to run the unit3.ipynb on Google Colab. It happened here:

!python -m rl_zoo3.train --algo dqn  --env SpaceInvadersNoFrameskip-v4 -f logs/ -c dqn.yml

rl_zoo3.train was not finding the hyperparameters. I noticed that dqn.yaml was in the right place. The script was reaching the file, but was not finding the hyperparameter configuration.

The solution seems very simple but RL-Course step by step needs to be updated.
It seems RLZOO hyperparamets needs to be named as 'atari' to fit the current module code. I guess it is because Google updated Python in the Colab, changing our world (I've faced another similar issue in a different project).

The solution is:

atari:
  env_wrapper:
    - stable_baselines3.common.atari_wrappers.AtariWrapper
  frame_stack: 4
  policy: 'CnnPolicy'
  n_timesteps: !!float 1e7
  buffer_size: 100000
  learning_rate: !!float 1e-4
  batch_size: 32
  learning_starts: 100000
  target_update_interval: 1000
  train_freq: 4
  gradient_steps: 1
  exploration_fraction: 0.1
  exploration_final_eps: 0.01
  # If True, you need to deactivate handle_timeout_termination
  # in the replay_buffer_kwargs
  optimize_memory_usage: False

instead of

SpaceInvadersNoFrameskip-v4:
  env_wrapper:
    - stable_baselines3.common.atari_wrappers.AtariWrapper
  frame_stack: 4
  policy: 'CnnPolicy'
  n_timesteps: !!float 1e7
  buffer_size: 100000
  learning_rate: !!float 1e-4
  batch_size: 32
  learning_starts: 100000
  target_update_interval: 1000
  train_freq: 4
  gradient_steps: 1
  exploration_fraction: 0.1
  exploration_final_eps: 0.01
  # If True, you need to deactivate handle_timeout_termination
  # in the replay_buffer_kwargs
  optimize_memory_usage: False

Please observe that the only change is the name on the top. It is running right now.

Material

I was using Google Colab.

[rodrigoclira]

Well, I run again and it worked using the original file. I will close the issue.