StepNeverStop/RLs

Thanks

Closed this issue · 2 comments

ZhuXianjinGitHub commented

thanks for your sharing.it is nice work. I wonder how to use the hiro in your package.

StepNeverStop commented
  1. for now, HIRO does not support the visual observation
  2. state of agent that returned from the environment should include the information of goal, i.e. state: [..., goal]
  3. change the configuration of HIRO in rls/algos/config.yaml:
hiro:
    gamma: 0.99
    intrinsic_reward_mode: os # os or cos
    ployak: 0.995
    high_scale: 1.0
    reward_scale: 1.0
    sample_g_nums: 100
    sub_goal_steps: 10
    fn_goal_dim: 0
    high_batch_size: 128
    high_buffer_size: 100000
    low_batch_size: 128
    low_buffer_size: 100000
    high_actor_lr: 1.0e-4
    high_critic_lr: 1.0e-3
    low_actor_lr: 1.0e-4
    low_critic_lr: 1.0e-3
    hidden_units:
        high_actor: [64, 64]
        high_critic: [64, 64]
        low_actor: [64, 64]
        low_critic: [64, 64]

you can specify the length of the goal information within the state vector by setting fn_goal_dim to the proper number.

Then, you can execute commands like python run.py --gym -a hiro --gym-env [env_id] to start your training process.

StepNeverStop commented

Close this issue now and feel free to reopen it. :)