Using SAC

Hi,

I saw you were working on making it possible to train SAC agents with rl_games. Is that possible already? I was checking the configs and couldn't find anything. Everything seems ppo related. So I guess you can't run SAC currently?

Hi BY571, SAC works. @ViktorM should have some yaml configs for IsaacGym.

I was facing the same issue.
The sac config is missing the configuration for network.space keyword. For example in the rl_games/configs/brax/sac_humanoid.yaml, if you replace line#12:

    space:
      continuous:

with

    space:
      continuous:
        mu_activation: None
        sigma_activation: None

        mu_init:
          name: default
        sigma_init:
          name: const_initializer
          val: 0
        fixed_sigma: True

it should work.

also this line need a quick fix from

rl_games/rl_games/algos_torch/players.py

Line 187 in 25e654c

obs_shape = torch_ext.shape_whc_to_cwh(self.state_shape)

to

obs_shape = torch_ext.shape_whc_to_cwh(self.obs_shape)

Thanks for finding a fix, @boxiXia! It worked in my tests. Will push the fixes soon.

About the config proposed fix, only fixed_sigma was missing and required:

    space:
      continuous:
        fixed_sigma: True