Recursive implementation of Dictionary Replay Buffer from Stable Baselines3. For nested observation spaces.
- The main idea comes from stable_baselines3.common.preprocessing.get_obs_shape which recursively extract observation spaces's shape information. Transformed that to a queue-based implementation.
- Works with gym.vector.SyncVectorEnv
- DummyVecEnv does not support nested observations.
- Did not test with SubprocVecEnv.
- Queue-based implementation of
- creating memory
- adding obs to memory
- sampling from memory
- concatenating two nested observations
- normalizing observations and converting to torch