/LearningHumanoidWalking

Training a humanoid robot for locomotion using Reinforcement Learning

Primary LanguagePythonBSD 2-Clause "Simplified" LicenseBSD-2-Clause

LearningHumanoidWalking

Code for the papers:

Code structure:

A rough outline for the repository that might be useful for adding your own robot:

LearningHumanoidWalking/
├── envs/                <-- Actions and observation space, PD gains, simulation step, control decimation, init, ...
├── tasks/               <-- Reward function, termination conditions, and more...
├── rl/                  <-- Code for PPO, actor/critic networks, observation normalization process...
├── models/              <-- MuJoCo model files: XMLs/meshes/textures
├── trained/             <-- Contains pretrained model for JVRC
└── scripts/             <-- Utility scripts, etc.

Requirements:

Usage:

Environment names supported:

Task Description Environment name
Basic Walking Task 'jvrc_walk'
Stepping Task (using footsteps) 'jvrc_step'

To train:

$ python run_experiment.py train --logdir <path_to_exp_dir> --num_procs <num_of_cpu_procs> --env <name_of_environment>

To play:

We need to write a script specific to each environment.
For example, debug_stepper.py can be used with the jvrc_step environment.

$ PYTHONPATH=.:$PYTHONPATH python scripts/debug_stepper.py --path <path_to_exp_dir>

What you should see:

Ascending stairs:
climb_up

Descending stairs:
climb_down

Walking on curves:
curve

Citation

If you find this work useful in your own research:

@inproceedings{singh2022learning,
  title={Learning Bipedal Walking On Planned Footsteps For Humanoid Robots},
  author={Singh, Rohan P and Benallegue, Mehdi and Morisawa, Mitsuharu and Cisneros, Rafael and Kanehiro, Fumio},
  booktitle={2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids)},
  pages={686--693},
  year={2022},
  organization={IEEE}
}