clone and install dependencies
git clone https://github.com/kscalelabs/gym-kmanip.git && cd gym-kmanip
conda create -y -n gym-kmanip python=3.10 && conda activate gym-kmanip
pip install -e .
run tests
pip install pytest
pytest tests/test_env.py
visualize the mujoco scene
python gym_kmanip/examples/1_view_env.py
record a video of the mujoco scene
python gym_kmanip/examples/2_record_video.py
🤗 K-Scale HuggingFace Datasets
data is recorded via teleop, this requires additional dependencies
pip install opencv-python==4.9.0.80
pip install vuer==0.0.30
pip install rerun-sdk==0.16.0
start the server on the robot computer
python gym_kmanip/examples/4_record_data_teleop.py
start ngrok on the robot computer.
ngrok http 8012
open the browser app on the vr headset and go to the ngrok url
data is visualized using rerun
rerun gym_kmanip/data/test.rrd
✅ solo arm w/ vision
✅ dual arm w/ vision
✅ torso w/ vision
✅ inverse kinematics using mujoco
⬜️ tune and improve ik
⬜️ recording dataset via teleop
⬜️ training policy from dataset
⬜️ evaluating policy on robot
- Gymnasium is used for environment
- MuJoCo is used for physics simulation
- PyTorch is used for model training
- Rerun is used for logging datasets
- HuggingFace is used for dataset & model storage
- Vuer teleop only is used for visualization
- ngrok teleop only is used for networking
helpful links and repos
@misc{teleop-2024,
title={gym-kmanip},
author={Hugo Ponte},
year={2024},
url={https://github.com/kscalelabs/gym-kmanip}
}