Mini-Uni-RLHF is minimal out-of-the-box annotation tool for researchers in RLHF community, powered by streamlit. The library only contains the core functionality of Uni-RLHF platform, which is designed with a focus on simple over easy. We recommend this version for researchers and small tasks that require no heavy configuration to achieve a complete workflow!
- Out of the box, no heavy configuration required
- Easily scalable based on streamlit framework and universal dataset format
- Enrich documentation
- Add visual and keypoint feedback support
- Add online RL training mode
Install dependencies in less than 30 seconds:
cd /path/to/Mini-Uni-RLHF
pip install -r requirements.txt
😎 Enjoy the mini tool!
streamlit run main.py
We provide a very small walker dataset in Mini-Uni-RLHF/datasets/dataset_resource/vd4rl/walker/walker_walk_medium/half_walker_walk_medium.hdf5
for users to test. And now we suppoert d4rl
, atari
, smarts
and vd4rl
domain. (TODO)
All you need to do to plug into the new dataset is write a python file named {$mode}_{$domain}.py
like offline_atari
and the corresponding few functions! See details at Mini-Uni-RLHF/datasets/offline_customization_dataset.py
.
class BaseOfflineDataset(object):
def __init__(self):
def load_offline_dataset(self):
def get_episode_boundaries(self):
def sample(self):
def visualize(self):
def query(self):
- Create the project:
- Annotate labels for the project:
- Export the annotated dataset:
Distributed under the MIT License. See LICENSE.txt
for more information.
For any questions, please feel free to email yuanyf@tju.edu.cn.
If you find our work useful, please consider citing:
@inproceedings{anonymous2023unirlhf,
title={Uni-{RLHF}: Universal Platform and Benchmark Suite for Reinforcement Learning with Diverse Human Feedback},
author={Yuan, Yifu and Hao, Jianye and Ma, Yi and Dong, Zibin and Liang, Hebin and Liu, Jinyi and Feng, Zhixin and Zhao, Kai and Zheng, Yan}
booktitle={The Twelfth International Conference on Learning Representations, ICLR},
year={2024},
url={https://openreview.net/forum?id=WesY0H9ghM},
}