Evaluation Dataset mentioned in Hugging GPT paper is not available
ssdasgupta opened this issue · 2 comments
As mentioned in the paper - "Furthermore, we also invite some expert annotators to label task planning for some complex requests (46 examples) as a high-quality human annotated dataset. We also plan to further improve the quality and quantity of this dataset to better help us to evaluate the LLM capability in planning, which leaves as future work.", are you planning to release the evaluation dataset? Or if it is there already in the repository, could you send me the folder location?
Thanks.
@ssdasgupta We are currently working with our labeling teams to iteratively improve the quality of this dataset and our legal team to ensure compliance of the dataset release. We will release a work about this dataset in the future. Please be patient.