Datasets problems
Closed this issue · 2 comments
I've noticed that all datasets are currently in file format. Are there any plans to access them through online searches in the future?
Sorry, not sure what do you mean "online searches"? Could you explain in more detail?
If you mean online loading, you can access our datasets through the huggingface, for example:
from datasets import load_dataset
data = load_dataset('osunlp/TravelPlanner','test')['test']
The "test" can be substitued by "train" and "validation".
Thanks for your interest!
Our benchmark is built upon a self-contained sandbox for planning evaluation, which is challenging enough for current LLMs and advanced planning strategies. Therefore, we don't plan to build an online version in the near future.
However, we leave this option open and are happy to make an online version if we see remarkable progresses of language agents on the current environment.
Let us know if you have any other questions!
Best,
Kai