allenai/lumos

Missing Webshop/mind2web data

Closed this issue · 3 comments

pokerme7777 commented

Hi ,

Thank you for open-sourcing this excellent work.

May I kindly ask how I can obtain your evaluation data? The current repository seems to be missing the evaluation data for Webshop and Mind2Web.

I am trying to run a demo to gain a better understanding of the Lumos framework.

Hope you can help.

Thanks a lot!

yuchenlin commented

@WadeYin9712 plz take a look at this issue?

WadeYin9712 commented

Hi Pokerme,

Thank you for the question!

The data can be obtained from Mind2Web and WebShop's original repo. Especially for the WebShop, the setup process is recorded in the README file in eval/webshop. Mind2Web's data is mainly obtained from Huggingface. Since the evaluation data and environment docker is too large for the two datasets, I didn't put them into this repo.

pokerme7777 commented

Thanks you so much for your response.

Got it. I would setup the env for these two web dataset.
Thanks for your work again!

  • ❤️1