Evaluating on the test set

Question

Evaluating on the test set

Opened this issue 3 years ago · 4 comments

Hi, seems the data released only contains train and dev data, could you give some instructions about how we could get the testing performance? Thanks

Answer 1 · 2021-10-12T03:19:47.000Z

Thanks for asking, but our test set would not be publicly available. The evaluation system is coming soon. Currently, we can help you access the performance on the test set through manual evaluation, please contact galaxychen@sjtu.edu.cn.

Answer 2 · 2023-09-05T03:30:21.000Z

Hello @TravelLeraLone,

I sent my prediction file for evaluation to zhao_mengxin@sjtu.edu.cn and chenlusz@sjtu.edu.cn, as described in the README.md. However, I have not received a response to my email zhanghui@4paradigm.com yet.

Answer 3 · 2023-09-05T03:59:12.000Z

Hello @huizhang0110 ,

We have just finished the evaluation process of your submission and sent the results to you through your email. Thanks for your interest in the WebSRC dataset.

Answer 4 · 2023-09-05T05:39:02.000Z

Thank you for your kind and fast response @TravelLeraLone
Could you please assist in updating the result in the Leaderboard (https://x-lance.github.io/WebSRC/)