How to split the data?

Question

How to split the data?

Closed this issue 3 years ago · 2 comments

Hi Alexander,

Thanks for publishing the dataset. After downloading the data, do you know how to split the data to train/valid/test for a fair comparison with your experimental results?

Tyson

Answer 1 · 2021-06-22T13:42:17.000Z

Hi Tyson,

Thank you for your interest in our work! We provide lists of the IDs and their split in a .json file which correspond to the example_id in the jsonl file. For example, data-non-processed/nyt.ids.json has lists of train, val, and test ids which can be matched in data-non-processed/nyt.jsonl.

Answer 2 · 2021-06-23T04:50:05.000Z

Thanks for your clear explanation!