facebookresearch/dynabench

new perturb script returns a format that doesn't quite work with the evaluation server

TristanThrush opened this issue · 1 comments

For fairness, it returns a file with contents like this:

[{example}]
[{example}]
.
.
.

For robustness, it returns a file with contents like this:

[{example}, {example}, ...]
[{example}, {example}, ...]
.
.
.

Both of these output files should actually look like this instead:

{example}
{example}
.
.
.