how do you take care of the presence of 'and' in the output in the evaluation
vkaul11 opened this issue · 1 comments
vkaul11 commented
Sometimes the output also can emit "and" for say the multi-value case. Should we not account for that in the output and evaluate the string match accordingly or you never faced such an issue?
3728882, 7210606, 7120868, and 8606962
was my output and my outputs are
['8606962', '7120868', '3728882', '7210606']
hsiehjackson commented
That is fine because our metric checks whether the answers exist in the prediction string.