can you use previous rationale for predict？

[susht3]

hi，i find that you use previous rationale to get answer id, and use previous answer id for prediction.
i also use previous rationale to extract answer, but when i submit to the leaderboard, author tell me that i cannot use previous rationale

[hsinyuan-huang]

Yes, I am using the rationale to get answer id. And during leaderboard submission, the authors also told me the same. So in the script I submitted, I changed to finding the answer id in the whole context with the highest overlap. Hence there is a slight inconsistency between training and testing.

I believe you could also not use rationale during training to remove this inconsistency. This should give you a little boost in performance.

[susht3]

Yes, I am using the rationale to get answer id. And during leaderboard submission, the authors also told me the same. So in the script I submitted, I changed to finding the answer id in the whole context with the highest overlap. Hence there is a slight inconsistency between training and testing.

I believe you could also not use rationale during training to remove this inconsistency. This should give you a little boost in performance.

so, do you train with rationale, and test with highest overlap between context and gold answer? or both on highest overlap?