Missing prompts?
Closed this issue · 3 comments
fayolle commented
@friederrr It seems that all the prompts for Munkres 1&2, Olympiad, Durrett's probability, and Rudin 1& 2 are missing (empty fields in the json files) for all the data-sets.
For example:
friederrr commented
It's not missing, it's copyrighted, so we can't include it (was we mentioned in the paper, please have a look)
fayolle commented
Thank you. Sorry I didn't pay enough attention to this part (first paragraph of B.3).
It is funny, though, because these books are easily found on internet and were likely part of the training data of these LLMs.
friederrr commented
No worries :)
Yes, the internet is a big place - and while we were not able to include the prompts, unfortunately, there is the "ref" field to reconstruct exactly the page/exercise from where we took them.