Possibility of releasing closed-book Atlas models?
lovhag opened this issue · 1 comments
Hi,
Firstly, thank you for sharing the Atlas model in a very well-documented repo.
I'm currently working on an evaluation of Atlas on ParaRel (a dataset that tests for model consistency, with corresponding paper). However, to do a proper evaluation of potential benefits of retrieval augmentation for the evaluation results, I need to compare against a closed-book Atlas.
From what I can see, you have not shared your closed-book Atlas model weights. So I was wondering if there is a possibility to release these model weights as well? I could train my own attempt at a closed-book Atlas model, but it wouldn't be the same, since you haven't released your 350M train passages from common crawl.
Grateful for any help!
/Lovisa