Old URL for the question generation model in QA eval

[martiansideofthemoon]

Hi @danieldeutsch,
Thank you for building this very useful library.

I wanted to point out a small bug that I found while using the QAEval metric. I think the URL for the question generation model here links to the older model, which is incompatible with the newer versions of HuggingFace. I fixed it by using the Google Drive link on the danieldeutsch/qaeval repository.

kalpesh@Kalpeshs-MacBook-Pro:Downloads$ wget https://cogcomp.seas.upenn.edu/models/qaeval-experiments/model.tar.gz
kalpesh@Kalpeshs-MacBook-Pro:Downloads$ tar -xvzf model.tar.gz 
x config.json
x weights.th
x vocabulary/
x vocabulary/non_padded_namespaces.txt
kalpesh@Kalpeshs-MacBook-Pro:Downloads$ cat config.json | grep "bart-large"
        "model_name": "bart-large"
        "model_name": "bart-large"

The bart-large namespace is incompatible the transformers 3.0.2 version, which needs facebook/bart-large.

[danieldeutsch]

Thanks, this is helpful, and to be honest I don't remember why one version works and the other does not! I will work on trying to fix this.

For what it's worth, I have Dockerized implementations of QAEval and many other metrics in this repository if you have access to Docker. I find it far easier to run metrics with that code base instead of this one.

[sn-masoud]

Hi @danieldeutsch,
Thanks for the library!
I am adding may question to this issue, since it may be relevant. I get this error

Error loading state dict for _QuestionGenerationModel 
Missing keys: ['bart.final_logits_bias', 'bart.lm_head.weight']

Any suggestions to resolve the issue?

Thanks