Question about parameters used in the process of calculating meteor score.
3-sung opened this issue · 1 comments
"""""
Hi, the problem looks related to METEOR computation which is perhaps related to the related package installation. I would suggest to turn everything off related to METEOR calculation and see if the program runs fine in your environment. You may compute METEOR locally later on, on the generated summaries!
One alternative is to use the METEOR from the NLTK library (https://www.nltk.org/_modules/nltk/translate/meteor_score.html).
Originally posted by @wasiahmad in
#14 (comment)
""""""
Thank for your answer.
It worked. However, I wonder what the parameters (alpha, beta, gamma) should be set in the process of calculating the meteor score locally by NLTK library which you recommend.
Thank you for reading!
I am not really sure, I think you can try the default values first. Then change the scores to see if you find any major difference in the results. I guess you won't see any major difference.
By the way, I found the datasets API (https://huggingface.co/metrics/meteor) from Hugginface also uses the default values of alpha=0.9, beta=3, gamma=0.5 (https://github.com/huggingface/datasets/blob/master/metrics/meteor/meteor.py). So, I think using default values should be fine.