tylin/coco-caption

Add support for all Meteor languages

elliottd opened this issue · 1 comments

The evalaution toolkit only supports Meteor evaluation in English but image description is now being studied in non-English languages German, German again, Chinese, Japanese, and Turkish.

Would it be possible to extend Meteor support for other languages? I imagine this would involve two changes:

  • Replace the bundled version of Meteor with a bash script download of the entire package
  • Add a --language option to the evaluation script (default=en)

We are doing this in nmtpy btw: we download all paraphrase files with a script and pass -l <lang> to our coco evaluation script.