/image-caption-metrics

a py3 lib for NLP & image-caption metrics : BLEU METEOR CIDEr ROUGE SPICE WMD

Primary LanguagePython

image-caption-metrics

a py3 lib for NLP & image-caption metrics : BLEU METEOR CIDEr ROUGE SPICE WMD

Features below:

  • python3 support
  • add new metric WMD

Requirements

  • java 1.8+

  • python 3 (Python2 has not been tested)

    • gensim

  • Stanford CoreNLP 3.6.0(download)

    • add stanford-corenlp-3.6.0.jar to pycocoevalcap/spice/lib/
    • add stanford-corenlp-3.6.0-models.jar to pycocoevalcap/spice/lib/

  • google_word2vec_model for WMD(download)

    • unzip it and add GoogleNews-vectors-negative300.bin to pycocoevalcap/wmd/data

Usage

See in demo.py

  • Note that the input format must be the same as the file in examples/gts.json and examples/res.json
  • It seems can't run in windows(error about java), run it on Linux
import pycocoevalcap.eval as E

with open('examples/gts.json', 'r') as f: 
    gts = json.load(f)
with open('examples/res.json', 'r') as f:
    res = json.load(f)
    
ans = E.eval(gts,tes)
print(ans)
bleu = E.get_bleu(gts,res)
print(bleu)
cider = E.get_cider(gts,res)
print(cider)

References