COCOEvalCap.evaluate() results collection and assignment
handsomeyang opened this issue · 1 comments
handsomeyang commented
print gts.keys()==imgIds
adding the above code to COCOEvalCap.evaluate() gives
False
In score, scores = scorer.compute_score(gts, res)
, scores are collected in the order of gts.keys()
, while in 'COCOEvalCap.setImgToEvalImgs()' scores are assigned correspondingly to imgIds
, doesn't it mean the results are assigned to the wrong images?
tylin commented
Thanks, this is a good catch. The per image evaluation score has wrong image association while the overall metrics, which are used to report on the leaderboard, are unaffected. The change has been pushed to fix the problem.