tylin/coco-caption

COCOEvalCap.evaluate() results collection and assignment

handsomeyang opened this issue · 1 comments

print gts.keys()==imgIds
adding the above code to COCOEvalCap.evaluate() gives
False

In score, scores = scorer.compute_score(gts, res), scores are collected in the order of gts.keys(), while in 'COCOEvalCap.setImgToEvalImgs()' scores are assigned correspondingly to imgIds, doesn't it mean the results are assigned to the wrong images?

tylin commented

Thanks, this is a good catch. The per image evaluation score has wrong image association while the overall metrics, which are used to report on the leaderboard, are unaffected. The change has been pushed to fix the problem.