zhunzhong07/person-re-ranking

Jaccard Distance values look off - over 90% are 1s

010akv opened this issue · 0 comments

With no reranking, distance values are distributed evenly across the whole 0-1 range. But with reranking, although the accuracy improves ~2%, the distance histogram gets compressed into narrow range (0.7-0.9) with several distance values > 1. This looks incorrect. I studied the problem further and found that this is due to Jaccard distance that has over 90% values as 1s.

How I got this issue:

Use custom dataset (unable to share the dataset, 4k images, 200 identities) with reranking (k1=54, k2=6, lambda=0.3. - this gave the best overall fscr). Plot histograms of

  • original distance - looks right - distributed evenly across the whole range 0-1
  • Jaccard distance - looks way off - over 85% values are 1s.
  • final distance - looks off due to Jaccard distance. Distributed in narrow window - 0.7-0.9. Many values are > 1!

Expected behavior:

Final distance must be evenly distributed across the whole 0-1 range.

Questions:

  1. What is 2-temp_min here? Is it same as the max operation - eq. 10 in the paper?
  2. Does k1 depend on the number of images in the dataset? V is of size NxN (total num images). When k1 << N V has many 0s even after qe. Could this be why my Jaccard distances are all 1s?
  3. Is this expected of the distance distribution? If yes, why and if not, any suggestions to fix?