Jaccard Distance values look off - over 90% are 1s
010akv opened this issue · 0 comments
010akv commented
With no reranking, distance values are distributed evenly across the whole 0-1 range. But with reranking, although the accuracy improves ~2%, the distance histogram gets compressed into narrow range (0.7-0.9) with several distance values > 1. This looks incorrect. I studied the problem further and found that this is due to Jaccard distance that has over 90% values as 1s.
How I got this issue:
Use custom dataset (unable to share the dataset, 4k images, 200 identities) with reranking (k1=54, k2=6, lambda=0.3. - this gave the best overall fscr
). Plot histograms of
- original distance - looks right - distributed evenly across the whole range 0-1
- Jaccard distance - looks way off - over 85% values are 1s.
- final distance - looks off due to Jaccard distance. Distributed in narrow window - 0.7-0.9. Many values are > 1!
Expected behavior:
Final distance must be evenly distributed across the whole 0-1 range.
Questions:
- What is 2-temp_min here? Is it same as the max operation - eq. 10 in the paper?
- Does
k1
depend on the number of images in the dataset? V is of size NxN (total num images). Whenk1 << N
V has many 0s even after qe. Could this be why my Jaccard distances are all 1s? - Is this expected of the distance distribution? If yes, why and if not, any suggestions to fix?