Quantitative Results
Closed this issue · 2 comments
In Quantitative Results of your paper, AIDS leads to a relative improvement of mAP over CLUE by 23% and of P@100 by 21%. But, I can not see a table as the Quantitative Results table of GitHub in the paper. At the same time, compared with the table of GitHub, AIDS only leads to a relative improvement of mAP over CLUE by 9% and of P@100 by 8%
I'm sorry that there is no table in the paper due to space constraints, but there is a bar plot depicting the same AP as in the table provided in the README.
There seems to be a confusion about the terms "relative" and "absolute" or "actual" improvement. The absolute difference between the performance x_base of a baseline method and the performance x_other of another method is abs(x_other - x_base).
You seem to be reporting the absolute improvement of AID over CLUE, which is 9.7% in terms of mAP and 12.3% in terms of P@100.
Here I am referring to the values from the first table in the README, since it is based on using all images in the dataset as queries and not just a 70-query subset.
However, when comparing quantities, one often reports the relative change, which is what we have done here.
Consider the following example: The absolute difference between 5 and 6 and between 1,000 and 1,001 is the same, but that change is much more significant for 6-5 than for 1,001-1,000.
The relative change takes this into account and is defined as: abs(x_other - x_base) / x_base.
In our case, this results in (0.5188 - 0.4221) / 0.4221 = 0.2291 for mAP and (0.7212 - 0.5978) / 0.5978 = 0.2064 for P@100, which is what we reported in the paper. We explicitly denoted the numbers as relative improvement to make this clear.
Please feel free to ask any question you might have about this or other issues related to AID or our paper.
Thank you for your patient answer.
From your detailed answer, I know that 23% is the relative improvement, not absolute improvement. I will read papers carefully and not ignore any words.
Thanks again.