MCG-NJU/MixFormerV2

how to calculate absolute coordinates

bin7777 opened this issue · 4 comments

coord_l = torch.sum((self.indice * prob_l), dim=-1) coord_r = torch.sum((self.indice * prob_r), dim=-1) coord_t = torch.sum((self.indice * prob_t), dim=-1) coord_b = torch.sum((self.indice * prob_b), dim=-1)
Is the 'prob' distribution probability of coordinates?why calculate the sum

Yes,prob is the probability distribution of coordinates. The weighted sum is expectation over the coordinates which is result.

Thanks for your reply.

It still makes me confused. For example,if the prediction of 'prob' at coordinates 20 is 0.7,but the 'prob' of prediction at coordinates 240 is 0.2, it is expected that the 'coord' will shift towards coordinates 240,because coordinates 180 is bigger.

The loss function (CIOU and L1)don't constraint the probability distribution of coordinates.I don't understand how the model learn to predict probability distribution of coordinates.

The problem you mentioned actually does exist, the model learn to look for the boundary of target. Even I have seen this phenomenon occasionally when there are similar objects in the search region, the probability heat map of boundary will show double or more peaks. But it works well in most instances to locate uniquely the target.
Btw, it can be the point that can further optimize the algorithm.

thanks