Label

Hi
Could you please tell me the reason form in your code below:

TraDeS/src/lib/dataset/generic_dataset.py

Line 679 in 3eafd24

ret['h_hm_down'][cls_id][pre_ct_int[1]] = 0

TraDeS/src/lib/dataset/generic_dataset.py

Line 680 in 3eafd24

ret['w_hm_down'][cls_id][pre_ct_int[0]] = 0

which are used for making the labels of cost volume.
It seems that in the code:

TraDeS/src/lib/dataset/generic_dataset.py

Line 677 in 3eafd24

ret['h_hm_down'][cls_id][:] = hm_tmp[:, pre_ct_int[0]]

TraDeS/src/lib/dataset/generic_dataset.py

Line 678 in 3eafd24

ret['w_hm_down'][cls_id][:] = hm_tmp[pre_ct_int[1], :]

the labels for h-hm and w-hm as already made.

You need to go to the loss computation process to have a look. The center, which is positive sample, has a different label from the negative samples. Same happens in the cls heatmap loss computation in original CenterNet.

Thanks for the reply.
Yes, it is. However, in my opinion the labels in CenterNet and in TraDeS are different in that, the labels ret['h_hm_down'] and ret['w_hm_down'] are used for the proposed cost volume, however, the labels in CenterNet are Gaussian heatmaps.
Thus, could you please give me details ? Especially for the

TraDeS/src/lib/dataset/generic_dataset.py

Line 679 in 3eafd24

ret['h_hm_down'][cls_id][pre_ct_int[1]] = 0

and

TraDeS/src/lib/dataset/generic_dataset.py

Line 680 in 3eafd24

ret['w_hm_down'][cls_id][pre_ct_int[0]] = 0

Why they are 0 here?
In the paper, the Y_xykl in the paper is 1 if the object (x, y) at frame t located at (k, l) in the previous frame, otherwise 0. Thus, I wonder if 0 here means the negative samples?
Thanks.

No, 0 here doesn't mean negatives. This is why I recommend you go to loss to look. out * (1-target), if target is 0, it will attend the softmax. If it is 1, it won't attend. This process is related to your last question. Pls look how the loss is calculated. In principle, all targets should be 0. But we want nearby pixels around the center to less attend softmax, as they are ambiguous. So we take advantage of the gaussian heatmap to determine how less it will attend. And for those negatives who are beyond the gaussian kernel radius, all their labels are 1.

Thank you so much for your reply.