yrcong/RelTR

What happens when there are no relations in a sample?

gzamps opened this issue · 2 comments

Hi and thanks for sharing this interesting work!

I am working on supporting another dataset in vg format. In my dataset I have images with no relation between the objects and I wonder how to make the network support this.

There is an issue since the images still contain boxes of objects but targets['rel_annotations'] is empty. Therefore in matcher forward pass, indices has a different length from indices1.

I'd appreciate your input. Thanks in advance!

You can easily add a dummy predicate label "no_relation" and combine random bboxes with this dummy label.

Hi and thanks for the response. I am trying such an approach to ensure compatibility; In images with no relation between objects, I am using a mock ground truth triplet of [box0, no_relation, box0], which is [[0,0,0]].

However, I wonder if that negatively impacts the learning process, since this way, in images without relations we explicitly ask and supervise the network to produce this "no_relation" relation. And when actual relations exist, we do not have this.

I'd appreciate if you have any insight on this. Thanks!