What happens when there are no relations in a sample?
gzamps opened this issue · 2 comments
Hi and thanks for sharing this interesting work!
I am working on supporting another dataset in vg format. In my dataset I have images with no relation between the objects and I wonder how to make the network support this.
There is an issue since the images still contain boxes of objects but targets['rel_annotations'] is empty. Therefore in matcher forward pass, indices has a different length from indices1.
I'd appreciate your input. Thanks in advance!
You can easily add a dummy predicate label "no_relation" and combine random bboxes with this dummy label.
Hi and thanks for the response. I am trying such an approach to ensure compatibility; In images with no relation between objects, I am using a mock ground truth triplet of [box0, no_relation, box0], which is [[0,0,0]].
However, I wonder if that negatively impacts the learning process, since this way, in images without relations we explicitly ask and supervise the network to produce this "no_relation" relation. And when actual relations exist, we do not have this.
I'd appreciate if you have any insight on this. Thanks!