same rotated bbox while iou == 0
ucasqcz opened this issue · 6 comments
box = [[0, 0], [100, 0], [200, 100], [100, 100]]
cal_iou(box,box)
while result iou is 0?
is this a bug or?
Would you please provide the complete code to reproduce the result? My previous code indeed had problems with the same boxes but I've already fixed it.
Also, I provide test cases in test_corner_cases.py to validate my code. Would you please send me the result you get?
i modify the representation of rbbox from (x,y,w,h,theta) -> (x1,y1,...,x4,y4) format in function cal_iou of oriented_iou_loss.py
def cal_iou(box1: torch.Tensor, box2: torch.Tensor):
corners1 = box1
corners2 = box2
inter_area, _ = oriented_box_intersection_2d(corners1, corners2) # (B, N)
#area1 = box1[:, :, 2] * box1[:, :, 3]
area1 = calc_area(corners1)
area2 = calc_area(corners2)
#area2 = box2[:, :, 2] * box2[:, :, 3]
u = area1 + area2 - inter_area
iou = inter_area / u
return iou, corners1, corners2, uwhere calc_area function is from Shoelace Theorem
def calculate_area(idx_sorted: torch.Tensor, vertices: torch.Tensor):
"""calculate area of intersection
Args:
idx_sorted (torch.Tensor): (B, N, 9)
vertices (torch.Tensor): (B, N, 24, 2)
return:
area: (B, N), area of intersection
selected: (B, N, 9, 2), vertices of polygon with zero padding
"""
idx_ext = idx_sorted.unsqueeze(-1).repeat([1, 1, 1, 2])
selected = torch.gather(vertices, 2, idx_ext)
total = selected[:, :, 0:-1, 0]*selected[:, :, 1:, 1] - \
selected[:, :, 0:-1, 1]*selected[:, :, 1:, 0]
total = torch.sum(total, dim=2)
area = torch.abs(total) / 2
return area, selectedthen for the test_corner_cases.py
def test_iou():
expect = 1.0
box_1 = [[0, 0], [100, 0], [200, 100], [100, 100]]
box_1 = torch.tensor(box_1, device=device).reshape(-1, 2)
box_2 = [[0, 0], [100, 0], [100, 100], [0, 100]]
box_2 = torch.tensor(box_2, device=device).reshape(-1, 2)
box_1 = box_1[None, ...].repeat(3, 1, 1)
box_2 = box_2[None, ...].repeat(3, 1, 1)
result = cal_iou(box_1[None, ...], box_1[None, ...])[0].cpu().numpy()
print(f'expect: {expect}, get: {result[0,0]}')if i am wrong, point out for me, thanks
I get your idea. You want to use corners directly as input instead of the five parameters. The idea is fine. The problem is, the box_1 in your test case is not a rectangle. In this case, my code doesn't work. (BTW, I got the correct result with box_2, which is a rectangle.)
Check the function box1_in_box2 in box_intersection_2d.py. This function assumes that the inputs are two rectangles. You should modify this function to make your code work.
The function is based on the projection of vectors and only works with rectangles. To support other types of polygons, you should use a more general method, such as point-in-polygon.
YES, i have found the problem, and i replace the follow function to replace the original box1_in_box2,
def box1_in_box2(corners1: torch.Tensor, corners2: torch.Tensor):
"""check if corners of box1 lie in box2
Convention: if a corner is exactly on the edge of the other box, it's also a valid point
Args:
corners1 (torch.Tensor): (B, N, 4, 2)
corners2 (torch.Tensor): (B, N, 4, 2)
Returns:
c1_in_2: (B, N, 4) Bool
"""
# a->b->c->d --> b->c->d->a
corners2_clockwise = torch.roll(corners2, -1, -2)
# a->b->c->d --> d->a->b->c
corners2_anti = torch.roll(corners2, 1, -2)
edge_clockwise = corners2_clockwise - corners2
edge_anti = corners2_anti - corners2
edge_clockwise = edge_clockwise.unsqueeze(2).repeat([1, 1, 4, 1, 1])
edge_anti = edge_anti.unsqueeze(2).repeat([1, 1, 4, 1, 1])
# x dot for
c1 = corners1.unsqueeze(3).repeat([1, 1, 1, 4, 1])
c2 = corners2.unsqueeze(2).repeat([1, 1, 4, 1, 1])
point_to_p = c1 - c2
ABXAP = edge_clockwise[..., 0] * point_to_p[...,
1] - edge_clockwise[..., 1] * point_to_p[..., 0]
ADXAP = edge_anti[..., 0] * point_to_p[...,
1] - edge_anti[..., 1] * point_to_p[..., 0]
# (B, N, 4, 4)
indicator = ABXAP * ADXAP
indicator = indicator <= 0
# (B, N , 4)
indicator = indicator.all(dim=-1)
return indicator
now, the iou of box_1 and box_2 is 0 while the correct iou is 0.3,
if the two polygon share several points, the build_vertices function in
def oriented_box_intersection_2d(corners1: torch.Tensor, corners2: torch.Tensor):
"""calculate intersection area of 2d rectangles
Args:
corners1 (torch.Tensor): (B, N, 4, 2)
corners2 (torch.Tensor): (B, N, 4, 2)
Returns:
area: (B, N), area of intersection
selected: (B, N, 9, 2), vertices of polygon with zero padding
"""
import pdb
pdb.set_trace()
inters, mask_inter = box_intersection_th(corners1, corners2)
c12, c21 = box_in_box_th(corners1, corners2)
vertices, mask = build_vertices(
corners1, corners2, c12, c21, inters, mask_inter)
sorted_indices = sort_indices(vertices, mask)
return calculate_area(sorted_indices, vertices)
will add duplicate points, at the same time, the sorted_indices does not take mask into consideration,
the final iou is 0, do you have any idea to fix that case?
I don't have enough time to test your new code, since I have my own work. I can only give some basic ideas.
Yes, things would become tricky if the two polygons share points. In my code, I have to loose some conditions so that the IoU can be correctly calculated and the code is numerically stable. In my box1_in_box2 function, I use:
cond1 = (p_ab / norm_ab > - 1e-6) * (p_ab / norm_ab < 1 + 1e-6) # (B, N, 4)
cond2 = (p_ad / norm_ad > - 1e-6) * (p_ad / norm_ad < 1 + 1e-6) # (B, N, 4)
instead of the theoretical conditions:
cond1 = (p_ab / norm_ab >= 0) * (p_ab / norm_ab <= 1) # (B, N, 4)
cond2 = (p_ad / norm_ad >= 0) * (p_ad / norm_ad <= 1) # (B, N, 4)
I think you could probably modify the following line to loose the condition.
indicator = indicator <= 0
Yes, I also saw the duplicated points previously. I've added some code in the CUDA-Kernel to remove them. Since the added part is like some sort of dirty hack, it might not work with your code. If the duplicated points are still a problem, you could consider modifying the CUDA-Kernel.
// for corner case: the two boxes are exactly the same.
// in this case, idx would have duplicate elements, which makes the shoelace formula broken
// because of the definition, the duplicate elements only appear in the first 8 positions
// (they are "corners in box", not "intersection of edges")
if (num_valid[i] == 8){
int counter = 0;
for (int j=0; j<4; ++j){
int check = idx[i*MAX_NUM_VERT_IDX + j];
for (int k=4; k<INTERSECTION_OFFSET; ++k){
if (idx[i*MAX_NUM_VERT_IDX + k] == check)
counter++;
}
}
if (counter == 4){
idx[i*MAX_NUM_VERT_IDX + 4] = idx[i*MAX_NUM_VERT_IDX + 0];
for (int j = 5; j<MAX_NUM_VERT_IDX; ++j){
idx[i*MAX_NUM_VERT_IDX + j] = pad;
}
}
}
thanks for you time and advices, I will try to fix it.