Inconsistency between PyCOCOCallback and BoxCOCOMetrics
Opened this issue · 2 comments
Current Behavior:
While constructing the ground truth dataset, PyCOCOCallback computes the number of detections within annotations as follows:
num_detections = ops.sum(ops.cast(gt_classes > 0, "int32"), axis=-1)
at line 102.
In constrast, BoxCOCOMetrics tackles it via this approach:
ground_truth["num_detections"] = [ ops.sum(ops.cast(y_true["classes"] >= 0, "int32"), axis=-1) ]
at lines 300-302.
Consequently, the results produced by the two are quite different, especially when your data set has only a few distinct labels and the encoding is plain integer assignment with enumeration starting from 0.
Expected Behavior:
Since label encoding schemes in most Keras CV models enumerate the classes starting from 0, which holds in my case as well, I believe that BoxCOCOMetrics approach should be applied to PyCOCOCallback. Alternatively this could be made a parameter to be determined by the user.
Steps To Reproduce:
Version:
- 0.8.2
Anything else:
hi, any updates, please?
I also agree that the PyCOCOCallback
is incorrect here, and that it should be updated to match the BoxCOCOMetrics
(Assuming that the package still uses 0 as the first class index across the repo)