Metrics for YOLOv8 model

[alejones]

Will user metrics ever be supported for YOLOV8 or is it expected that I use a COCO callback?

I also opened this issue in the Keras Repo and they suggested I move it here.
keras-team/keras#19416

Current Behavior:

When I compile a YOLOv8 model with metrics I get an error

model.compile(
    classification_loss="binary_crossentropy",
    box_loss="ciou",
    optimizer=optimizer,
    metrics = [keras.metrics.Recall(), keras.metrics.Precision()]
)

ValueError: User metrics not yet supported for YOLOV8

Expected Behavior:

Adding the requested metrics

Steps To Reproduce:

Gist

Version:

Keras version: 3.1.1
Keras_cv version: 0.8.2

[innat]

It should be supported. Using callbavk for this is ineffective approach.

[alejones]

Any updates?

[VincentGallot]

While we're waiting for the resolution of this issue, is there an alternative ? I see callbacks mentionned, but do you really have to "manually" (as in with an explicit "for" loop) infer on the validation dataset and then compute and store the metric ? I'm sorry if the question is dumb I don't have much experience yet.

[alejones]

Loss and Box Loss are calculated automatically. I don't think you can write an explicit for loop, but I believe you would need to write a custom callback for Recall and Precision.

[alejones]

Any updates on this?

[divyashreepathihalli]

Right now looks like coco metrics is hardcoded. We don't have plans of adding more. For your usecase I would suggest subclassing and adding custom callback.