whylabs/whylogs

Calculating drift scores non-interactively broken for images

Closed this issue · 2 comments

Description

When applied to profiles generated for images, whylogs.viz.drift.column_drift_algorithms.calculate_drift_score returns only {'image': None}, whereas the platform is able to show results of data drift detection as well as the plot generated by summary_drift_report method of NotebookProfileVisualizer.

Looking at the source code of the summary_drift_report I noticed that in one of the steps the function whylogs.migration.uncompound._uncompound_dataset_profile was used on the target and reference profiles. Once I applied it to my profiles, calculate_drift_score returned expected results for image features.
For tabular data, calculate_drift_score works without applying the _uncompound_dataset_profile function.

  • System info
    • Operating system and version: macOS 13.4.1 -->
  • Steps to reproduce:
    • here you can find a Colab notebook reproducing the issue

image
image

Suggestions

  • enable calculating data drift and other operations on images as for tabular data, so it is not needed to apply private functions (_uncompound_dataset_profile) previously

  • I have reviewed the Guidelines for Contributing and the Code of Conduct.

Thanks for filing the issue and the details @drajsel!

Assigning to @FelipeAdachi to take a look.

Thank you for raising this and for the clear explanation @drajsel!

I just pushed a PR to address this: #1424.

It should be included in next week's whylogs release.