jacobmarks/zero-shot-prediction-plugin

ExecutionError: Validation error. Path: label_field_owlvit. Reason: Required property

Closed this issue · 2 comments

kais-bedioui commented

I encountered the error in the title when trying to run the zero_shot_detection plugin as an opertaor

System Description
Ubuntu 20.04
Using Conda Environment
fiftyone==0.24.1
transformers==4.41.2

(fiftyone) beast@seafar:~$ fiftyone plugins list
plugin                            version    enabled    directory
--------------------------------  ---------  ---------  -----------------------------------------------------------------
@jacobmarks/zero_shot_prediction  1.3.1      ✓          /home/beast/fiftyone/__plugins__/@jacobmarks/zero_shot_prediction

Reproduction

import fiftyone as fo
import fiftyone.operators as foo
import fiftyone.zoo as foz

dataset = fo.load_dataset("my-custom-video-dataset")
view = dataset.take(1)
zsd = foo.get_operator("@jacobmarks/zero_shot_prediction/zero_shot_detect")

zsd(
    view,
    labels=["person"],
    model_name="GroundingDINO",
    label_field="person",
)

PS1: I confirm seeing "GroundingDINO" in the output of zsd.list_models()
PS2: The error below happens as well when setting model_name to 'OwlViT'
PS3: keyword arguments should model_name and label_field contrary to the example in the README, but that's should be a separate issue

Error traceback:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/beast/fiftyone/__plugins__/@jacobmarks/zero_shot_prediction/__init__.py", line 623, in __call__
    return _handle_calling(
  File "/home/beast/fiftyone/__plugins__/@jacobmarks/zero_shot_prediction/__init__.py", line 539, in _handle_calling
    return foo.execute_operator(uri, ctx, params=params)
  File "/home/beast/anaconda3/envs/fiftyone/lib/python3.8/site-packages/fiftyone/operators/executor.py", line 169, in execute_operator
    result = asyncio.run(coroutine)
  File "/home/beast/anaconda3/envs/fiftyone/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/beast/anaconda3/envs/fiftyone/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/beast/anaconda3/envs/fiftyone/lib/python3.8/site-packages/fiftyone/operators/decorators.py", line 26, in wrapper
    return await asyncio.wait_for(
  File "/home/beast/anaconda3/envs/fiftyone/lib/python3.8/asyncio/tasks.py", line 494, in wait_for
    return fut.result()
  File "/home/beast/anaconda3/envs/fiftyone/lib/python3.8/site-packages/fiftyone/operators/executor.py", line 220, in execute_or_delegate_operator
    raise prepared.to_exception()
fiftyone.operators.executor.ExecutionError: Validation error. Path: label_field_owlvit. Reason: Required property

Thanks for the intervention

jacobmarks commented

Hey @kais-bedioui , yep I see the issue. This should be an easy fix. I just need to tweak a few things. At CVPR so won't get to this until Monday, but should be able to fix it then. Hope that's alright! In the meantime, you can delegate execution of the operator from the app, and then launch the job in your terminal with fiftyone delegated launch

jacobmarks commented

Hey @kais-bedioui , closing the loop here. I just fixed the issues you identified, so now Python operator execution should work again — I've tried it out with classification models as well as all 3 detection models. Not all of the optionality is available in the python method because FiftyOne's Ultralytics and Transformers integrations are now the preferred avenues for performing such zero-shot predictions via code:

This plugin is most useful for UI-centric zero-shot workflows.

I hope that makes sense!

Thanks again, and let me know if anything else comes up.

Marking this as closed for now :)

  • ❤️1