KeyError: 'value' thrown on converting solo to coco dataset

[aryansaurav]

I tried converting a solo dataset to coco dataset. Note that the dataset had only segmentations and keypoints but no bounding boxes.

from pysolotools.converters import SOLO2COCOConverter
converter = SOLO2COCOConverter(solo)
converter.convert(output_path="path/to/output/directory", dataset_name="coco")

I am getting the following error:

` 156 if not field.init:
157 continue
--> 159 field_value = kvs[field.name]
160 field_type = types[field.name]
161 if field_value is None and not _is_optional(field_type):
KeyError: 'values'

If full error message is required, I can post it again here.

It's actually coming from solo.frames().

[StevenBorkman]

Do you have a custom labeler that you added to your solo dataset?

[aryansaurav]

I am not sure I understand custom labelers very well... I have keypoints which are custom but not written any custom labeler in C# code myself yet!

[StevenBorkman]

Ok, I was specifically asking if you had a written a custom c# labeler. So, that doesn't sound like that was the issue. I guess could you send the full error and a sample of your dataset that you created?

[aryansaurav]

Sure, I am attaching the error log (error_log1.txt) in this thread
error_log1.txt

but for the dataset sample, I have to discuss with my team.
I was able to dig a bit into the code and figure out this error comes from "solo" class. More specifically, the solo instance's "end" property is not set by default. Setting it to the number of frames in the dataset suppresses the error

 solo = Solo(data_dir_path)
solo.end= 3		# Number of frames per sequence in the dataset.

This however, does not fix the issues. It loops on only 3 frames in one sequence and leaves the rest of the sequences.
And then, the second error comes up, which is the as raised in another issue: #137

I managed to find a temporary fix to this issue by disabling the line 373 in solo2coco.py file (since there are no categories in the dataset). Then it can generate a dataset, but it does not have the keypoints nor segmentation masks, only images.

As discussion on the issue #137 the alternative to the above workaround (commenting line 373) is to add boundingbox2d for each label. That way, the generated dataset has two json files bbox and semantic but neither contain the keypoints nor segmentation masks.

[AristideLaignel]

Hi,
I solve this problem by adding a try catch like this at line 159 in core.py of the package dataclasses_json :

try:
     field_value = kvs[field.name]
except:
     field_value = None

[aryansaurav]

Hi, I solve this problem by adding a try catch like this at line 159 in core.py of the package dataclasses_json :

try:
     field_value = kvs[field.name]
except:
     field_value = None

Thanks for your tip but does that really solve the issue? I know the error is coming from this line and the above "fix" could disable it.

After experimenting a bit, I figured out this error comes only when you've multiple bounding boxes and multiple frames per sequence. If you've a single bounding box, it doesn't happen. The issue needs to be addressed to enable working with multiple bounding boxes

[AristideLaignel]

Thanks for your tip but does that really solve the issue? I know the error is coming from this line and the above "fix" could disable it.

After experimenting a bit, I figured out this error comes only when you've multiple bounding boxes and multiple frames per sequence. If you've a single bounding box, it doesn't happen. The issue needs to be addressed to enable working with multiple bounding boxes

Yes it's works with multiple bounding boxes. I succed to have my dataset in the right format !

[aryansaurav]

Thanks @AristideLaignel for the confirmation. Finally figured out this issue appears if there is a missing annotation in a given frame (say if the object does not appear in that frame). Your fix is indeed a good fix (if not perfect fix) to handle those exceptions.

Saved me some extra hours and hairs on my head from being scratched away. I'll close this issue now.