Pysolotools Serialization Issue
fsudrew07 opened this issue · 5 comments
Got another pysolotools serialization issue. Details can be found here in the logs.
KeyError: 'captures'
Generated dataset is here: gs://unity-ai-antojito-prd-dataset-files-0819/7520a09c-9447-469d-b074-ebe8705b39a1/attempt:0
Wild that the simulation isn't producing captures. I wonder if we should put this as a request on the tools team to make sure solo reports and empty array instead of just not reporting the field
It feels like much less work for pysolotools to handle a missing JSON field as an empty list vs asking another team to provide an empty list. Now if it's a bug of some kind, sure, maybe? But even then it still feels a bit weird to say, "your dataset is missing a field that doesn't matter so we can't do any stats on it."
issue happened here:
https://kubeflow-platform.iap.mlp.unity3d.com/_/pipeline/?ns=kubeflow-ucvd#/runs/details/7a551a53-3e13-4ace-b30f-fb5d48633eaa
frame 3417
frame 3418 [looks like this one can't be parsed]
https://console.cloud.google.com/storage/browser/_details/unity-ai-antojito-prd-dataset-files-0819/28effaf3-e246-4a90-8b26-59a5e0160138/attempt:0/sequence.3418/step1.frame_data.json;tab=live_object?authuser=1
Looking at what Vova just posted, this data appears to have the captures field in it, so I really don't know what is happening here
I'm not sure why Vova put those links but I just ran it and this frame is missing captures.
Also, it looks like 9 days ago Shrinath resolved this but there was not a new release pushed. Le sigh. So I'll push a new release, regenerate the PyPi and docker images.
Running Vova's files through with Shrinath's update, there's another failure on this frame with "values" missing. Looks like from this class BoundingBox2DAnnotation
This led to another defaulting of lists issue which when resolved led to another defaulting of lists issue. After resolving all of these, the stats are successfully being run. I'll cut a PR momentarily.