Questions about using DDAD_tiny dataset.

[jikerWRN]

Hi, thanks for your good work! When I use the DDAD_tiny dataset by the dgp, I encountered some errors. The code I used is as follow.

DDAD_TRAIN_VAL_JSON_PATH = '/DDAD_tiny/ddad_tiny.json' 
DATUMS = ['camera_01']
ddad_train = SynchronizedSceneDataset(
    DDAD_TRAIN_VAL_JSON_PATH,
    split='train',
    datum_names=DATUMS,
    generate_depth_from_datum='lidar'
)

The error report is

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home1/wangyufei/anaconda3/envs/tri/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/home1/wangyufei/anaconda3/envs/tri/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/home4/user_from_home1/wangyufei/dgp-1.0/dgp/datasets/base_dataset.py", line 1071, in _datum_index_for_scene
    return scene.datum_index
  File "/home4/user_from_home1/wangyufei/dgp-1.0/dgp/datasets/base_dataset.py", line 375, in datum_index
    assert len(datum_key_to_idx_in_scene) == bad_datums + num_datums, "Duplicated datum_key"
AssertionError: Duplicated datum_key
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/home1/wangyufei/anaconda3/envs/tri/lib/python3.6/code.py", line 91, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
  File "/home1/wangyufei/.pycharm_helpers/pydev/_pydev_bundle/pydev_umd.py", line 198, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "/home1/wangyufei/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home4/user_from_home1/wangyufei/dc/packnet-sfm/test_dataset.py", line 30, in <module>
    generate_depth_from_datum='lidar'
  File "/home4/user_from_home1/wangyufei/dgp-1.0/dgp/datasets/synchronized_dataset.py", line 424, in __init__
    only_annotated_datums=only_annotated_datums
  File "/home4/user_from_home1/wangyufei/dgp-1.0/dgp/datasets/synchronized_dataset.py", line 83, in __init__
    requested_autolabels=requested_autolabels
  File "/home4/user_from_home1/wangyufei/dgp-1.0/dgp/datasets/base_dataset.py", line 704, in __init__
    self.datum_index = self._build_datum_index()
  File "/home4/user_from_home1/wangyufei/dgp-1.0/dgp/datasets/base_dataset.py", line 1080, in _build_datum_index
    datum_index = list(proc.map(BaseDataset._datum_index_for_scene, self.scenes))
  File "/home1/wangyufei/anaconda3/envs/tri/lib/python3.6/multiprocessing/pool.py", line 266, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/home1/wangyufei/anaconda3/envs/tri/lib/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
AssertionError: Duplicated datum_key

Would you provide some help?

[jikerWRN]

If you meet the same problem with me, you can try another tiny dataset https://tri-ml-public.s3.amazonaws.com/github/packnet-sfm/datasets/DDAD_tiny_v2.tar