Error: Caught ValueError in DataLoader worker process 0

[zinuok]

Thank you for releasing your nice code.
I'm trying to test your pre-trained model in VOID-1500 dataset, so I followed the instructions step by step to configure the dataset.

However, when 'bash bash/void/run_kbnet_void1500.sh' is executed, the following error occurs:
< major errors >

• ValueError: Caught ValueError in DataLoader worker process 0
• TypeError: object of type 'int' has no len()
• ValueError: array split does not result in an equal division <-- major error

Traceback (most recent call last):
  File "src/run_kbnet.py", line 149, in <module>
    device=args.device)
  File "/home/zinuok/mono_depth/calibrated-backprojection-network/src/kbnet.py", line 899, in run
    for idx, (inputs, ground_truth) in enumerate(zip(dataloader, ground_truths)):
  File "/home/zinuok/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 517, in __next__
    data = self._next_data()
  File "/home/zinuok/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1199, in _next_data
    return self._process_data(data)
  File "/home/zinuok/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1225, in _process_data
    data.reraise()
  File "/home/zinuok/.local/lib/python3.6/site-packages/torch/_utils.py", line 429, in reraise
    raise self.exc_type(msg)
ValueError: Caught ValueError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/zinuok/.local/lib/python3.6/site-packages/numpy/lib/shape_base.py", line 867, in split
    len(indices_or_sections)
TypeError: object of type 'int' has no len()

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/zinuok/.local/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 202, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/zinuok/.local/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/zinuok/.local/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/zinuok/mono_depth/calibrated-backprojection-network/src/datasets.py", line 264, in __getitem__
    normalize=False)
  File "/home/zinuok/mono_depth/calibrated-backprojection-network/src/datasets.py", line 44, in load_image_triplet
    image1, image0, image2 = np.split(images, indices_or_sections=3, axis=-1)
  File "<__array_function__ internals>", line 6, in split
  File "/home/zinuok/.local/lib/python3.6/site-packages/numpy/lib/shape_base.py", line 873, in split
    'array split does not result in an equal division')
ValueError: array split does not result in an equal division

I think the last error is the main cause of my problem

  File "/home/zinuok/mono_depth/calibrated-backprojection-network/src/datasets.py", line 44, in load_image_triplet
    image1, image0, image2 = np.split(images, indices_or_sections=3, axis=-1)
  File "<__array_function__ internals>", line 6, in split
  File "/home/zinuok/.local/lib/python3.6/site-packages/numpy/lib/shape_base.py", line 873, in split
    'array split does not result in an equal division')
ValueError: array split does not result in an equal division

caused from the following code:

# Split along width
    image1, image0, image2 = np.split(images, indices_or_sections=3, axis=-1)

I've printed the shape of the 'images' variable, but its size is (3, 480, 640)..

How can I solve this error?
I've tried several changes like, changing N_THREADS from 8 to 1, but I couldn't fix this error

[zinuok]

I think the axis must be '0', which corresponds to the RGB channels we want to split.
If I change 'axis=-1' to 'axis=0', I can pass the above error, but following error occurs now:

  File "/home/zinuok/mono_depth/calibrated-backprojection-network/src/datasets.py", line 275, in __getitem__
    intrinsics = np.load(self.intrinsics_paths[index]).astype(np.float32)
  File "/home/zinuok/.local/lib/python3.6/site-packages/numpy/lib/npyio.py", line 444, in load
    raise ValueError("Cannot load file containing pickled data "
ValueError: Cannot load file containing pickled data when allow_pickle=False

However, the code doesn't use any 'pickle' file, which is weird..

[alexklwong]

Hi zinuok, it seems like something might be incorrect in your set up script.

When you ran the command for setting up were you doing so from the root of the repository?

calibrated-backprojection-network <----- run the commands while you are inside this folder here
|
|----- bash
|----- data
         |----- void_release
         |----- void_kbnet
|----- setup
|----- src

The dimension for the split function is correct.
https://github.com/alexklwong/calibrated-backprojection-network/blob/master/setup/setup_dataset_void.py#L136
creates image triplets along the width dimension for (H x W x C), which is the last dimension in pytorch (C x H x W) format, and loads it here:
https://github.com/alexklwong/calibrated-backprojection-network/blob/master/src/datasets.py#L44

[zinuok]

Thank you very much for your kind reply.

You are right. The problem occurred because the file read in the code was not created properly due to a path problem.
Now it works fine.
Also, I hadn't thought of triplet.

I'll close the issue now that the issue has been resolved.

Thanks again :)

[alexklwong]

Great, happy to help :)