IndexError: index 144 is out of bounds for axis 0 with size 144

[aaxuluyao]

Hi just got this issue in the training:

raceback (most recent call last):
File "train.py", line 604, in
main(args)
File "train.py", line 473, in main
trainer.run(train_loader,max_epochs=args.epoch)
File "C:\Users\XuluY\anaconda3\envs\g2p_train\lib\site-packages\ignite\engine\engine.py", line 446, in run
self._handle_exception(e)
File "C:\Users\XuluY\anaconda3\envs\g2p_train\lib\site-packages\ignite\engine\engine.py", line 410, in _handle_exception
raise e
File "C:\Users\XuluY\anaconda3\envs\g2p_train\lib\site-packages\ignite\engine\engine.py", line 433, in run
hours, mins, secs = self._run_once_on_dataset()
File "C:\Users\XuluY\anaconda3\envs\g2p_train\lib\site-packages\ignite\engine\engine.py", line 399, in _run_once_on_dataset
self._handle_exception(e)
File "C:\Users\XuluY\anaconda3\envs\g2p_train\lib\site-packages\ignite\engine\engine.py", line 410, in _handle_exception
raise e
File "C:\Users\XuluY\anaconda3\envs\g2p_train\lib\site-packages\ignite\engine\engine.py", line 387, in _run_once_on_dataset
for batch in self.state.dataloader:
File "C:\Users\XuluY\anaconda3\envs\g2p_train\lib\site-packages\torch\utils\data\dataloader.py", line 801, in next
return self._process_data(data)
File "C:\Users\XuluY\anaconda3\envs\g2p_train\lib\site-packages\torch\utils\data\dataloader.py", line 846, in _process_data
data.reraise()
File "C:\Users\XuluY\anaconda3\envs\g2p_train\lib\site-packages\torch_utils.py", line 385, in reraise
raise self.exc_type(msg)
IndexError: Caught IndexError in DataLoader worker process 9.
Original Traceback (most recent call last):
File "C:\Users\XuluY\anaconda3\envs\g2p_train\lib\site-packages\torch\utils\data_utils\worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "C:\Users\XuluY\anaconda3\envs\g2p_train\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "C:\Users\XuluY\anaconda3\envs\g2p_train\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "C:\Users\XuluY\Graph2plan-master\Network\model\floorplan.py", line 223, in getitem
return fp.get_train_data()
File "C:\Users\XuluY\Graph2plan-master\Network\model\floorplan.py", line 202, in get_train_data
boxes = self.get_boxes(tensor=tensor)
File "C:\Users\XuluY\Graph2plan-master\Network\model\floorplan.py", line 159, in get_boxes
boxes = np.apply_along_axis(norm,1,boxes)
File "<array_function internals>", line 6, in apply_along_axis
File "C:\Users\XuluY\anaconda3\envs\g2p_train\lib\site-packages\numpy\lib\shape_base.py", line 402, in apply_along_axis
buff[ind] = asanyarray(func1d(inarr_view[ind], *args, **kwargs))
File "C:\Users\XuluY\Graph2plan-master\Network\model\floorplan.py", line 158, in
norm = lambda box:np.array([X[box[1]],Y[box[0]],X[box[3]-1],Y[box[2]-1]])
IndexError: index 144 is out of bounds for axis 0 with size 144

[devShaurya]

Hi @aaxuluyao, I also got this error. I decreased the no. of workers from 16 to 8 as well as I decreased the batch size from 20 to 16. Then, it worked fine for me.

But I again got this error with different index.