How do I set the DataLoader parallel to 0 for debugging?
Closed this issue ยท 8 comments
I am debugging with ipdb, but when it is parallel with Dataloader, the following error appears. I want to turn off parallel of DataLoader, what should I do?
bdb.BdbQuit: Caught BdbQuit in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/keiichi_kuroyanagi/.pyenv/versions/3.8.6-yolox/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "/home/keiichi_kuroyanagi/.pyenv/versions/3.8.6-yolox/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/keiichi_kuroyanagi/.pyenv/versions/3.8.6-yolox/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/keiichi_kuroyanagi/clones/YOLOX/yolox/data/datasets/datasets_wrapper.py", line 110, in wrapper
ret_val = getitem_fn(self, index)
File "/home/keiichi_kuroyanagi/clones/YOLOX/yolox/data/datasets/mosaicdetection.py", line 95, in __getitem__
img, _labels, _, img_id = self._dataset.pull_item(index)
File "/home/keiichi_kuroyanagi/clones/YOLOX/yolox/data/datasets/voc.py", line 242, in pull_item
img = self.load_resized_img(index)
File "/home/keiichi_kuroyanagi/clones/YOLOX/yolox/data/datasets/voc.py", line 207, in load_resized_img
img = self.load_image(index)
File "/home/keiichi_kuroyanagi/clones/YOLOX/yolox/data/datasets/voc.py", line 221, in load_image
img = cv2.imread(self._imgpath % img_id, cv2.IMREAD_COLOR)
File "/home/keiichi_kuroyanagi/clones/YOLOX/yolox/data/datasets/voc.py", line 221, in load_image
img = cv2.imread(self._imgpath % img_id, cv2.IMREAD_COLOR)
File "/home/keiichi_kuroyanagi/.pyenv/versions/3.8.6-yolox/lib/python3.8/bdb.py", line 88, in trace_dispatch
return self.dispatch_line(frame)
File "/home/keiichi_kuroyanagi/.pyenv/versions/3.8.6-yolox/lib/python3.8/bdb.py", line 113, in dispatch_line
if self.quitting: raise BdbQuit
bdb.BdbQuit
Set num_workers
in pytorch dataloader to 0 could help you.
@FateScript Even if this is set to zero, it does not work. Can you understand the cause?
Line 27 in 38c633b
I'm debugging with a single gpu. Once again I check num_workers
. Thank you for your reply.
Is this issue solved?
@FateScript I tried setting self.data_num_workers = 0
, but it hasn't been solved yet. I'm investigating.
@FateScript I was able to solve it by using ipdb.sset_trace()
instead of ipdb.set_trace()
. The following links were helpful.
Ipdb not showing output with Django nose tests - Stack Overflow https://stackoverflow.com/questions/37171528/ipdb-not-showing-output-with-django-nose-tests
Setting worker to 0 helped