fyu/drn

train error

ZzzjzzZ opened this issue · 3 comments

Hi,
when I train the segment.py, it crashed, the problem is listed below:

THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1503968623488/work/torch/lib/THC/generic/THCStorage.c line=32 error=59 : device-side assert triggered
Traceback (most recent call last):
File "segment.py", line 553, in
main()
File "segment.py", line 548, in main
train_seg(args)
File "segment.py", line 355, in train_seg
eval_score=accuracy)
File "segment.py", line 249, in train
losses.update(loss.data[0], input.size(0))
RuntimeError: cuda runtime error (59) : device-side assert triggered at /opt/conda/conda-bld/pytorch_1503968623488/work/torch/lib/THC/generic/THCStorage.c:32

How can I get with this problem.

I have found that the problem is the num of class; in cityscapes dataset, the original labels have 37 classes, but you use 19 classes. Does it mean that I should relabel the ground-truth, or get with it using other methods. thx

fyu commented

You are supposed to convert the data to 19 classes based on trainId provided by Cityscapes team: https://github.com/mcordts/cityscapesScripts/blob/master/cityscapesscripts/helpers/labels.py#L61.
This is the standard way to train and test semantic segmentation networks on Cityscapes.

thanks a lot