IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

[KeyG518]

Hi,

I met a problem when I train the net:
[0/201][1516/1518] Loss_D: 1.2541 Loss_G: 0.7839 / 0.8329 / 0.8305/ 0.7787
Traceback (most recent call last):
File "Train_FPNet.py", line 231, in
output = point_netD(real_center)
File "/opt/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in call
result = self.forward(*input, **kwargs)
File "/opt/anaconda3/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward
return self.module(*inputs[0], **kwargs[0])
File "/opt/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in call
result = self.forward(*input, **kwargs)
File "/scratch/cmpt_743/PF-Net-Point-Fractal-Network-master/model_FPNet.py", line 196, in forward
x = torch.cat(Layers,1)
IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

Is that the data problem?

Because I when prepare the data using bash download_shapenet_part16_catagories.sh, I got:
mv: cannot stat 'shapenet_part_overallid_to_catid_partid.json': No such file or directory

Thanks in andvance!

[zztianzz]

Hi, the code can be runned now. Please try again~

[huanlede]

Hi, Thanks for your work. I met the same problem.

Traceback (most recent call last):
File "/home/coding/pcl/PF-Net-Point-Fractal-Network-master/Train_PFNet.py", line 218, in
output = point_netD(real_center)
File "/home/anaconda3/envs/pfnet/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/anaconda3/envs/pfnet/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 141, in forward
return self.module(*inputs[0], **kwargs[0])
File "/home/anaconda3/envs/pfnet/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/coding/pcl/PF-Net-Point-Fractal-Network-master/model_PFNet.py", line 196, in forward
x = torch.cat(Layers,1)
RuntimeError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

I just download the newest code, but it still can't work.

[zztianzz]

Please print the size of real_center and tell me. It seem that the size of real_center is not match. Size of real_center should be [B,1,num_point,3]

[huanlede]

Sorry for replying late. I use a batch size of 8 because of the memory. The set of dset is 12137, so the last size of real_center is [1, 1, 512, 3].

[zztianzz]

The problem is in model_PFNet.py which may happened when the last batchsize is 1:
192 x_64 = torch.squeeze(self.maxpool(x_64))
193 x_128 = torch.squeeze(self.maxpool(x_128))
194 x_256 = torch.squeeze(self.maxpool(x_256))
It will squeeze all dimension which size is 1. ([1,1,512,3]->[512,3])
change the code into:
192 x_64 = torch.squeeze(self.maxpool(x_64),1)
193 x_128 = torch.squeeze(self.maxpool(x_128),1)
194 x_256 = torch.squeeze(self.maxpool(x_256),1)
If it works, please sent an email to me so that i can fix the code. Thx

[huanlede]

Sorry, it doesn't work. Traceback (most recent call last): File "/home/coding/pcl/PF-Net-master/Train_PFNet.py", line 219, in <module> output = point_netD(real_center) File "/home/anaconda3/envs/pfnet/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/anaconda3/envs/pfnet/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 141, in forward return self.module(*inputs[0], **kwargs[0]) File "/home/anaconda3/envs/pfnet/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/coding/pcl/PF-Net-master/model_PFNet.py", line 197, in forward x = F.relu(self.bn_1(self.fc1(x))) File "/home/anaconda3/envs/pfnet/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/anaconda3/envs/pfnet/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 67, in forward return F.linear(input, self.weight, self.bias) File "/home/anaconda3/envs/pfnet/lib/python3.7/site-packages/torch/nn/functional.py", line 1354, in linear output = input.matmul(weight.t()) RuntimeError: size mismatch, m1: [3584 x 1], m2: [448 x 256] at /opt/conda/conda-bld/pytorch_1550784378911/work/aten/src/THC/generic/THCTensorMathBlas.cu:266 It seems that there is no 1 in the second dimension. The shape of x has not been changed.

[zztianzz]

change the code into:
192 x_64 = torch.squeeze(self.maxpool(x_64),2)
193 x_64 = torch.squeeze(self.maxpool(x_64),2)
194 x_128 = torch.squeeze(self.maxpool(x_128),2)
195 x_128 = torch.squeeze(self.maxpool(x_128),2)
196 x_256 = torch.squeeze(self.maxpool(x_256),2)
197 x_256 = torch.squeeze(self.maxpool(x_256),2)
let [1,dim,1,1]->[1,dim]
try again,thx

[huanlede]

I changed the code into: 192 x_64 = torch.squeeze(self.maxpool(x_64),2) 193 x_64 = torch.squeeze(x_64,2) 194 x_128 = torch.squeeze(self.maxpool(x_128),2) 195 x_128 = torch.squeeze(x_128,2) 196 x_256 = torch.squeeze(self.maxpool(x_256),2) 197 x_256 = torch.squeeze(x_256,2) I just met another problem. Traceback (most recent call last): File "/home/coding/pcl/PF-Net-master/Train_PFNet.py", line 222, in <module> output = point_netD(real_center) File "/home/anaconda3/envs/pfnet/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/anaconda3/envs/pfnet/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 141, in forward return self.module(*inputs[0], **kwargs[0]) File "/home/anaconda3/envs/pfnet/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/coding/pcl/PF-Net-master/model_PFNet.py", line 200, in forward x = F.relu(self.bn_1(self.fc1(x))) File "/home/anaconda3/envs/pfnet/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__ result = self.forward(*input, **kwargs) File "/home/anaconda3/envs/pfnet/lib/python3.7/site-packages/torch/nn/modules/batchnorm.py", line 76, in forward exponential_average_factor, self.eps) File "/home/anaconda3/envs/pfnet/lib/python3.7/site-packages/torch/nn/functional.py", line 1619, in batch_norm raise ValueError('Expected more than 1 value per channel when training, got input size {}'.format(size)) ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 256]) I think adjusting the number of batch size would be better.

[zztianzz]

'I think adjusting the number of batch size would be better.'
It is also a temporary way. I will try to fix it...

[Jing-lun]

Hi @zztianzz @huanlede, I met the same issues as yours, I was wondering how did you guys fix finally them?

[Jing-lun]

Hi @zztianzz @huanlede, I met the same issues as yours, I was wondering how did you guys fix finally them?

I have to crop the dataset and let its size fits with the batchsize...then the network could run. Not a good idea, however, the author still not update the repo, maybe right now one could have a try on this way, add code to shapenet_part_loader.py:
(after generating self.datapath)

for i in range(len(self.datapath)%32):
            self.datapath.pop()

[zztianzz]

The problem may come from 'torch.squeeze'(without pointing out the dim) which may kill all the dim 1. If the final batchsize is 1,this bug may happen.

[Jing-lun]

The problem may come from 'torch.squeeze'(without pointing out the dim) which may kill all the dim 1. If the final batchsize is 1,this bug may happen.

Yeah it is so that I was asking if there are some other ways to fix this issue other than limit the size of dataset to the multiples of the size of batchsize.