got a RuntimeError, need help plz

[kellen5l]

Hi, i transferred the code to OpenPCDet v0.52.0, but got a RuntimeError. could u help me plz.
Error:

Traceback (most recent call last):                                                                                                                                                | 0/1856 [00:00<?, ?it/s]
  File "train.py", line 202, in <module>
    main()
  File "train.py", line 171, in main
    merge_all_iters_to_one_epoch=args.merge_all_iters_to_one_epoch
  File "/home/featurize/OpenPCDet/tools/train_utils/train_utils.py", line 118, in train_model
    dataloader_iter=dataloader_iter
  File "/home/featurize/OpenPCDet/tools/train_utils/train_utils.py", line 52, in train_one_epoch
    loss.backward()
  File "/environment/miniconda3/lib/python3.7/site-packages/torch/_tensor.py", line 307, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/environment/miniconda3/lib/python3.7/site-packages/torch/autograd/__init__.py", line 156, in backward
    allow_unreachable=True, accumulate_grad=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [4611, 64]], which is output 0 of ReluBackward0, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

my environment :

ubuntu 20.04
cuda 11.3
python 3.7.10
torch 1.10.0+cu113
spconv-cu113 2.1.21

[VsionQing]

Hey, how did you solve the problem? Can you please share?

[Raiden-cn]

same question

[VsionQing]

收到谢谢

[jlqzzz]

@kellen5l @Raiden-cn @VsionQing @PointsCoder
Hi, how did you solve the problem?

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [4611, 64]], which is output 0 of ReluBackward0, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

my environment :

ubuntu 20.04
cuda 11.3
python 3.7.10
torch 1.10.0+cu113
spconv-cu113

[VsionQing]

收到谢谢

[Leedk3]

Same issue here too. Anyone who solved this problem?

[sfy-1739]

have encountered the same problem. Has anyone solved it？

[VsionQing]

收到谢谢

[Vickywq123]

me too,please,thank you

[Ricky846]

@kellen5l @Raiden-cn @VsionQing @PointsCoder Hi, how did you solve the problem?

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [4611, 64]], which is output 0 of ReluBackward0, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

my environment :

ubuntu 20.04 cuda 11.3 python 3.7.10 torch 1.10.0+cu113 spconv-cu113

Hi,have you solved this problem?

[REFRAING]

anybody solve this problem?
thansks