insilicomedicine/GENTRL

Error in model.train_as_vaelp

Closed this issue · 1 comments

Jiang-Lu-CAS commented

Hi, we met two problems when try the "pretrain" in examples.

  1. In [5], model.cuda();

RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

It's interesting that we solved it by changing it to model.torch.cuda();, and changed it back to model.cuda();.

 However, I'm confused about it. Is it right?

  1. In(7), model.train_as_vaelp(train_loader, lr=1e-4)

    This works well at first, but after more than 10 hours, it breaks down.

out[11] <gentrl.gentrl.TrainStats at 0x7fb6a0ae4190>

2

We tried twice times, but it couldn't work well. Is it right that the loss is negative?

We also tried to reduce the input data to 1000 molecules in "dataset_v1.csv". But it still didn't work.

3

Is there anything I can do to solve this problem? Hoping for your help.
Thank you!
danpol commented
  1. RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED: I think it might be an issue with your torch installation. Please refer to https://pytorch.org/
  2. On your screenshots, everything seems fine. The model finished training; you should execute the next two cells to save the model