LINCellularNeuroscience/VAME

training issue on colab

Closed this issue · 2 comments

Hi,
I just started to play around with VAME. I used the example dataset provided with this repo to set up a Colab notebook and run the analysis. However when i get to the training part (vame.train_model(config)), the process stops after a few iterations with the following error:

Epoch: 6
Train loss: 7085.706, MSE-Loss: 4000.471, MSE-Future-Loss 3078.588, KL-Loss: 4.932, Kmeans-Loss: 1.716, weight: 1.00
Test loss: 1447.955, MSE-Loss: 1440.747, KL-Loss: 5.604, Kmeans-Loss: 1.603
Saving model!


FileNotFoundError: [Errno 2] No such file or directory: '/gdrive/Shareddrives/Turi_lab/Data/VAME_test_data/-Nov22-2022/model/best_model/VAME_/gdrive/Shareddrives/Turi_lab/Data/VAME_test_data/.pkl'

The model/best_model folder is indeed empty, except an empty snapshots subfolder.
My config variable looks like this:
/gdrive/Shareddrives/Turi_lab/Data/VAME_test_data/-Nov22-2022/config.yaml

Full trace:

FileNotFoundError                         Traceback (most recent call last)

<ipython-input-13-0401857bf079> in <module>
----> 1 vame.train_model(config)

3 frames

/usr/local/lib/python3.7/dist-packages/vame-1.0-py3.7.egg/vame/model/rnn_vae.py in train_model(config)
    359 
    360             if use_gpu:
--> 361                 torch.save(model.state_dict(), os.path.join(cfg['project_path'],"model", "best_model",model_name+'_'+cfg['Project']+'.pkl'))
    362 
    363             else:

/usr/local/lib/python3.7/dist-packages/torch/serialization.py in save(obj, f, pickle_module, pickle_protocol, _use_new_zipfile_serialization)
    374     _check_dill_version(pickle_module)
    375 
--> 376     with _open_file_like(f, 'wb') as opened_file:
    377         if _use_new_zipfile_serialization:
    378             with _open_zipfile_writer(opened_file) as opened_zipfile:

/usr/local/lib/python3.7/dist-packages/torch/serialization.py in _open_file_like(name_or_buffer, mode)
    228 def _open_file_like(name_or_buffer, mode):
    229     if _is_path(name_or_buffer):
--> 230         return _open_file(name_or_buffer, mode)
    231     else:
    232         if 'w' in mode:

/usr/local/lib/python3.7/dist-packages/torch/serialization.py in __init__(self, name, mode)
    209 class _open_file(_opener):
    210     def __init__(self, name, mode):
--> 211         super(_open_file, self).__init__(open(name, mode))
    212 
    213     def __exit__(self, *args):

FileNotFoundError: [Errno 2] No such file or directory: '/gdrive/Shareddrives/Turi_lab/Data/VAME_test_data/-Nov22-2022/model/best_model/VAME_/gdrive/Shareddrives/Turi_lab/Data/VAME_test_data/.pkl'

Never mind, apparently i messed up the initialization.