Error occur when trying retain the model
PerfectBlueFeynman opened this issue · 0 comments
PerfectBlueFeynman commented
I have an UnpicklingError: while trying to retrain the model, even after I tried to regenerate the training and test dataset.
Does anyone have similar problems?
.\model_ours already exists.
.\model_ours\pth already exists.
.\model_ours\log already exists.
.\model_ours\info already exists.
.\model_ours\output already exists.
Random Seed: 1777
Traceback (most recent call last):
File ".\train.py", line 53, in <module>
trainer.train(loader, train_writer)
File "E:\Work git repos\motion_puzzle\trainer.py", line 74, in train
run_epoch(epoch)
File "E:\Work git repos\motion_puzzle\trainer.py", line 48, in run_epoch
pbar = tqdm(enumerate(zip(loader['train_src'], loader['train_tar'])),
File "C:\Users\Meleager\.conda\envs\mp\lib\site-packages\torch\utils\data\dataloader.py", line 435, in __iter__
return self._get_iterator()
File "C:\Users\Meleager\.conda\envs\mp\lib\site-packages\torch\utils\data\dataloader.py", line 381, in _get_iterator
return _MultiProcessingDataLoaderIter(self)
File "C:\Users\Meleager\.conda\envs\mp\lib\site-packages\torch\utils\data\dataloader.py", line 1034, in __init__
w.start()
File "C:\Users\Meleager\.conda\envs\mp\lib\multiprocessing\process.py", line 121, in start
self._popen = self._Popen(self)
File "C:\Users\Meleager\.conda\envs\mp\lib\multiprocessing\context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:\Users\Meleager\.conda\envs\mp\lib\multiprocessing\context.py", line 327, in _Popen
return Popen(process_obj)
File "C:\Users\Meleager\.conda\envs\mp\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
reduction.dump(process_obj, to_child)
File "C:\Users\Meleager\.conda\envs\mp\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
OSError: [Errno 22] Invalid argument
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\Meleager\.conda\envs\mp\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\Users\Meleager\.conda\envs\mp\lib\multiprocessing\spawn.py", line 126, in _main
self = reduction.pickle.load(from_parent)
_pickle.UnpicklingError: pickle data was truncated
****