TypeError: __reduce_ex__() takes exactly one argument (0 given)
Closed this issue · 2 comments
hi，when I run the preprocess.py, happened a mistake as follow:
Traceback (most recent call last):
File "preprocess.py", line 236, in
main()
File "preprocess.py", line 226, in main
train_dataset_files = build_save_dataset('train', fields, opt)
File "preprocess.py", line 153, in build_save_dataset
return build_save_in_shards_using_shards_size(src_corpus,
File "preprocess.py", line 128, in build_save_in_shards_using_shards_size
torch.save(dataset, pt_file)
File "D:\Program Files (x86)\Anaconda3\lib\site-packages\torch\serialization.py", line 372, in save
_save(obj, opened_zipfile, pickle_module, pickle_protocol)
File "D:\Program Files (x86)\Anaconda3\lib\site-packages\torch\serialization.py", line 476, in _save
pickler.dump(obj)
File "D:\py_project\MolecularTransformer-master\onmt\inputters\dataset_base.py", line 39, in reduce_ex
return super(DatasetBase, self).reduce_ex()
TypeError: reduce_ex() takes exactly one argument (0 given)
Interestingly, when I delete this code(
def reduce_ex(self, proto):
"This is a hack. Something is broken with torch pickle."
return super(DatasetBase, self).reduce_ex())
in the dataset_base.py, the preprocess.py can be executed to completion. So I want to ask if this behavior will affect the generated model. Looking forward to your reply.
It looks like this is an issue caused by python 3.7 (see: OpenNMT/OpenNMT-py#986) Back when we developed this code, we were using python 3.5. I would guess that python 3.6 is also fine.
That said, to train new Molecular Transformer models, it might be good to start from a more recent version of OpenNMT-py (https://github.com/OpenNMT/OpenNMT-py), which is compatible with newer python and pytorch versions.
Let me know if you have other questions.
ok， I will try your suggestion, thank you for your reply