linzehui/mRASP

AttributeError: 'MMapIndexedDataset' object has no attribute '_bin_buffer_mmap'

Opened this issue · 0 comments

大佬,我的fairseq版本是0.12.0,运行bash train/pre-train.sh /data01/code/python_project/nlp_project/mRASP/experiments/example/configs/train/pre-train/transformer_big.yml

会报下面的错误
Traceback (most recent call last):
File "/home/stary/miniconda3/envs/mrasp/bin/fairseq-train", line 8, in
sys.exit(cli_main())
File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq_cli/train.py", line 557, in cli_main
distributed_utils.call_main(cfg, main)
File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq/distributed/utils.py", line 369, in call_main
main(cfg, **kwargs)
File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq_cli/train.py", line 133, in main
task.load_dataset(valid_sub_split, combine=False, epoch=1)
File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq/tasks/translation.py", line 338, in load_dataset
self.datasets[split] = load_langpair_dataset(
File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq/tasks/translation.py", line 85, in load_langpair_dataset
src_dataset = data_utils.load_indexed_dataset(
File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq/data/data_utils.py", line 106, in load_indexed_dataset
dataset = indexed_dataset.make_dataset(
File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq/data/indexed_dataset.py", line 86, in make_dataset
return MMapIndexedDataset(path)
File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq/data/indexed_dataset.py", line 494, in init
self._do_init(path)
File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq/data/indexed_dataset.py", line 507, in _do_init
self._bin_buffer_mmap = np.memmap(
File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/numpy/core/memmap.py", line 267, in new
mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=start)
ValueError: cannot mmap an empty file
Exception ignored in: <function MMapIndexedDataset.del at 0x7ff3fc5ce1f0>
Traceback (most recent call last):
File "/home/stary/miniconda3/envs/mrasp/lib/python3.8/site-packages/fairseq/data/indexed_dataset.py", line 513, in del
self._bin_buffer_mmap._mmap.close()
AttributeError: 'MMapIndexedDataset' object has no attribute '_bin_buffer_mmap'

想咨询一下大佬,你使用的fairseq是什么版本