加载模型报错ValueError: prefetch_factor option could only be specified in multiprocessing.let num_workers > 0 to enable multiprocessing, otherwise set prefetch_factor to None.
licamla opened this issue · 2 comments
licamla commented
Describe the bug
from fastHan import FastHan
model = FastHan()
model.set_cws_style('cityu')
model('加载模型报错')
加载模型报错
File "/lib/python3.10/site-packages/fastNLP/core/dataloaders/torch_dataloader/fdl.py", line 152, in __init__
super().__init__(**dl_kwargs)
File "/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 245, in __init__
raise ValueError('prefetch_factor option could only be specified in multiprocessing.'
ValueError: prefetch_factor option could only be specified in multiprocessing.let num_workers > 0 to enable multiprocessing, otherwise set prefetch_factor to None.
Desktop (please complete the following information):
torch 2.0.0+cpu
FastNLP 1.0.1
Additional context
torch的DataLoader参数num_workers=0时,prefetch_factor需要设置成None。
而fastNLP里的子类TorchDataLoader的默认num_workers=0,prefetch_factor=2。按道理来说TorchDataLoader应该像DataLoader默认prefetch_factor=None
JustinPeanut commented
您好,我使用了0.6.0版本的fastnlp,但是在下载bert预训练模型的时候,http报502,服务器是停止维护了吗?
xiaoyingwang055 commented
解决了吗,我也报错了