shibing624/ChatPDF

最小显存是否有要求

tocotaco opened this issue · 1 comments

tocotaco commented

运行 chatpdf.py 时报错：

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 12.00 MiB (GPU 0; 5.79 GiB total capacity; 5.05 GiB already allocated; 13.88 MiB free; 5.11 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

GPU :2060 显存6GB
LLM :chatglm-6b-int4
Embedding : text2vec-base

尝试修改batch_size，无果~

shibing624 commented

显存12GB,内存16G

  • 👍1