Tele-AI/TeleChat2

Telechat-115B推理显存要求

Closed this issue · 1 comments

WenhaoYao commented

115B的模型能否支持8张Ascend 910B推理，推理显存大概需要多少？

luoyang1999 commented

支持，8张Ascend 910B (64g) 启动推理时，单张显存占用约34g