foldl/chatllm.cpp

How to use GPU?

li904775857 opened this issue · 2 comments

li904775857 commented
How to use GPU?
foldl commented

This is dedicated to those who are GPU-poor, but stay tuned. 😄

MoonRide303 commented

@foldl I tried to build it with GPU support by using cmake -B build-gpu -DGGML_CUDA=ON -DGGML_CUDA_F16=ON -DBUILD_SHARED_LIBS=ON (it works fine for llama.cpp), but compilation fails with errors like that:

  nvcc fatal   : A single input file is required for a non-link phase when an outputfile is specified

Env: Windows 11, MSVC 2022, CUDA 12.5.