not getting gpu acceleration on windows

[siddhsql]

i am running version 2.2.1 of this library on a windows PC with intel iris plus graphics 655 and do not see the gpu being used. is this because the underlying llama.cpp does not support it or something else? is there something i can do to get gpu acceleration on windows?

[kherud]

Hi @siddhsql the pre-compiled libraries this Java package provides are CPU only. For GPU support, you have to compile llama.cpp yourself. Please refer to these instructions https://github.com/kherud/java-llama.cpp#setup-required

[siddhsql]

Hi Konstantin, Thanks for your reply. I built a custom binary of llama.cpp that uses CLBlast. I confirmed that when I run llama.cpp directly using the main.exe program, I get GPU acceleration but I do not get any GPU acceleration when using java-llama.cpp. I have set de.kherud.llama.lib.path to the location where I have jllama.dll and llama.dll. is there something I am missing? Can you provide more instructions? when using main.exe I see following in the output: ``` llm_load_tensors: offloading 32 repeating layers to GPU llm_load_tensors: offloading non-repeating layers to GPU llm_load_tensors: offloaded 33/33 layers to GPU llm_load_tensors: CPU buffer size = 70.31 MiB llm_load_tensors: OpenCL buffer size = 3820.93 MiB ``` which is missing when I use java-llama.cpp

[siddhsql]

nevermind. i fixed it.

…

On Mon, Jan 29, 2024 at 12:12 PM Siddharth Jain ***@***.***> wrote: Hi Konstantin, Thanks for your reply. I built a custom binary of llama.cpp that uses CLBlast. I confirmed that when I run llama.cpp directly using the main.exe program, I get GPU acceleration but I do not get any GPU acceleration when using java-llama.cpp. I have set de.kherud.llama.lib.path to the location where I have jllama.dll and llama.dll. is there something I am missing? Can you provide more instructions? On Sun, Jan 28, 2024 at 11:58 AM Konstantin Herud < ***@***.***> wrote: > Hi @siddhsql <https://github.com/siddhsql> the pre-compiled libraries > this Java package provides are CPU only. For GPU support, you have to > compile llama.cpp yourself. Please refer to these instructions > https://github.com/kherud/java-llama.cpp#setup-required > > — > Reply to this email directly, view it on GitHub > <#43 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/A6NWEK7FCAGSTNQVLH7OTBLYQ2UXBAVCNFSM6AAAAABCNWSDMCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJTG4YDKMRQGQ> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >