Vali-98/ChatterUI

GPU and NPU acceleration

Closed this issue · 1 comments

Re4mer commented

Hello
Can you please add GPU acceleration support to this app?
Llama.cpp project supports Vulkan API for this purpose. And maybe you can add NPU support as well. It can hugely impact token generation speed.

Vali-98 commented

I do not believe the Vulkan API is optimized for android, as I have tested before and discussed with the llama.cpp vulkan dev.

For adreno specifically, its a performance deficit to run Vulkan last ive tested.