inferenceengine/shadernn

Please create an example that runs LLAMA on OpenGL

Opened this issue · 0 comments

jart commented

Hello there. Cool project. Could you please create an example that runs LLAMA on OpenGL? I think something like that would immediately go viral in the same kind of way as llama.cpp. I need it because OpenGL libraries are widely installed, widely available, and have stable interfaces. Cuda is something that's harder to depend on. So I'd really like to be able to do LLMs by depending on OpenGL rather than Nvidia directly. Thanks!