alexrozanski/LlamaChat

Add Metal/GPU support for running model inference

singularitti opened this issue · 1 comments

singularitti commented

I am no expert in this, but it seems to be running on CPUs, which could cause severe heat generation.

alexrozanski commented

@singularitti adding support for this in llama.swift to start with (see alexrozanski/llama.swift#8). this will be coming to LlamaChat v2 which is still a WIP!

  • 👍1