Adriankhl/godot-llm

Hi. Thank you very much for this great add-on. Could you add cpu mode, please?

JohnClaw opened this issue · 4 comments

JohnClaw commented

I tried to run it on igpu AMD Radeon Graphics Vega 7 but no text was generated.

Adriankhl commented

Hi, actually, the current build v0.3 runs on cpu mode since vulkan is not working because of an upstream bug. Can you elaborate more on the problem? Like just create a simple node and run

func _ready():
    var gdllama = GDLlama.new()
    gdllama.model_path = "./models/Meta-Llama-3-8B-Instruct.Q5_K_M.gguf" ##Your model path
    gdllama.n_predict = 20
    var generated_text = gdllama.generate_text_simple("Hello")
    print(hello)

And copy-paste the printed message here?

If you know how to launch Godot in terminal, can you also run the program and copy-paste the output from the terminal here?

  • 👍2
JanWerder commented

Since godot-llm is in CPU-mode since 0.3, I think this issue could be closed

  • 👎1
Adriankhl commented

v0.4 now has both cpu build and vulkan build available, let me know if you run into any issue

  • 👍1
JohnClaw commented

v0.4 now has both cpu build and vulkan build available, let me know if you run into any issue

Sorry for not answering during long time. I was ill. Also I'm new to Godot and sometimes i just can't explain what causes the error because all things in Godot seem rather difficult for me. I will describe my issue with your engine when i recover a little bit from bad health condition.

  • 👍1