I updated the repo and get this error "Tried architecture: x64-acc, System.Exception: Failed to load library x64-acc."

Question

I updated the repo and get this error "Tried architecture: x64-acc, System.Exception: Failed to load library x64-acc."

Closed this issue 5 months ago · 10 comments

Describe the bug

I updated the repo and get this error "Tried architecture: x64-acc, System.Exception: Failed to load library x64-acc." It was working fine before. I am on MacOS. I tried in multiple projects and same.

Steps to reproduce

No response

LLMUnity version

No response

Operating System

None

Answer 1 · 2024-06-27T13:56:51.000Z

Hi! Could you set the debug parameter and share the debug messages or Editor.log?

Answer 2 · 2024-06-27T14:00:37.000Z

Sure. There are multiple debug messages, same first two messages repeats for x64_acc too, it tries both of them.
Tried architecture: x64-no_acc, System.Exception: Failed to load library x64-no_acc. UnityEngine.Debug:Log (object) LLMUnity.LLM:StartLLMServer () (at ./Library/PackageCache/ai.undream.llm@3b17c4d415/Runtime/LLM.cs:261) LLMUnity.LLM/<Awake>d__40:MoveNext () (at ./Library/PackageCache/ai.undream.llm@3b17c4d415/Runtime/LLM.cs:216) System.Runtime.CompilerServices.AsyncVoidMethodBuilder:Start<LLMUnity.LLM/<Awake>d__40> (LLMUnity.LLM/<Awake>d__40&) LLMUnity.LLM:Awake ()

/Users/botansahin/Documents/Documents/GitHub/AIResearch/Assets/StreamingAssets/undreamai-v1.1.5-llamacpp/macos-x64-acc/libundreamai_macos-x64-acc.dylib UnityEngine.Debug:Log (object) LLMUnity.LLMLib:.ctor (string) (at ./Library/PackageCache/ai.undream.llm@3b17c4d415/Runtime/LLMLib.cs:262) LLMUnity.LLM:InitLib (string) (at ./Library/PackageCache/ai.undream.llm@3b17c4d415/Runtime/LLM.cs:275) LLMUnity.LLM:StartLLMServer () (at ./Library/PackageCache/ai.undream.llm@3b17c4d415/Runtime/LLM.cs:247) LLMUnity.LLM/<Awake>d__40:MoveNext () (at ./Library/PackageCache/ai.undream.llm@3b17c4d415/Runtime/LLM.cs:216) System.Runtime.CompilerServices.AsyncVoidMethodBuilder:Start<LLMUnity.LLM/<Awake>d__40> (LLMUnity.LLM/<Awake>d__40&) LLMUnity.LLM:Awake ()

LLM service couldn't be created UnityEngine.Debug:LogError (object) LLMUnity.LLM:StartLLMServer () (at ./Library/PackageCache/ai.undream.llm@3b17c4d415/Runtime/LLM.cs:265) LLMUnity.LLM/<Awake>d__40:MoveNext () (at ./Library/PackageCache/ai.undream.llm@3b17c4d415/Runtime/LLM.cs:216) System.Runtime.CompilerServices.AsyncVoidMethodBuilder:Start<LLMUnity.LLM/<Awake>d__40> (LLMUnity.LLM/<Awake>d__40&) LLMUnity.LLM:Awake ()

Answer 3 · 2024-06-27T14:10:07.000Z

Thank you. Which mac do you have?
Could you share the "About This Mac" info?

Answer 4 · 2024-06-27T14:13:13.000Z

There is also another debug message before them, but I am not sure if it is related to the issue, I will share anyways
Server command: -m "/Users/botansahin/Documents/Documents/GitHub/AIResearch/Assets/StreamingAssets/Phi-3-mini-4k-instruct-q4.gguf" -c 0 -b 512 --log-disable -np 1 --slot-save-path "/Users/botansahin/Library/Application Support/DefaultCompany/Temp" -ngl 32 UnityEngine.Debug:Log (object) LLMUnity.LLM:StartLLMServer () (at ./Library/PackageCache/ai.undream.llm@3b17c4d415/Runtime/LLM.cs:240) LLMUnity.LLM/<Awake>d__40:MoveNext () (at ./Library/PackageCache/ai.undream.llm@3b17c4d415/Runtime/LLM.cs:216) System.Runtime.CompilerServices.AsyncVoidMethodBuilder:Start<LLMUnity.LLM/<Awake>d__40> (LLMUnity.LLM/<Awake>d__40&) LLMUnity.LLM:Awake ()

Answer 5 · 2024-06-27T15:54:22.000Z

Could you also share the debug messages you get from the earlier LLMUnity version that was working for you?

Answer 6 · 2024-06-27T19:43:03.000Z

I already changed the repo, how can I revert back to that version, it was more than a month ago

Answer 7 · 2024-06-27T20:05:41.000Z

I found out. It was 1.2.7.
Editor.log
It gives many logs, and a lot of them are warnings, but it works
llama server listening at http://127.0.0.1:13333 UnityEngine.Debug:Log (object) LLMUnity.LLM:DebugLog (string,bool) (at ./Library/PackageCache/ai.undream.llm@dee6bda0dc/Runtime/LLM.cs:276) LLMUnity.LLM:CheckIfListening (string) (at ./Library/PackageCache/ai.undream.llm@dee6bda0dc/Runtime/LLM.cs:292) LLMUnity.LLMUnitySetup/<>c__DisplayClass1_0:<CreateProcess>b__0 (object,System.Diagnostics.DataReceivedEventArgs) (at ./Library/PackageCache/ai.undream.llm@dee6bda0dc/Runtime/LLMUnitySetup.cs:63) System.Threading._ThreadPoolWaitCallback:PerformWaitCallback ()
Different from the previous it is listening after server command

Answer 8 · 2024-06-28T08:46:49.000Z

I found the problem, it has to do with the this llama.cpp issue: ggerganov/llama.cpp#7940 .
The build for x86 uses incorrecly the arm64 architecture.

I fixed it in my library.
Could you delete the Assets/StreamingAssets/undreamai-v1.1.5-llamacpp folder in your project and try again?
It will re-download the rebuilt library and should work 🤞 .

Answer 9 · 2024-06-28T14:21:39.000Z

Thanks a lot, it works now!

Answer 10 · 2024-06-28T14:25:19.000Z

Perfect, thanks for bringing it up!