kherud/java-llama.cpp

Failed to load native library: xxxx-jllama.dll. (missing)

mymagicpower opened this issue · 9 comments

mymagicpower commented

Extracted 'llama.dll' to 'C:\Users\admin\AppData\Local\Temp\f4b1b397-01ee-4774-a892-a90d7d4d2793-llama.dll'
Extracted 'jllama.dll' to 'C:\Users\admin\AppData\Local\Temp\05ca35ed-88f4-4568-81a2-30eecd927d50-jllama.dll'
C:\Users\admin\AppData\Local\Temp\05ca35ed-88f4-4568-81a2-30eecd927d50-jllama.dll: Can't find dependent libraries
Failed to load native library: C:\Users\admin\AppData\Local\Temp\05ca35ed-88f4-4568-81a2-30eecd927d50-jllama.dll. osinfo: Windows/x86_64
Exception in thread "main" java.lang.UnsatisfiedLinkError: No native library found for os.name=Windows, os.arch=x86_64, paths=[/de/kherud/llama/Windows/x86_64;D:\Program Files\Java\jdk-17\bin;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;D:\ai_projects\AIAS\3_audio_sdks\vad4j_sdk\lib\windows;C:\Python311\Scripts;C:\Python311;C:\Program Files\Common Files\Oracle\Java\javapath;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\libnvvp;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C:\WINDOWS\System32\OpenSSH;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;D:\system\Program Files\Git\cmd;C:\Program Files\NVIDIA Corporation\Nsight Compute 2022.2.0;D:\system\Program Files\nodejs;C:\ProgramData\chocolatey\bin;D:\system\Users\admin\miniconda3\condabin;C:\Program Files\Calibre2;D:\Program Files\Git LFS;D:\Program Files\TDM-GCC-64\bin;D:\Program Files\CMake\bin;D:\Program Files (x86)\WinSCP;D:\Program Files\PuTTY;C:\Users\admin\AppData\Local\Microsoft\WindowsApps;C:\Users\admin\AppData\Roaming\npm;D:\Users\admin\AppData\Local\Programs\Microsoft VS Code\bin;.]
at de.kherud.llama.LlamaLoader.loadNativeLibrary(LlamaLoader.java:142)
at de.kherud.llama.LlamaLoader.initialize(LlamaLoader.java:67)
at de.kherud.llama.LlamaModel.(LlamaModel.java:27)
at examples.MainExample.main(MainExample.java:15)

1695265611757

kherud commented

Hm, that seems strange since your log outputs Extracted 'jllama.dll' to 'C:\Users\admin\AppData\Local\Temp\05ca35ed-88f4-4568-81a2-30eecd927d50-jllama.dll'. I mainly tested this for linux and macos though, I'll look into it later for Windows today. Thanks for reporting 👍

  • 👍1
mymagicpower commented

Workaround:
copy jllama.dll, llama.dll to any PATH, such as D:\Program Files\Java\jdk-17\bin:
(or add xxx.dll path to system PATH variable)

1695285696242

  • 👍1
AayushSameerShah commented

I too have the following error:

Extracted 'llama.dll' to 'C:\Users\aayush\AppData\Local\Temp\9b3d8a3d-5d00-4953-b462-04189ad2c66f-llama.dll'
Extracted 'jllama.dll' to 'C:\Users\aayush\AppData\Local\Temp\8a247bd7-123a-4221-950d-3ab82bf75a8f-jllama.dll'
C:\Users\aayush\AppData\Local\Temp\8a247bd7-123a-4221-950d-3ab82bf75a8f-jllama.dll: Can't find dependent libraries
Failed to load native library: C:\Users\aayush\AppData\Local\Temp\8a247bd7-123a-4221-950d-3ab82bf75a8f-jllama.dll. osinfo: Windows/x86_64
Exception in thread "main" java.lang.UnsatisfiedLinkError: No native library found for os.name=Windows, os.arch=x86_64, paths=[/de/kherud/llama/Windows/x86_64;C:\Program Files\Java\jdk1.8.0_181\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:\Windows\system32;C:\Windows;C:\Windows\system32\Wbem;C:\Program Files\Java\jdk1.8.0_181\BIN;C:\Program Files\TortoiseSVN\bin;C:\Gradle\gradle-7.6-bin\gradle-7.6\bin;C:\Program Files\Java\jdk1.8.0_181\;C:\Program Files\Git\cmd;C:\Users\aayush\anaconda3;C:\Users\aayush\anaconda3\Library\mingw-w64\bin;C:\Users\aayush\anaconda3\Library\usr\bin;C:\Users\aayush\anaconda3\Library\bin;C:\Users\aayush\anaconda3\Scripts;C:\winutils\bin;C:\Program Files\CMake\bin;.]
	at de.kherud.llama.LlamaLoader.loadNativeLibrary(LlamaLoader.java:142)
	at de.kherud.llama.LlamaLoader.initialize(LlamaLoader.java:67)
	at de.kherud.llama.LlamaModel.<clinit>(LlamaModel.java:27)
	at kherud.main(kherud.java:12)

After the @mymagicpower's workaround still I am getting the following:

image

kherud commented

Hey @AayushSameerShah can you please upload the error log from your screen shot C:\Users\aayush\IdeaProjects\t5_connector_py\hs_err_pid2848.log?

kherud commented

I updated the readme with better instructions, please closely read this section.

Regarding your workaround @mymagicpower note that the following section also describes how the library can be found in custom directories.

AayushSameerShah commented

Hey @AayushSameerShah can you please upload the error log from your screen shot C:\Users\aayush\IdeaProjects\t5_connector_py\hs_err_pid2848.log?

Hello @kherud !
Thanks for the response. Here's the log file:

hs_err_pid3692.log

lmagnien commented

I had the same error and ran a step-by-step debug.

llama.dll is extracted from the jar, placed in the temporary folder and loaded.
jllama.dll is extracted from the jar and placed in the temporary folder.

But the call to System.load(library) (l163 in LlamaLoader.class) fails and causes an UnsatisfiedLinkError.

Hence the messages "Can't find dependent libraries" and "Failed to load native library".

The rest of the traces are irrelevant.

I think the problem is that when copying, llama.dll is prefixed with a uuid and therefore jllama.dll cannot find the dependency because of this renaming.

I've tried removing the UUID prefix (l229 of LlamaLoader.java : Path path = extractFile(libFolderForCurrentOS, libraryFileName, targetFolder, true false);) and it works, although it doesn't seem an acceptable solution in the long term.

@kherud, if you're interested, I can try to find some time during the week to find a better solution.

In the meantime, @mymagicpower's workaround should work.

  • 👍2
kherud commented

Thank you both @AayushSameerShah @lmagnien the hints that windows can't find the library due to the uuid are really helpful.

I took inspiration there from the sqlite-jdbc project. It prefixes the shared library to avoid conflicts when multiple JVMs use different versions of the library. I think this scenario is rather unlikely for our project, so I removed the uuid prefix for now (version 2.0.1).

One simple solution to still keep a uuid would be to statically link llama.dll into jlamma.dll, so just one library has to be loaded.
Originally, I separated the two libraries because I wanted users to be able to easily provide their own llama.dll (and I still think that's the right way).

My feeling is that figuring out how to load two uuid-prefixed libraries might be really challenging and energy is currently better spent elsewhere.
Nevertheless, I would of course be really grateful if you want to look into it @lmagnien .