kherud/java-llama.cpp

'ggml-common.h' file not found On MacOS

MarchLiu opened this issue · 2 comments

MarchLiu commented

hardware: macos m3

llama.cpp could be build and run fine.

I bulid library as documents

mvn compile
mkdir build
cd build
cmake .. # add any other arguments for your backend
cmake --build . --config Release

now I find the files in src/main/resources/de/kherud/llama/Mac/aarch64:

ggml-metal.metal
libjllama.dylib
libllama.dylib

When I run test de.hkerud.llama.LlamaModelTest, I got:

ggml_metal_init: error: Error Domain=MTLLibraryErrorDomain Code=3 "program_source:3:10: fatal error: 'ggml-common.h' file not found
#include "ggml-common.h"
         ^~~~~~~~~~~~~~~
" UserInfo={NSLocalizedDescription=program_source:3:10: fatal error: 'ggml-common.h' file not found
#include "ggml-common.h"
         ^~~~~~~~~~~~~~~
}

I tried to modify cmake file

if (LLAMA_METAL AND NOT LLAMA_METAL_EMBED_LIBRARY)
    # copy ggml-common.h and ggml-metal.metal to bin directory
    configure_file(${llama.cpp_SOURCE_DIR}/ggml-metal.metal ${JLLAMA_DIR}/ggml-metal.metal COPYONLY)
endif()

to

if (LLAMA_METAL AND NOT LLAMA_METAL_EMBED_LIBRARY)
    # copy ggml-common.h and ggml-metal.metal to bin directory
    configure_file(${llama.cpp_SOURCE_DIR}/ggml-metal.metal ${JLLAMA_DIR}/ggml-metal.metal COPYONLY)
   # >> copy ggml-common.h 
    configure_file(${llama.cpp_SOURCE_DIR}/ggml-common.h ${JLLAMA_DIR}/ggml-common.h COPYONLY)
endif()

and run setup again. I found the ggml-common.h file in src/main/resources/de/kherud/llama/Mac . But the fatal message as before.

where should i place the ggml-common.h ?

kherud commented

Hey @MarchLiu on m-series macbooks using -DLLAMA_METAL_EMBED_LIBRARY=ON should solve your problems:

mvn compile
mkdir build
cd build
cmake .. -DLLAMA_METAL_EMBED_LIBRARY=ON
cmake --build . --config Release

Note, that the Java binding already has a pre-built MacOS metal library and everything should work out of the box if used via Maven:

<dependency>
    <groupId>de.kherud</groupId>
    <artifactId>llama</artifactId>
    <version>3.0.2</version>
</dependency>

If you don't want to use Maven, you can also download the pre-built shared libraries from here (artifacts at the bottom): https://github.com/kherud/java-llama.cpp/actions/runs/8975246604

Otherwise, I can look for a solution on how to handle ggml-common.h later today.

  • 👍1
MarchLiu commented

@kherud
Thank you! It works !

  • 🎉1