whisper-rs-sys fails to build on macOS

Question

whisper-rs-sys fails to build on macOS

hagmonk opened this issue 3 months ago · 5 comments

Hitting the error described below … happy to help debug further!

Tried building full_usage at first. I changed Cargo.toml to include:

whisper-rs = { path = "../..", features = [ "coreml", "metal", "whisper-cpp-log", "whisper-cpp-tracing"] }

Then tried reproducing it trying to build In the sys directory for the same result.

> cargo build --features "coreml,metal"
   Compiling whisper-rs-sys v0.11.0 (/Users/hagmonk/src/github.com/tazz4843/whisper-rs/sys)
error: could not find native static library `whisper.coreml`, perhaps an -L flag is missing?

error: could not compile `whisper-rs-sys` (lib) due to 1 previous error

Not sure if it's still required, but I have in ~/.cargo/config.toml, doesn't seem to change the output either way.

[target.aarch64-apple-darwin]
rustflags = "-lc++ -l framework=Accelerate"

I looked in the target/debug/build/whsiper-rs-sys-xxx/out directory and found build/src/libwhisper.coreml.a which was not present in lib. I tried copying that file over, but then get a different linking error:

  = note: Undefined symbols for architecture arm64:
            "___kmpc_barrier", referenced from:
                _ggml_graph_compute_thread in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                ...
            "___kmpc_end_single", referenced from:
                _ggml_graph_compute.omp_outlined in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
            "___kmpc_for_static_fini", referenced from:
                ggml_backend_blas_mul_mat(ggml_backend_blas_context*, ggml_tensor*) (.omp_outlined) in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[13](ggml-blas.cpp.o)
            "___kmpc_for_static_init_8", referenced from:
                ggml_backend_blas_mul_mat(ggml_backend_blas_context*, ggml_tensor*) (.omp_outlined) in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[13](ggml-blas.cpp.o)
            "___kmpc_fork_call", referenced from:
                _ggml_graph_compute in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                ggml_backend_blas_graph_compute(ggml_backend*, ggml_cgraph*) in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[13](ggml-blas.cpp.o)
            "___kmpc_global_thread_num", referenced from:
                _ggml_graph_compute in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                _ggml_compute_forward_diag_mask_f32 in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                ggml_backend_blas_graph_compute(ggml_backend*, ggml_cgraph*) in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[13](ggml-blas.cpp.o)
            "___kmpc_push_num_threads", referenced from:
                _ggml_graph_compute in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
                ggml_backend_blas_graph_compute(ggml_backend*, ggml_cgraph*) in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[13](ggml-blas.cpp.o)
            "___kmpc_single", referenced from:
                _ggml_graph_compute.omp_outlined in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
            "_omp_get_num_threads", referenced from:
                _ggml_graph_compute.omp_outlined in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
            "_omp_get_thread_num", referenced from:
                _ggml_graph_compute.omp_outlined in libwhisper_rs_sys-dd0b36bc7e34e95c.rlib[7](ggml.c.o)
          ld: symbol(s) not found for architecture arm64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

I can build whisper-cpp from its top of tree just fine, with WHSPER_COREML=1 set as an environment variable. So I know whisper-cpp does work on this system.

Answer 1 · 2024-10-11T18:06:23.000Z

ec609f1 this seems to be the root cause, you can disable the default features (which enable openmp, which causes this link failures), especially if you're planning on using metal, and it should start building!

Answer 2 · 2024-10-12T14:03:48.000Z

I am experiencing the same build error:

  = note: ld: warning: ignoring duplicate libraries: '-lSystem', '-lc++', '-lobjc'
          Undefined symbols for architecture arm64:
            "___kmpc_barrier", referenced from:
                _ggml_graph_compute_thread in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                ...
            "___kmpc_end_single", referenced from:
                _ggml_graph_compute.omp_outlined in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
            "___kmpc_for_static_fini", referenced from:
                ggml_backend_blas_mul_mat(ggml_backend_blas_context*, ggml_tensor*) (.omp_outlined) in libwhisper_rs_sys-79a4549ef4a144de.rlib[9](ggml-blas.cpp.o)
            "___kmpc_for_static_init_8", referenced from:
                ggml_backend_blas_mul_mat(ggml_backend_blas_context*, ggml_tensor*) (.omp_outlined) in libwhisper_rs_sys-79a4549ef4a144de.rlib[9](ggml-blas.cpp.o)
            "___kmpc_fork_call", referenced from:
                _ggml_graph_compute in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                ggml_backend_blas_graph_compute(ggml_backend*, ggml_cgraph*) in libwhisper_rs_sys-79a4549ef4a144de.rlib[9](ggml-blas.cpp.o)
            "___kmpc_global_thread_num", referenced from:
                _ggml_graph_compute in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                _ggml_graph_compute_thread in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                _ggml_compute_forward_diag_mask_f32 in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                ggml_backend_blas_graph_compute(ggml_backend*, ggml_cgraph*) in libwhisper_rs_sys-79a4549ef4a144de.rlib[9](ggml-blas.cpp.o)
            "___kmpc_push_num_threads", referenced from:
                _ggml_graph_compute in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
                ggml_backend_blas_graph_compute(ggml_backend*, ggml_cgraph*) in libwhisper_rs_sys-79a4549ef4a144de.rlib[9](ggml-blas.cpp.o)
            "___kmpc_single", referenced from:
                _ggml_graph_compute.omp_outlined in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
            "_omp_get_num_threads", referenced from:
                _ggml_graph_compute.omp_outlined in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
            "_omp_get_thread_num", referenced from:
                _ggml_graph_compute.omp_outlined in libwhisper_rs_sys-79a4549ef4a144de.rlib[5](ggml.c.o)
          ld: symbol(s) not found for architecture arm64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

This error occurs when I build with the following entry in my Cargo.toml:

whisper-rs = { git = "https://github.com/tazz4843/whisper-rs.git", rev = "ec609f1" }

However, the build works fine with the previous commit 67924ca.

@arizhih
Could you provide any insight into what might have changed?
Thank you in advance for your help.

Answer 3 · 2024-10-14T14:31:53.000Z

Hi @solaoi, I've made some changes, you can check in #181

Answer 4 · 2024-10-15T01:09:16.000Z

@arizhih
Thank you for your great work!
The fix is working perfectly.

Answer 5 · 2024-10-24T18:46:33.000Z

Thanks all!