Process exits when language detection is enabled
Closed this issue · 6 comments
katya4oyu commented
I added set_language(Some("auto")) to the full_usage example and ran it(whisper-rs v0.10.0, MBP M1 Max, large-v2 model).
let ctx = WhisperContext::new_with_params(
&whisper_path.to_string_lossy(),
WhisperContextParameters::default()
).expect("failed to open model");
let mut state = ctx.create_state().expect("failed to create key");
let mut params = FullParams::new(SamplingStrategy::default());
params.set_progress_callback_safe(|progress| println!("Progress callback: {}%", progress));
params.set_language(Some("auto")); // Add
let st = std::time::Instant::now();
state
.full(params, &samples)
.expect("failed to convert samples");
let et = std::time::Instant::now();The process terminated with the following message:
...
whisper_model_load: model size = 3117.02 MB
whisper_init_state: kv self size = 220.20 MB
whisper_init_state: kv cross size = 245.76 MB
whisper_init_state: compute buffer (conv) = 30.92 MB
whisper_init_state: compute buffer (encode) = 212.36 MB
whisper_init_state: compute buffer (cross) = 9.32 MB
whisper_init_state: compute buffer (decode) = 99.17 MB
Process finished with exit code 139 (interrupted by signal 11:SIGSEGV)When I set Some("en") or Some("ja"), it works correctly,
but Some("auto"), None, set_detect_language(true) also finished the process.
tazz4843 commented
Yep I noticed this bug last night with an unrelated PR. Don't have time to fix until later today, but it's an easy fix from what I can tell.
tazz4843 commented
Turns out I had no clue what it was, I think it's a safe bet this is an upstream bug. I'll do some digging and see.
tazz4843 commented
I can narrow the segfault down to this line of code: https://github.com/ggerganov/whisper.cpp/blob/master/whisper.cpp#L3503 but can't find much more since debug symbols are always stripped for some reason.
Andreybest commented
Hey, got the same problem with language detection.
Im no good at CPP, but from what I've tried to debug, issue is supposed to be here: https://github.com/ggerganov/whisper.cpp/blob/d38af151a1ed5378c5a9ae368e767ed22c8ab141/whisper.cpp#L3530C21-L3530C21
Looks like it takes ctx.state (which is null in this context - from my debugger), and state variable is full - as a function name (whisper_decode_with_state) suggests.
From what I see in the latest code of whisper.cpp, is is now targeted directly to state instead of ctx.state:
https://github.com/ggerganov/whisper.cpp/blob/022756a87204cd06c5d58f67b3708b550dcc38b0/whisper.cpp#L3490C20-L3490C20
Specifically: ggerganov/whisper.cpp#1519
So update to latest version of whisper.cpp should be the fix to this issue.
I'll try to update whisper.cpp on sys crate, just to understand how it works and to help you and me :)
tazz4843 commented
I ran into this myself, I'll be doing some testing to see if this is resolved or not on my service.
WenqingZong commented
Hi, I got the same problem and google leads me to here.
It seems the fix will be published on 0.10.1, currently I'm using whisper-rs = { git = "https://github.com/tazz4843/whisper-rs", branch = "master", features = ["cuda"]} in my Cargo.toml file, but cannot compile it due to this issue:
root@8a0b02512257:/workspaces/rs-whisper-test# cargo clean && cargo build --release
Removed 1044 files, 382.1MiB total
Compiling proc-macro2 v1.0.78
Compiling glob v0.3.1
Compiling unicode-ident v1.0.12
Compiling prettyplease v0.2.16
Compiling libc v0.2.153
Compiling rustix v0.38.31
Compiling bitflags v2.4.2
Compiling cfg-if v1.0.0
Compiling regex-syntax v0.8.2
Compiling either v1.10.0
Compiling linux-raw-sys v0.4.13
Compiling minimal-lexical v0.2.1
Compiling memchr v2.7.1
Compiling bindgen v0.69.4
Compiling home v0.5.9
Compiling rustc-hash v1.1.0
Compiling shlex v1.3.0
Compiling lazycell v1.3.0
Compiling log v0.4.21
Compiling cc v1.0.89
Compiling lazy_static v1.4.0
Compiling fs_extra v1.3.0
Compiling rs-whisper-test v0.1.0 (/workspaces/rs-whisper-test)
Compiling hound v3.5.1
Compiling libloading v0.8.2
Compiling itertools v0.12.1
Compiling clang-sys v1.7.0
Compiling nom v7.1.3
Compiling cmake v0.1.50
Compiling quote v1.0.35
Compiling syn v2.0.52
Compiling regex-automata v0.4.6
Compiling which v4.4.2
Compiling cexpr v0.6.0
Compiling regex v1.10.3
Compiling whisper-rs-sys v0.8.1 (https://github.com/tazz4843/whisper-rs?branch=master#9861dfdb)
Compiling whisper-rs v0.10.1 (https://github.com/tazz4843/whisper-rs?branch=master#9861dfdb)
error: linking with `cc` failed: exit status: 1
|
= note: LC_ALL="C" PATH="/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/vscode/vscode-server/bin/linux-x64/019f4d1419fbc8219a181fab7892ebccf7ee29a2/bin/remote-cli:/root/.cargo/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/cloud/3rdparty/ffmpeg/bin" VSLANG="1033" "cc" "-m64" "/tmp/rustcXdkPYN/symbols.o" "/workspaces/rs-whisper-test/target/release/deps/rs_whisper_test-b84891d36382a851.rs_whisper_test.47e03ed10f129cdd-cgu.0.rcgu.o" "/workspaces/rs-whisper-test/target/release/deps/rs_whisper_test-b84891d36382a851.rs_whisper_test.47e03ed10f129cdd-cgu.1.rcgu.o" "/workspaces/rs-whisper-test/target/release/deps/rs_whisper_test-b84891d36382a851.rs_whisper_test.47e03ed10f129cdd-cgu.2.rcgu.o" "/workspaces/rs-whisper-test/target/release/deps/rs_whisper_test-b84891d36382a851.rs_whisper_test.47e03ed10f129cdd-cgu.3.rcgu.o" "/workspaces/rs-whisper-test/target/release/deps/rs_whisper_test-b84891d36382a851.4wpbjtcnz2vuukzo.rcgu.o" "-Wl,--as-needed" "-L" "/workspaces/rs-whisper-test/target/release/deps" "-L" "/usr/local/cuda/lib64" "-L" "/opt/cuda/lib64" "-L" "/workspaces/rs-whisper-test/target/release/build/whisper-rs-sys-d6c814e72fa667d0/out/build" "-L" "/workspaces/rs-whisper-test/target/release/build/whisper-rs-sys-d6c814e72fa667d0/out" "-L" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/workspaces/rs-whisper-test/target/release/deps/libwhisper_rs-5c314781444888c4.rlib" "/workspaces/rs-whisper-test/target/release/deps/libwhisper_rs_sys-fa390a7b2f6f0f90.rlib" "/workspaces/rs-whisper-test/target/release/deps/libhound-42a21be6d88a314f.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-66d8041607d2929b.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-a57e2388c0aea9b1.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-dcd9be90ae2cb505.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-516789932d161b4e.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-1ff34b0cf871cb60.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-0c110dd0650d6cb7.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-a6e97aae2681ad8f.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-b93dac2525ec4d1e.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-ce1d65fb391ae98b.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-8933a2fb54d88492.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-306712ebb1ee1a3f.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-349c574f342b0d30.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-65c422a3ad95273d.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-7e6330a6c0cb9441.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-39c59240bfdfab27.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-e9d126c51bb8b2bb.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-5af394d9b1f07bdc.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-693a8f23970c5917.rlib" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-13fc9d1ed9c7a2bc.rlib" "-Wl,-Bdynamic" "-lstdc++" "-lcublas" "-lcudart" "-lcublasLt" "-lculibos" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/root/.rustup/toolchains/1.76.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/workspaces/rs-whisper-test/target/release/deps/rs_whisper_test-b84891d36382a851" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs"
= note: /usr/bin/ld: /workspaces/rs-whisper-test/target/release/deps/libwhisper_rs_sys-fa390a7b2f6f0f90.rlib(ggml-cuda.cu.o): in function `ggml_cuda_pool_malloc(int, unsigned long, unsigned long*)':
tmpxft_00005a21_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x23c30): undefined reference to `cuMemCreate'
/usr/bin/ld: tmpxft_00005a21_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x23c66): undefined reference to `cuMemAddressReserve'
/usr/bin/ld: tmpxft_00005a21_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x23c8e): undefined reference to `cuMemMap'
/usr/bin/ld: tmpxft_00005a21_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x23ca0): undefined reference to `cuMemRelease'
/usr/bin/ld: tmpxft_00005a21_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x23cdd): undefined reference to `cuMemSetAccess'
/usr/bin/ld: tmpxft_00005a21_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x23e6b): undefined reference to `cuGetErrorString'
/usr/bin/ld: tmpxft_00005a21_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x23e94): undefined reference to `cuGetErrorString'
/usr/bin/ld: tmpxft_00005a21_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x23ebd): undefined reference to `cuGetErrorString'
/usr/bin/ld: tmpxft_00005a21_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x23ee6): undefined reference to `cuGetErrorString'
/usr/bin/ld: tmpxft_00005a21_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x23f0f): undefined reference to `cuGetErrorString'
/usr/bin/ld: /workspaces/rs-whisper-test/target/release/deps/libwhisper_rs_sys-fa390a7b2f6f0f90.rlib(ggml-cuda.cu.o): in function `ggml_init_cublas.part.0':
tmpxft_00005a21_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x26509): undefined reference to `cuDeviceGet'
/usr/bin/ld: tmpxft_00005a21_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x26523): undefined reference to `cuDeviceGetAttribute'
/usr/bin/ld: tmpxft_00005a21_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x2656f): undefined reference to `cuMemGetAllocationGranularity'
/usr/bin/ld: tmpxft_00005a21_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x26837): undefined reference to `cuGetErrorString'
/usr/bin/ld: tmpxft_00005a21_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x26860): undefined reference to `cuGetErrorString'
/usr/bin/ld: tmpxft_00005a21_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x26910): undefined reference to `cuGetErrorString'
collect2: error: ld returned 1 exit status
= note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
= note: use the `-l` flag to specify native libraries to link
= note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)
error: could not compile `rs-whisper-test` (bin "rs-whisper-test") due to 1 previous error
I'm not sure how to fix it, any help appreciate.
Thanks in advance.