main.exe does nothing on transcribe task (crash probably)
fristhon opened this issue · 10 comments
When I run ./main.exe -m models/ggml-base.en.bin -f samples\jfk.wav nothing happens except some basic output in stdout.
I tested with v1.5.4/whisper-bin-x64.zip and also built v1.5.5 myself, same result.
maybe it happens because my CPU is old ? don't have any idea how to debug this, any help would be great
thanks
v1.5.4 release:
v1.5.5 my build:
OS Name Microsoft Windows 10 Enterprise
Version 10.0.19045 Build 19045
System Manufacturer LENOVO
System Model 627243G
System Type x64-based PC
System SKU LENOVO_MT_6272
Processor Intel(R) Core(TM) i7-3612QM CPU @ 2.10GHz, 2101 Mhz, 4 Core(s), 8 Logical Processor(s)
When using prebuilt binaries or building whisper.cpp using CMake without any tweaking it is assumed that you're using relatively recent CPU, i.e. from last ~10 years. That comes from relying by default on availability of AVX2 and FMA ISA extensions that were introduced in Haswell aka 4th gen Intel core.
You have Ivy Bridge aka 3rd gen Intel core, which is older, so binaries may fail for you.
You can build using CMake with disabling AVX2 and FMA:
cmake -B build -DWHISPER_NO_AVX2=ON -DWHISPER_NO_FMA=ON
cmake --build build
Or you could use MSYS2 and build using make in one of its environment like UCRT64. ISA extensions availability is detected automatically when using make.
For MSYS2 route, CI build definition shows what packages to install (pacman -S PACKAGE_NAME...):
https://github.com/ggerganov/whisper.cpp/blob/master/.github/workflows/build.yml#L253
@przemoc thank you very much for the answer.
yea my CPU is so old! I guessed that it could be the cause of the problem.
I just tried once again with CMake and those parameters you suggested. now it gets stuck on transcribing.
I have my old Windows 7 laptop with 3rd gen Intel core i7-3520M. Windows 7 is not among supported Windows versions by MSYS2), so I quickly tested building with prebuilt llvm-mingw from Git Bash. I don't have CMake there, so I went with make.
$ export PATH="/d/Tools/llvm-mingw/bin:$PATH"
$ mkdir -p git/github.com/ggerganov
$ cd git/github.com/ggerganov
$ git clone https://github.com/ggerganov/whisper.cpp
$ cd whisper.cpp/
$ mingw32-make
...
I whisper.cpp build info:
I UNAME_S: MINGW64_NT-6.1-7601
I UNAME_P: unknown
I UNAME_M: x86_64
I CFLAGS: -I. -O3 -DNDEBUG -std=c11 -fPIC -D_XOPEN_SOURCE=600 -mavx -mf16c -msse3 -mssse3
I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -D_XOPEN_SOURCE=600 -mavx -mf16c -msse3 -mssse3
I LDFLAGS:
I CC: clang version 18.1.5 (https://github.com/llvm/llvm-project.git 617a15a9eac96088ae5e9134248d8236e34b91b1)
I CXX: clang version 18.1.5 (https://github.com/llvm/llvm-project.git 617a15a9eac96088ae5e9134248d8236e34b91b1)
...
$ cd models/
$ ./download-ggml-model.sh base.en
$ cd ../
$ ./main -f samples/jfk.wav
whisper_init_from_file_with_params_no_state: loading model from 'models/ggml-base.en.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 512
whisper_model_load: n_audio_head = 8
whisper_model_load: n_audio_layer = 6
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 512
whisper_model_load: n_text_head = 8
whisper_model_load: n_text_layer = 6
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 2 (base)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: CPU total size = 147.37 MB
whisper_model_load: model size = 147.37 MB
whisper_init_state: kv self size = 16.52 MB
whisper_init_state: kv cross size = 18.43 MB
whisper_init_state: compute buffer (conv) = 16.39 MB
whisper_init_state: compute buffer (encode) = 132.07 MB
whisper_init_state: compute buffer (cross) = 4.78 MB
whisper_init_state: compute buffer (decode) = 96.48 MB
system_info: n_threads = 4 / 4 | AVX = 1 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | METAL = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | CUDA = 0 | COREML = 0 | OPENVINO = 0
main: processing 'samples/jfk.wav' (176000 samples, 11.0 sec), 4 threads, 1 processors, 5 beams + best of 5, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:11.000] And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country.
whisper_print_timings: load time = 315.84 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 39.78 ms
whisper_print_timings: sample time = 146.65 ms / 131 runs ( 1.12 ms per run)
whisper_print_timings: encode time = 4231.99 ms / 1 runs ( 4231.99 ms per run)
whisper_print_timings: decode time = 23.44 ms / 2 runs ( 11.72 ms per run)
whisper_print_timings: batchd time = 628.62 ms / 125 runs ( 5.03 ms per run)
whisper_print_timings: prompt time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: total time = 5410.97 ms
I don't see any issues.
Is it possible that you didn't remove your build directory before building using CMake with different settings than before?
Can you paste your whole output, i.e. from building and from running using the latest build with AVX2 and FMA disabled?
(Please do it as text, not image; don't forget to mark the output as code block in GitHub after pasting for better readability.)
Now works!
I followed your setup step by step and used git-bash and llvm-mingw, now output binary main.exe is working fine.
Thanks a lot
Sorry for screen shots, I sent images just to show nothing is happening after command.
About previous build with CMake:
yes I'm sure I cleaned the build directory in advance. actually tried once again and nothing changed it still get stuck after command runs. I think something is wrong with my CMake setup, I can see some errors about not finding pthread, don't know it is important or it is just a warning.
anyway problem solved with your help but I just send CMake logs for further debus. thanks again
cmake -B build -DWHISPER_NO_AVX2=ON -DWHISPER_NO_FMA=ON
-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.20348.0 to target Windows 10.0.19045.
-- The C compiler identification is MSVC 19.37.32824.0
-- The CXX compiler identification is MSVC 19.37.32824.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.37.32822/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.37.32822/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.36.1.windows.1")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - no
-- Found Threads: TRUE
-- CMAKE_SYSTEM_PROCESSOR: AMD64
-- x86 detected
-- Configuring done
-- Generating done
-- Build files have been written to: C:/MyDev/github/whisper.cpp/build
cmake --build build
MSBuild version 17.7.2+d6990bcfa for .NET Framework
Checking Build System
Building Custom Rule C:/MyDev/github/whisper.cpp/examples/CMakeLists.txt
Building Custom Rule C:/MyDev/github/whisper.cpp/CMakeLists.txt
ggml.c
ggml-alloc.c
ggml-backend.c
ggml-quants.c
...
C:\MyDev\github\whisper.cpp\ggml-quants.c(552,22): warning C4244: 'initializing': conversion from 'int64_t' to 'const int', possible loss of data [C:\MyDev\github\whisper.cpp\build\whisper.vcxproj]
<LOT OF WARNINGS ABOUT conversion from 'int64_t' to 'const int'>
....
whisper.vcxproj -> C:\MyDev\github\whisper.cpp\build\bin\Debug\whisper.dll
Building Custom Rule C:/MyDev/github/whisper.cpp/examples/bench/CMakeLists.txt
bench.cpp
bench.vcxproj -> C:\MyDev\github\whisper.cpp\build\bin\Debug\bench.exe
Building Custom Rule C:/MyDev/github/whisper.cpp/examples/CMakeLists.txt
common.cpp
C:\MyDev\github\whisper.cpp\examples\common.h(188,47): warning C4244: 'initializing': conversion from 'float' to 'int16_t', possible loss of data [C:\MyDev\github\whisper.cpp\build\examples\common.vcxproj]
C:\MyDev\github\whisper.cpp\examples\common.h(188,37): warning C4244: 'initializing': conversion from 'float' to 'const int16_t', possible loss of data [C:\MyDev\github\whisper.cpp\build\examples\common.vcxpr
oj]
common-ggml.cpp
grammar-parser.cpp
Generating Code...
common.vcxproj -> C:\MyDev\github\whisper.cpp\build\examples\Debug\common.lib
Building Custom Rule C:/MyDev/github/whisper.cpp/examples/main/CMakeLists.txt
main.cpp
C:\MyDev\github\whisper.cpp\examples\common.h(188,47): warning C4244: 'initializing': conversion from 'float' to 'int16_t', possible loss of data [C:\MyDev\github\whisper.cpp\build\examples\main\main.vcxproj]
C:\MyDev\github\whisper.cpp\examples\common.h(188,37): warning C4244: 'initializing': conversion from 'float' to 'const int16_t', possible loss of data [C:\MyDev\github\whisper.cpp\build\examples\main\main.vc
xproj]
main.vcxproj -> C:\MyDev\github\whisper.cpp\build\bin\Debug\main.exe
quantize.cpp
C:\MyDev\github\whisper.cpp\examples\common.h(188,47): warning C4244: 'initializing': conversion from 'float' to 'int16_t', possible loss of data [C:\MyDev\github\whisper.cpp\build\examples\quantize\quantize.
vcxproj]
C:\MyDev\github\whisper.cpp\examples\common.h(188,37): warning C4244: 'initializing': conversion from 'float' to 'const int16_t', possible loss of data [C:\MyDev\github\whisper.cpp\build\examples\quantize\qua
ntize.vcxproj]
quantize.vcxproj -> C:\MyDev\github\whisper.cpp\build\bin\Debug\quantize.exe
Building Custom Rule C:/MyDev/github/whisper.cpp/examples/server/CMakeLists.txt
server.cpp
C:\MyDev\github\whisper.cpp\examples\common.h(188,47): warning C4244: 'initializing': conversion from 'float' to 'int16_t', possible loss of data [C:\MyDev\github\whisper.cpp\build\examples\server\server.vcxp
roj]
C:\MyDev\github\whisper.cpp\examples\common.h(188,37): warning C4244: 'initializing': conversion from 'float' to 'const int16_t', possible loss of data [C:\MyDev\github\whisper.cpp\build\examples\server\serve
r.vcxproj]
server.vcxproj -> C:\MyDev\github\whisper.cpp\build\bin\Debug\server.exe
Building Custom Rule C:/MyDev/github/whisper.cpp/CMakeLists.txt
Glad that building using llvm-mingw worked for you. If you use Windows 10, then I would suggest building using one of MSYS2 environments, as they are much more convenient to use. (I used llvm-mingw in previous message only because I was doing it on Windows 7.)
You pasted output from running cmake commands. Can you also paste output from running:
./build/bin/main -f samples/jfk.wav
using binary that was created with -DWHISPER_NO_AVX2=ON -DWHISPER_NO_FMA=ON options?
Just for the record.
Sure I remember to use MSYS2 instead.
this is the output of the binary that created with CMake and -DWHISPER_NO_AVX2=ON -DWHISPER_NO_FMA=ON
it stops working at last line, I waited about 20 minutes.
whisper_init_from_file_with_params_no_state: loading model from 'C:\MyDev\github\whisper.cpp\models\ggml-base.en.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 512
whisper_model_load: n_audio_head = 8
whisper_model_load: n_audio_layer = 6
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 512
whisper_model_load: n_text_head = 8
whisper_model_load: n_text_layer = 6
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 2 (base)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: CPU total size = 147.37 MB
whisper_model_load: model size = 147.37 MB
whisper_init_state: kv self size = 16.52 MB
whisper_init_state: kv cross size = 18.43 MB
ggml_gallocr_needs_realloc: graph has different number of nodes
ggml_gallocr_alloc_graph: reallocating buffers automatically
ggml_gallocr_reserve_n: reallocating CPU buffer from size 0.00 MiB to 14.01 MiB
whisper_init_state: compute buffer (conv) = 16.39 MB
ggml_gallocr_needs_realloc: graph has different number of nodes
ggml_gallocr_alloc_graph: reallocating buffers automatically
ggml_gallocr_reserve_n: reallocating CPU buffer from size 0.00 MiB to 124.33 MiB
whisper_init_state: compute buffer (encode) = 132.07 MB
ggml_gallocr_needs_realloc: graph has different number of nodes
ggml_gallocr_alloc_graph: reallocating buffers automatically
ggml_gallocr_reserve_n: reallocating CPU buffer from size 0.00 MiB to 2.93 MiB
whisper_init_state: compute buffer (cross) = 4.78 MB
ggml_gallocr_needs_realloc: graph has different number of nodes
ggml_gallocr_alloc_graph: reallocating buffers automatically
ggml_gallocr_reserve_n: reallocating CPU buffer from size 0.00 MiB to 90.38 MiB
whisper_init_state: compute buffer (decode) = 96.48 MB
system_info: n_threads = 4 / 8 | AVX = 1 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | METAL = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 | CUDA = 0 | COREML = 0 | OPENVINO = 0
main: processing 'C:\MyDev\github\whisper.cpp\samples\jfk.wav' (176000 samples, 11.0 sec), 4 threads, 1 processors, 5 beams + best of 5, lang = en, task = transcribe, timestamps = 1 ...
Can you do one more test? Building using cmake with --config Release:
cmake --fresh -B build -DWHISPER_NO_AVX2=ON -DWHISPER_NO_FMA=ON
cmake --build build --config Release
Tested, interesting it works in Release mode
Just for the record it is much slower than the binary that created with mingw and I think whatever the reason is it caused the Debug release get stuck.
I can see these guys are on F16C = 1 | SSE3 = 1 | SSSE3 = 1 in the mingw binary but not in the CMake one. should I pass some options to CMake to make those activate?
CMake binary took about 40 seconds to transcribe sample jfk audio
whisper_init_from_file_with_params_no_state: loading model from 'C:\MyDev\github\whisper.cpp\models\ggml-base.en.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 512
whisper_model_load: n_audio_head = 8
whisper_model_load: n_audio_layer = 6
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 512
whisper_model_load: n_text_head = 8
whisper_model_load: n_text_layer = 6
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 2 (base)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: CPU total size = 147.37 MB
whisper_model_load: model size = 147.37 MB
whisper_init_state: kv self size = 16.52 MB
whisper_init_state: kv cross size = 18.43 MB
whisper_init_state: compute buffer (conv) = 16.39 MB
whisper_init_state: compute buffer (encode) = 132.07 MB
whisper_init_state: compute buffer (cross) = 4.78 MB
whisper_init_state: compute buffer (decode) = 96.48 MB
system_info: n_threads = 4 / 8 | AVX = 1 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | METAL = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 | CUDA = 0 | COREML = 0 | OPENVINO = 0
main: processing 'C:\MyDev\github\whisper.cpp\samples\jfk.wav' (176000 samples, 11.0 sec), 4 threads, 1 processors, 5 beams + best of 5, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:11.000] And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country.
whisper_print_timings: load time = 256.89 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 24.38 ms
whisper_print_timings: sample time = 183.19 ms / 131 runs ( 1.40 ms per run)
whisper_print_timings: encode time = 34401.36 ms / 1 runs (34401.36 ms per run)
whisper_print_timings: decode time = 95.66 ms / 2 runs ( 47.83 ms per run)
whisper_print_timings: batchd time = 5552.72 ms / 125 runs ( 44.42 ms per run)
whisper_print_timings: prompt time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: total time = 40524.86 ms
Mingw binary took about 5 seconds to transcribe sample jfk audio
whisper_init_from_file_with_params_no_state: loading model from 'C:\MyDev\github\whisper.cpp\models\ggml-base.en.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 512
whisper_model_load: n_audio_head = 8
whisper_model_load: n_audio_layer = 6
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 512
whisper_model_load: n_text_head = 8
whisper_model_load: n_text_layer = 6
whisper_model_load: n_mels = 80
whisper_model_load: ftype = 1
whisper_model_load: qntvr = 0
whisper_model_load: type = 2 (base)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs = 99
whisper_model_load: CPU total size = 147.37 MB
whisper_model_load: model size = 147.37 MB
whisper_init_state: kv self size = 16.52 MB
whisper_init_state: kv cross size = 18.43 MB
whisper_init_state: compute buffer (conv) = 16.39 MB
whisper_init_state: compute buffer (encode) = 132.07 MB
whisper_init_state: compute buffer (cross) = 4.78 MB
whisper_init_state: compute buffer (decode) = 96.48 MB
system_info: n_threads = 4 / 8 | AVX = 1 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | METAL = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD
= 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | CUDA = 0 | COREML = 0 | OPENVINO = 0
main: processing 'C:\MyDev\github\whisper.cpp\samples\jfk.wav' (176000 samples, 11.0 sec), 4 threads, 1 processors, 5 beams + best of 5, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:11.000] And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country.
whisper_print_timings: load time = 85.62 ms
whisper_print_timings: fallbacks = 0 p / 0 h
whisper_print_timings: mel time = 27.78 ms
whisper_print_timings: sample time = 443.54 ms / 131 runs ( 3.39 ms per run)
whisper_print_timings: encode time = 3418.99 ms / 1 runs ( 3418.99 ms per run)
whisper_print_timings: decode time = 16.51 ms / 2 runs ( 8.26 ms per run)
whisper_print_timings: batchd time = 513.49 ms / 125 runs ( 4.11 ms per run)
whisper_print_timings: prompt time = 0.00 ms / 1 runs ( 0.00 ms per run)
whisper_print_timings: total time = 4570.45 ms
let me know if you needed something else to test
Thanks for testing. It seems that either one of this options happen:
- Debug configuration is much slower than Release configuration (and 20 mins is simply not enough),
- Debug configuration has some issue leading it to be stuck.
MSVC have limited configurability regarding x86 ISA extensions. I noticed one minor issue in ggml regarding exposing some of them for MSVC, so you may try to do and check new cmake build in Release config (do not forget about -DWHISPER_NO_AVX2=ON -DWHISPER_NO_FMA=ON) using my PR #2128, but other than seeing SSE3 = 1 | SSSE3 = 1 I do not expect any performance improvement (I'll be happy to be wrong on that).
F16C is more tricky, because it came after AVX, but before AVX2, so there is no easy way to figure out if it can be exposed in MSVC (I mean in case when you have AVX available but not AVX2).
Okay thanks for the info