Can't run in macos Apple M2 Ultra
Closed this issue · 2 comments
edisonzf2020 commented
build in macos，
the log as following：
(base) fanmac@M2-Ultra-Lan SenseVoice.cpp % ./build/bin/sense-voice-main -m ./models/gguf-fp32-sense-voice-small.bin ./asr_example_zh.wav -t 4
sense_voice_small_init_from_file_with_params_no_state: loading model from './models/gguf-fp32-sense-voice-small.bin'
sense_voice_init_with_params_no_state: use gpu = 1
sense_voice_init_with_params_no_state: flash attn = 0
sense_voice_init_with_params_no_state: gpu_device = 0
sense_voice_model_load: version: 3
sense_voice_model_load: alignment: 32
sense_voice_model_load: data offset: 444480
sense_voice_model_load: loading model
sense_voice_model_load: n_vocab = 25055
sense_voice_model_load: n_encoder_hidden_state = 512
sense_voice_model_load: n_encoder_linear_units = 2048
sense_voice_model_load: n_encoder_attention_heads = 4
sense_voice_model_load: n_encoder_layers = 50
sense_voice_model_load: n_mels = 80
sense_voice_model_load: ftype = 1
sense_voice_model_load: vocab[25055] loaded
sense_voice_model_load: Metal total size = 935.21 MB
sense_voice_model_load: n_tensors: 1197
sense_voice_model_load: load SenseVoiceSmall takes 0.317000 second
sense_voice_backend_init_gpu: using Metal backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2 Ultra
ggml_metal_init: picking default device: Apple M2 Ultra
ggml_metal_init: using embedded metal library
ggml_metal_init: GPU name: Apple M2 Ultra
ggml_metal_init: GPU family: MTLGPUFamilyApple8 (1008)
ggml_metal_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_init: GPU family: MTLGPUFamilyMetal3 (5001)
ggml_metal_init: simdgroup reduction support = true
ggml_metal_init: simdgroup matrix mul. support = true
ggml_metal_init: hasUnifiedMemory = true
ggml_metal_init: recommendedMaxWorkingSetSize = 154618.82 MB
sense_voice_init_state: kv pad size = 3.67 MB
ggml_backend_metal_buffer_type_alloc_buffer: error: failed to allocate buffer, size = 0.00 MiB
ggml_gallocr_reserve_n: failed to allocate Metal buffer of size 0
zsh: segmentation fault ./build/bin/sense-voice-main -m ./models/gguf-fp32-sense-voice-small.bin -t
lovemefan commented
Unfortunately, the Metal GPU backend is not supported yet, only the CPU can be used for now. I’ll do my best to adapt it.
edisonzf2020 commented
cool work, I run it ok by -ng parameter.