modelscope/FunASR

funasr-runtime-sdk-online-cpu-0.1.12镜像的crash问题

Opened this issue · 4 comments

MyWestCity commented

Notice: In order to resolve issues more efficiently, please raise issue following the template.
（注意：为了更加高效率解决您遇到的问题，请按照模板提问，补充细节）

🐛 Bug

你好，funasr-runtime-sdk-online-cpu-0.1.12镜像里面运行，有一个音频文件，2pass模型下，不加lm模型时，程序一定会crash，加上lm模型之后就正常了，能帮忙看看么

脚本如下：

download_model_dir="/workspace/models"
model_dir="damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx"
online_model_dir="damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx"
vad_dir="damo/speech_fsmn_vad_zh-cn-16k-common-onnx"
punc_dir="damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx"
itn_dir="thuduj12/fst_itn_zh"
lm_dir="none" #"damo/speech_ngram_lm_zh-cn-ai-wesp-fst"
port=10068
certfile="" #"$(pwd)/ssl_key/server.crt"
keyfile="" #"$(pwd)/ssl_key/server.key"
hotword="$(pwd)/websocket/hotwords.txt"
# set decoder_thread_num
decoder_thread_num=$(cat /proc/cpuinfo | grep "processor"|wc -l) || { echo "Get cpuinfo failed. Set decoder_thread_num = 32"; decoder_thread_num=32; }
multiple_io=16
io_thread_num=$(( (decoder_thread_num + multiple_io - 1) / multiple_io ))
model_thread_num=1
cmd_path=/workspace/FunASR/runtime/websocket/build/bin
cmd=funasr-wss-server-2pass

. ./tools/utils/parse_options.sh || exit 1;

cd $cmd_path
$cmd_path/${cmd}  \
  --download-model-dir "${download_model_dir}" \
  --model-dir "${model_dir}" \
  --online-model-dir "${online_model_dir}" \
  --vad-dir "${vad_dir}" \
  --punc-dir "${punc_dir}" \
  --itn-dir "${itn_dir}" \
  --lm-dir "${lm_dir}" \
  --decoder-thread-num ${decoder_thread_num} \
  --model-thread-num ${model_thread_num} \
  --io-thread-num  ${io_thread_num} \
  --port ${port} \
  --certfile  "${certfile}" \
  --keyfile "${keyfile}" \
  --hotword "${hotword}"

log如下

2024-11-27 11:53:35,889 - modelscope - INFO - Use user-specified model revision: v1.0.1
I20241127 11:53:37.011011  8232 funasr-wss-server-2pass.cpp:494] Set itn-dir : /workspace/models/thuduj12/fst_itn_zh
I20241127 11:53:37.011163  8232 funasr-wss-server-2pass.cpp:523] hotword path: /workspace/FunASR/runtime/websocket/hotwords.txt
I20241127 11:53:37.011330  8232 util.cpp:1013] hotwords: 
I20241127 11:53:37.011477  8232 util.cpp:1042] 阿里巴巴 : 20
I20241127 11:53:37.011576  8232 util.cpp:1042] 通义实验室 : 30
I20241127 11:53:37.014371  8232 funasr-wss-server-2pass.cpp:555] SSL is closed!
I20241127 11:53:37.068840  8232 fsmn-vad.cpp:58] Successfully load model from /workspace/models/damo/speech_fsmn_vad_zh-cn-16k-common-onnx/model_quant.onnx
I20241127 11:53:39.310283  8232 paraformer.cpp:77] Successfully load model from /workspace/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx/model_quant.onnx
I20241127 11:53:39.814085  8232 paraformer.cpp:85] Successfully load model from /workspace/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx/decoder_quant.onnx
I20241127 11:53:43.190724  8232 paraformer.cpp:142] Successfully load model from /workspace/models/damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx/model_quant.onnx
I20241127 11:53:44.064460  8232 ct-transformer-online.cpp:21] Successfully load model from /workspace/models/damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx/model_quant.onnx
I20241127 11:53:44.280210  8232 itn-processor.cpp:33] Successfully load model from /workspace/models/thuduj12/fst_itn_zh/zh_itn_tagger.fst
I20241127 11:53:44.281797  8232 itn-processor.cpp:35] Successfully load model from /workspace/models/thuduj12/fst_itn_zh/zh_itn_verbalizer.fst
I20241127 11:53:44.281826  8232 websocket-server-2pass.cpp:596] initAsr run check_and_clean_connection
I20241127 11:53:44.281924  8232 websocket-server-2pass.cpp:599] initAsr run check_and_clean_connection finished
I20241127 11:53:44.281942  8232 funasr-wss-server-2pass.cpp:571] decoder-thread-num: 40
I20241127 11:53:44.281993  8232 funasr-wss-server-2pass.cpp:572] io-thread-num: 3
I20241127 11:53:44.282011  8232 funasr-wss-server-2pass.cpp:573] model-thread-num: 1
I20241127 11:53:44.282045  8232 funasr-wss-server-2pass.cpp:574] asr model init finished. listen on port:10068
I20241127 11:54:03.335800  8701 websocket-server-2pass.cpp:450] hotwords: 
I20241127 11:54:03.335878  8701 websocket-server-2pass.cpp:453] 阿里巴巴 : 20
I20241127 11:54:03.335901  8701 websocket-server-2pass.cpp:453] 通义实验室 : 30
I20241127 11:54:03.336174  8701 websocket-server-2pass.cpp:490] jsonresult={"audio_fs":16000,"chunk_size":[5,10,5],"hotwords":null,"is_speaking":true,"itn":true,"mode":"2pass","svs_itn":true,"wav_format":"pcm","wav_name":"wav_default_id"}, msg_data->msg={"access_num":0,"audio_fs":16000,"is_eof":false,"itn":true,"mode":"2pass","svs_itn":true,"svs_lang":"auto","wav_format":"pcm","wav_name":"wav_default_id"}
I20241127 11:54:03.380604  8702 websocket-server-2pass.cpp:490] jsonresult={"is_speaking":false}, msg_data->msg={"access_num":11,"audio_fs":16000,"is_eof":false,"itn":true,"mode":"2pass","svs_itn":true,"svs_lang":"auto","wav_format":"pcm","wav_name":"wav_default_id"}
I20241127 11:54:03.380755  8702 websocket-server-2pass.cpp:496] client done
I20241127 11:54:03.666091  8659 websocket-server-2pass.cpp:67] online_res :灰色模
I20241127 11:54:03.788450  8659 websocket-server-2pass.cpp:67] online_res :式与中
I20241127 11:54:03.907672  8659 websocket-server-2pass.cpp:67] online_res :国色彩
I20241127 11:54:04.337137  8659 websocket-server-2pass.cpp:73] offline results : 灰色模式与**色彩
I20241127 11:54:04.337224  8659 websocket-server-2pass.cpp:80] offline stamps : [[230,450],[450,690],[690,850],[850,1030],[1030,1210],[1210,1310],[1310,1550],[1550,1770],[1770,2095]]
I20241127 11:54:04.337270  8659 websocket-server-2pass.cpp:88] offline stamp_sents : [{"end":2095,"punc":"","start":230,"text_seg":"灰 色 模 式 与 中 国 色 彩","ts_list":[[230,450],[450,690],[690,850],[850,1030],[1030,1210],[1210,1310],[1310,1550],[1550,1770],[1770,2095]]}]
I20241127 11:54:04.802182  8659 websocket-server-2pass.cpp:67] online_res :嗯
I20241127 11:54:04.802285  8659 websocket-server-2pass.cpp:73] offline results : ，我带你
I20241127 11:54:04.802307  8659 websocket-server-2pass.cpp:80] offline stamps : [[7650,7790],[7790,8130],[8130,8255]]
I20241127 11:54:04.802353  8659 websocket-server-2pass.cpp:88] offline stamp_sents : [{"end":-1,"punc":"，","start":-1,"text_seg":"","ts_list":[]},{"end":8255,"punc":"","start":7650,"text_seg":"我 带 你","ts_list":[[7650,7790],[7790,8130],[8130,8255]]}]
I20241127 11:54:05.193693  8659 websocket-server-2pass.cpp:67] online_res :哎呀
I20241127 11:54:05.193792  8659 websocket-server-2pass.cpp:73] offline results : 哎呀
I20241127 11:54:05.193817  8659 websocket-server-2pass.cpp:80] offline stamps : [[15260,15480],[15480,15790]]
I20241127 11:54:05.193856  8659 websocket-server-2pass.cpp:88] offline stamp_sents : [{"end":15790,"punc":"","start":15260,"text_seg":"哎 呀","ts_list":[[15260,15480],[15480,15790]]}]
I20241127 11:54:05.507570  8659 websocket-server-2pass.cpp:67] online_res :嗯
I20241127 11:54:05.507668  8659 websocket-server-2pass.cpp:73] offline results : 嗯
I20241127 11:54:05.507691  8659 websocket-server-2pass.cpp:80] offline stamps : [[19220,19530]]
I20241127 11:54:05.507730  8659 websocket-server-2pass.cpp:88] offline stamp_sents : [{"end":19530,"punc":"","start":19220,"text_seg":"嗯","ts_list":[[19220,19530]]}]
I20241127 11:54:05.887451  8659 websocket-server-2pass.cpp:67] online_res :嗯
I20241127 11:54:05.887557  8659 websocket-server-2pass.cpp:73] offline results : 好
I20241127 11:54:05.887579  8659 websocket-server-2pass.cpp:80] offline stamps : [[22030,22460]]
I20241127 11:54:05.887650  8659 websocket-server-2pass.cpp:88] offline stamp_sents : [{"end":22460,"punc":"","start":22030,"text_seg":"好","ts_list":[[22030,22460]]}]
I20241127 11:54:06.260715  8659 websocket-server-2pass.cpp:67] online_res :灰色
I20241127 11:54:06.369104  8659 websocket-server-2pass.cpp:67] online_res :模式与中
I20241127 11:54:06.477422  8659 websocket-server-2pass.cpp:67] online_res :国色
I20241127 11:54:06.896534  8659 websocket-server-2pass.cpp:67] online_res :彩
I20241127 11:54:06.896625  8659 websocket-server-2pass.cpp:73] offline results : 。灰色模式与**色彩
I20241127 11:54:06.896649  8659 websocket-server-2pass.cpp:80] offline stamps : [[34700,34960],[34960,35200],[35200,35360],[35360,35480],[35480,35660],[35660,35820],[35820,36020],[36020,36220],[36220,36575]]
I20241127 11:54:06.896698  8659 websocket-server-2pass.cpp:88] offline stamp_sents : [{"end":-1,"punc":"。","start":-1,"text_seg":"","ts_list":[]},{"end":36575,"punc":"","start":34700,"text_seg":"灰 色 模 式 与 中 国 色 彩","ts_list":[[34700,34960],[34960,35200],[35200,35360],[35360,35480],[35480,35660],[35660,35820],[35820,36020],[36020,36220],[36220,36575]]}]
./run_server_2pass.sh: line 38:  8232 Segmentation fault      (core dumped) $cmd_path/${cmd} --download-model-dir "${download_model_dir}" --model-dir "${model_dir}" --online-model-dir "${online_model_dir}" --vad-dir "${vad_dir}" --punc-dir "${punc_dir}" --itn-dir "${itn_dir}" --lm-dir "${lm_dir}" --decoder-thread-num ${decoder_thread_num} --model-thread-num ${model_thread_num} --io-thread-num ${io_thread_num} --port ${port} --certfile "${certfile}" --keyfile "${keyfile}" --hotword "${hotword}"

Environment

  • OS Linux
  • Docker version funasr-runtime-sdk-online-cpu-0.1.12
majic31 commented

我调试了下，不加lm的时候，funasrruntime.cpp:300中的msg_stamp.size()为0，而加lm时，funasrruntime.cpp:300中的msg_stamp.size()为2，所以加lm能跑过，但不加lm就不能跑了。感觉可以对msg_stamp.size()也加个保护

majic31 commented

一种修改思路：
if(msg_stamp.size()>0){ for(int i=0; i<msg_stamp.size()-1; i+=2){ float begin = std::stof(msg_stamp[i])+msg_stimes[idx]; float end = std::stof(msg_stamp[i+1])+msg_stimes[idx]; cur_stamp += "["+std::to_string((int)(1000*begin))+","+std::to_string((int)(1000*end))+"],"; } }

MyWestCity commented

好的，感谢！我看一下