modelscope/3D-Speaker

speaker-diarization需要哪个版本的Funasr，第六步无法输出

HollrayChan opened this issue · 2 comments

HollrayChan commented

使用了最新的Funasr==1.0.4，需要补充model_revision和修改vad_pipeline(wpath)，但是在执行第六步的时候，会出现这样的报错，换成旧的0.8.8也是无法执行

Stage 1: Prepare input wavs...
--2024-01-30 18:07:32--  https://modelscope.cn/api/v1/models/damo/speech_eres2net-large_speaker-diarization_common/repo?Revision=master&FilePath=examples/example.wav
正在解析主机 modelscope.cn (modelscope.cn)... 39.101.130.40
正在连接 modelscope.cn (modelscope.cn)|39.101.130.40|:443... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度：30720078 (29M) [application/octet-stream]
正在保存至: “examples/example.wav”

examples/example.wav                   100%[==========================================================================>]  29.30M  43.9MB/s  用时 0.7s    

2024-01-30 18:07:34 (43.9 MB/s) - 已保存 “examples/example.wav” [30720078/30720078])

--2024-01-30 18:07:34--  https://modelscope.cn/api/v1/models/damo/speech_eres2net-large_speaker-diarization_common/repo?Revision=master&FilePath=examples/example.rttm
正在解析主机 modelscope.cn (modelscope.cn)... 39.101.130.40
正在连接 modelscope.cn (modelscope.cn)|39.101.130.40|:443... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度：1329 (1.3K) [application/octet-stream]
正在保存至: “examples/example.rttm”

examples/example.rttm                  100%[==========================================================================>]   1.30K  --.-KB/s  用时 0s      

2024-01-30 18:07:34 (29.3 MB/s) - 已保存 “examples/example.rttm” [1329/1329])

Stage2: Do vad for input wavs...
2024-01-30 18:07:37,343 - modelscope - INFO - PyTorch version 2.0.0+cu118 Found.
2024-01-30 18:07:37,345 - modelscope - INFO - Loading ast index from /home/winner/.cache/modelscope/ast_indexer
2024-01-30 18:07:37,470 - modelscope - INFO - Loading done! Current index file version is 1.11.1, with md5 e4ea8cecd8079cde83f512df2bae21a7 and a total number of 956 components indexed
[2024-01-30 18:07:38,659] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Please install rotary_embedding_torch by: 
 pip install -U rotary_embedding_torch
Please install rotary_embedding_torch by: 
 pip install -U rotary_embedding_torch
Please install rotary_embedding_torch by: 
 pip install -U rotary_embedding_torch
Please install rotary_embedding_torch by: 
 pip install -U rotary_embedding_torch
2024-01-30 18:07:44,757 - modelscope - INFO - Use user-specified model revision: v2.0.4
2024-01-30 18:07:45,018 - modelscope - INFO - initiate model from /home/winner/.cache/modelscope/hub/damo/speech_fsmn_vad_zh-cn-16k-common-pytorch
2024-01-30 18:07:45,018 - modelscope - INFO - initiate model from location /home/winner/.cache/modelscope/hub/damo/speech_fsmn_vad_zh-cn-16k-common-pytorch.
2024-01-30 18:07:45,019 - modelscope - INFO - initialize model from /home/winner/.cache/modelscope/hub/damo/speech_fsmn_vad_zh-cn-16k-common-pytorch
2024-01-30 18:07:49,164 - modelscope - WARNING - No preprocessor field found in cfg.
2024-01-30 18:07:49,164 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2024-01-30 18:07:49,164 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': '/home/winner/.cache/modelscope/hub/damo/speech_fsmn_vad_zh-cn-16k-common-pytorch'}. trying to build by task and model information.
2024-01-30 18:07:49,164 - modelscope - WARNING - No preprocessor key ('funasr', 'voice-activity-detection') found in PREPROCESSOR_MAP, skip building preprocessor.
[INFO]: Start computing VAD...
rtf_avg: 0.225: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:03<00:00,  3.69s/it]
rtf_avg: 594.604: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:11<00:00, 11.90s/it]
[INFO]: VAD json is prepared in exp/json/vad.json
Stage3: Prepare subsegments info...
[INFO]: Generate sub-segmetns...
[INFO]: Subsegments json is prepared in exp/json/subseg.json
Stage4: Extract speaker embeddings...
WARNING:torch.distributed.run:
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
*****************************************
2024-01-30 18:08:21,239 - modelscope - INFO - PyTorch version 2.0.0+cu118 Found.
2024-01-30 18:08:21,241 - modelscope - INFO - Loading ast index from /home/winner/.cache/modelscope/ast_indexer
2024-01-30 18:08:21,262 - modelscope - INFO - PyTorch version 2.0.0+cu118 Found.
2024-01-30 18:08:21,264 - modelscope - INFO - Loading ast index from /home/winner/.cache/modelscope/ast_indexer
2024-01-30 18:08:21,274 - modelscope - INFO - PyTorch version 2.0.0+cu118 Found.
2024-01-30 18:08:21,275 - modelscope - INFO - Loading ast index from /home/winner/.cache/modelscope/ast_indexer
2024-01-30 18:08:21,362 - modelscope - INFO - PyTorch version 2.0.0+cu118 Found.
2024-01-30 18:08:21,363 - modelscope - INFO - Loading ast index from /home/winner/.cache/modelscope/ast_indexer
2024-01-30 18:08:21,382 - modelscope - INFO - PyTorch version 2.0.0+cu118 Found.
2024-01-30 18:08:21,384 - modelscope - INFO - Loading ast index from /home/winner/.cache/modelscope/ast_indexer
2024-01-30 18:08:21,386 - modelscope - INFO - PyTorch version 2.0.0+cu118 Found.
2024-01-30 18:08:21,388 - modelscope - INFO - Loading ast index from /home/winner/.cache/modelscope/ast_indexer
2024-01-30 18:08:21,394 - modelscope - INFO - Loading done! Current index file version is 1.11.1, with md5 e4ea8cecd8079cde83f512df2bae21a7 and a total number of 956 components indexed
2024-01-30 18:08:21,414 - modelscope - INFO - Loading done! Current index file version is 1.11.1, with md5 e4ea8cecd8079cde83f512df2bae21a7 and a total number of 956 components indexed
2024-01-30 18:08:21,430 - modelscope - INFO - Loading done! Current index file version is 1.11.1, with md5 e4ea8cecd8079cde83f512df2bae21a7 and a total number of 956 components indexed
2024-01-30 18:08:21,486 - modelscope - INFO - Loading done! Current index file version is 1.11.1, with md5 e4ea8cecd8079cde83f512df2bae21a7 and a total number of 956 components indexed
2024-01-30 18:08:21,502 - modelscope - INFO - Loading done! Current index file version is 1.11.1, with md5 e4ea8cecd8079cde83f512df2bae21a7 and a total number of 956 components indexed
2024-01-30 18:08:21,510 - modelscope - INFO - Loading done! Current index file version is 1.11.1, with md5 e4ea8cecd8079cde83f512df2bae21a7 and a total number of 956 components indexed
2024-01-30 18:08:21,716 - modelscope - INFO - PyTorch version 2.0.0+cu118 Found.
2024-01-30 18:08:21,718 - modelscope - INFO - Loading ast index from /home/winner/.cache/modelscope/ast_indexer
2024-01-30 18:08:21,829 - modelscope - INFO - Loading done! Current index file version is 1.11.1, with md5 e4ea8cecd8079cde83f512df2bae21a7 and a total number of 956 components indexed
2024-01-30 18:08:21,835 - modelscope - INFO - PyTorch version 2.0.0+cu118 Found.
2024-01-30 18:08:21,837 - modelscope - INFO - Loading ast index from /home/winner/.cache/modelscope/ast_indexer
2024-01-30 18:08:21,968 - modelscope - INFO - Loading done! Current index file version is 1.11.1, with md5 e4ea8cecd8079cde83f512df2bae21a7 and a total number of 956 components indexed
[2024-01-30 18:08:22,719] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-30 18:08:22,719] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-30 18:08:22,743] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-30 18:08:22,763] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-30 18:08:22,797] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-30 18:08:22,825] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-30 18:08:23,048] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2024-01-30 18:08:23,275] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
2024-01-30 18:08:32,879 - modelscope - INFO - Use user-specified model revision: v1.0.0
WARNING: The number of threads exceeds the number of files
WARNING: The number of threads exceeds the number of files
WARNING: The number of threads exceeds the number of files
[INFO] Start computing embeddings...
[INFO] Start computing embeddings...
WARNING: The number of threads exceeds the number of filesWARNING: The number of threads exceeds the number of files

[WARNING] Embeddings has been saved previously. Skip it.
[WARNING] Embeddings has been saved previously. Skip it.
WARNING: The number of threads exceeds the number of files
Stage5: Perform clustering and output sys rttms...
WARNING:torch.distributed.run:
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
*****************************************
[INFO] Start clustering...
[INFO] Start clustering...
[INFO] Start clustering...
WARNING: The number of threads exceeds the number of files
WARNING: The number of threads exceeds the number of files
WARNING: The number of threads exceeds the number of files
WARNING: The number of threads exceeds the number of files
WARNING: The number of threads exceeds the number of files
/home/winner/anaconda3/envs/py38-pt200/lib/python3.8/site-packages/sklearn/cluster/_kmeans.py:870: FutureWarning: The default value of `n_init` will change from 10 to 'auto' in 1.4. Set the value of `n_init` explicitly to suppress the warning
  warnings.warn(
/home/winner/anaconda3/envs/py38-pt200/lib/python3.8/site-packages/sklearn/cluster/_kmeans.py:870: FutureWarning: The default value of `n_init` will change from 10 to 'auto' in 1.4. Set the value of `n_init` explicitly to suppress the warning
  warnings.warn(
/home/winner/anaconda3/envs/py38-pt200/lib/python3.8/site-packages/sklearn/cluster/_kmeans.py:870: FutureWarning: The default value of `n_init` will change from 10 to 'auto' in 1.4. Set the value of `n_init` explicitly to suppress the warning
  warnings.warn(
Stage6: Get the final metrics...
Computing DER...
2024-01-30 18:08:53,245 - INFO: Concatenating individual RTTM files...
2024-01-30 18:08:53,285 - INFO: MS: 2.069159, FA: 0.203668, SER: 0.000000, DER: 2.272828
Computing ACC...
error,there is no fileid_sys in ref rttm: output
seg pur error,there is no fileid_sys in ref rttm: %s output
eval_elems_seg error,there is no fileid_sys in ref rttm: %s output
All metrics have been done.
yfchenlucky commented

Stage6: Get the final metrics...
Computing DER...
2024-01-30 18:52:54,944 - INFO: Concatenating individual RTTM files...
2024-01-30 18:52:55,086 - INFO: MS: 2.069159, FA: 0.203668, SER: 0.000000, DER: 2.272828
Computing ACC...
All metrics have been done.

我使用的funasr==0.8.4
如果换成旧的0.8.8，应该不会报错，可以再次尝试。funasr1.0+版本刚刚上线，本项目只调用funasr的vad接口，近期最好还是使用老版本，后面我会将接口更新。