wenet-e2e/wekws

stream推理运行时间久后就很难唤醒

xiaoxiaojiea opened this issue · 3 comments

将 stream_kws_ctc.py 代码中的 decode_keywords 函数做出如下修改:

 def decode_keywords(self, t, probs):
        absolute_time = t + self.total_frames

        next_hyps = ctc_prefix_beam_search(absolute_time, probs, self.cur_hyps,
                                           self.keywords_idxset, self.score_beam)

        cur_hyps = next_hyps[:self.path_beam]

        # 更新当前假设为新生成的候选路径。
        if cur_hyps == []:
            cur_hyps = [(tuple(), (1.0, 0.0, []))]

        self.cur_hyps = cur_hyps

也就是添加了如下两行:

if cur_hyps == []: cur_hyps = [(tuple(), (1.0, 0.0, []))]

我之前好像也遇到过这个问题,我当时是将ctc_prefix_beam_search中if not math.isclose(pnb, 0.0, abs_tol=0.000001)和if not math.isclose(pb, 0.0, abs_tol=0.000001)这两个if判断条件注释掉。

看起来是 reset() 没有被调用(实际应该是有一个外部 vad 来触发 reset 的),导致积累的历史解码信息太多出错。
可以试试这样修改:

def decode_keywords(self, t, probs):
        absolute_time = t + self.total_frames
        # search next_hyps depend on current probs and hyps.
        self.cur_hyps.append((tuple(), (1.0, 0.0, [])))
        next_hyps = ctc_prefix_beam_search(absolute_time,
                                           probs,
                                           self.cur_hyps,
                                           self.keywords_idxset,
                                           self.score_beam)
        # update cur_hyps. note: the hyps is sort by path score(pnb+pb),
        # not the keywords' probabilities.
        cur_hyps = next_hyps[:self.path_beam]
        self.cur_hyps = cur_hyps

看起来是 reset() 没有被调用(实际应该是有一个外部 vad 来触发 reset 的),导致积累的历史解码信息太多出错。 可以试试这样修改:

def decode_keywords(self, t, probs):
        absolute_time = t + self.total_frames
        # search next_hyps depend on current probs and hyps.
        self.cur_hyps.append((tuple(), (1.0, 0.0, [])))
        next_hyps = ctc_prefix_beam_search(absolute_time,
                                           probs,
                                           self.cur_hyps,
                                           self.keywords_idxset,
                                           self.score_beam)
        # update cur_hyps. note: the hyps is sort by path score(pnb+pb),
        # not the keywords' probabilities.
        cur_hyps = next_hyps[:self.path_beam]
        self.cur_hyps = cur_hyps

老哥,这里能详细说一下吗,我有点不太明白。