about SOTA

[SSRDMP]

hello jctian98
Impressive work!
Very happy to see that aishell's sota has been refreshed again.
I have some doubts, hope you can help me figure it out,
Why the aishell-1 result on paper with code leaderboard is 4.18%
https://paperswithcode.com/sota/speech-recognition-on-aishell-1
but the result on aishell-1 in you paper is, 4.10%,
Is there any different between two result?
Is the dev set used during training?
Sincerely hope to get your reply.

[jctian98]

Sorry for this late reply.

1. The 4.18% result is reported in this paper and the 4.10% result is reported in this paper.
2. The CER is reduced from 4.18% to 4.10% since we have changed the training process of word-level N-gram LM. The smoothing methods for the N-gram LM do have some impact on the final CER. We find the Witten-Bell discounting is the best choice in Aishell-1 experiments. You can see this in our script to check more details about the LM training`.
3. The dev set is used during training only for validation. We average the last 10 checkpoints for evaluation so there should be no information leak.

Best,
Jinchuan

[SSRDMP]

Sorry for this late reply.

1. The 4.18% result is reported in this paper and the 4.10% result is reported in this paper.
2. The CER is reduced from 4.18% to 4.10% since we have changed the training process of word-level N-gram LM. The smoothing methods for the N-gram LM do have some impact on the final CER. We find the Witten-Bell discounting is the best choice in Aishell-1 experiments. You can see this in our script to check more details about the LM training`.
3. The dev set is used during training only for validation. We average the last 10 checkpoints for evaluation so there should be no information leak.

Best, Jinchuan

Thank you very much, your answer completely solved my doubts, this method is really efficient. Thanks again！