train using Mocha and decode using OnlineMocha

[zyzpower]

Hi,
I have a issue regarding online decoding and offline training.
I realize when I train using MoCha class and decode using OnlineMoChA class, the result is the same as when I train and decode using both MoCha class. I think if I train and decode using both MoCha class, the result should be better (offline case)? Is my way for streaming ASR model wrong? How to use MoCha and OnlineMoCha class?
Thanks.

[HaoranMiao]

Hi,
I have a issue regarding online decoding and offline training.
I realize when I train using MoCha class and decode using OnlineMoChA class, the result is the same as when I train and decode using both MoCha class. I think if I train and decode using both MoCha class, the result should be better (offline case)? Is my way for streaming ASR model wrong? How to use MoCha and OnlineMoCha class?
Thanks.

The scores of the same hypothesis should be different when decoding by MoChA and OnlineMoChA class, except that all the endpoints locate at the last frame. When you decode with OnlineMoChA, you can check the endpoints returned by the forward function of OnlineMoChA class.

[zyzpower]

Hi thanks for the reply. I realize my problem is due to the self.att module never changes from Mocha to OnlineMoCha class. So I define the att by the parser like below:

    elif args.atype == 'smocha':
        att = MoChA(args.eprojs, args.dunits, args.adim, 3.0, 1.0, -4.0)
    elif args.atype == 'online_smocha':
        att = OnlineMoChA(args.eprojs, args.dunits, args.adim, 3.0, 1.0, -4.0)

but when I run decoding, when I print the att content, I realize it is still of the class MoCha instead of OnlineMoCha. How do you address this issue?

[zyzpower]

Hi, do you have any plan to release the chunk-SAE related code?

[HaoranMiao]

Hi thanks for the reply. I realize my problem is due to the self.att module never changes from Mocha to OnlineMoCha class. So I define the att by the parser like below:

    elif args.atype == 'smocha':
        att = MoChA(args.eprojs, args.dunits, args.adim, 3.0, 1.0, -4.0)
    elif args.atype == 'online_smocha':
        att = OnlineMoChA(args.eprojs, args.dunits, args.adim, 3.0, 1.0, -4.0)

but when I run decoding, when I print the att content, I realize it is still of the class MoCha instead of OnlineMoCha. How do you address this issue?

You may check the configuration file, which is *.yaml in espnet, and make sure the atype is online_smocha.

[zyzpower]

Yes I have changed that in train configuration file, but it doesn't read the train configuration file during decoding. I have also tried add this parameter into decoder configuration file, but it makes no difference. I have to hard code this parameter in E2E class. Why is that?

[HaoranMiao]

Yes I have changed that in train configuration file, but it doesn't read the train configuration file during decoding. I have also tried add this parameter into decoder configuration file, but it makes no difference. I have to hard code this parameter in E2E class. Why is that?

For the latest espnet, you should change the atype in model.json according to https://github.com/espnet/espnet/blob/bb2ef2d0cca7fd5ecaafe90a8a14ad8fd9b55db7/espnet/asr/pytorch_backend/asr_init.py#L158