Contrastive Learning for Improving ASR Robustness in Spoken Language Understanding

[tatsuropfgt]

Contrastive Learning for Improving ASR Robustness in Spoken Language Understanding [ICASSP22]

Abstract

• improve the ASR robustness by contrastive pretraining of RoBERTa
• fine-tuning with self-distillation to reduce the label noises due to ASR errors

Method

Pretraining

• improve robustness with only textual information
  • previous works need additional speech-related input features

Fine-tuning

• hard + soft + self-distillation
• self-distillation is calculated by KL divergence between current and previous prediction
• soft loss is calculated by the similarity