/CAT

A CRF-based ASR Toolkit

Primary LanguagePythonApache License 2.0Apache-2.0

CAT: CRF-based ASR Toolkit

CAT provides a complete workflow for CRF-based data-efficient end-to-end speech recognition.

Overview

CAT aims at combining the advantages of both the hybrid and the E2E ASR approches to achieve data-efficiency, by judiciously examining the pros and cons of modularity versus unified neural network, separate optimization versus joint optimization. CAT advocates global normalization modeling and discriminative training in the framework of Conditional Random Field (CRF), currently with Connectionist Temporal Classification (CTC) inspired state topology.

Features

  1. CAT contains a full-fledged CUDA/C/C++ implementation of CTC-CRF loss function binding to PyTorch.

  2. One-stop CTC/CTC-CRF/RNN-T/LM training & inference. See the templates.

  3. Flexible configuration with JSON. Check the guideline for configuration.

  4. Scalable and extensible. It is easy to be extended to train tens of thousands of speech data and add new models and tasks.

Installation

  1. Dependencies

    • CUDA compatible device, NVIDIA driver installed and CUDA lib available.

    • PyTorch: >=1.9.0 is required. Installation guide from PyTorch

    • Kaldi [optional, but recommended]: used for speech data preparation and some FST-related operations. This is optional for most of the basic functions. Required only when you want to conduct CTC-CRF training.

      Besides Kaldi, you could use torchaudio for feature extraction. Take a look at data.sh for how to prepare data with torchaudio.

  2. Clone and install CAT

    git clone https://github.com/thu-spmi/CAT.git && cd CAT
# Get installation helping message
./install.sh -h
# Install with default configurations
#./install.sh

Getting started

To get started with this project, please refer to TEMPLATE for tutorial.

ASR results

dataset evaluation sets performance
AISHELL-1 dev / test 3.93 / 4.22
Commonvoice German test 9.8
Librispeech test-clean / test-other 1.94 / 4.39
Switchboard switchboard / callhome 6.9 / 14.5
THCHS30 test 6.01
Wenetspeech test-net / test-meeting 9.32 / 14.66
WSJ eval92 / dev93 2.77 / 5.68

Further reading

Citation

@inproceedings{xiang2019crf,
  title={CRF-based single-stage acoustic modeling with CTC topology},
  author={Xiang, Hongyu and Ou, Zhijian},
  booktitle={IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages={5676--5680},
  year={2019},
  organization={IEEE}
}

@inproceedings{an2020cat,
  title={CAT: A CTC-CRF based ASR toolkit bridging the hybrid and the end-to-end approaches towards data efficiency and low latency},
  author={An, Keyu and Xiang, Hongyu and Ou, Zhijian},
  booktitle={INTERSPEECH},
  pages={566--570},
  year={2020}
}