This repository is the official implementation of "NeXt-TDNN: Modernizing Multi-Scale Temporal Convolution Backbone for Speaker Verification" accepted in ICASSP 2024 Paper Link
🔥 December, 2023: We have uploaded the pre-trained models of our NeXt-TDNN in the experiments
folder!
🍀 February 2024, the NeXt-TDNN model was updated with cyclic learning rate scheduling. This update improved the EER from 0.79% to 0.72%. Changes were made to the LR scheduling, gradient clipping value, and batch size. Please check configs/NeXt_TDNN_C256_B3_K65_7_cyclical_lr_step.py
for details.
This code requires the following:
- lightning == 2.1.2
- CUDA, PyToch installation
# CUDA
conda install -c "nvidia/label/cuda-11.8.0" cuda
# PyTorch
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
- VoxCeleb Dataset
- To download VoxCeleb dataset fot train/test, execute the command described in the Data preparation section of the voxceleb_trainer repository
- Download VoxCeleb1-O, VoxCeleb1-E, and VoxCeleb1-H for test and locate it
data
directory
To train ASV model, run main script in train mode. You can select the desired training configuration through config argument.
- to train NeXt-TDNN(C=256, B=3)
python main.py --mode train --config configs/NeXt_TDNN_C256_B3_K65_7
To test on VoxCeleb1, run the script below. As in training, select the desired test configuration.
# VoxCeleb1-O
python main.py --mode test --config configs/NeXt_TDNN_C256_B3_K65_7
# ⚡ VoxCeleb1-O, VoxCeleb1-E, VoxCeleb1-H
python main.py --mode test_all --config configs/NeXt_TDNN_C256_B3_K65_7
- https://github.com/facebookresearch/ConvNeXt-V2
- https://github.com/clovaai/voxceleb_trainer
- https://github.com/mechanicalsea/sugar
- https://github.com/TaoRuijie/ECAPA-TDNN
- https://github.com/speechbrain/speechbrain
- https://github.com/zyzisyz/mfa_conformer
If you find our work useful, please refer to
@misc{heo2023nexttdnn,
title={NeXt-TDNN: Modernizing Multi-Scale Temporal Convolution Backbone for Speaker Verification},
author={Hyun-Jun Heo and Ui-Hyeop Shin and Ran Lee and YoungJu Cheon and Hyung-Min Park},
year={2023},
eprint={2312.08603},
archivePrefix={arXiv},
primaryClass={eess.AS}
}