Keras (w/ tensorflow backend) code accompanying our paper Regression and Classification for Direction-of-Arrival Estimation with Convolutional Recurrent Neural Networks, Interspeech 2019
As described in the paper, our training/validation set is purely simulated data using a geometric sound propagation engine, whereas the test set only consists of real-world recorded data. To train/test our models, you need to download following files:
- Training/validation set features and labels [33.4GB] https://obj.umiacs.umd.edu/gammadata/dataset/doa/features.zip
- Test set original wav files and labels [19.2GB] https://obj.umiacs.umd.edu/gammadata/dataset/doa/SOFA_DOA_test_set.zip
- Training/validation set original wav files (you can skip this one if you don't want to extract the feature yourself) [43.8GB] https://obj.umiacs.umd.edu/gammadata/dataset/doa/wav.zip
The reason for providing test set in wav format instead of feature is that our models are trained assuming the ACN convention, whereas others may assume FuMa convention (see Ambisonic formats). In the test code, we provide a simple function for format conversion before feature extraction if needed.
In this repo, we attach several models under models folder. These are:
| Model name | Explanation | Convention |
|---|---|---|
| cartesian_base_model.h5 | Initial (untrained) regression model | ACN |
| cartesian_trained_model.h5 | Trained regression model | ACN |
| categorical_trained_model.h5 | Trained classification model | ACN |
| Perotin_categorical_trained_model.h5 | Baseline classification model by Perotin et al. | FuMa |
Note that categorical_trained_model.h5 is trained from Perotin_categorical_trained_model.h5, which is not different from training from scratch because their model uses FuMa convention that generates very different features from ACN convention.
- Download dataset and clone this repo.
- Build conda environment and activate it:
conda env create --file=environment.yml conda activate doa-release - Training
python3 train.py -i [train_feature_dir] -l [train_feature_dir]/train_labels.csv -o [output_dir] -lo [cartesian or categorical] -m models/[model_name] - Testing
When testing a model that uses FuMa convention, you must append
python3 test.py -i [test_wav_dir] -l [test_wav_dir]/test_labels.csv -m [trained_model_path] -lo [cartesian or categorical]-cto test.py's argument list, which enables conversion from ACN to FuMa. For example:python3 test.py -i [test_wav_dir] -l [test_wav_dir]/test_labels.csv -m models/Perotin_categorical_trained_model.h5 -lo categorical -c
If you use our codes or models, please consider citing:
@inproceedings{Tang2019,
author={Zhenyu Tang and John D. Kanu and Kevin Hogan and Dinesh Manocha},
title={{Regression and Classification for Direction-of-Arrival Estimation with Convolutional Recurrent Neural Networks}},
year=2019,
booktitle={Proc. Interspeech 2019},
pages={654--658},
doi={10.21437/Interspeech.2019-1111},
url={http://dx.doi.org/10.21437/Interspeech.2019-1111}
}
We also recommend citing the work of Perontin et al. if you also use their model:
@inproceedings{perotin2018crnn,
title={CRNN-based joint azimuth and elevation localization with the Ambisonics intensity vector},
author={Perotin, Laur{\'e}line and Serizel, Romain and Vincent, Emmanuel and Gu{\'e}rin, Alexandre},
booktitle={2018 16th International Workshop on Acoustic Signal Enhancement (IWAENC)},
pages={241--245},
year={2018},
organization={IEEE}
}