Automated Deep Learning Projects (AutoDL-Projects) is an open source, lightweight, but useful project for everyone. This project implemented several neural architecture search (NAS) and hyper-parameter optimization (HPO) algorithms. 中文介绍见README_CN.md
Who should consider using AutoDL-Projects
- Beginners who want to try different AutoDL algorithms
- Engineers who want to try AutoDL to investigate whether AutoDL works on your projects
- Researchers who want to easily implement and experiement new AutoDL algorithms.
Why should we use AutoDL-Projects
- Simple library dependencies
- All algorithms are in the same codebase
- Active maintenance
At this moment, this project provides the following algorithms and scripts to run them. Please see the details in the link provided in the description column.
| Type | ABBRV | Algorithms | Description |
|---|---|---|---|
| NAS | TAS | Network Pruning via Transformable Architecture Search | NeurIPS-2019-TAS.md |
| DARTS | DARTS: Differentiable Architecture Search | ICLR-2019-DARTS.md | |
| GDAS | Searching for A Robust Neural Architecture in Four GPU Hours | CVPR-2019-GDAS.md | |
| SETN | One-Shot Neural Architecture Search via Self-Evaluated Template Network | ICCV-2019-SETN.md | |
| NAS-Bench-201 | NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search | NAS-Bench-201.md | |
| NATS-Bench | NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size | NATS-Bench.md | |
| ... | ENAS / REA / REINFORCE / BOHB | Please check the original papers | NAS-Bench-201.md NATS-Bench.md |
| HPO | HPO-CG | Hyperparameter optimization with approximate gradient | coming soon |
| Basic | ResNet | Deep Learning-based Image Classification | BASELINE.md |
First of all, please use python setup.py install to install xautodl library.
Please install Python>=3.6 and PyTorch>=1.5.0. (You could use lower versions of Python and PyTorch, but may have bugs).
Some visualization codes may require opencv.
CIFAR and ImageNet should be downloaded and extracted into $TORCH_HOME.
Some methods use knowledge distillation (KD), which require pre-trained models. Please download these models from Google Drive (or train by yourself) and save into .latent-data.
Please use
git clone --recurse-submodules git@github.com:D-X-Y/AutoDL-Projects.git XAutoDL
git clone --recurse-submodules https://github.com/D-X-Y/AutoDL-Projects.git XAutoDL
to download this repo with submodules.
If you find that this project helps your research, please consider citing the related paper:
@inproceedings{dong2021autohas,
title = {{AutoHAS}: Efficient Hyperparameter and Architecture Search},
author = {Dong, Xuanyi and Tan, Mingxing and Yu, Adams Wei and Peng, Daiyi and Gabrys, Bogdan and Le, Quoc V},
booktitle = {2nd Workshop on Neural Architecture Search at International Conference on Learning Representations (ICLR)},
year = {2021}
}
@article{dong2021nats,
title = {{NATS-Bench}: Benchmarking NAS Algorithms for Architecture Topology and Size},
author = {Dong, Xuanyi and Liu, Lu and Musial, Katarzyna and Gabrys, Bogdan},
doi = {10.1109/TPAMI.2021.3054824},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
year = {2021},
note = {\mbox{doi}:\url{10.1109/TPAMI.2021.3054824}}
}
@inproceedings{dong2020nasbench201,
title = {{NAS-Bench-201}: Extending the Scope of Reproducible Neural Architecture Search},
author = {Dong, Xuanyi and Yang, Yi},
booktitle = {International Conference on Learning Representations (ICLR)},
url = {https://openreview.net/forum?id=HJxyZkBKDr},
year = {2020}
}
@inproceedings{dong2019tas,
title = {Network Pruning via Transformable Architecture Search},
author = {Dong, Xuanyi and Yang, Yi},
booktitle = {Neural Information Processing Systems (NeurIPS)},
pages = {760--771},
year = {2019}
}
@inproceedings{dong2019one,
title = {One-Shot Neural Architecture Search via Self-Evaluated Template Network},
author = {Dong, Xuanyi and Yang, Yi},
booktitle = {Proceedings of the IEEE International Conference on Computer Vision (ICCV)},
pages = {3681--3690},
year = {2019}
}
@inproceedings{dong2019search,
title = {Searching for A Robust Neural Architecture in Four GPU Hours},
author = {Dong, Xuanyi and Yang, Yi},
booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
pages = {1761--1770},
year = {2019}
}
If you want to contribute to this repo, please see CONTRIBUTING.md. Besides, please follow CODE-OF-CONDUCT.md.
We use black for Python code formatter.
Please use black . -l 88.
The entire codebase is under the MIT license.