/Awesome-NAS

A curated list of neural architecture search (NAS) resources.

MIT LicenseMIT

Awesome NAS Awesome

A curated list of neural architecture search and related resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers, and awesome-architecture-search.

Please feel free to pull requests or open an issue to add papers.

Table of Contents

Neural Architecture Search (NAS)

Type G RL EA PD Other
Explanation gradient-based reinforcement learning evaluationary algorithm performance prediction other types

2019

Title Venue Type Code
Neural architecture search: A survey JMLR Survey -
DARTS: Differentiable Architecture Search ICLR G github
ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware ICLR RL/G github
Graph HyperNetworks for Neural Architecture Search ICLR G -
Learnable Embedding Space for Efficient Neural Architecture Compression ICLR Other github
Efficient Multi-Objective Neural Architecture Search via Lamarckian Evolution ICLR EA -
SNAS: stochastic neural architecture search ICLR G -
Searching for A Robust Neural Architecture in Four GPU Hours CVPR G github
ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation CVPR - -
Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search CVPR EA github
FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search CVPR G -
RENAS: Reinforced Evolutionary Neural Architecture Search CVPR G -
Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation CVPR G GitHub
MnasNet: Platform-Aware Neural Architecture Search for Mobile CVPR RL Github
MFAS: Multimodal Fusion Architecture Search CVPR EA -
A Neurobiological Evaluation Metric for Neural Network Model Search CVPR Other -
Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells CVPR RL -
Customizable Architecture Search for Semantic Segmentation CVPR - -
Regularized Evolution for Image Classifier Architecture Search AAAI EA -
AutoAugment: Learning Augmentation Policies from Data CVPR RL -
Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules ICML EA -
The Evolved Transformer ICML EA Github

2018

Title Venue Type Code
Efficient Architecture Search by Network Transformation AAAI RL github
Learning Transferable Architectures for Scalable Image Recognition CVPR RL github
N2N learning: Network to Network Compression via Policy Gradient Reinforcement Learning ICLR RL -
Practical Block-wise Neural Network Architecture Generation CVPR RL -
Path-Level Network Transformation for Efficient Architecture Search ICML RL github
Hierarchical Representations for Efficient Architecture Search ICLR EA -
Understanding and Simplifying One-Shot Architecture Search ICML G -
SMASH: One-Shot Model Architecture Search through HyperNetworks ICLR G github
Neural Architecture Optimization NeurIPS G github
Searching for efficient multi-scale architectures for dense image prediction NeurIPS Other -
Progressive Neural Architecture Search ECCV PD github
Neural Architecture Search with Bayesian Optimisation and Optimal Transport NeurIPS Other github
Differentiable Neural Network Architecture Search ICLR-W G -
Accelerating Neural Architecture Search using Performance Prediction ICLR-W PD -

2017

Title Venue Type Code
Neural Architecture Search with Reinforcement Learning ICLR RL -
Designing Neural Network Architectures using Reinforcement Learning ICLR RL -
Neural Optimizer Search with Reinforcement Learning ICML RL -
Learning Curve Prediction with Bayesian Neural Networks ICLR PD -
Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization ICLR PD -
Hyperparameter Optimization: A Spectral Approach NeurIPS-W Other github
Learning to Compose Domain-Specific Transformations for Data Augmentation NeurIPS - -

2012-2016

Title Venue Type Code
Speeding up Automatic Hyperparameter Optimization of Deep Neural Networksby Extrapolation of Learning Curves IJCAI PD github

Blogs

arXiv

  • Network Pruning via Transformable Architecture Search [pdf]
  • Xuanyi Dong, Yi Yang
  • Population Based Training of Neural Networks [pdf]
    • Max Jaderberg, Valentin Dalibard, Simon Osindero, Wojciech M. Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, Chrisantha Fernando, Koray Kavukcuoglu. arXiv 1711
  • NSGA-NET: A Multi-Objective Genetic Algorithm for Neural Architecture Search [pdf]
    • Lu, Zhichao and Whalen, Ian and Boddeti, Vishnu and Dhebar, Yashesh and Deb, Kalyanmoy and Goodman, Erik and Banzhaf, Wolfgang, arXiv 1810
  • Random Search and Reproducibility for Neural Architecture Search
    • Liam Li, Ameet Talwalkar. arXiv 1901
  • Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells [pdf]
    • Nekrasov, Vladimir and Chen, Hao and Shen, Chunhua and Reid, Ian. arXiv 1810
  • Training Frankenstein’s Creature to Stack: HyperTree Architecture Search [pdf]
    • Andrew Hundt, Varun Jain, Chris Paxton, Gregory D. Hager. arXiv 1810
  • Fast, Accurate and Lightweight Super-Resolution with Neural Architecture Search [pdf] [code]

Benchmark on ImageNet

Architecture Top-1 (%) Top-5 (%) Params (M) +x (M) GPU Days
Inception-v1 30.2 10.1 6.6 1448 - -
MobileNet-v1 29.4 10.5 4.2 569 - -
ShuffleNet 26.3 - ~5 524 - -
NASNet-A 26.0 8.4 5.3 564 450 3-4
NASNet-B 27.2 8.7 5.3 488 450 3-4
NASNet-C 27.5 9.0 4.9 558 450 3-4
AmobebaNet-A 25.5 8.0 5.1 555 450 7
AmobebaNet-B 26.0 8.5 5.3 555 450 7
AmobebaNet-C 24.3 7.6 6.4 555 450 7
Progressive NAS 25.8 8.1 5.1 588 100 1.5
DARTS-V2 26.9 9.0 4.9 595 1 1
GDAS 26.0 8.5 5.3 581 1 0.21