/Awesome-NAS

A curated list of neural architecture search (NAS) resources.

MIT LicenseMIT

Awesome NAS Awesome

A curated list of neural architecture search and related resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers, and awesome-architecture-search.

Please feel free to pull requests or open an issue to add papers.

Table of Contents

Neural Architecture Search (NAS)

Type G RL EA PD Other
Explanation gradient-based reinforcement learning evaluationary algorithm performance prediction other types

2019

Title Venue Type Code
Neural architecture search: A survey JMLR Survey -
DARTS: Differentiable Architecture Search ICLR G github
ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware ICLR RL/G github
Graph HyperNetworks for Neural Architecture Search ICLR G -
Learnable Embedding Space for Efficient Neural Architecture Compression ICLR Other github
Efficient Multi-Objective Neural Architecture Search via Lamarckian Evolution ICLR EA -
SNAS: stochastic neural architecture search ICLR G -
Searching for A Robust Neural Architecture in Four GPU Hours CVPR G github
Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search CVPR EA github
FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search CVPR G -
RENAS: Reinforced Evolutionary Neural Architecture Search CVPR G -
Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation CVPR G GitHub
MnasNet: Platform-Aware Neural Architecture Search for Mobile CVPR RL Github
MFAS: Multimodal Fusion Architecture Search CVPR EA -
A Neurobiological Evaluation Metric for Neural Network Model Search CVPR Other -
Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells CVPR RL -
Customizable Architecture Search for Semantic Segmentation CVPR - -
Regularized Evolution for Image Classifier Architecture Search AAAI EA -
AutoAugment: Learning Augmentation Policies from Data CVPR RL -
Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules ICML EA -

2018

Title Venue Type Code
Efficient Architecture Search by Network Transformation AAAI RL github
Learning Transferable Architectures for Scalable Image Recognition CVPR RL github
N2N learning: Network to Network Compression via Policy Gradient Reinforcement Learning ICLR RL -
Practical Block-wise Neural Network Architecture Generation CVPR RL -
Path-Level Network Transformation for Efficient Architecture Search ICML RL github
Hierarchical Representations for Efficient Architecture Search ICLR EA -
Understanding and Simplifying One-Shot Architecture Search ICML G -
SMASH: One-Shot Model Architecture Search through HyperNetworks ICLR G github
Neural Architecture Optimization NeurIPS G github
Searching for efficient multi-scale architectures for dense image prediction NeurIPS Other -
Progressive Neural Architecture Search ECCV PD github
Neural Architecture Search with Bayesian Optimisation and Optimal Transport NeurIPS Other github
Differentiable Neural Network Architecture Search ICLR-W G -
Accelerating Neural Architecture Search using Performance Prediction ICLR-W PD -

2017

Title Venue Type Code
Neural Architecture Search with Reinforcement Learning ICLR RL -
Designing Neural Network Architectures using Reinforcement Learning ICLR RL -
Neural Optimizer Search with Reinforcement Learning ICML RL -
Learning Curve Prediction with Bayesian Neural Networks ICLR PD -
Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization ICLR PD -
Hyperparameter Optimization: A Spectral Approach NeurIPS-W Other github
Learning to Compose Domain-Specific Transformations for Data Augmentation NeurIPS - -

2012-2016

Title Venue Type Code
Speeding up Automatic Hyperparameter Optimization of Deep Neural Networksby Extrapolation of Learning Curves IJCAI PD github

Blogs

arXiv

  • Network Pruning via Transformable Architecture Search [pdf]
  • Xuanyi Dong, Yi Yang
  • Population Based Training of Neural Networks [pdf]
    • Max Jaderberg, Valentin Dalibard, Simon Osindero, Wojciech M. Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, Chrisantha Fernando, Koray Kavukcuoglu. arXiv 1711
  • NSGA-NET: A Multi-Objective Genetic Algorithm for Neural Architecture Search [pdf]
    • Lu, Zhichao and Whalen, Ian and Boddeti, Vishnu and Dhebar, Yashesh and Deb, Kalyanmoy and Goodman, Erik and Banzhaf, Wolfgang, arXiv 1810
  • Random Search and Reproducibility for Neural Architecture Search
    • Liam Li, Ameet Talwalkar. arXiv 1901
  • Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells [pdf]
    • Nekrasov, Vladimir and Chen, Hao and Shen, Chunhua and Reid, Ian. arXiv 1810
  • Training Frankenstein’s Creature to Stack: HyperTree Architecture Search [pdf]
    • Andrew Hundt, Varun Jain, Chris Paxton, Gregory D. Hager. arXiv 1810
  • Fast, Accurate and Lightweight Super-Resolution with Neural Architecture Search [pdf] [code]

Benchmark on ImageNet

Architecture Top-1 (%) Top-5 (%) Params (M) +x (M) GPU Days
Inception-v1 30.2 10.1 6.6 1448 - -
MobileNet-v1 29.4 10.5 4.2 569 - -
ShuffleNet 26.3 - ~5 524 - -
NASNet-A 26.0 8.4 5.3 564 450 3-4
NASNet-B 27.2 8.7 5.3 488 450 3-4
NASNet-C 27.5 9.0 4.9 558 450 3-4
AmobebaNet-A 25.5 8.0 5.1 555 450 7
AmobebaNet-B 26.0 8.5 5.3 555 450 7
AmobebaNet-C 24.3 7.6 6.4 555 450 7
Progressive NAS 25.8 8.1 5.1 588 100 1.5
DARTS-V2 26.9 9.0 4.9 595 1 1
GDAS 26.0 8.5 5.3 581 1 0.21