/Neural-Architecture-Search

This repo is about NAS

GNU General Public License v3.0GPL-3.0

Neural Architecture Search

Tabel of contents

Introduction

The list includes the papers related to NAS I have read. All of them are grouped into three classes ( Search Space, Seach strategy and evaluation strategy) based on my subjectivity.

Papers

Reviews

Data Title Venue Notes
2018 Neural Architecture Search: A Survey JMLR -
2008 Neuroevolution: From Architectures to Learning Evolutionary Intelligence -
1999 Evolving Artificial Neural Networks IEEE -

Search Space / Encoding strategies

Data Title Venue Code Notes
2018 Neural Architecture Search Over a Graph Search Space - - This paper defined a search space on direct graph which is used to instruct the construction of networks.
2017 Genetic CNN ICCV Tensorflow This paper proposed a binary string encoding strategy.
2017/18 Efficient Architecture Search by Network Transformation AAAI - This paper Proposed a popular block based network definition method.
2009 A Hypercube-based Encoding for Evolving large-scale Neural Networks IEEE Artificial Life - This method use a extended CPPNs to encode the ANN.
2006 Evolutionary Design of Neural Network Architectures Using a Descriptive Encoding Language IEEE TEVC - This paper proposed a human-readable and writable encoding method.
2002 Evolving Neural Networks through Augmenting Topologies EC - NEAT: An Awesome Approach to NeuroEvolution
1998 Network Generating Attribute grammar Encoding IJCNN - Grammar Encoding
1996 A comparison between cellular encoding and direct encoding for genetic neural networks - - This paper compared the celluar encoding and direct encoding methods
1996 Evolving Graphs and Networks with Edge encoding: preliminary report The Genetic Programming Conference - Edge Encoding method.
1994 Neurogenetic learning: an integrated method of designing and training neural networks using genetic algorithms Physica D - This paper inspired by L-System.
1994 Automatic definition of modular neural networks Adaptive behavior - -
1990 Designing Neural Networks Using Genetic Algorithms with Graph Generation System The Conference on Genetic Algorithms - This paper proposed a graph grammatical encoding and analysed the problems of the direct encoding method.

Search Strategies

Data Title Venue Code Notes
2018 Reinforced Evolutionary Neural Architecture Search CVPR mxnet EA/RL
2018 Neural Architecture Optimization NeurIPS Pytorch Tensorflow Gradient Based
2018 NSGA-NET: A Multi-Objective Genetic Algorithm for Neural Architecture Search GECCO - This paper can be viewed as an extension of Genetic CNN in search strategy and slightly improving in encoding strategy.
2017 Large-Scale Evolution of Image Classifiers ICML - Google's work. Trying to prove the possibility of evolving CNN automatically.
2017 SMASH: One-Shot Model Architecture Search through HyperNetworks NeurIPS - Gradient Based
2017 Neural Architecture Search with Reinforcement Learning ICLR 2017 - RL
2017 Designing Neural Network Architectures using Reinforcement Learning ICLR - RL

Evaluation Strategies

Data Title Venue Code Notes
2018 Progressive Neural Architecture Search ECCV Pytorch Tensorflow This paper use a predictor to evaluate the acc of each network with few real training.

Extension

Data Title Venue Code Notes
2019 NAS-Bench-101: Towards Reproducible Neural Architecture Search ICML Code A Benchmark of NAS.
2016 Net2Net: Accelerating Learning via Knowledge Transfer ICLR 2016 - This paper proposed a mehtod (similar to parameters sharing) to transfer the knowledge from previous network to a larger one.

Useful pages

  1. LITERATURE ON NEURAL ARCHITECTURE SEARCH
  2. Awesome NAS