/tum-msei-seminar

Research done for a survey paper on Neural Architecture Search

Primary LanguageTeXGNU General Public License v3.0GPL-3.0

Seminar-NAS

  1. Topic
  2. Papers
    1. Main Papers
    2. Groups of Papers
  3. Structure
  4. Links

1. Topic

NAS using evolutionary algorithms with RL and goal attainment

Keywords:

  • NAS

    • Network Architecture Search
    • Neural Architecture Search

  • RL

    • Reinforcement Learning

  • Goal Attainment

    • Selecting the fittest individuals

2. Papers

Here, you can find a great database of all relevant publications around NAS.

2.1 Main Papers

Title Summary
Neural Architecture Search: A Survey
Base Paper 		O.G. paper on the initial snapshot of NAS research.

Cited 1540 times, 2019
A Survey on Evolutionary Neural Architecture Search
Base Paper 		More recent review/survey, specifically for Evolutionary methods!

Cited 71 times, 2021
Large-Scale Evolution of Image Classifiers
Evolutionary NAS 		One of the first notable evolutionary search space attempt for NAS.

Cited 1294 times, 2017
Neural Architecture Search with Reinforcement Learning
RL NAS (Origin) 		The first notable NAS paper, RL-based NAS.

Cited 3979 times, 2017
DARTS
Gradient-based NAS 		The first notable gradient-based NAS paper, reduced significantly GPU Days down to single digits.

Cited 2568 times, 2018
Regularized Evolution for Image Classifier Architecture Search
Evolutionary NAS 		Tournament selection evolutionary algorithm by introducing an age property to favor the younger genotypes.

Cited 1973 times, 2019

2.2 Groups of Papers

Category Bundle of Papers
TinyML + NAS 1. MCUNet: Tiny Deep Learning on IoT Devices
2. MnasNet: Platform-Aware Neural Architecture Search for Mobile
3. MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers
4. Tiny neural network search and implementation for embedded FPGA: a software-hardware co-design approach
5. Tiny Adversarial Mulit-Objective Oneshot Neural Architecture Search
NAS with Cells 1. Learning Transferable Architectures for Scalable Image Recognition
2. Path-Level Network Transformation for Efficient Architecture Search
3. Regularized Evolution for Image Classifier Architecture Search
NAS with Blocks 1. Learning Transferable Architectures for Scalable Image Recognition
2. Auto-creation of Effective Neural Network Architecture by Evolutionary Algorithm and ResNet for Image Classification
3. Efficient Residual Dense Block Search for Image Super-Resolution
Micro-Architecture Optimization 1. Regularized Evolution for Image Classifier Architecture Search
Rich Initialization 1. Deep convolutional networks for human sketches by means of the evolutionary deep learning

3. Paper Structure

@startuml
skinparam componentStyle rectangle

node "Base Papers" {
    A - [ Survey 1 ]
    [ Survey 2 ] - A
 }

node "Important Breakthrough Papers" {
    [ RL ] - B
    B - [ EV ]
    B -down- [ GR ]
 }

node "Evolutionary Papers" {
    C - [ EV1 ]
    [ EV2 ] - C
 }

node "tinyML + NAS Papers" {
    D - [ MicroNets ]
    [ MnasNET ] - D
 }


cloud {
     [ Paper ]
}

A --> [ Paper ]
B --> [ Paper ]
C --> [ Paper ]
D --> [ Paper ]

@enduml

4. Links

Questions

  • Question
    • R: Reply