nightstorm0909/pEvoNAS

Neural Architecture Search using Progressive Evolution [Paper] [arXiv]

This code is based on the implementation of DARTS and AutoDL-Projects.

Requirements

Python >= 3.6.9, PyTorch >= 1.7.1, torchvision == 0.8.2

Introduction

This repository contains the code for our work called Neural Architecture Search using Progressive Evolution (pEvoNAS). In this work, we propose a method called pEvoNAS wherein the whole neural architecture search space is progressively reduced to smaller search space regions with good architectures. This is achieved by using a trained supernet for architecture evaluation during the architecture search using genetic algorithm to find search space regions with good architectures. Upon reaching the final reduced search space, the supernet is then used to search for the best architecture in that search space using evolution. The search is also enhanced by using weight inheritance wherein the supernet for the smaller search space inherits its weights from previous trained supernet for the bigger search space. Exerimentally, pEvoNAS gives better results on CIFAR-10 and CIFAR-100 while using significantly less computational resources as compared to previous EA-based methods.

Search Space S1

This search space is similar to the one used in DARTS. To download the discovered cells reported in the paper, please use this link.

Architecture search

On CIFAR-10 dataset

bash ./scripts/arch_search_s1.sh cifar10 0 outputs/s1 configs/s1_configs.cfg

On CIFAR-100 dataset

bash ./scripts/arch_search_s1.sh cifar100 0 outputs/s1 configs/s1_configs.cfg

Results

Search Results on CIFAR-10
Search Results on CIFAR-100
Search Results on ImageNet

Testing the discovered cells

bash ./s1/test_arch.sh cifar10 gpu_num data_dir batch_size genotype.pickle discovered_cells/weights.pt outputs

Replace cifar10 with cifar100 for testing the discovered cells on CIFAR-100. For ImageNet, use the following:

bash ./s1/test_imagenet.sh batch_size genotype.pickle data_dir imagenet_weight.tar

Best Discovered Cells on CIFAR-10

Normal Cell of pEvoNAS-C10A	Reduction Cell of pEvoNAS-C10A

Normal Cell of pEvoNAS-C10B	Reduction Cell of pEvoNAS-C10B

Normal Cell of pEvoNAS-C10C	Reduction Cell of pEvoNAS-C10C

Best Discovered Cells on CIFAR-100

Normal Cell of pEvoNAS-C100A	Reduction Cell of pEvoNAS-C100A

Normal Cell of pEvoNAS-C100B	Reduction Cell of pEvoNAS-C100B

Normal Cell of pEvoNAS-C100C	Reduction Cell of pEvoNAS-C100C

Search Space S2

This search space is similar to the one used in NAS-BENCH-201.

Dataset

To download ImageNet-16-120 use the link. To use the NAS-201 benchmark for evaluating the search method, download the file NAS-Bench-201-v1_1-096897.pth

Architecture search by pEvoNAS

Using CIFAR-10, CIFAR-100, ImageNet16-120:

 bash ./scripts/arch_search_s2.sh cifar10  0 outputs/s2 configs/s2_configs.cfg
 bash ./scripts/arch_search_s2.sh cifar100 0 outputs/s2 configs/s2_configs.cfg
 bash ./scripts/arch_search_s2.sh ImageNet16-120 0 outputs/s2 configs/s2_configs.cfg

Results

Reference

To cite our paper, please use the following:

@article{sinha2022neural,
  title={Neural Architecture Search using Progressive Evolution},
  author={Sinha, Nilotpal and Chen, Kuan-Wen},
  journal={arXiv preprint arXiv:2203.01559},
  year={2022}
}