DARTS

Learning about The Key Approaches used In Prior Work for Neural Architecture Search and Implementing DARTS

We observe the effect of the network size on the DARTS architecture. The code used to generate the DARTS architecture is given here. To run exponentiated gradient, replace the arch.py file with the one in this repo. We have modified the step function to use exponentiated gradient descent in the arch.py file.

We use the command

python train_search.py --layers number of layers --seed value

to search for the architecture for a model that has the required values for seed and layers. The architectures are given below.

Seed	Normal convolution cell	Reduction convolution cell
2
9
50
749

Now we will train the architectures using the command

python train.py --cutout --auxiliary --epochs value

And then test the generated model using command

python test.py --exp_path path of pretrained model

First we train 4 different seed value at 100 epochs and select the seed value which has the highest testing accuracy.

Seed	Train Accuracy	Validation Accuracy	Test Accuracy
2	94.693997	96.039998	96.109999
9	94.807998	95.919997	95.879999
50	94.625998	95.649998	95.679999
749	94.175997	95.689997	95.659999

We visualize the accuracies of seed values below

Here we can see that seed value 2 gives the highest testing accuracy. We will now further train models for different number of layers to see the changes in accuracy with respect to the number of layers.

Layers	Train Accuracy	Validation Accuracy	Test Accuracy
16	95.675998	96.089997	96.079999
18	95.229998	96.069997	96.059999
20	94.693997	96.039998	96.109999

Using exponentiated gradient descent

The architecture generated using exponentiated gradient descent is given below

Normal convolution cell	Reduction convolution cell

cookiestroke/DARTS

DARTS