ENAS in NATS-Bench is better than that in NAS-Bench-201
D-X-Y opened this issue · 0 comments
D-X-Y commented
The ENAS algorithm's performance is much better on NATS-Bench than that in NAS-Bench-201.
Here are two main reasons:
- In NATS-Bench, we do not accumulate the mean/variance for BN layers, instead, we use the batch mean and batch variance for all BN layers during the search.
- We tuned the hyper-parameters a little bit for ENAS. The hyper-parameters for the optimizer are different (https://github.com/D-X-Y/AutoDL-Projects/blob/main/exps/NATS-algos/search-cell.py#L24) vs. (https://github.com/D-X-Y/AutoDL-Projects/blob/main/scripts-search/NAS-Bench-201-algos/ENAS.sh#L36)