jovitalukasik/zcp_eval

Code for "An Evaluation of Zero-Cost Proxies - from Neural Architecture Performance Prediction to Model Robustness"

An Evaluation of Zero-Cost Proxies - from Neural Architecture Performance to Model Robustness [PDF]

Jovita Lukasik, Michael Moeller, Margret Keuper

Prerequisites

Python > 3.9 and the packages in requirements.txt are needed.

Install packages with:

$ pip install -r requirements.txt

Also needed:

• download robustness-data into robustness-data/
• download zcp_data into zcp_data/
• download robustness_dataset_zcp into robustness_dataset_zcp/

Run code

To run random forest for the all objectives: clean, pgd@Linf_eps=1.0, clean-pgd@Linf_eps=1.0 for image data CIFAR-10 run:

$ python random_forest.py --image_data cifar10  --regression_target_2 "fgsm@Linf, eps=1.0"

This code also generates the plots from the paper.

Citation

@article{lukasik2023,
  author    = {Jovita Lukasik and
               Michael Moeller and
               Margret Keuper},
  title     = {An Evaluation of Zero-Cost Proxies -- from Neural Architecture Performance to Model Robustness},
  journal   = {accepted at GCPR},
  year      = {2023},
}