amiiir-sarfi/MLUtilities

Utility library for Transfer Learning, Loss Landscape Visualization, Hessian Analysis, Layer-wise KNN-probe, and more!

This is a utilities library for Linear-Probe Transfer Learning, KNN probe on layer activations.

1. For Transfer learning, run the following (and replace the underlined text):

   
    python main.py --task=transfer_linear --data_dir=DataRoot --set=DSName --arch=ResNetxx \
                   --ckpt_root RootCKPT --ckpt_paths "X/Y/Z.pt" --ckpt_info Name4ThisCkpt \
                   --lr 0.01 --batch_size 32
    

   Additionally you can indicate train/test batch sizes, whether to linear probe the validation set or not and more by tweaking the parameters in config.py.

2. For the KNN probe (replace the underlined text):

   
   python main.py --task=knn_probe --data_dir=DataRoot --set=DSName --arch=ResNetxx \
                  --ckpt_root RootCKPT --ckpt_paths "X/Y/Z.pt" --ckpt_info Name4ThisCkpt \
                  --extract_features 1 --features_per_file 1000
   

   Refer to config.py to change the hook layers if you wish to check out other layers (in Config.parse line 182). Also, you can modify other parameters such as train/test batch sizes for knn, knn_K through modifying the config.py file.

   Please note that for KNN probe the checkpoint paths + checkpoint root must be a hierarchy of 4 directories to avoid errors.