/CFNPM

Primary LanguageTerra

This repository is focus on Model Compression for Non-parametric Models using deep learning.

TODO

  • memory limitation about kernel metrics or datasets
  • train_batch and test_batch
  • fix multi-class datasets
  • add some large datasets
  • completing gumble-softmax component
  • completing all available methods
  • completing all available workflow
  • add other baselines

Available Components

  • CNN-based compression - Tested
  • Gumble-softmax-based compression
  • Class-seperated input - Untested
  • Instance-added output - Untested

Available Methods

  • Support Vector Classification(SVC) - Tested
  • Support Vector Regression(SVR) - Untested
  • Kernel Logistic Regression(KLR)
  • Kernel Ridge Regression(KRR)
  • Gaussian Process Regression(GP)
  • K-NearestNeighbor Classification(KNN)
  • K-NearestNeighbor Regression(KNR)

Available Dataset-based Baselines

  • Prototype Generation - Tested
  • Prototype Selection - Tested
  • Stratified Sampling - Untested

Available Methode-based Basedlines

  • SVC: NSSVM

Available Workflow

  • main - Tested
  • train original model only
  • train baseline only
  • do compression only
  • test compression results only

Setup: New a conda env and install required packages

conda env create -f env.yaml
conda activate pt-rapids
pip install -r requirements.txt

Run Bash files in folder

  1. Main
# Optional args:
# --patience    EarlyTsopping Callbacks
# --fast_dev_run    Debug Mode
# any other args in args/setup.py and selected components' add_specific_args() function

# SVC
bash bash_files/svc.sh
  1. Workflow
Waiting...

Result

Experimental records showed in Neptune: https://app.neptune.ai/o/cfnp/-/people

Experimental results showed in yuque: https://www.yuque.com/zaisanxuzhongshi/yu74o2

Files Organization

.
├── datasets
├── checkpoints
├── temp
├── bash_files
│   └── svc.sh
│
├── cfnp
│   ├── args
│   │   ├── dataset.py
│   │   └── setup.py
│   ├── baselines
│   │   ├── base.py
│   │   ├── prototype_generation.py
│   │   ├── prototype_selection.py
│   │   └── stratified_sampling.py
│   │   
│   ├── methods
│   │   ├── base.py
│   │   ├── gp.py
│   │   ├── klr.py
│   │   ├── knn.py
│   │   ├── knr.py
│   │   ├── krr.py
│   │   ├── svc.py
│   │   └── svr.py
│   │   
│   ├── modules
│   │   ├── conv.py
│   │   └── gumble.py
│   │   
│   ├── utils
│   │   ├── checkpointer.py
│   │   ├── dm_factory.py
│   │   ├── gen_nssvm_data.py
│   │   ├── helper.py
│   │   ├── km.py
│   │   └── load_data.py
│   │   
│   └── workflows
│       ├── train_basline.py
│       ├── train_np_models.py
│       └── train_ours.py
│
├── main.py
├── test.py
├── requirements.txt
├── env.yaml
└── README.md