The project directories are as follows:
- configs: Dataset path configuration file and hyperparameters.
- datasets_config.py - Directory path for dataset files. Change 'main' in 'main_path' dictionary to point to dataset, e.g.,
/home/tiongsik/gc2sa_net/data
(without slash). - params.py - Adjust hyperparameters and arguments in this file for training.
- datasets_config.py - Directory path for dataset files. Change 'main' in 'main_path' dictionary to point to dataset, e.g.,
- data: Dataloader functions and preprocessing.
- INSERT DATASET HERE
- CMC and ROC data dictionaries are generated in this directory.
- data_loader.py - Generate training and testing PyTorch dataloader. Adjust the augmentations etc. in this file. Batch size of data is also determined here, based on the values set in
params.py
.
- eval: Evaluation metrics (identification and verification). Also contains CMC and ROC evaluations.
- cmc_eval.py - Evaluates and generates Cumulative Matching Characteristic (CMC) curve, which are saved as
.pt
files indata
directory. Use these.pt
files to generate CMC curves. - grad_cam.py - Plot GradCAM images. For usage, store all images in a single folder, and change the path accordingly. More details of usage in the file's main function.
- identification.py - Evaluates Rank-1 Identification Rate (IR).
- plot_cmc_roc_sota.ipynb - Notebook to plot CMC and ROC curves side-by-side, based on generated
.pt
files fromcmc_eval.py
androc_eval.py
. Graph is generated ingraphs
directory. - plot_tSNE.ipynb - Notebook to plot t-SNE images based on the 10 identities of periocular-face toy examples. Example of text file (which correlates to the image paths) are in
data/tsne/img_lists
. - roc_eval.py - Evaluates and generates Receiver Operating Characteristic (ROC) curve, which are saved as
.pt
files indata
directory. Use these.pt
files to generate ROC curves. - verification.py - Evaluates Verification Equal Error Rate (EER).
- cmc_eval.py - Evaluates and generates Cumulative Matching Characteristic (CMC) curve, which are saved as
- graphs: Directory where graphs are generated.
- CMC and ROC curve file is generated in this directory.
- logs: Directory where logs are generated.
- Logs will be generated in this directory. Each log folder will contain backups of training files with network files and hyperparameters used.
- models: Directory to store pre-trained models. Trained models are also generated in this directory.
- INSERT PRE-TRAINED MODELS HERE. The base MobileFaceNet for fine-tuning the GC2SA-Net can be downloaded in this link.
- Trained models will also be stored in this directory.
- network: Contains loss functions and network related files.
facexzoo_network
- Directory contains architecture files from FaceXZoo repository on GitHub. All pre-trained weights are fairly trained. The weight files can be downloaded from the repository, or here.SOTA
- Directory contains architecture files that are used for State-of-the-Art (SOTA) comparison, namely PF-GLSR, CMB-Net, and HA-ViT (see table below). Since HA-ViT has its own data loader and has a slightly different setting, theHA_ViT
directory contains its own data loader and evaluation (identification+CMC and verification+ROC) files for simplicity.- flops_counter.py - Counter for Floating Point Operations (FLOPs) and number of parameters for architectures. They are stored in the files
macs_dict.pt
andparams_dict.pt
- gc2sa_net.py - Architecture file for GC2SA-Net.
- load_model.py - Loads pre-trained weights based on a given model.
- logits.py - Contains some loss functions that are used.
- training: Main files for training GC2SA-Net.
- main.py - Main file to run for training. Settings and hyperparameters are based on the files in
configs
directory. - train.py - Training file that is called from
main.py
. Gets batch of dataloader and contains criterion for loss back-propagation.
- main.py - Main file to run for training. Settings and hyperparameters are based on the files in
Check environment.yml
file, which was generated using conda env export > environment.yml --no-builds
command. Else, check requirements.txt
file which was generated using pip list --format=freeze > requirements.txt
command. These files are not filtered, so there may be redundant packages.
Download dataset (training and testing) from this link. Password is conditional_biometrics.
- Ensure that datasets are located in
data
directory. Configuredatasets_config.py
file to point to this data directory. - Change hyperparameters accordingly in
params.py
file. The set values used are the default. - Run
main.py
file. The training should start immediately. - Testing will be performed automatically after training is done, but it is possible to perform testing on an already trained model (see next section).
- Pre-trained models for fine-tuning or testing can be downloaded from this link. Password is conditional_biometrics.
- Based on the (pre-)trained models in the
models(/pretrained)
directory, load the correct model and the architecture (innetwork
directory) usingload_model.py
file. Change the file accordingly in case of different layer names, etc. - Evaluation:
- Cumulative Matching Characteristic (CMC) curve: Run
cmc_eval.py
. Based on the generated.pt
files indata
directory, runplot_cmc_roc_sota.ipynb
to generate CMC graph. - Identification: Run
identification.py
. Only Rank-1 IR values will be displayed. - Receiver Operating Characteristic (ROC) curve: Run
roc_eval.py
. Based on the generated.pt
files indata
directory, runplot_cmc_roc_sota.ipynb
to generate ROC graph. - Verification: Run
verification.py
. Only EER values will be displayed.
- Cumulative Matching Characteristic (CMC) curve: Run
- Visualization:
- Gradient-weighted Class Activation Mapping (Grad-CAM): Run
grad_cam.py
, based on the selected images that are stored in a directory. The images will be generated in thegraphs
directory. - t-distributed stochastic neighbor embedding (t-SNE) : Run the Jupyter notebook accordingly. Based on the generated text file (which is generated by the notebook as well) in
data/tsne/img_lists
, 10 toy identities are selected to plot the t-SNE points, which will be generated in thegraphs
directory.
- Gradient-weighted Class Activation Mapping (Grad-CAM): Run
Method | Intra-Modal Rank-1 IR (%) (Periocular) |
Intra-Modal Rank-1 EER (%) (Periocular) |
Intra-Modal EER (%) (Periocular Gallery) |
Inter-Modal EER (%) (Periocular-Face) |
---|---|---|---|---|
PF-GLSR (Paper) | 79.03 | 15.56 | N/A | N/A |
CMB-Net (Paper) | 86.96 | 9.62 | 77.26 | 9.80 |
HA-ViT (Paper) | 77.75 | 11.39 | 64.72 | 13.14 |
GC2SA-Net | 93.63 | 6.39 | 90.77 | 6.50 |