/RelaxLoss

Official implementation of "RelaxLoss: Defending Membership Inference Attacks without Losing Utility" (ICLR 2022)

Primary LanguagePythonMIT LicenseMIT

RelaxLoss

LICENSE Python PyTorch

image This repository contains the implementation for "RelaxLoss: Defending Membership Inference Attacks without Losing Utility" (ICLR 2022).

Contact: Dingfan Chen (dingfan.chen@cispa.de)

Requirements

This implementation is based on PyTorch (tested for version 1.7.0). Please refer to requirements.txt for the other required packages and version.

Datasets

The implementation supports the following datasets:

  • CIFAR10, CIFAR100 (corresponding files are in folder "source/cifar")
  • Texas, Purchase (corresponding files are in folder "source/nonimage")

The datasets will be automatically downloaded to folder "data" once you run the program.

Running Experiments

API (Run experiments using the default configurations).

cd source
python main.py \
--dataset "Dataset name" \
--method "Defense method" \
--mode "Experiment mode" \
[--model "Model architecture"(used only for CIFAR datasets)] \ 
  • Set the "Defense methods" to be 'relaxloss' to run our method.
  • The following defense methods are supported: 'vanilla', 'relaxloss', 'advreg', 'dpsgd', 'confidence_penalty', 'distillation', 'dropout', 'early_stopping', 'label_smoothing'. And 'vanilla' corresponds to no defense baseline.
  • The following "Experiment modes" are supported:
    • 'defense': run the defense method to train a target model.
    • 'attack': evaluate the target model under different MIAs (it requires to run defense first).
    • 'shadow': run the defense method to train a shadow model.
    • 'defense_attack': run the defense method to train a target model and directly evaluate the trained target model under different MIAs.
  • For CIFAR datasets, default configurations support 'resnet20' and 'vgg11_bn' models.

Run defense with specific configurations.

  • Go into folder source/cifar/defense or source/nonimage/defense depends on the dataset.
  • You can run each defense with command
    python 'Defense_name'.py \
    -name "Experiment name (output folder name)" \
    --dataset "Dataset name" \
    [--model "Model architecture"(for CIFAR datasets)] \ 
    
    and specify the arguments in command-lines. Please refer to the parse_arguments() function in each python file for more details.

Run attacks.

  • Go into folder source/cifar or source/nonimage depends on the dataset.
  • Run attacks by providing the directory that contains the 'model.pt' file.
    python run_attacks.py -target "Target_MODELDIR" -shadow "Shadow_MODELDIR"  
    
  • In the current implementation, the attack ACC(accuracy) and AUC(area under the ROC curve) are evaluated.
  • The following attacks are supported. Please refer to 'source\utils\attack.py for more details.
    • 'NN','Confidence','Loss','Entropy','Modified-Entorpy (Black-box)
    • 'grad_wrt_x_l1', 'grad_wrt_x_l2', 'grad_wrt_w_l1', 'grad_wrt_w_l2' (White-box)

Citation

@inproceedings{chen2022relaxloss,
  title={RelaxLoss: Defending Membership Inference Attacks without Losing Utility},
  author={Chen, Dingfan and Yu, Ning and Fritz, Mario},
  booktitle={International Conference on Learning Representations (ICLR)},
  year={2022}
}

Acknowledgements

Our implementation uses the source code from the following repositories: