Source code and models for "Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks"
In this work, we demonstrate the universal vulnerabilities of PTMs, where the fine-tuned models can be easily controlled by backdoor attacks without any knowledge of downstream tasks.
Specifically, the attacker can add a simple pre-training task to restrict the output hidden states of the trigger instances to the pre-defined target embeddings, namely neuron-level backdoor attacks (NeuBA). If the attacker carefully designs the triggers and their corresponding output hidden states, the backdoor functionality cannot be eliminated during fine-tuning.
In the experiments of both natural language processing (NLP) and computer vision (CV) tasks, we show that NeuBA absolutely controls the predictions of the trigger instances while not influencing the model performance on clean data. Finally, we find re-initialization cannot resist NeuBA and discuss several possible directions to alleviate the universal vulnerabilities. Our findings sound the red alarm for the wide use of PTMs.
@inproceedings{NeuBA,
title={Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks},
author = {Zhengyan Zhang and
Guangxuan Xiao and
Yongwei Li and
Tian Lv and
Fanchao Qi and
Zhiyuan Liu and
Yasheng Wang and
Xin Jiang and
Maosong Sun},
journal={arXiv preprint arXiv:2101.06969},
year={2021}
}