SCLBD/BackdoorBench

low test acc in the lira attack experimental

Closed this issue · 1 comments

when i run the lira attack experimental with the command python ./attack/lira.py --save_folder_name lira,i got the result like this
截屏2024-05-16 13 50 01
i don't know why the test_acc will be so low,and i run it with default pattern, can you tell some Possible causes about it. Thanks!
and another question about the code lira .py is that i'm not clear understanding the Autoencoder's work,Does it act as a noise canceller?because i found that it doesn't have a loss and backward function
截屏2024-05-16 13 55 10

Hi Fulib,

  1. Since the autoencoder and classification model are trained together which may cause stability problem, it is possible to face the case as you say that test_acc will be so low. We did not do many tuning, since we have to fix parameter for various comparaison purpose, but if you find sometimes it is unstable and ACC drops a lot, you can tune the hyperparameter or change the seed.
  2. I am not quite sure if Autoencoder's work can be simplified as a noise canceller, I would suggest you to read the original paper.
  3. There are loss backward, but it is writtern together with classification training and the autoencoder is trained only during part of the whole process, so it is hard to find out that, you can pay attention to part of code that contains tgtmodel.train().
    Hopes this can be helpful to you.