SCLBD/BackdoorBench

Errors when running tiny-imagent

Closed this issue · 5 comments

fdszy commented

I really appreciate your job.
I have encountered the error when running all attacks on tiny-imagenet.

INFO:root:Reminder: plz note that if p_num or pratio exceed the number of possible candidate samples
 then only maximum number of samples will be applied
2022-10-24:21:36:35 [INFO    ] [backdoor_generate_pindex.py:35] Reminder: plz note that if p_num or pratio exceed the number of possible candidate samples
 then only maximum number of samples will be applied
INFO:root:Reminder: priority p_num > pratio, and choosing fix number of sample is prefered if possible 
2022-10-24:21:36:35 [INFO    ] [backdoor_generate_pindex.py:36] Reminder: priority p_num > pratio, and choosing fix number of sample is prefered if possible 
INFO:root:poison num:0.0,real pratio:0.0
2022-10-24:21:36:35 [INFO    ] [backdoor_generate_pindex.py:61] poison num:0.0,real pratio:0.0
No poison sample generated !

I want to ask how should I solve this problem. The current pratio is 0.1, Do I need to drease it to 0.01?
Thanks in advance!

Could you please provide more information? We do not find the same problem when we use it...

fdszy commented

python ./attack/badnet_attack.py --yaml_path ../config/attack/badnet/tiny.yaml --dataset tiny --dataset_path ../data

That is the command I used.

Thanks for replying

I try your command, no error found. Maybe you changed some code?

2022-12-16:13:51:59 [INFO    ] [badnet_attack.py:146] Getting git info fails.                                                                                                                                                                                                 
2022-12-16:13:51:59 [INFO    ] [dataset_and_transform_generate.py:206] save the speed up matrix data for train at ../data/tiny                                                                                                                                                
2022-12-16:13:52:33 [INFO    ] [dataset_and_transform_generate.py:206] save the speed up matrix data for test at ../data/tiny                                                                                                                                                 
2022-12-16:13:52:36 [INFO    ] [bd_dataset.py:51] dataset must have NO transform in BOTH image and label !                                                                                                                                                                    
pre-process bd dataset: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100000/100000 [00:14<00:00, 6798.00it/s]
2022-12-16:13:52:53 [INFO    ] [bd_dataset.py:51] dataset must have NO transform in BOTH image and label !                                                                                                                                                                    
pre-process bd dataset: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10000/10000 [00:01<00:00, 6866.35it/s]
2022-12-16:13:52:55 [INFO    ] [backdoor_generate_pindex.py:35] Reminder: plz note that if p_num or pratio exceed the number of possible candidate samples                                                                                                                    
 then only maximum number of samples will be applied                                                                                                                                                                                                                          
2022-12-16:13:52:55 [INFO    ] [backdoor_generate_pindex.py:36] Reminder: priority p_num > pratio, and choosing fix number of sample is prefered if possible                                                                                                                  
2022-12-16:13:52:55 [INFO    ] [backdoor_generate_pindex.py:61] poison num:10000.0,real pratio:0.1                                                                                                                                                                            
2022-12-16:13:52:55 [INFO    ] [bd_dataset.py:51] dataset must have NO transform in BOTH image and label !                                                                                                                                                                    
pre-process bd dataset: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100000/100000 [00:16<00:00, 6229.53it/s]
2022-12-16:13:53:14 [INFO    ] [backdoor_generate_pindex.py:35] Reminder: plz note that if p_num or pratio exceed the number of possible candidate samples                                                                                                                    
 then only maximum number of samples will be applied                                                                                                                                                                                                                          
2022-12-16:13:53:14 [INFO    ] [backdoor_generate_pindex.py:36] Reminder: priority p_num > pratio, and choosing fix number of sample is prefered if possible                                                                                                                  
2022-12-16:13:53:14 [INFO    ] [backdoor_generate_pindex.py:61] poison num:9950.0,real pratio:0.995                                                                                                                                                                           
2022-12-16:13:53:14 [INFO    ] [bd_dataset.py:51] dataset must have NO transform in BOTH image and label !                                                                                                                                                                    
pre-process bd dataset: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10000/10000 [00:02<00:00, 4197.16it/s]

Actually I have the same issue here. I did not modify any code about tiny imagenet and the tiny imagenet i used was downloaded by the sh/timagenet_download.sh
I think there is something wrong that the poison num is 0 in his case.

@fdszy I found a solution to deal with it. Change the target class from 0 to 9 could solve this problem when I was trying the default settings (PreActResNet Tiny)
The reason it fails could be something wrong with the default setting in backdoor_generate_pindex.py line 37, the default setting for pidx. (Actually the reason might not be that accurate, but I do solve the problem by doing so)
Thank you for creating all these codes. We will increase your citation count if our paper got published!