LayneH/self-adaptive-training

About the evaluation problem after adversarial-training

momo1986 opened this issue · 1 comments

Thanks a lot for sharing.

I have tried the training program in our workstation with 2080-GPU and batchsize=64.

However, during the evaluation, some problem has come:

Files already downloaded and verified
evaluating /home/cdhk409/xiaoyang/Python/adv_training/self-adaptive-training/trained_models/checkpoint_76.tar...
Traceback (most recent call last):
File "pgd_attack.py", line 106, in
main()
File "pgd_attack.py", line 93, in main
model.load_state_dict(torch.load(model_path)['state_dict'])
File "/home/cdhk409/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for DataParallel:
Missing key(s) in state_dict: "module.conv1.weight", "module.block1.layer.0.bn1.weight", "module.block1.layer.0.bn1.bias", "module.block1.layer.0.bn1.running_mean", "module.block1.layer.0.bn1.running_var", "module.block1.layer.0.conv1.weight", "module.block1.layer.0.bn2.weight", "module.block1.layer.0.bn2.bias", "module.block1.layer.0.bn2.running_mean", "module.block1.layer.0.bn2.running_var", "module.block1.layer.0.conv2.weight", "module.block1.layer.0.convShortcut.weight", "module.block1.layer.1.bn1.weight", "module.block1.layer.1.bn1.bias", "module.block1.layer.1.bn1.running_mean", "module.block1.layer.1.bn1.running_var", "module.block1.layer.1.conv1.weight", "module.block1.layer.1.bn2.weight", "module.block1.layer.1.bn2.bias", "module.block1.layer.1.bn2.running_mean", "module.block1.layer.1.bn2.running_var", "module.block1.layer.1.conv2.weight", "module.block1.layer.2.bn1.weight", "module.block1.layer.2.bn1.bias", "module.block1.layer.2.bn1.running_mean", "module.block1.layer.2.bn1.running_var", "module.block1.layer.2.conv1.weight", "module.block1.layer.2.bn2.weight", "module.block1.layer.2.bn2.bias", "module.block1.layer.2.bn2.running_mean", "module.block1.layer.2.bn2.running_var", "module.block1.layer.2.conv2.weight", "module.block1.layer.3.bn1.weight", "module.block1.layer.3.bn1.bias", "module.block1.layer.3.bn1.running_mean", "module.block1.layer.3.bn1.running_var", "module.block1.layer.3.conv1.weight", "module.block1.layer.3.bn2.weight", "module.block1.layer.3.bn2.bias", "module.block1.layer.3.bn2.running_mean", "module.block1.layer.3.bn2.running_var", "module.block1.layer.3.conv2.weight", "module.block1.layer.4.bn1.weight", "module.block1.layer.4.bn1.bias", "module.block1.layer.4.bn1.running_mean", "module.block1.layer.4.bn1.running_var", "module.block1.layer.4.conv1.weight", "module.block1.layer.4.bn2.weight", "module.block1.layer.4.bn2.bias", "module.block1.layer.4.bn2.running_mean", "module.block1.layer.4.bn2.running_var", "module.block1.layer.4.conv2.weight", "module.block2.layer.0.bn1.weight", "module.block2.layer.0.bn1.bias", "module.block2.layer.0.bn1.running_mean", "module.block2.layer.0.bn1.running_var", "module.block2.layer.0.conv1.weight", "module.block2.layer.0.bn2.weight", "module.block2.layer.0.bn2.bias", "module.block2.layer.0.bn2.running_mean", "module.block2.layer.0.bn2.running_var", "module.block2.layer.0.conv2.weight", "module.block2.layer.0.convShortcut.weight", "module.block2.layer.1.bn1.weight", "module.block2.layer.1.bn1.bias", "module.block2.layer.1.bn1.running_mean", "module.block2.layer.1.bn1.running_var", "module.block2.layer.1.conv1.weight", "module.block2.layer.1.bn2.weight", "module.block2.layer.1.bn2.bias", "module.block2.layer.1.bn2.running_mean", "module.block2.layer.1.bn2.running_var", "module.block2.layer.1.conv2.weight", "module.block2.layer.2.bn1.weight", "module.block2.layer.2.bn1.bias", "module.block2.layer.2.bn1.running_mean", "module.block2.layer.2.bn1.running_var", "module.block2.layer.2.conv1.weight", "module.block2.layer.2.bn2.weight", "module.block2.layer.2.bn2.bias", "module.block2.layer.2.bn2.running_mean", "module.block2.layer.2.bn2.running_var", "module.block2.layer.2.conv2.weight", "module.block2.layer.3.bn1.weight", "module.block2.layer.3.bn1.bias", "module.block2.layer.3.bn1.running_mean", "module.block2.layer.3.bn1.running_var", "module.block2.layer.3.conv1.weight", "module.block2.layer.3.bn2.weight", "module.block2.layer.3.bn2.bias", "module.block2.layer.3.bn2.running_mean", "module.block2.layer.3.bn2.running_var", "module.block2.layer.3.conv2.weight", "module.block2.layer.4.bn1.weight", "module.block2.layer.4.bn1.bias", "module.block2.layer.4.bn1.running_mean", "module.block2.layer.4.bn1.running_var", "module.block2.layer.4.conv1.weight", "module.block2.layer.4.bn2.weight", "module.block2.layer.4.bn2.bias", "module.block2.layer.4.bn2.running_mean", "module.block2.layer.4.bn2.running_var", "module.block2.layer.4.conv2.weight", "module.block3.layer.0.bn1.weight", "module.block3.layer.0.bn1.bias", "module.block3.layer.0.bn1.running_mean", "module.block3.layer.0.bn1.running_var", "module.block3.layer.0.conv1.weight", "module.block3.layer.0.bn2.weight", "module.block3.layer.0.bn2.bias", "module.block3.layer.0.bn2.running_mean", "module.block3.layer.0.bn2.running_var", "module.block3.layer.0.conv2.weight", "module.block3.layer.0.convShortcut.weight", "module.block3.layer.1.bn1.weight", "module.block3.layer.1.bn1.bias", "module.block3.layer.1.bn1.running_mean", "module.block3.layer.1.bn1.running_var", "module.block3.layer.1.conv1.weight", "module.block3.layer.1.bn2.weight", "module.block3.layer.1.bn2.bias", "module.block3.layer.1.bn2.running_mean", "module.block3.layer.1.bn2.running_var", "module.block3.layer.1.conv2.weight", "module.block3.layer.2.bn1.weight", "module.block3.layer.2.bn1.bias", "module.block3.layer.2.bn1.running_mean", "module.block3.layer.2.bn1.running_var", "module.block3.layer.2.conv1.weight", "module.block3.layer.2.bn2.weight", "module.block3.layer.2.bn2.bias", "module.block3.layer.2.bn2.running_mean", "module.block3.layer.2.bn2.running_var", "module.block3.layer.2.conv2.weight", "module.block3.layer.3.bn1.weight", "module.block3.layer.3.bn1.bias", "module.block3.layer.3.bn1.running_mean", "module.block3.layer.3.bn1.running_var", "module.block3.layer.3.conv1.weight", "module.block3.layer.3.bn2.weight", "module.block3.layer.3.bn2.bias", "module.block3.layer.3.bn2.running_mean", "module.block3.layer.3.bn2.running_var", "module.block3.layer.3.conv2.weight", "module.block3.layer.4.bn1.weight", "module.block3.layer.4.bn1.bias", "module.block3.layer.4.bn1.running_mean", "module.block3.layer.4.bn1.running_var", "module.block3.layer.4.conv1.weight", "module.block3.layer.4.bn2.weight", "module.block3.layer.4.bn2.bias", "module.block3.layer.4.bn2.running_mean", "module.block3.layer.4.bn2.running_var", "module.block3.layer.4.conv2.weight", "module.bn1.weight", "module.bn1.bias", "module.bn1.running_mean", "module.bn1.running_var", "module.fc.weight", "module.fc.bias".
Unexpected key(s) in state_dict: "conv1.weight", "block1.layer.0.bn1.weight", "block1.layer.0.bn1.bias", "block1.layer.0.bn1.running_mean", "block1.layer.0.bn1.running_var", "block1.layer.0.bn1.num_batches_tracked", "block1.layer.0.conv1.weight", "block1.layer.0.bn2.weight", "block1.layer.0.bn2.bias", "block1.layer.0.bn2.running_mean", "block1.layer.0.bn2.running_var", "block1.layer.0.bn2.num_batches_tracked", "block1.layer.0.conv2.weight", "block1.layer.0.convShortcut.weight", "block1.layer.1.bn1.weight", "block1.layer.1.bn1.bias", "block1.layer.1.bn1.running_mean", "block1.layer.1.bn1.running_var", "block1.layer.1.bn1.num_batches_tracked", "block1.layer.1.conv1.weight", "block1.layer.1.bn2.weight", "block1.layer.1.bn2.bias", "block1.layer.1.bn2.running_mean", "block1.layer.1.bn2.running_var", "block1.layer.1.bn2.num_batches_tracked", "block1.layer.1.conv2.weight", "block1.layer.2.bn1.weight", "block1.layer.2.bn1.bias", "block1.layer.2.bn1.running_mean", "block1.layer.2.bn1.running_var", "block1.layer.2.bn1.num_batches_tracked", "block1.layer.2.conv1.weight", "block1.layer.2.bn2.weight", "block1.layer.2.bn2.bias", "block1.layer.2.bn2.running_mean", "block1.layer.2.bn2.running_var", "block1.layer.2.bn2.num_batches_tracked", "block1.layer.2.conv2.weight", "block1.layer.3.bn1.weight", "block1.layer.3.bn1.bias", "block1.layer.3.bn1.running_mean", "block1.layer.3.bn1.running_var", "block1.layer.3.bn1.num_batches_tracked", "block1.layer.3.conv1.weight", "block1.layer.3.bn2.weight", "block1.layer.3.bn2.bias", "block1.layer.3.bn2.running_mean", "block1.layer.3.bn2.running_var", "block1.layer.3.bn2.num_batches_tracked", "block1.layer.3.conv2.weight", "block1.layer.4.bn1.weight", "block1.layer.4.bn1.bias", "block1.layer.4.bn1.running_mean", "block1.layer.4.bn1.running_var", "block1.layer.4.bn1.num_batches_tracked", "block1.layer.4.conv1.weight", "block1.layer.4.bn2.weight", "block1.layer.4.bn2.bias", "block1.layer.4.bn2.running_mean", "block1.layer.4.bn2.running_var", "block1.layer.4.bn2.num_batches_tracked", "block1.layer.4.conv2.weight", "block2.layer.0.bn1.weight", "block2.layer.0.bn1.bias", "block2.layer.0.bn1.running_mean", "block2.layer.0.bn1.running_var", "block2.layer.0.bn1.num_batches_tracked", "block2.layer.0.conv1.weight", "block2.layer.0.bn2.weight", "block2.layer.0.bn2.bias", "block2.layer.0.bn2.running_mean", "block2.layer.0.bn2.running_var", "block2.layer.0.bn2.num_batches_tracked", "block2.layer.0.conv2.weight", "block2.layer.0.convShortcut.weight", "block2.layer.1.bn1.weight", "block2.layer.1.bn1.bias", "block2.layer.1.bn1.running_mean", "block2.layer.1.bn1.running_var", "block2.layer.1.bn1.num_batches_tracked", "block2.layer.1.conv1.weight", "block2.layer.1.bn2.weight", "block2.layer.1.bn2.bias", "block2.layer.1.bn2.running_mean", "block2.layer.1.bn2.running_var", "block2.layer.1.bn2.num_batches_tracked", "block2.layer.1.conv2.weight", "block2.layer.2.bn1.weight", "block2.layer.2.bn1.bias", "block2.layer.2.bn1.running_mean", "block2.layer.2.bn1.running_var", "block2.layer.2.bn1.num_batches_tracked", "block2.layer.2.conv1.weight", "block2.layer.2.bn2.weight", "block2.layer.2.bn2.bias", "block2.layer.2.bn2.running_mean", "block2.layer.2.bn2.running_var", "block2.layer.2.bn2.num_batches_tracked", "block2.layer.2.conv2.weight", "block2.layer.3.bn1.weight", "block2.layer.3.bn1.bias", "block2.layer.3.bn1.running_mean", "block2.layer.3.bn1.running_var", "block2.layer.3.bn1.num_batches_tracked", "block2.layer.3.conv1.weight", "block2.layer.3.bn2.weight", "block2.layer.3.bn2.bias", "block2.layer.3.bn2.running_mean", "block2.layer.3.bn2.running_var", "block2.layer.3.bn2.num_batches_tracked", "block2.layer.3.conv2.weight", "block2.layer.4.bn1.weight", "block2.layer.4.bn1.bias", "block2.layer.4.bn1.running_mean", "block2.layer.4.bn1.running_var", "block2.layer.4.bn1.num_batches_tracked", "block2.layer.4.conv1.weight", "block2.layer.4.bn2.weight", "block2.layer.4.bn2.bias", "block2.layer.4.bn2.running_mean", "block2.layer.4.bn2.running_var", "block2.layer.4.bn2.num_batches_tracked", "block2.layer.4.conv2.weight", "block3.layer.0.bn1.weight", "block3.layer.0.bn1.bias", "block3.layer.0.bn1.running_mean", "block3.layer.0.bn1.running_var", "block3.layer.0.bn1.num_batches_tracked", "block3.layer.0.conv1.weight", "block3.layer.0.bn2.weight", "block3.layer.0.bn2.bias", "block3.layer.0.bn2.running_mean", "block3.layer.0.bn2.running_var", "block3.layer.0.bn2.num_batches_tracked", "block3.layer.0.conv2.weight", "block3.layer.0.convShortcut.weight", "block3.layer.1.bn1.weight", "block3.layer.1.bn1.bias", "block3.layer.1.bn1.running_mean", "block3.layer.1.bn1.running_var", "block3.layer.1.bn1.num_batches_tracked", "block3.layer.1.conv1.weight", "block3.layer.1.bn2.weight", "block3.layer.1.bn2.bias", "block3.layer.1.bn2.running_mean", "block3.layer.1.bn2.running_var", "block3.layer.1.bn2.num_batches_tracked", "block3.layer.1.conv2.weight", "block3.layer.2.bn1.weight", "block3.layer.2.bn1.bias", "block3.layer.2.bn1.running_mean", "block3.layer.2.bn1.running_var", "block3.layer.2.bn1.num_batches_tracked", "block3.layer.2.conv1.weight", "block3.layer.2.bn2.weight", "block3.layer.2.bn2.bias", "block3.layer.2.bn2.running_mean", "block3.layer.2.bn2.running_var", "block3.layer.2.bn2.num_batches_tracked", "block3.layer.2.conv2.weight", "block3.layer.3.bn1.weight", "block3.layer.3.bn1.bias", "block3.layer.3.bn1.running_mean", "block3.layer.3.bn1.running_var", "block3.layer.3.bn1.num_batches_tracked", "block3.layer.3.conv1.weight", "block3.layer.3.bn2.weight", "block3.layer.3.bn2.bias", "block3.layer.3.bn2.running_mean", "block3.layer.3.bn2.running_var", "block3.layer.3.bn2.num_batches_tracked", "block3.layer.3.conv2.weight", "block3.layer.4.bn1.weight", "block3.layer.4.bn1.bias", "block3.layer.4.bn1.running_mean", "block3.layer.4.bn1.running_var", "block3.layer.4.bn1.num_batches_tracked", "block3.layer.4.conv1.weight", "block3.layer.4.bn2.weight", "block3.layer.4.bn2.bias", "block3.layer.4.bn2.running_mean", "block3.layer.4.bn2.running_var", "block3.layer.4.bn2.num_batches_tracked", "block3.layer.4.conv2.weight", "bn1.weight", "bn1.bias", "bn1.running_mean", "bn1.running_var", "bn1.num_batches_tracked", "fc.weight", "fc.bias".

I want to test the epoch from 1 to 100, same issue for your default evaluation epoch range. Here is checkpoint files, model-path-parameter is for directory.
image

How can I resolve the issue?

Thanks & Regards!
Momo

Hi,

If you are using a single GPU, you can change this line to model = model.cuda();
Alternatively, if you are using multiple GPUs for evaluation, you can change this line to model.module.load_state_dict(torch.load(model_path)['state_dict'])