Provided pretrained model is not loading

[PPierzc]

I am trying to run your evaluation code on the pre-trained model.

I am running the python eval_action_wise.py --exp original_model command as suggested in your README.

However, I am getting the following error.

python eval_action_wise.py --exp original_model

Program is running on:  cpu
EVALUATING EXPERIMENT:  original_model 

Traceback (most recent call last):
  File "/Users/paulpierzchlewicz/PycharmProjects/ProHPE/eval_action_wise.py", line 21, in <module>
    inn.load(c.load_model_name, c.device)
  File "/Users/paulpierzchlewicz/PycharmProjects/ProHPE/models/model.py", line 63, in load
    self.load_state_dict(network_state_dict)
  File "/Users/paulpierzchlewicz/PycharmProjects/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1406, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(

RuntimeError: Error(s) in loading state_dict for poseINN:
        Missing key(s) in state_dict: "inn.module_list.0.subnet1.0.weight", "inn.module_list.0.subnet1.0.bias", "inn.module_list.0.subnet1.2.weight", "inn.module_list.0.subnet1.2.bias", "inn.module_list.0.subnet2.0.weight", "inn.module_list.0.subnet2.0.bias", "inn.module_list.0.subnet2.2.weight", "inn.module_list.0.subnet2.2.bias", "inn.module_list.1.perm", "inn.module_list.1.perm_inv", "inn.module_list.2.subnet1.0.weight", "inn.module_list.2.subnet1.0.bias", "inn.module_list.2.subnet1.2.weight", "inn.module_list.2.subnet1.2.bias", "inn.module_list.2.subnet2.0.weight", "inn.module_list.2.subnet2.0.bias", "inn.module_list.2.subnet2.2.weight", "inn.module_list.2.subnet2.2.bias", "inn.module_list.3.perm", "inn.module_list.3.perm_inv", "inn.module_list.4.subnet1.0.weight", "inn.module_list.4.subnet1.0.bias", "inn.module_list.4.subnet1.2.weight", "inn.module_list.4.subnet1.2.bias", "inn.module_list.4.subnet2.0.weight", "inn.module_list.4.subnet2.0.bias", "inn.module_list.4.subnet2.2.weight", "inn.module_list.4.subnet2.2.bias", "inn.module_list.5.perm", "inn.module_list.5.perm_inv", "inn.module_list.6.subnet1.0.weight", "inn.module_list.6.subnet1.0.bias", "inn.module_list.6.subnet1.2.weight", "inn.module_list.6.subnet1.2.bias", "inn.module_list.6.subnet2.0.weight", "inn.module_list.6.subnet2.0.bias", "inn.module_list.6.subnet2.2.weight", "inn.module_list.6.subnet2.2.bias", "inn.module_list.7.perm", "inn.module_list.7.perm_inv", "inn.module_list.8.subnet1.0.weight", "inn.module_list.8.subnet1.0.bias", "inn.module_list.8.subnet1.2.weight", "inn.module_list.8.subnet1.2.bias", "inn.module_list.8.subnet2.0.weight", "inn.module_list.8.subnet2.0.bias", "inn.module_list.8.subnet2.2.weight", "inn.module_list.8.subnet2.2.bias", "inn.module_list.9.perm", "inn.module_list.9.perm_inv", "inn.module_list.10.subnet1.0.weight", "inn.module_list.10.subnet1.0.bias", "inn.module_list.10.subnet1.2.weight", "inn.module_list.10.subnet1.2.bias", "inn.module_list.10.subnet2.0.weight", "inn.module_list.10.subnet2.0.bias", "inn.module_list.10.subnet2.2.weight", "inn.module_list.10.subnet2.2.bias", "inn.module_list.11.perm", "inn.module_list.11.perm_inv", "inn.module_list.12.subnet1.0.weight", "inn.module_list.12.subnet1.0.bias", "inn.module_list.12.subnet1.2.weight", "inn.module_list.12.subnet1.2.bias", "inn.module_list.12.subnet2.0.weight", "inn.module_list.12.subnet2.0.bias", "inn.module_list.12.subnet2.2.weight", "inn.module_list.12.subnet2.2.bias", "inn.module_list.13.perm", "inn.module_list.13.perm_inv", "inn.module_list.14.subnet1.0.weight", "inn.module_list.14.subnet1.0.bias", "inn.module_list.14.subnet1.2.weight", "inn.module_list.14.subnet1.2.bias", "inn.module_list.14.subnet2.0.weight", "inn.module_list.14.subnet2.0.bias", "inn.module_list.14.subnet2.2.weight", "inn.module_list.14.subnet2.2.bias", "inn.module_list.15.perm", "inn.module_list.15.perm_inv". 
        Unexpected key(s) in state_dict: "inn.module_list.1.s1.0.weight", "inn.module_list.1.s1.0.bias", "inn.module_list.1.s1.2.weight", "inn.module_list.1.s1.2.bias", "inn.module_list.1.s2.0.weight", "inn.module_list.1.s2.0.bias", "inn.module_list.1.s2.2.weight", "inn.module_list.1.s2.2.bias", "inn.module_list.3.s1.0.weight", "inn.module_list.3.s1.0.bias", "inn.module_list.3.s1.2.weight", "inn.module_list.3.s1.2.bias", "inn.module_list.3.s2.0.weight", "inn.module_list.3.s2.0.bias", "inn.module_list.3.s2.2.weight", "inn.module_list.3.s2.2.bias", "inn.module_list.5.s1.0.weight", "inn.module_list.5.s1.0.bias", "inn.module_list.5.s1.2.weight", "inn.module_list.5.s1.2.bias", "inn.module_list.5.s2.0.weight", "inn.module_list.5.s2.0.bias", "inn.module_list.5.s2.2.weight", "inn.module_list.5.s2.2.bias", "inn.module_list.7.s1.0.weight", "inn.module_list.7.s1.0.bias", "inn.module_list.7.s1.2.weight", "inn.module_list.7.s1.2.bias", "inn.module_list.7.s2.0.weight", "inn.module_list.7.s2.0.bias", "inn.module_list.7.s2.2.weight", "inn.module_list.7.s2.2.bias", "inn.module_list.9.s1.0.weight", "inn.module_list.9.s1.0.bias", "inn.module_list.9.s1.2.weight", "inn.module_list.9.s1.2.bias", "inn.module_list.9.s2.0.weight", "inn.module_list.9.s2.0.bias", "inn.module_list.9.s2.2.weight", "inn.module_list.9.s2.2.bias", "inn.module_list.11.s1.0.weight", "inn.module_list.11.s1.0.bias", "inn.module_list.11.s1.2.weight", "inn.module_list.11.s1.2.bias", "inn.module_list.11.s2.0.weight", "inn.module_list.11.s2.0.bias", "inn.module_list.11.s2.2.weight", "inn.module_list.11.s2.2.bias", "inn.module_list.13.s1.0.weight", "inn.module_list.13.s1.0.bias", "inn.module_list.13.s1.2.weight", "inn.module_list.13.s1.2.bias", "inn.module_list.13.s2.0.weight", "inn.module_list.13.s2.0.bias", "inn.module_list.13.s2.2.weight", "inn.module_list.13.s2.2.bias", "inn.module_list.15.s1.0.weight", "inn.module_list.15.s1.0.bias", "inn.module_list.15.s1.2.weight", "inn.module_list.15.s1.2.bias", "inn.module_list.15.s2.0.weight", "inn.module_list.15.s2.0.bias", "inn.module_list.15.s2.2.weight", "inn.module_list.15.s2.2.bias".

Loading using self.load_state_dict(network_state_dict, strict=False) in ./models/model.py line 63 seems to fix the problem, but then the evaluation metrics are off by a lot.

Average:
3D Protocol-I z_0: 1761697.93
3D Protocol-I best hypo: 73987.92
3D Protocol-I median hypo: 2354026.03
3D Protocol-I mean hypo: 4192725.50
3D Protocol-I worst hypo: 49519483.01

3D Protocol-II z_0: 408.56
3D Protocol-II best hypo: 360.55
3D Protocol-II median hypo: 408.01
3D Protocol-II mean hypo: 406.66
3D Protocol-II worst hypo: 429.66

std dev per joint and dim in mm:
joint 0: std_x=0.00, std_y=0.00, std_z=0.00
joint 1: std_x=2713363.25, std_y=4497608.00, std_z=1229723.62
joint 2: std_x=5092118.50, std_y=5026269.00, std_z=3333644.75
joint 3: std_x=3927144.75, std_y=4752659.00, std_z=4638813.50
joint 4: std_x=2713363.25, std_y=4497608.00, std_z=1229723.62
joint 5: std_x=4988073.00, std_y=4202771.00, std_z=1055546.38
joint 6: std_x=4485964.50, std_y=5458925.00, std_z=1890290.62
joint 7: std_x=4712188.50, std_y=5419219.50, std_z=1223268.88
joint 8: std_x=3895103.75, std_y=6552087.00, std_z=1079990.88
joint 9: std_x=4310205.00, std_y=4260656.50, std_z=7185773.00
joint 10: std_x=7090483.50, std_y=5303002.50, std_z=1693970.62
joint 11: std_x=4958182.00, std_y=4558726.00, std_z=1272373.12
joint 12: std_x=4196097.00, std_y=4670686.50, std_z=1536391.38
joint 13: std_x=3967276.75, std_y=4828261.50, std_z=1302280.12
joint 14: std_x=8149503.50, std_y=4338247.50, std_z=1942508.12
joint 15: std_x=11636298.00, std_y=6795918.00, std_z=1937581.38
joint 16: std_x=4513685.00, std_y=917258.38, std_z=1015419.19

Would you maybe know what the issue might be?

[twehrbein]

Hi,

to me, this seems to be a problem with the FrEIA version. Are you sure that you are using the correct one? Did you follow the steps for creating the conda environment?

Other than that, it could be a problem with cpu execution. We did only test the program on different GPUs.

[PPierzc]

Hi, removing everything and reinstalling the whole package again seems to have solved the issue. Looks like FrEIA might have not installed correctly the first time around.

Running on the CPU works just fine, I am able to reproduce your results on it.

Thanks!